Documentation:SQL Server change tracking

Background

The MDrivenServer has build in change tracking. It's implemented as a "dirty list" of objects that all clients can ask for. When the client gets this list, it markes all matched objects in memory as "dirty" and discards them. If any of these objects are present in a user interface (that is currently shown), these objects will be immediately reloaded from the MDrivenServer/ SQL Server.

MDrivenServer's dirty list is not a part of SQL Server and out of the box the MDrivenServer won't find out if some other client or service changes the underlying tables directly in the SQL Server.

The need for external dirty list handling (aka. change tracking) is often needed only when there's legacy system reading and writing to the same SQL database. Because of this, it's usually not a good idea to have it on all tables in the database. You probably only want change tracking on the tables that can be affected by the non-MDriven system.

MDriven implementation

Support for change tracking in SQL Server

Tracking of changes is available from SQL Server 2008 and in Azure SQL. It's available in the Express editions, so you can use it in the built in Turnkey database.

Background about this on MS site

https://docs.microsoft.com/en-us/sql/relational-databases/track-changes/about-change-tracking-sql-server

Article with a lot of details: https://docs.microsoft.com/en-us/sql/relational-databases/track-changes/work-with-change-tracking-sql-server

How we propose you use the MDriven features to do change tracking

We want to put as little extra load as possible on the SQL Server. We therefore use to different jobs to detect that something changed and then what has changed

• Something changed: The stored procedure CHANGE_TRACKING_CURRENT_VERSION() can be used to know if ANY change has occurred. We compare this version value with the value when we last processed change information.

• What has changed: If there's a change, we iterate change information for all tables we're interested in.

Both of these methods use sqlpassthroughobjects to directly use SQL to return information.

It then uses SuspectExternalUpdateInvalidate to "invalidate", make dirty, the changes objects.

Support with other database servers

The MDriven implementation is not dependent on HOW you find out what has changed. As long as you can find the primary key of changed rows, you can use this functionality.

Applying change tracking

Preparing your SQL-server

The suggested pattern to allow for MDriven Applications to effectively discover changes to data made by others is this. Turn on change tracking on the db and on the tables you want to discover foreign changes to

To turn change tracking on, do like this

ALTER DATABASE <database name>
SET CHANGE_TRACKING = ON  
(CHANGE_RETENTION = 2 DAYS, AUTO_CLEANUP = ON) 

Turn on change tracking for each table you want to track

ALTER TABLE <table name>
ENABLE CHANGE_TRACKING

You do NOT need column track, we track whole rows/objects.

Model additions

Add attributes in SysSingleton

• CurrentVersionNumberFromChangeTracking: Int64?

• LastHandledVersionNumberFromChangeTracking: Int64?

Viewmodel additions

Read more on Serverside actions

Create a MDrivenServer server side job that asks sql server for the current change track version

Create a viewmodel SS_GetCurrentTrackingVersion rooted in SysSingleton

Add the CurrentVersionNumberFromChangeTracking attribute to the viewmodel

Add action called GetCurrentVersion with this EAL

self.CurrentVersionNumberFromChangeTracking := SysSingleton.sqlpassthrough( 
'-- Get the current tracking version number
  DECLARE @CurrentVersionNumberFromChangeTracking bigint;
  SET @CurrentVersionNumberFromChangeTracking = CHANGE_TRACKING_CURRENT_VERSION();
  select @CurrentVersionNumberFromChangeTracking', Int64 )->first.Part1

Add critera for serverside execution to the viewmodel with an PS Ocl like this

SysSingleton.allInstances

Create a MDrivenServer server side job that invalidates changed objects

• Create a viewmodel SS_InvalidateExternalChanges rooted in SysSingleton

• Add the CurrentVersionNumberFromChangeTracking and LastHandledVersionNumberFromChangeTracking attributes to the viewmodel

• Add variable vHandleToVersionNumber as Int64

• Add action called Invalidate with this EAL

-- Move to variable avoiding change of version during execution
vHandleToVersionNumber := vCurrent_SS_InvalidateExternalChanges.CurrentVersionNumberFromChangeTracking;
-- Invalidate changed objects
selfVM.SuspectExternalUpdateInvalidate(<classname>.sqlpassthroughobjects( 'SELECT <primarykeyattributename> FROM CHANGETABLE (CHANGES <tablename>, ' + LastHandledVersionNumberFromChangeTracking.toString + ') AS C') );
selfVM.SuspectExte....
selfVM.SuspectExte....
selfVM.SuspectExte....
-- Update last handled change number
self.LastHandledVersionNumberFromChangeTracking := vHandleToVersionNumber

The selfVM.SuspectExternalUpdateInvalidate..... is the one you need to adapt to your tables that you are tracking.

One row for each of the classes/tables you're tracking.

• Add critera for serverside execution to the viewmodel with an PS Ocl like this

SysSingleton.allInstances->select(ss|ss.CurrentVersionNumberFromChangeTracking <> ss.LastHandledVersionNumberFromChangeTracking)

This means that the job isn't even invoked is there's nothing to do.

Recommended interval

Interval has to be adjusted depending on need and load on the server.

But it can be quite low, because it's a very fast operation. For example 5 seconds, or what suits your need of discovery.

The detection takes about 10 ms to run and getting changes for 40 tables (without changes) around 100 ms.