As your Turnkey or other MDriven project grows in popularity you may want to scale out to multiple servers. Even if the load of a particular server is not the reason, regional distance might be. With distance comes latency and latency kills the joy of almost any system.
To work around load and latency we now offer a distribution mechanisms in MDrivenServer.
The strategy behind the MDrivenServer distribution is based on the assumption that data reads are much more common than data writes. Luckily this is the case in all administrative applications I can come to think of.
Distribution normally pose a couple of problems:
- There is a tradeoff between distribution and consistency – meaning if you have the same data in multiple places far apart – you cannot always be sure it is exactly equal given a point in time.
- A distributed solution is harder to maintain because there are more nodes involved in the solution.
MDrivenServer solves both these problems in a rather elegant way.
The number 1 issue; consistency
The number 1 issue is a fact of physics and we just need to make sure that we minimize the suffering due to it. MDrivenServer solves this by appointing only 1 MDrivenServer as Master. It is only the Master that actually receives data writes from users. All MDriverServer Slaves are connected to this Master and when users of a Slave is saving data – the MDrivenServer Slave route that datablock to the its Master. Once the Master accepts the message the Slave asks the Master if there are any news ready for distribution. The Master then answers with the newly commited datablocks and the Slave merge these to its own database.
The whole operation is done in the matter of split seconds. The Slave continuously poll the Master for changes – so given enough time and clear internet pathways the Slave will have the same content as the Master.
In the event of network failure so that the Slave cannot reach the Master – then users will not be able to save data – and the Slave will not get updated data from the Master. This will all rectify itself once the Slave can reach the Master again.
It is always the Slave that initiates the communication. The Master just accepts save requests and deliver committed updates.
MDrivenServers that participate in this cluster all need an extra table added to their database. The table make sure that the commitblocks that has been applied to the Master is backed up along the rest of the data in the database. The transaction updating the business data is also used for the commitblock information. This way the Master is consistent even when restored from a backup.
Since the extra table structure is equal for the Master and Slave(s) it is easy to use a backup of a Master-db to deploy it as a new Slave.
The number 2 issue; maintenance
Your system is constantly evolved – and the model is changed on a monthly, weekly or even daily basis. The data in one given commitblock is created in the context of one model version only. This means that Slaves cannot be allowed to have a different model than the Master. As you may have a Slave in every part of the world you can imagine the administrative work to keep this cluster configured correctly.
To manage this we write a checksum of the current model along with each commitblock we persist in the Master. As a Slave receives newly created commitblocks in may discover that the model checksum used to create the commitblock differs from the model the slave currently use.
If the Slave discovers that it uses the wrong model – it asks the Master for the model needed – using the model checksum passed along the commitblock. The Master then returns this model to the Slave and the Slave engage in an automatic evolve of its database. Once the evolve is fulfilled the commitblock can be safely applied.
The described mechanism makes it possible for you to sit back and deploy to one Master node and trust that Slaves you have distributed over the world or stacked next to each other to handle load – will be maintained automatically.
Q: Can this be used with my own server built with the MDriven Framework? Are there any reusable components that I can use?
The main thing is the ability to catch the commitblocks used for SyncHandler inside the db transaction that will apply it:
var res = new Eco.Persistence.SyncHandler(); res.OnSubmittedCommitBlock += Res_OnSubmittedCommitBlock;
Once you are able to do that you can catch the commitblock and save it to the same db – inside the same transaktion:
var updP = (e.OperationsParams as TUpdateParameters);
if (updP != null)
{
foreach (IDatabase db in updP.Databases)
{
var query = db.GetExecQuery();
try
{
query.AssignSqlText(“insert into MDrivenServerSynk (Time,CommitBlock,modelid,slaveMergeTime) values (:inserttime,:CommitBlock,:modelid,:slaveMergeTime) “);
And once you can do that you have your slaves copy these commitblocks and apply them to their own db.
The other thing is to be able to catch all updating calls to the slave and reroute those to the master instead. A commitblock will be created and applied by the well behaving slave.
The third thing is to have the Slave discovering Master model changes – and auto-evolve so that the commitblocks are applied to the correct environment at all times
MDrivenServer does these 3 things for you.