Today I’ve seen a very interesting post in MongoDb blog:
What’s interesting about it is that in our current project, we’ve been going to implement the same scheme for quite some time, but still haven’t got around to do it. It seems that other people did, though, so let’s hope we will too – apparently it’s humanely possible.
The idea is simple: if you update your data in large batches, there’s going to be SOME downtime. Specifically, if the schema changes, the whole table will be locked for the duration of a DDL operation, and if it’s a large table, then it’s going to be a LONG downtime. And once your business starts growing and expanding into different parts of the world, the time window for maintenance decreases, as it’s always business time SOMEWHERE – right?
So, that’s where MongoDb comes into play, – because it’s schemaless. So, you don’t have DDL operations! Still, you might have heavy updates, that is, time-consuming DML operations, and they too are not a joke. And that’s where the main concept of single-item updates appears.
The concept works the following way:
- each entity has a version number (if it’s not set, consider it to be 0);
- when entity is read from DB, it’s updated on-the-fly and the version is incremented;
- when entity is saved back to DB, it’s saved with that new version;
- for this to be feasible, all work on the entity should be done from one place (i.e. a DAO layer class);
- this means another layer of abstraction – a migration layer – between your DAO and business logic;
- the rest of the data (untouched, unused) can be updated in small batches with a recurring background thread untill all the entities have a desired version.
The author of the original article, Sean Reilly, also points out some things that should be attended to, like, what to do when a primary key changes or gets renamed, or when a field that’s part of a unique index gets renamed (effectively making the value of the index field to be NULL, and only one NULL value is allowed for a unique index, so this is where one might make use of sparse indexes).
All in all, a very good article! We can definitely make use of it.