Continuous Integration with MongoDb – MongoDB, build parties, and deploying your web application at 11am on a Wednesday

Today I’ve seen a very interesting post in MongoDb blog:

MongoDB, build parties, and deploying your web application at 11am on a Wednesday

What’s interesting about it is that in our current project, we’ve been going to implement the same scheme for quite some time, but still haven’t got around to do it. It seems that other people did, though, so let’s hope we will too – apparently it’s humanely possible.

The idea is simple: if you update your data in large batches, there’s going to be SOME downtime. Specifically, if the schema changes, the whole table will be locked for the duration of a DDL operation, and if it’s a large table, then it’s going to be a LONG downtime. And once your business starts growing and expanding into different parts of the world, the time window for maintenance decreases, as it’s always business time SOMEWHERE – right?

So, that’s where MongoDb comes into play, – because it’s schemaless. So, you don’t have DDL operations! Still, you might have heavy updates, that is, time-consuming DML operations, and they too are not a joke. And that’s where the main concept of single-item updates appears.

The concept works the following way:

  • each entity has a version number (if it’s not set, consider it to be 0);
  • when entity is read from DB, it’s updated on-the-fly and the version is incremented;
  • when entity is saved back to DB, it’s saved with that new version;
  • for this to be feasible, all work on the entity should be done from one place (i.e. a DAO layer class);
  • this means another layer of abstraction – a migration layer – between your DAO and business logic;
  • the rest of the data (untouched, unused) can be updated in small batches with a recurring background thread untill all the entities have a desired version.

The author of the original article, Sean Reilly, also points out some things that should be attended to, like, what to do when a primary key changes or gets renamed, or when a field that’s part of a unique index gets renamed (effectively making the value of the index field to be NULL, and only one NULL value is allowed for a unique index, so this is where one might make use of sparse indexes).

All in all, a very good article! We can definitely make use of it.


About Maryna Cherniavska

I have productively spent 10+ years in IT industry, designing, developing, building and deploying desktop and web applications, designing database structures and otherwise proving that females have a place among software developers. And this is a good place.
This entry was posted in Uncategorized and tagged . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s