Thinking Inside a Bigger Box

As I release more frequently, I start to focus on automating the actual process of deploying a release. One of the most powerful steps of automating deployment is to automatically upgrade the database schema.

This technique first saw mainstream use with the Ruby-on-Rails framework. Today, there are several mature tools that will help you organize and execute database changes (Scala Migrations, Ruby-on-Rails Migrations, dbdeploy, Liquibase, dbmaintain). And if none fit you perfectly, it’s easy to create your own.

In my current project, we have rolled our own solutions for this:

• All changes to the database are stored as SQL-files that are packaged into the deployment unit (in our case, a WAR-file). These files will usually contain statements like “ALTER TABLE ADD COLUMN” and “CREATE TABLE”. To get the files executed in the right order, we name the files with an increasing sequence number, like 012-add_payment_type_to_customer.sql.
• Whenever the application is started, it looks for a table named “MIGRATIONS” in the database and creates it if it doesn’t exist.
• At startup, the application looks through the list of migration files it has been packaged with and sees which file names don’t have an entry in the MIGRATIONS table.
• The application executes all the scripts that haven’t been executed already. If any script fails to execute, it makes a note of the error in the MIGRATIONS table and refuses to start the application

We run the migration procedure every time we start up the application, whether it is in test or production. Even the JUnit tests that access the database will run any pending migrations before starting. The result is that any database change that we intent to roll out into production will at the very least be executed once on each developers private copy of the database, as well as once on the continuous integration server. By the time they get executed in a controlled testing environment, we’re pretty confident that they work as intended.

Some migration tools use a more friendly (and portable) syntax than SQL DDL statements. Many allow for rollback of migrations. Most don’t automatically execute the pending migrations on application start, but require a separate command to execute them.

Your first step towards automating database migrations is to make sure that every change in the database is represented by some sort of script and that all these scripts are versioned with the rest of your code. From there, you can improve your process when you notice a step in the process that seems to involve too much work or risk.

Automating the deployment process will reduce the need for documentation and the opportunity for errors during one of the most critical times in the project. It is especially important to reduce the possibility of miscommunication and mistyping if the people responsible for deployment are in a separate organizational unit, which often seems to be the case. Make their job as easy as possible!