You should care about Configuration Management
There’s one question you really should be able to answer about your software project, whether you’re a project manager, a tester, an architect or a developer: “How are we going to put this safely into production and get it out again if we need to?” I think this is what Configuration Management tries to answer, but it gets bogged down in so many details that I’ve never been able to say that I’m able to understand what configuration management is. I think many “configuration management people” like to perpetuate the myth of “you don’t understand what configuration management is, so don’t even try.” As getting software into and out of production is a question of vital importance to everyone working with it, I think this is a really bad situation.
How do we answer the question, then?
There are a few things we need to find out about our software in order to get it safely into and out of production. First, we need to know what it is. That is, what pieces (“artifacts”) do we need to install, copy, edit, replace or otherwise modify in order to get the new (or first) version into production? Second, we need to know how we do it, that is, what commands and tools do we need to use to complete the job. Lastly, we need to answer the question of how do we do it safely. In this blog post, I will focus on the first two questions, leaving the last question for later. “Fools ignore complexity; pragmatists suffer it; experts avoid it; geniuses remove it.” – Alan Perlis The best way to manage the artifacts you need to put into production, is to reduce them in number. Avoiding unnecessary configuration, packaging things together, and avoiding complex installation procedures goes a long way to making the configuration manageable. When I control my environment, this is how I set a new version of software into production (slightly simplified): On all server hosts: 1. Unpack this zip-file to this location, 2. update this symbolic link (or system property), 3. execute this command to stop the old version, 4. execute that command to start the new version. Rolling back should be executing steps 2-4 again, but with the versions swapped around. The advantage if this approach is that it is not incremental. We don’t have to care much about what is already on the system when we install. Realistically, you can’t hope for you application to be as easy as I describe, but you can usually get by with just having to deal with the database in addition. A trick that works in many situations, is to make sure that all database changes are backwards compatible, and upgrade the database a week before you upgrade the application. Beyond the database, you can create a configuration management as streamlined as I describe! But when I don’t have control over the environment, painful things happen. My experience with Java EE is different. With out current WebSphere environment, we have to “rebuild” the configuration on the server cluster, run a very shaky deployment + configuration script and/or go through a multi-step web-application to install the new version. The old version has to be stopped as much as 30 minutes while this process goes on. The “software” consists of application server configuration, ear-files, and for reasons I never understand, sundry other files that people toss around. This is not good enough, and as a result, people very often upgrade parts of the application at a time, which means that you often lose track of what’s in production. My experience with Java SE servers vary, but in the worst situations, there’s nothing that you could call configuration management, various versions of jar-files (often renamed to foo.jar-old, foo.jar-backup, foo.jar-1, try and guess which one was the last version!), ditto configuration files, scripts and other programs. You need to be very thorough when you describe the upgrade procedures for such a system. The final problem is the myriad of tools that seems to give no thought to configuration management. Especial visual development tools, Enterprise Portals and Enterprise Application Integration platforms seem to be totally unaware of this problem. With no clear and automated way to propagate changes from a development to a test to a production environment, these tools seem to cost much more in operation than I have ever witnessed that they can save in development. I have seen many organizations evaluate such tools only on how they look in a demonstration setting, and not on how they affect your ability to safely upgrade and rollback software in production environments. I didn’t recognize why at the time, but the fact that Ruby-on-Rails addresses this question from the very beginning (via Capistrano), is one of the big reasons that I love it so much. My plea is that you don’t suffer a complex configuration management in order to satisfy the demands of some tool or standard. Second, start thinking about the steps you need to install or upgrade your server now! How you put software into (and out of) production is possibly the most critical question you have to ask. Make sure you can understand the answer, and don’t let a “configuration management guru” tell you otherwise!
Comments:
[Tomas] - Apr 10, 2007
Thanks, Johannes!
You had a small typo - “Capstrano” should be “Capistrano”
Capistrano http://manuals.rubyonrails.com/read/chapter/97#page256