Thinking Inside a Bigger Box

Warning: This article requires a lot of editing love before it is very useful. It might be somewhat incoherent. Read at your own risk. ;-)

Silo (software): A silo system cannot easily integrate with any other system.

In software, the term “silo” is used to refer to a system that is constructed as one unit from the front end to the data storage. Everything is tied together to work with the rest of the silo, but not with other elements.

This is considered a bad thing.

The problem comes when we want to integrate the system with other systems or reuse parts of the system.

Many of the new ideas in software development has as one of their big goals to try and rectify the silo problem. In general this is achieved by splitting up the system into services that may or may not be distributed across different computers.

But the badness of the silo hinges on two claims: First, that the applications built as an integrated stack cannot be integrated with, and second, that a system of distributed services can be integrated with.

Fact 1: Layers reduce understanding

One way that services are introduced is as an abstraction on top of an existing system. More often that we’d think, that system is again an abstraction upon an older system. Each layer was added in an attempt to make it easier for someone to integrate with the system. Each layer makes the whole harder to understand.

Just a few weeks ago, I experienced a profound sense of deja vu as I had gradually peeled of layers of a set of services and discovered that integrating directly to the ultimate data source was the easiest option. Each layer had added confusion, complexity, delays and unneeded constraints. Integrating with the “silo” was easier than integrating with the service layers on top of it.

Fact 2: Silos often aren’t

We’re currently building Java-systems that will extend and ultimately (hopefully) replace systems running on old mainframes. The common perception is that the mainframe apps are silos. They are hard to understand and impossible to split up.

Have you tried? As it turned out, of our many Java-developers, almost none have dared to started looking into how the mainframe works. When someone finally did, he found that it was fairly easy to integrate with. It requires hard work, but when did integration not?

Fact 3: Services might not be

“Fine,” you say, “but what about large grained services?” My experience is that even these are hits and misses. For example, one would think that distributing information via postal mail, electronic channels and fax might be a reusable service. It can be, but it could also be a cost driver.

Even when it comes to large scale services, reuse is costly. If your “enterprise architecture” focuses on what services should be reused by whom, you might be wasting a lot of time for a lot of people.

Ode to the silo

Maybe the classical silo isn’t as bad as we’re often told. Focusing on just what the system needs to do and dealing with the integration by talking to others sure is expensive. But maybe trying to architect a reusable set of services is just as expensive.