The Myth of the Silo
Warning: This article requires a lot of editing love before it is very useful. It might be somewhat incoherent. Read at your own risk. ;-) Silo (software): A silo system cannot easily integrate with any other system.
In software, the term “silo” is used to refer to a system that is constructed as one unit from the front end to the data storage. Everything is tied together to work with the rest of the silo, but not with other elements.
This is considered a bad thing.
The problem comes when we want to integrate the system with other systems or reuse parts of the system.
Many of the new ideas in software development has as one of their big goals to try and rectify the silo problem. In general this is achieved by splitting up the system into services that may or may not be distributed across different computers.
But the badness of the silo hinges on two claims: First, that the applications built as an integrated stack cannot be integrated with, and second, that a system of distributed services can be integrated with.
Fact 1: Layers reduce understanding
One way that services are introduced is as an abstraction on top of an existing system. More often that we’d think, that system is again an abstraction upon an older system. Each layer was added in an attempt to make it easier for someone to integrate with the system. Each layer makes the whole harder to understand. Just a few weeks ago, I experienced a profound sense of deja vu as I had gradually peeled of layers of a set of services and discovered that integrating directly to the ultimate data source was the easiest option. Each layer had added confusion, complexity, delays and unneeded constraints. Integrating with the “silo” was easier than integrating with the service layers on top of it.
Fact 2: Silos often aren’t
We’re currently building Java-systems that will extend and ultimately (hopefully) replace systems running on old mainframes. The common perception is that the mainframe apps are silos. They are hard to understand and impossible to split up. Have you tried? As it turned out, of our many Java-developers, almost none have dared to started looking into how the mainframe works. When someone finally did, he found that it was fairly easy to integrate with. It requires hard work, but when did integration not?
Fact 3: Services might not be
“Fine,” you say, “but what about large grained services?” My experience is that even these are hits and misses. For example, one would think that distributing information via postal mail, electronic channels and fax might be a reusable service. It can be, but it could also be a cost driver. Even when it comes to large scale services, reuse is costly. If your “enterprise architecture” focuses on what services should be reused by whom, you might be wasting a lot of time for a lot of people.
Ode to the silo
Maybe the classical silo isn’t as bad as we’re often told. Focusing on just what the system needs to do and dealing with the integration by talking to others sure is expensive. But maybe trying to architect a reusable set of services is just as expensive.
Comments:
[Eivind] - Jun 16, 2008
Audacious assertions! Fact 1: “Each layer was added in an attempt to make it easier for someone to integrate with the system. Each layer makes the whole harder to understand.” Would you call this a silo? Each layer is probably added for, and probably serves, its purpose of adaption or adding value. Different layers may exist in parallell or on top of each other. New and different integration needs may solved on top of previous, lower layers. That seems natural. Unless you only conceive the top as being “the service”. Fact 2: Important learning. Silos may be only apparent and present integration interfaces. Thanks. Fact 3: Yes, few examples of successful, euphoric, large-grained integration and reuse exist, and probably for good reasons. But isn’t there an agile, medium-grained middle way between that and silo thinking? Ode to silos that in fact aren’t and to agile, timely integration?
Johannes Brodwall - Jun 16, 2008
I don’t think of the layered system that I’m talking about as a silo. Rather, I think of it as a business level service composed of several component services. What characterizes such system is a great deal of fragmented responsibility.
The people responsible for a single layer generally seem to consider themselves service providers for the layers above.
This is not a agile way of creating software. But it’s a common implementation of medium-grained SOA services.
Thanks for the kind words, Eivind. Your closing remark is exactly what I wanted to express.