The madness of layered architecture
I once visited a team that had fifteen layers in their code. That is: If you wanted to display some data in the database in a web page, that data passed through 15 classes in the application. What did these layers do? Oh, nothing much. They just copied data from one object to the next. Or sometimes the “access object layer” would perform a check that objects were valid. Or perhaps the check would be done in the “boundary object layer”. It varied, depending on which part of the application you looked.
Puzzled (and somewhat annoyed), I asked the team why they had constructed their application this way. The answer was simple enough: They had been told so by the expensive consultant who had been hired to advice on the organization’s architecture.
I asked the team what rationale the consultant had given. They just shrugged. Who knows?
Today, I often visit teams who have three to five layers in their code. When asked why, the response is usually the same: This is the advice they have been given. From a book, a video or a conference talk. And the rationale remains elusive or muddled at best.
Why do we construct layered applications?
There’s an ancient saying in the field of computing: Any problem in computer science can be solved by adding a layer of indirection.
Famously, this is the guiding principle behind our modern network stack. In web services SOAP performs method calls on top of HTTP. HTTP sends requests and receives responses on top of TCP. TCP streams data in two directions on top of IP. IP routes packets of bits through a network on top of physical protocols like Ethernet. Ethernet broadcasts packets of bits with a destination address to all computers on a bus.
Each layer performs a function that lets the higher layer abstract away the complexities of for example resending lost packets or routing packets through a globally interconnected network.
The analogy is used to argue for layers in enterprise application architecture.
But enterprise applications are not like network protocols. Every layer in most enterprise application operates at the same level of abstraction.
To pick on a popular example: John Papa’s video on Single Page Applications uses the following layers on the server side (and a separate set on the client side): Controllers, UnitOfWork, Repository, Factories and EntityFramework. So for example the AttendanceRepository property in CodeCamperUnitOfWork returns a AttendanceRepository to the AttendanceController, which calls GetBySessionId() method in AttendanceRepository layer, which finally calls DbSet.Where(ps => ps.SessionId == sessionId) on EntityFramework. And then there’s the RepositoryFactories layers. Whee!
And what does it all do? It filters an entity based on a parameter. Wat?!
(A hint that this is going off the rails is that discussion in the video presentation starts with the bottom and builds up to the controllers instead of outside in)
In a similar Java application, I have seen - and feel free to skip these tedious details - the SpeakersController.findByConference calls SpeakersService.findByConference, which calls SpeakersManager.findByConference, which calls SpeakersRepository.findByConference, which constructs a horrific JPAQL query which nobody can understand. JPA returns an @Entity which is mapped to the database, and the Repository, or perhaps the Manager, Service or Controller, or perhaps two or three of these, will transform from Speaker-class to another.
Why is this a problem?
The cost of code: A reasonable conjecture would be that the cost of developing and maintaining an application grows with the size of the application. Adding code without value is waste.
Single responsibility principle: In the above example, the SpeakerService will often contain all functionality associated with speakers. So if adding a speaker requires you to select a conference from a drop-down list, the SpeakerService will often have a findAllConferences method, so that SpeakersController doesn’t need to also have a dependency on ConferenceService. However, this makes the classes into functionality magnets. The symptom is low coherence: the methods of one class can be divided into distinct sets that are never used at the same time.
Dumb services: “Service” is a horrible name for a class - a service is a more or less coherent collection of functions. A more meaningful name would be a “repository” for a service that stores and retrieves objects, a Query is a service that selects objects based on a criteria (actually it’s a command, not a service), a Gateway is a service that communicates with another system, a ReportGenerator is a service that creates a report. Of course, the fact that a controller may have references to a repository, a report generator and a gateway should be quite normal if the controller fetches data from the database to generate a report for another system.
Multiple points of extension: If you have a controller that calls a service that calls a manager that calls a repository and you want to add some validation that the object you are saving is consistent, where would you add it? How much would you be willing to bet that the other developers on the team would give the same answer? How much would you be willing to bet that you would give the same answer in a few months?
Catering to the least common denominator: In the conference application we have been playing with, DaysController creates and returns the days available for a conference. The functionality needed for DaysController is dead simple. On the other hand TalksController has a lot more functionality. Even though these controllers have vastly different needs, they both get the same (boring) set of classes: A Controller, a UnitOfWork, a Repository. There is no reason the DaysController couldn’t use EntityFramework directly, other than the desire for consistency.
Most applications have a few functional verticals that contain the meat of the application and a lot of small supporting verticals. Treating them the same only creates more work and more maintenance effort.
So how can you fix it?
The first thing you must do is to build your application from the outside in. If your job is to return a set of objects, with .NET EntityFramework you can access the DbSet directly - just inject IDbSet in your controller. With Java JPA, you probably want a Repository with a finder method to hide the JPAQL madness. No service, manager, worker, or whatever is needed.
The second thing you must do is to grow your architecture. When you realize that there’s more responsibilities in your controller than deciding what to do with a user request, you must extract new classes. You may for example need a PdfScheduleGenerator to create a printable schedule for your conference. If you’re using .NET entity framework, you many want to create some LINQ extension methods on e.g. IEnumerable (which is extended by IDbSet)
The third and most important thing you must do is to give your classes names that reflect their responsibilities. A service should not just be a place to dump a lot of methods.
Every problem in computer science can be solved by adding a layer of indirection, but most problems in software engineering can be solved by removing a misplaced layer.
Let’s build leaner applications!
Comments:
[Henrik Eriksson] - Jul 21, 2014
Not sure why there’s a difference or if it really matters, package is just a technical level of abstraction (depending on layered context), not function. Also by not separating it you miss out on, let’s say OSGI or Jigsaw or other class loading/jar bundling techniques. I do agree that packages are a bit lame (particulary the language support could be better, again jigsaw) but in a OOP language you need a hierarchical structure (other than field,method,class) to be able to help you get things right.
I do however to agree with your statement of building applications outside in, but just by doing that you are and must create layers (you’ll have at least one). I do believe that creating layers because the sake of layers it’s never a good thing, the context of a layer shouldn’t give appearance of that there are more layers. And I think most of the layered applications out there are broken because of mixup of where the code should actually reside and most of the solutions out there are too imperative and not conveying any sort of functional context. I usually speak of that a layer is some sort of DMZ of the relayed function.
The example of SpeakerX.findByConference() seems to be a valid case of wasted code. But in general one has to look at the code to make that decision and then it turns tricky.
Johannes Brodwall - Jul 19, 2014
To some extent it may be an instance of the phenomenon you describe.
On the other hand, I see a lot of respected thought leaders advocating layers as a primary categorization of code. In many technologies, this is a practice that leads to lots of redundant code (as you also describe).
The fact that this is actively advocated make me feel that it may be more than a poor execution.
J. B. Rainsberger - Jul 18, 2014
I agree, but now we’re back to the age-old problem of “Many people don’t do X well, so X must be a bad idea.” I don’t think layering, the concept, is madness, unless we combine it with something like the Surgical Team (only one person understands the point of the layers) or Code Ownership (I can’t collapse or bypass a layer when it makes sense) or… you get the idea.
Johannes Brodwall - Jul 18, 2014
I think that the idea of uniform layering in an application - that is layers for the sake of layers - is a mistake and one that hurts many software systems today.
In Java, a way of thinking about this is the package structure. If the main organizing elements in the source code is layers (e.g. com.company.application.web, com.company.application.persistence), the application is suffering from this. I’ve also seen the Mock object situation that you use and used the same criteria myself.
Johannes Brodwall - Jul 18, 2014
Having a “repository” class is a good way to encapsulate responsibility and improve maintainability. Having a “repository” package (or namespace) is not.
[Henrik Eriksson] - Jul 18, 2014
I do think you actually need layers otherwise you are not coding correctly and therefore providing to a unmaintainable system. However it feels like most of the time people focus on to create abstractions rather than create context. A layer should convey a meaning otherwise it’s not possible to provide an abstraction. Also I think there is a misunderstanding or abuse of interfaces, which adds to the mess of layers. And to make it more complex, layers are not specific to a system but rather to a function and how a layer should look like depends on the context of the function.
[Glenn Bech] - Aug 7, 2014
What is a layer, really?
I have never encountered runtime enforcement of call stack correctness nor software designs that stops arbitrary instantiations and method invocations. So, I see layering to be a property of the design- not of the software itself. They exist in our minds ( and are usually spawned in the dark pits of software architecture documents).
If we apply the Single Responsibility Principle to our design, layers become clusters of classes with similar responsibilities. This makes sense. A layered design also add the sometimes wanted rigidity of a constraint on direction. This can help avoid unwanted dependency cycles. So, I don’t agree that layers are always a bad thing. But, Mindless software design is always a bad. I think we,as developers, should ask the very provocative question of “why” more often. Two colleague of mine recently blogged about related topics (here - http://www.inbeta.no/2014/05/09/hvorfor/ - and here http://www.inbeta.no/2014/05/30/a-vaere-en-rebell/)
We must dare to stand up and speak when the team is running over the cliff like lemmings, wearing t-shirts saying “this is how we do it around here”.
[cblin] - Jul 14, 2014
indeed a great article to say that we should focus on business components instead of technical layers.
that is to say, when you are performing very simple CRUD, there is no value to separate data access from business logic since there is nearly zero logic…
on the contrary, when you perform a very hard algorithm that is the heart of your system, you should take care to isolate it so that you can
so layers are not useless, but you should take care to add one only when the business use case needs it (and because one use case needs it does not mean all the application needs it)
[BK] - Jul 13, 2014
I think that’s a key - if the same concepts appear in each layer, something is wrong.
Layers can be very useful - particularly when you mix “hard” and “soft” layers. But they should address genuinely different issues.
Look at the numpy stack, for example.One layer for memory handling and optimised array calculations; the layer on top of that for expressing the application logic.
[Jan Bols] - Jul 12, 2014
I’m currently working on a project that goes something like this: 1. A ComponentController has a method getComponent that takes a UUID in String format and returns a Component structure in json. 2. The ComponentController calls a ComponentService and gets a ComponentForm containing only String properties that is converted into json by the Spring MVC framework. 3. The ComponentService calls a ComponentQueryService that returns a ComponentDTO. The ComponentDTO is converted into a ComponentForm. 4. The ComponentQueryService calls another ComponentService in another bounded context. It gets another ComponentDTO that is converted into this ComponentDTO. 5. The ComponentService in the other bounded context calls a ComponentRepository that returns a Component hibernate entity that is converted into the ComponentDTO. 6. Finally, the ComponentRepository calls hibernate to get the Component hibernate entity.
Wonderful!
However, there is a difference between a responsibilities of a controller and that of a repository or a service. The controller’s responsibility is to translate the http request into something http agnostic and translate the result of the service call into some form the client understands. The repository’s responsibility is to handle all nasty db details when getting and storing data. The result is that the caller of the repository is unaware of the persistence details. Separating these responsibilities into layers is still valuable IMHO.
The result of the layers is indeed a fragmentation of code among controller, service and repository but proper use of packages can still keep them close to each other.
J. B. Rainsberger - Jul 13, 2014
Needless indirection only adds cost, certainly. This article’s title, however, suggests the claim that layering (in general) is madness. I don’t think you mean that, do you?
I tend to use mock objects to help me identify wasteful layers: the resulting tests tend to look strange, such as would be the tests for SpeakersController.findByConference(), which could simply check either stub or set an expectation on SpeakersService.findByConference(), probably ensuring that the former passes its parameters directly to the latter. That the test appears not to check anything interesting points to indirection without abstraction. (Of course, in this case, I can’t imagine needing a mock object to tell me that the Controller -> Service interaction needs to go.)
[Kalanamith Mannapperuma] - Jul 10, 2014
Correct me if I am wrong, if go to lower level languages there is no concept called layering, but those applications are efficient and fast relative to hardware facilities which was available that time. Where is the bottleneck with modern higher level languages?
Johannes Brodwall - Jul 10, 2014
The bottleneck is in our brains. :-)
In old-time C programs there wouldn’t be layers in the way they are in modern C# and Java programs. What you would see is things like the OSI-model which employed layers in a meaningful way. Or the layer of memory management and the layer of logic on top of it.
But in such a system of layers, the same business concept (e.g. Customer) would never need to exist in more than one layer
[gamsjo] - Jul 11, 2014
Nice Johannes!
I remember this layering thinking made a lot of sense for the ISO-stack some 20 years ago..
The analogue hardware guys did their mysterious stuff at the bottom. Then the digital hardware did their signal processing using assembly or maybe some lines of c. Then the “proper programming” started at the Link layer, but still pretty low level. At the next levels the idea was that we could use special purpose techniques for the different abstractions. We dreamt of 4th gen and even 5th gen languages for the highest layers.
So I guess one origin for this thinking may have been specialisation; People are experts on their level - and should only need to talk to (and have some understanding of) the level above and beneath.