The [TestCase] lets us easily have several test cases implemented by the same method. FluentAssertions adds the Should() extension method to all objects (and primitives, too) that let’s us write things like ToRoman(3333).Should().Be("MMMCCCXXXIII").
Add some sugar to your C# tests today. What’s your favorite ways of writing sweeter tests?
After having worked with Scrum for a number of years, I still witness sprint reviews where the team’s demonstration of the product is confusing and the value produced in the sprint is unclear. The demo may consist of just a bunch of different functions and screens without any meaning. Or maybe the team is just talking about what happens behind the curtains in the database. Or maybe the demo just doesn’t display the value that the team was supposed to give to the stakeholders. Most teams have okay demos most of the time, but every now and then, it’s a complete train wreck.
If you’ve experienced a sprint like this, you probably noticed some problems from the very beginning. The sprint planning may have been chaotic and the work during the sprint may have felt purposeless. Chances are that the team was most of the time discussing technical terms that didn’t make much sense to the product owner.
If your team learns to be clear about the sprint goal, you can avoid anemic demos, unstructured planning and undirected work. “But wait”, you may say, “we had a sprint goal and our demo still sucked”. You may think you have a clear sprint goal, but very few teams know what a sprint goal looks like. You may have one, but you might have gotten there by accident.
Here is what a sprint goal looks like: At the end of this period of time, we will demonstrate something to our stakeholders. What we will demonstrate will tell a compelling story that demonstrates real value. We can create the first draft plan of what the demo will look like already at the sprint planning, and we can use this description both to verify our understanding and plan our work.
Let’s say that the product owner says “the goal for the next sprint is to verify the payment solution.” What would a plan for this sprint look like?
At a sprint planning meeting, after the product owner has described the goal, the team plans its work. Then they come back to the product owner to verify that their plan matches the goal. This is what a good plan may sound like: “We will set up so that all users on the web site has a random set of items in their shopping cart. Then we will go to the checkout page. Here, we will see that the shopping cart is displayed in a reasonable way. When we click the payment button, the user will be redirected to the test site for payment provider. We’ll input credit card details and pay. The user will be redirected back to the web site and the web site will display the success or the failure of the payment. We’ll also show the order along with the payment status in a early mockup of the order list page”
The product owner may agree to this sprint plan. If the team knows their technologies well, it is now easy to break this down into tasks, such as “create a shopping cart model”, “display shopping cart page”, “retrieve the payment status from the payment provider” and “store the payment status in the database”.
This demo script will guide the team both during the construction and during the actual sprint review. During the construction a team member is now in a position to solve the sprint goal in the simplest way possible. By focusing on how the team will demonstrate value instead of what technical tasks may or may not be required (e.g. “construct a new order facade service” – whatever that may mean!) we can dramatically cut down on wasteful and convoluted design.
Agile methods emphasize “adapting to change over following a plan”. The same holds true for a demo script. The purpose of the script is not to create a perfect plan (which is of limited value), but to get a clear picture of what we need to create and how we will demonstrate that we have indeed delivered real value.
When I learned math in elementary school, I would reach for my calculator. But my father stopped me: “You only get to use the calculator when you can do math without it.” As you can imagine, at the time I though this was unfair and unreasonable, but later I have discovered what advantage it is to understand the basics before you reach for a powerful tool.
Many developers are focused on what fancy framework they will learn next. Or what programming language might solve all their problems. Before we reach for the tools, it’s useful to learn how they really work. My motto is: “I will not use a framework I would be unable to recreate.” It is of course, too time consuming to create a framework as complete as many of those available, but I should at least be able to solve all the normal cases myself.
Having recreated the behavior of a framework means that I will understand how the framework is implemented. I get better intuition about how to use the framework, I understand more quickly what the problem might be when something doesn’t work, and last, but not least: I understand when the framework is harming me more than it’s helping me.
An example I have enjoyed using is to create a web application in Java without framework. I may use a simple domain like creating an address book where you can register your contacts and search for them. In honor of my C#-friends, I have solved the same task in C#: How to make a web application without MVC, without ASP.NET or even without IIS.
Test-driven development is an essential tool for me to think clearly. I’ve made an exception from the calculator rule above and used a few testing libraries: SimpleBrowser.WebDriver, FluentAssertions and NUnit. This test demonstrates the behavior that I want from the application when I’m done:
I add an empty class for My.Application.WebServer and the test will fail at the line browser.Url = server.BaseUrl as there is not real server.
To implement the server, I’m using a cute small class which is part of the .NET core library: System.Net.HttpListener. Here are the essentials:
Running the test again, I get one step further. This time, I am told that the test can’t find the link to “Add contact”. No big surprise, as I’m not serving any HTML! A small change in the WebServer code will fix this:
Then we just have to create a simple implementation for AddressBookController.Service:
Again, the test will get one step further. Now we can see that the main page is presented with the links “Add contact” and “Find contact“. After Click()ing “Add contact” the test will of course fail to find the field fullName as we have not created the form yet. The method HandleGetRequest inspects the URL to determine which page should be displayed:
We’re almost done saving contacts. The test fails to find the link “Find contact” after submitting the form. The method Service must be modified to handle POST requests and perform a redirect back to the menu:
All that remains to turn this into a real application is to use a real database and correct the obvious security vulnerability when we display contacts. The AddressBookWebServer could have a Main method to enable you to run the code. But I’ll leave these issues as an exercise to you, dear reader.
This article has showed how HTTP really works and how frameworks like ASP.NET MVC work behind the curtains. There are many details that we’re glad that the framework can fix for us, like the character encoding and reading the contents of a POST request. And there are many things that turn out to be not as hard as you’d think, like a real “redirect-on-post” pattern. In more than one project, I’ve realized that after spending a few days understanding the underlying technology, I could deliver the project much better and faster without the “obvious”, popular frameworks that everyone recommend that you use.
Did I reinvent the wheel in this article? You could argue that I did, but let me stretch the metaphor of “reinventing the wheel” as far as possible:
My experience is that many “cars” today have misaligned wheels where the axel isn’t mounted in the center. Maybe the wheel was poorly constructed, or maybe the car was just assembled wrong. Maybe we notice that the car is bouncing because two of the wheels have a misaligned axel. And then we spend a lot of work trying to adjust these wheels to synchronize the bouncing. Finally we publish articles about the nice even undulations of our car after aligning the errors in the wheels.
If we have some experience contructing one or two “wheels”, it’s possible that we’re able to identify the real problems with the “wheels” we were given, so we can determine which “wheels” are good and which “wheels” are bad. Not to mention: We may learn how to use them properly.
Reinvent wheels you don’t understand, don’t use a framework you couldn’t have made yourself and don’t use a calculatore before you understand math.
When a software architect gets a good idea or learns something new, he has a problem. The main job of the architect is to ensure that the right information in present inside the heads of the people who should build the application. Every new piece of information in the architect’s head represents a broader gap between his brain and that of the rest of the team.
The classical ways of adressing this gap is for the architect to write huge documents or sets of wiki pages. When they realize that there’s not sufficient time set aside in the project schedule for the developers to read all this information, the architect may present the material to developers who sit and nod their heads. But what did they really understand?
Instead of the read-listen-and-nod approach, I prefer an approach that I sometimes call “dragging the information throught the heads of the team and looking at what comes out in the other end.” I provide as little processed information as possible, but instead give the team a structured workshop to uncover and structure the information by asking me or business stakeholders. The outcomes of the workshop should be some tanglible results presented by the team. This result is always different from what I had in mind. Sometimes the difference shows a critical misunderstanding, which allows me to go more in depth in this area. Sometimes the difference represents a trivial misunderstanding or difference in opinion and the architect has the difficult task of accepting a small disgreement without distracting the team. Sometimes, the team has discovered something much smarter than the original idea of the architect.
I find it most useful to do workshops in small groups of three people per group. Each group should produce something that they can show to the whole team afterwards. Here are some examples of workshops that I run:
Divide in groups of three with the users/business and the developers represented in each group. Each group should discuss and fill in a template for the vision of the product being created: “For some user who performs some business function the name of system is a type of system which gives a capability related to the task. Unlike most interesting alternative our solution has an important advantage“. The groups get 10 minutes before a debrief with the whole team.
Each group then brainstorms a list of users, consumers and others affected by the system and write these on sticky notes. This should be about 20-30 roles. The whole team decides on a few interesting users and the groups then write down for some these: What characterizes the user, what tasks do they perform and what do they value?
Based on the list of tasks that stakeholders perform, we create a sketch of a usage flow. I like to refine the documented usage flow with a small task group which takes a few hours to prepare a description of the flow of interaction between the system and external actors
Groups of three go through the usage flow to come up with Actors (users and systems), Domain concepts (classes) or Containers (deployment diagram) mentioned or implied in the usage flow and write these on sticky notes. After showing the Actors, Concepts or Containers to the whole group, each workgroup then organizes these on flipcharts to create a Context Model, a Domain Model and a Deployment Model.
Many of these workshops can also be run with distributed groups over video conference and screen sharing.
I like to collect all of these artifacts (vision, users, usage flow, context model, domain model and deployment model) in a PowerPoint presentation so it can be easily showed by the team to external stakeholders. Sometimes someone on the team feel that photographed flipcharts with sticky notes are too informal and decide to draw something in Visio or another fancy tool. This is just a plus.
By asking the team to produce something and present it, rather than explaining the architecture to the team, I ensure that the information is really in their heads and not just my fooling myself by my own understanding.
Do you ever feel it’s hard to get real progress in a sprint towards the business goal? Do you feel the feedback from a iteration picks on all the details you didn’t mean to cover this sprint? Do you feel like sprint planning meetings are dragging out? Then a Rainbow Sprint Plan may be for you.
Here is an example of a Rainbow Sprint plan:
A customer wants cheap vacations
The customer signs up for daily or weekly notifications of special flight offers
Periodically the System checks which customers should get notifications
The System checks for offers that matches the customer’s travel preference by looking up flights with the travel provider system
The System notifies customer of any matching offers via SMS
Variation: The System notifies customer of any matching offers via email
The customer accepts the offer via SMS
The System books the tickets on behalf of the customer
The system confirms the booking by sending an SMS to the customer
The customer can at any point see their active offers and accepted offers on the system website
The customer enjoys a cheap vacation!
What you can see from this plan:
Use case overview: The plan gives a high-level picture of the next release. We can see how the work we are doing is fitting together and how it ends up satisfying a customer need. This is a requirement technique that is basically Use Cases as per Alistair Cockburn’s “Writing Effective Use Cases“. I’ve been writing use cases at this level for the last three years and found it to be a good way to understand requirements. The trick of good use cases is to stay at a the right level. In this example, each step is some interaction between the system and a user or the system and another system. How this communication is handled is something I find best to leave for an individual sprint.
Iterative completion: Each step has a color code:
Black: The team hasn’t started looking into this
Red: We have made something, but it’s a dummy version just to show something
Orange: We have made something, but we expect lots of work remaining
Yellow: We’re almost done, we’re ready to receive feedback
Green: Development is complete, we have done reasonable verification and documentation
So the plan accepts that we revisit a feature. As we get closer to the next release, things will move further and further into the rainbow. But we can choose whether we want to get everything to orange first, or whether we will leave some things at red (or even black) while bringing other steps all the way to green.
Demonstration script: When we get to the end of the sprint and demonstrate what we’ve created, this plan gives a pretty good idea of what the demo will look like: We will sign up the customer to a dummy signup page (red), we will register some flights in another dummy page (red), trigger the actual scheduling code (orange), then we will see that an SMS is received on an actual phone (yellow). Then we will simulate an SMS response (orange), see that they system made some communication to a dummy system (red), and send “ok” back as an SMS to the customer (orange). This will focus the team around a shared vision of what to do in this sprint.
I have been thinking in terms of a Rainbow Plan in my last projects, but I’ve never used the term before. I think the plan addresses three of the most common problems that I see in Scrum implementations:
The team doesn’t see where it’s going, because user stories are too fine grained to get the big picture. User story mapping and use cases address this, and rainbow plans put it into a sprint-context
The team dives into technical details during sprint planning. With rainbow plans, the sprint plan becomes the demo plan which coincides with the requirements.
The project has a purely incremental approach, where each feature should be completed in a single sprint. This means that it’s hard to keep the big picture and the product owner is forced to look for even small bugs in everything that’s done in a sprint. With rainbow plans, the team agrees on the completeness of each feature.
May you always become more goal oriented and productive in your sprints.
Sometimes, small questions lead to big answers. Sometimes these answers are controversial. One such question is “What does this warning about serialVersionUID mean”? All the advice out there basically is for developers who don’t know what’s going on to write code that will ignore errors when something unexpected happens. In my view – this is exactly the wrong approach. The safe way to act is to make sure that your program crashes if you don’t have control.
Java programmers usually get this warning when they write code that looks like this:
On stackoverflow this question has an answer that is extensive, well-written, accepted and, in my opinion, wrong. In short, the answer just recommends to implement something like the following:
Let’s dig deeper.
The reason we get this warning is that HttpServlet implements the interface Serializable. This was a design flaw in the first version of the servlet-api, and now we’re stuck with it. A serialized object can be written to and read from byte streams, such as a file. Here is an example:
In this case, everything is fine. But let’s imagine that some time passes between the write and the read. For example, what if we try to read the file with the next version of the program. A version in which the Person class is changed? For example, what if we changed the implementation of Person to store the name as a single field fullName, instead of two fields firstName and lastName?
In this case, we would get an error message like the following:
But wait, there’s another option. Let’s say we found this problem when we tested if our new version was backwards compatible. Now, we could just cut and paste the serialVersionUID from the stack trace. If we do test our software, this is just as easy as putting the serialVersionUID there in the first place. So, let’s fix the Person class:
Voila! Problem fixed. Our code will run again. Here’s the output of printing the object:
Whoops! By forcing different versions of class Person to have the same serialVersionUID, the code now has a serious bug. FullName should never be null (especially since it’s final!). And what’s worse, the bug is a silent one. If we don’t examine the contents of System.out (in this case), we might not catch it before it escapes into the wild.
When you’re not sure, the correct behavior should be to fail, not to silently do the wrong thing.
If you omit a serialVersionUID field from your class, many changes will cause serialized objects to no longer be readable. Even for trivial changes.
Sadly, classes like HttpServlet, AbstractAction and JFrame which are meant to be subclassed implements Serializable, even though they are almost never serialized in practice. Adding serialVersionUID to these classes would only be noise.
The serialVersionUID field can be added to a class afterward if you actually want to read old objects in a new version of the program. This leaves you no worse off than if you added the serialVersionUID field in the first place.
If the old and the new version of the class are deeply incompatible, giving the class a serialVersionUID when you first create it will cause silent faulty behavior.
I prefer loud, failing behavior that is easy to detect during testing over quiet, faulty behavior that may escape into production. I think you do, too.
I have tried to create a simple process for debriefing workshops. This is the current process I use, and I think it may be useful for others.
I give everyone sticky notes with three colors
I ask everyone to write “a thing that surprised you about the workshop”, “a thing that you learned today” and “a thing that you plan to do as a result of the workshop”. Each question goes on a different color sticky note.
Everyone puts their sticky notes on a flip chart. As people come up, I read through their notes and pick out some notes that I think were interesting to discuss
When everyone has put up their answers, I read up 1-3 notes about what surprised the participants, 1-3 notes about what they learned and 1-3 notes about what they plan to use it for. I comment on these results.
As I end the workshop, I tell everyone “now, pick up the note with your plan as you leave the room and put it in your pocket. When you find it in the future, ask yourself whether you have completed it, or whether you should put it back into your pocket
I feel this gives a nice closure to the workshop and a drive for people to apply what they’ve learned. Have you been to one of my workshops where I had this debriefing? I’d like to know whether you used the “what do you plan” note. :-)
I’m working on this idea, and I don’t know if it appeals to you guys. I’d like your input on whether this is something to explore further.
Here’s the deal: I’ve encountered teams who, when working with SOA technologies have been dragged into the mud by the sheer complexity of their tools. I’ve only seen this in Java, but I’ve heard from some C# developers that they recognize the phenomenon there as well. I’d like to explore an alternative approach.
This approach requires more hard work than adding a WSDL (web service definition language. Hocus pocus) file to your project and automatically generating stuff. But it comes with added understanding and increased testability. In the end, I’ve experienced that this has made me able to complete my tasks quicker, despite the extra manual labor.
The purpose of this blog post (and if you like it, it’s expansions) is to explore a more bare-bones approach to SOA in general and to web services specifically. I’m illustrating these principles by using a concrete example: Let users be notified when their currency drops below a threshold relative to the US dollar. In order to make the service technologically interesting, I will be using the IP address of the subscriber to determine their currency.
Step 1: Create your active services by mocking external interactions
Mocking the activity of your own services can help you construct the interfaces that define your interaction with external services.
Spoiler: I’ve recently started using random test data generation for my tests with great effect.
The Publisher has a number of Services that it uses. Let us focus on one service for now: The GeoLocationService.
Step 2: Create a test and a stub for each service – starting with GeoLocationService
The top level test shows what we need from each external service. Informed by this and reading (yeah!) the WSDL for a service, we can test drive a stub for a service. In this example, we actually run the test using HTTP by starting Jetty embedded inside the test.
Validate and create the XML payload
This is the first “bare-knuckled” bit. Here, I create the XML payload without using a framework (the groovy “$”-syntax is courtesy of the JOOX library, a thin wrapper on top of the built-in JAXP classes):
I add the XSD (more hocus pocus) for the actual service to the project and code to validate the message. Then I start building the XML payload by following the validation errors.
In this example, I get a little help (and a little pain) from the JOOX library for XML manipulation in Java. As XML libaries for Java are insane, I’m giving up with the checked exceptions, too.
Spoiler: I’m generally very unhappy with the handling of namespaces, validation, XPath and checked exceptions in all XML libraries that I’ve found so far. So I’m thinking about creating my own.
Of course, you can use the same approach with classes that are automatically generated from the XSD, but I’m not convinced that it really would help much.
Stream the XML over HTTP
Java’s built in HttpURLConnection is a clunky, but serviceable way to get the XML to the server (As long as you’re not doing advanced HTTP authentication).
Spoiler: This code should be expanded with logging and error handling and the validation should be moved into a decorator. By taking control of the HTTP handling, we can solve most of what people buy an ESB to solve.
Create the stub and parse the XML
The stub uses xpath to find the location in the request. It generates the response in much the same way as the ws client generated the request (not shown).
Spoiler: The stubs can be expanded to have a web page that lets me test my system without real integration to any external service.
Validate and parse the response
The ws client can now validate that the response from the stub complies with the XSD and parse the response. Again, this done using XPath. I’m not showing the code, as it’s just more of the same.
The real thing!
The code now verifies that the XML payload conforms to the XSD. This means that the ws client should now be usable with the real thing. Let’s write a separate test to check it:
Yay! It works! Actually, it failed the first time I tried it, as I didn’t have the correct country name for the IP address that I tested with.
This sort of point-to-point integration test is slower and less robust than my other unit tests. However, I don’t find make too big of a deal out of that fact. I filter the test from my Infinitest config and I don’t care much beyond that.
Fleshing out all the services
The SubscriptionRepository, CurrencyService and EmailService need to be fleshed out in the same way as the GeolocationService. However, since we know that we only need very specific interaction with each of these services, we don’t need to worry about everything that could possibly be sent or received as part of the SOAP services. As long as we can do the job that the business logic (CurrencyPublisher) needs, we’re good to go!
Demonstration and value chain testing
If we create web UI for the stubs, we can now demonstrate the whole value chain of this service to our customers. In my SOA projects, some of the services we depend on will only come online late in the project. In this case, we can use our stubs to show that our service works.
Spoiler: As I get tired of verifying that the manual value chain test works, I may end up creating a test that uses WebDriver to set up the stubs and verify that the test ran okay, just like I would in the manual test.
Taking the gloves off when fighting in an SOA arena
In this article, I’ve showed and hinted at more than half a dozen techniques to work with tests, http, xml and validation that don’t involve frameworks, ESBs or code generation. The approach gives the programmer 100% control over their place in the SOA ecosystem. Each of the areas have a lot more depth to explore. Let me know if you’d like to see it be explored.
Oh, and I’d also like ideas for better web services to use, as the Geolocated currency email is pretty hokey.
Changing your perspective even a small amount can have profound effects on how you approach your system.
Let’s say you’re writing a web application in Java. In the system you deal with orders, customers and products. As a web application, your classes include staples like PersonController, PersonRepository, CustomerController and OrderService. How do you organize your classes into packages?
There are two fundamental ways to structure your packages. Either you can focus on the logical tiers, like com.brodwall.myapp.controllers, com.brodwall.myapp.domain or perhaps com.brodwall.myapp.services.customer. Or you can focus on the domain contexts, like com.brodwall.myapp.customer, com.brodwall.myapp.orders and com.brodwall.myapp.products. The first approach is by far the most prevalent. In my view, it’s also the least helpful.
Here are some ways your thinking changes if you structure your packages around domain concepts, rather than technological tiers:
First, and most fundamentally, your mental model will now be aligned with that of the users of your system. If you’re asked to implement a typical feature, it is now more likely to be focused around a strict subset of the packages of your system. For example, adding a new field to a form will at least affect the presentation logic, entity and persistence layer for the corresponding domain concept. If your packages are organized around tiers, this change will hit all over your system. In a word: A system organized around features, rather than technologies, have higher coherence. This technical term means that a large percentage of a the dependencies of a class are located close to that class.
Secondly, organizing around domain concepts will give you more options when your software grows. When a package contains tens of classes, you may want to split it up in several packages. The discussion can itself be enlightening. “Maybe we should separate out the customer address classes into a com.brodwall.myapp.customer.address package. It seems to have a bit of a life on its own.” “Yeah, and maybe we can use the same classes for other places we need addresses, such as suppliers?” “Cool, so com.brodwall.myapp.address, then?” Or maybe you decide that order status codes and payment status codes deserve to be in the “com.brodwall.myapp.order.codes” package.
On the other hand, what options do you have for splitting up com.brodwall.myapp.controllers? You could create subpackages for customer, orders and products, but these subpackages may only have one or possibly two classes each.
Finally, and perhaps most intriguingly, using domain concepts for packages allows you to vary the design according on a case by case basis. Maybe you really need a OrderService which coordinates the payment and shipping of an order, while ProductController only needs basic create-retrieve-update-delete functionality with a repository. A ProductService would just get in the way. If ProductService is missing from the com.brodwall.myapp.services package, this may be confusing or at the very least give you a nagging feeling that something is wrong. On the other hand, if there’s no Controller in the com.brodwall.myapp.product package, it doesn’t matter much.
Also, most systems have some good parts and some not-so-good parts. If your Services package is not working for you, there’s not much you can do. But if the Products package is rotten, you can throw it out and reimplement it without the whole system being thrown into a state of chaos.
By putting the classes needed to implement a feature together with each other and apart from the classes needed to implement other features, developers can be pragmatic and innovative when developing one feature without negatively affecting other features.
The flip side of this is that most developers are more comfortable with some technologies in the application and less comfortable with other technologies. Organizing around features instead of technologies force each developer to consider a larger set of technological challenges. Some programmers take this as a motivating challenge to learn, while others, it seems, would rather not have to learn something new.
If it were my money being spend to create features, I know what kind of developer I would want.
Trivial changes can have large effects. By organizing your software around features, you get a more coherent system that allows for growth. It may challenge your developers, but it drives down the number of hand-offs needed to implement a feature and it challenges the developers to improve the parts of the application they are working on.