Teaser: Bare-knuckle SOA

I’m working on this idea, and I don’t know if it appeals to you guys. I’d like your input on whether this is something to explore further.

Here’s the deal: I’ve encountered teams who, when working with SOA technologies have been dragged into the mud by the sheer complexity of their tools. I’ve only seen this in Java, but I’ve heard from some C# developers that they recognize the phenomenon there as well. I’d like to explore an alternative approach.

This approach requires more hard work than adding a WSDL (web service definition language. Hocus pocus) file to your project and automatically generating stuff. But it comes with added understanding and increased testability. In the end, I’ve experienced that this has made me able to complete my tasks quicker, despite the extra manual labor.

The purpose of this blog post (and if you like it, it’s expansions) is to explore a more bare-bones approach to SOA in general and to web services specifically. I’m illustrating these principles by using a concrete example: Let users be notified when their currency drops below a threshold relative to the US dollar. In order to make the service technologically interesting, I will be using the IP address of the subscriber to determine their currency.

Step 1: Create your active services by mocking external interactions

Mocking the activity of your own services can help you construct the interfaces that define your interaction with external services.

Teaser:

Spoiler: I’ve recently started using random test data generation for my tests with great effect.

The Publisher has a number of Services that it uses. Let us focus on one service for now: The GeoLocationService.

Step 2: Create a test and a stub for each service - starting with GeoLocationService

The top level test shows what we need from each external service. Informed by this and reading (yeah!) the WSDL for a service, we can test drive a stub for a service. In this example, we actually run the test using HTTP by starting Jetty embedded inside the test.

Teaser:

Validate and create the XML payload

This is the first “bare-knuckled” bit. Here, I create the XML payload without using a framework (the groovy “$"-syntax is courtesy of the JOOX library, a thin wrapper on top of the built-in JAXP classes):

I add the XSD (more hocus pocus) for the actual service to the project and code to validate the message. Then I start building the XML payload by following the validation errors.

Teaser:

In this example, I get a little help (and a little pain) from the JOOX library for XML manipulation in Java. As XML libaries for Java are insane, I’m giving up with the checked exceptions, too.

Spoiler: I’m generally very unhappy with the handling of namespaces, validation, XPath and checked exceptions in all XML libraries that I’ve found so far. So I’m thinking about creating my own.

Of course, you can use the same approach with classes that are automatically generated from the XSD, but I’m not convinced that it really would help much.

Stream the XML over HTTP

Java’s built in HttpURLConnection is a clunky, but serviceable way to get the XML to the server (As long as you’re not doing advanced HTTP authentication).

Teaser:

Spoiler: This code should be expanded with logging and error handling and the validation should be moved into a decorator. By taking control of the HTTP handling, we can solve most of what people buy an ESB to solve.

Create the stub and parse the XML

The stub uses xpath to find the location in the request. It generates the response in much the same way as the ws client generated the request (not shown).

Spoiler: The stubs can be expanded to have a web page that lets me test my system without real integration to any external service.

Validate and parse the response

The ws client can now validate that the response from the stub complies with the XSD and parse the response. Again, this done using XPath. I’m not showing the code, as it’s just more of the same.

The real thing!

The code now verifies that the XML payload conforms to the XSD. This means that the ws client should now be usable with the real thing. Let’s write a separate test to check it:

Yay! It works! Actually, it failed the first time I tried it, as I didn’t have the correct country name for the IP address that I tested with.

This sort of point-to-point integration test is slower and less robust than my other unit tests. However, I don’t find make too big of a deal out of that fact. I filter the test from my Infinitest config and I don’t care much beyond that.

Fleshing out all the services

The SubscriptionRepository, CurrencyService and EmailService need to be fleshed out in the same way as the GeolocationService. However, since we know that we only need very specific interaction with each of these services, we don’t need to worry about everything that could possibly be sent or received as part of the SOAP services. As long as we can do the job that the business logic (CurrencyPublisher) needs, we’re good to go!

Demonstration and value chain testing

If we create web UI for the stubs, we can now demonstrate the whole value chain of this service to our customers. In my SOA projects, some of the services we depend on will only come online late in the project. In this case, we can use our stubs to show that our service works.

Spoiler: As I get tired of verifying that the manual value chain test works, I may end up creating a test that uses WebDriver to set up the stubs and verify that the test ran okay, just like I would in the manual test.

Taking the gloves off when fighting in an SOA arena

In this article, I’ve showed and hinted at more than half a dozen techniques to work with tests, http, xml and validation that don’t involve frameworks, ESBs or code generation. The approach gives the programmer 100% control over their place in the SOA ecosystem. Each of the areas have a lot more depth to explore. Let me know if you’d like to see it be explored.

Oh, and I’d also like ideas for better web services to use, as the Geolocated currency email is pretty hokey.

Comments:

[chris] - Jul 23, 2012

great article! thanks

[Geir Hedemark] - Jul 25, 2012

Great article. Can I suggest a step 0? “Sit down. Relax. Now, forget about all the tools and technologies you want to use. No, really. I mean it. The technologies won’t help you get the design of the service right, and it will take attention away from the design. I’ll be standing behind you with a big stick with a nail in it to help you out. Now sit down and think of kittens.”

Johannes Brodwall - Jul 25, 2012

I like that: “Step 0: Relax!”

Lars Reed - Aug 14, 2012

I am teased… :)

[Charlie] - Jul 30, 2012

Indeed the concepts that underpin SOA are far more interesting and useful than the technologies you can use to implement them. Like when is it appropriate to use SOA? What are the trade-offs? Why does temporal coupling affect overall system stability?

[RyanP] - Jul 31, 2012

Excellent.  BTW… What graphics tool did you use to create those diagrams?  I like the handwriting feel to them.

Johannes Brodwall - Jul 31, 2012

Thanks. The diagrams were created using yuml.me, a briliant text-based tool.