One of my favorite problems is that of creating a Java EE application from scratch. I call this kata “The Java EE Spike Kata”. In order to understand the role of frameworks, I use no web frameworks in the process of creating this application. I’ve completed this particular exercise about forty times on my own and more than ten times with various pair-programming partners. The whole exercise takes me about 90 minutes, and I still learn new things.

The total time of the screencasts is around 40 minutes, so you may want to pick and choose. Each section provides a link to the starting point for the source code if you want to follow along.

Please notice: The videos are accompanied by loud, pounding music. Keep you headphones on or your volume down if you share offices with someone else. Or mute the videos if you dislike the music.

Idea 1: Build your software from the outside-in

(10 minutes, github starting point)

I start building the application by writing tests that access the application over HTTP and looks at the resulting HTML. As you might have gathered, when I start this test, there is no web application. Only when the tests require a web application to continue do I start creating it. In this example, I had created a basic sketch of the interaction between the three web pages in the application before I started coding. No further design was necessary.

This particular approach uses WebDriver and Jetty to run. The cute assertion library that you may have noticed at the end of the video is FEST-assert.

Idea 2: Specify behavior rather than implementation

(6 minutes, github starting point)

I don’t make much of a distinction between different types of tests. All good tests try to describe what the software should do at some level, rather than how the software does it. But the how at some level may be the what at another level. My first test specified the interaction between the web browser and the server. In this test, one step may be to fill in the form element of a web page. This second video shows how this form works in terms of actual HTML. But the details of what framework (if any) is used, is not visible in the test.

The second thing you’ll notice in the video is that I run the tests more frequently. And each test run is much quicker. As our tests move close to the code, the rate of feedback improves.

This particular test uses Mockito to mock out the Servlet API. The assertions use FEST-assert.

Idea 3: Increase the rate of feedback

(5 minutes, github starting point)

This video illustrates the frequency of feedback. The example test-drives creating an equals-method. This task is often not worth test-driving. The resulting method is usually simple and/or generated by your IDE. But it is a good example to of how quick the cycle between test and production code can be when you’re writing tests that are close to the problem at hand. When I pair program this part of the kata, we usually use a technique called ping-pong programming: One programmer writes a failing test (or failing assertion) and hands the keyboard to his partner, the other programmer makes the test pass and writes another failing test before passing the keyboard back. On a good run, we will switch who’s got the keyboard more often than once per minute.

Notice that I also focus on the behavior of the equals-method in this test.

Idea 4: Grow the API rather than designing it up front

(8 minutes, github starting point)

As the web application grows under my fingers, I discover the need for a Data Access Object (DAO). However, as this represents a major internal interface in my application, I use Mockito to mock the implementation until I’m done with the behavior of my servlet. When this is done, I test-drive the implementation of the DAO in a separate test class.

The video also illustrates another important lesson: The code is getting ripe for a refactoring. But it’s important to resist the urge to refactor until the tests are green. If you refactor on red tests, you have much higher chances of running down a dead-end road and you’ll have to throw away your progress, wondering what went wrong.

The example uses Mockito to mock out the DAO API.

If you want to see how I implement the DAO with Hibernate, you can see the video on blip.tv.

Idea 5: Grow the design rather than speculating

(3 minutes, github starting point)

The video is only a partial example of this principle. Throughout the whole application, I’ve been refactoring, gradually pulling out structure to more well-structured methods and classes. The video illustrates some of the power of IDE’s when it comes to refactoring. Using the IDE to massage your code into a better design makes evolutionary design much easier to do in practice. Make sure to learn your IDE’s most useful refactorings!

The kata may seem like a non-realistic example at this time, but I’ve actually grown a very successful architecture on my current project using much the same approach. If you want to explore where to go next, the next step needed for this application is to factor out the views into separate classes and then use either a View Template language (like Velocity) or a View Transformer (using, for example dom4j) to generate the HTML. (Let me know if you’d like to see the screencast of this as well).

Idea 6: It’s supposed to work the first time around

(5 minutes, github starting point)

In this video, I return to the first test, PersonWebTest, to finish the configuration of the application. I discover a few mistakes I made in the web test as I complete the exercise. Then I try out the code in the browser. And all the scenarios I had planned for work out of the box.

When you try out your code for the first time, it should work. When you master test-driven development, you will probably forget how you programs didn’t use to work the first time. Only when you occasionally run into an unexpected error during manual testing it becomes clear how test-driven development changes you life.

If you want to see the whole, uninterrupted 75 minute code kata, the video is available at blip.tv. You can also take a look at the finished source code at github.

Happy programming!

A big thanks to Trond, Thomas, Ram and Christian who helped improve this post. Thank you to Finn-Robert, Øistein, Mats, Anders, Siv, Peyman, Ivar, Øystein, Cecilia, Nicolay, and Karianne who have pair-programmed this exercise with me. I especially appreciate how Ivar and Karianne both helped influence the way the application is wired together and showed me that even after 30 iterations, I still had things to learn; and how Øistein showed me how two trained developers could complete the exercise faster with pair-programming than alone. And thank you to Nicolay who graciously brought food to our pair-programming exercise.

The videos were made with the excellent (and free!) BB FlashBack Express. The keyboard echo is courtesy of KeyPosé by Magnus Jungsbluth.

I found both attempts lacking. However, while Ron Jeffries freely admitted that he didn’t even know the rules of Sudoku when he started, both Norvig himself and his readers fawn over his solution. I didn’t find it very understandable.

So I took it upon myself to examine the problem myself. I did some up-front thinking in the shower and on the subway, then attacked the problem with TDD. I ended up with a solution that works in all cases (unlike Norvig). My implementation has readable code, readable tests, and solves the problem reasonably fast.

Observations and conjectures

Here are a few things I learned from the exercise:

• When you’re using TDD to solve a tricky algorithm, you have to think about both the algorithm and the test approach.
• Solving a problem with a known algorithm using TDD gives more readable code than I otherwise would expect.
• When I solved the problem with TDD, running the solution on real problems worked the very first time I tried it.
• The trick to making TDD work is to work from the outside in.
• When creating a Sudoku solver, don’t think like a human! Think like a machine! The human algorithm is difficult to understand and likely to not work on all problems. This was the biggest problem with Norvig’s code

The journey

I decided on the following approach:

1. I had decided upon an initial design with a solver class and a board class. The solver should use a recursive depth first search. The solver asks the board what options exists per cell, but it has no knowledge of the rules of Sudoku (such as no duplicate numbers on the same row).
2. The first step was to get the solver (“the outside”) correct. For this step, I mocked out the board
3. The second step was to implement the interface that the solver needed for the board. Mainly, this is a matter of specifying the rules for what numbers can occur in which cell on a Sudoku board.
4. Finally, I wrote some code to read and write the Sudoku board. When trying the solver on real problems, it worked the first time, and solved 95 hard problems correct. It was somewhat slow, though.

After solving the problem the first time, I practices a few times and recorded a screen cast of the solution:

The solver

Testing the solver is a matter of creating a mock board and ensuring that the solver does the correct things. This is the most complex test case:

@Test
public void shouldBacktrackWhenNoMoreOptions() throws Exception {
    SudokuSolver solver = new SudokuSolver();
    SudokuBoard board = mock(SudokuBoard.class);
    when(board.getOptionsForCell(anyInt(), anyInt()))
            .thenReturn(singleOption());
 
    when(board.getOptionsForCell(8, 7))
            .thenReturn(moreOptions(1, 2));
    when(board.getOptionsForCell(8, 8))
            .thenReturn(noOptions())
            .thenReturn(singleOption());
 
    assertThat(solver.findSolution(board)).isTrue();
    InOrder order = inOrder(board);
    order.verify(board).setValueInCell(1, 8,7);
    order.verify(board).setValueInCell(2, 8,7);
}

It specifies that all cells, except (8,7) and (8,8) return exactly one option. (8,7) returns two options. (8,8) returns no options the first time it is called, and one option the second time. The test verifies that a solution is found, and the solver tries to set both options for (8,7).

This drives a rather simple algorithm. Here’s basically the whole algorithm:

public boolean findSolution(Board board, int cell) {
    if (cell == SIZE*SIZE) return true;
 
    boolean wasEmpty = board.isEmpty(row(cell), col(cell));
    for (Integer value : board.getCellOptions(row(cell), col(cell))) {
        board.setValueInCell(value, row(cell), col(cell));
        if (findSolution(board, cell+1)) return true;
    }
    if (wasEmpty) board.clearValueInCell(row(cell), col(cell));
 
    return false;
}

The algorithm tries all available options for a cell in order. If no solution works for the rest of the board, the algorithm returns false (for “no solution”).

The algorithm is not how a human would solve Sudoku. But then again, we’re not writing a tutorial on how to solve Sudoku, we’re writing a program that solves Sudoku.
The board

As I implemented the solver, the interface for the board started to emerge. At that point in time, I had to create tests for the Sudoku board itself. A typical test verifies that the board doesn’t allow duplicate values in a row:

@Test
public void shouldDisallowOptionsInSameRow() throws Exception {
    int row = 4;
    board.setValueInCell(1, row, 5);
    board.setValueInCell(2, row, 8);
    board.setValueInCell(3, row+1, 5);
    assertThat(board.getOptionsForCell(row, 0))
            .excludes(1,2).contains(3);
}

The essence of SudokuBoard is finding out what values are legal in an open cell:

public List getOptionsForCell(int row, int col) {
    if (!isEmpty(row,col)) return Arrays.asList(cells[row][col]);
    List result = allOptions();
    removeAllInRow(result, row);
    removeAllInCol(result, col);
    removeAllInBox(result, row, col);
    return result;
}

TDD as a design guide

I invite you to compare Peter Norvig’s solution to mine (you can find the full source code for my solution in my github repository).

It would probably have been possible for me to code the solution faster without tests, but it probably would not have worked the first time I tried it. I also would have much less confidence in the code. Finally, I think the design imposed by the tests made my code easier to understand.

This article is a summary of a seminar I had on the topic. If it seems like it’s a continuation of an existing discussion that’s because, to some extent, it is. If you haven’t been discussing exchanging your app server, this article probably isn’t very interesting to you.

By putting the application server inside my application instead of the other way around, I was able to leap tall buildings in a single bound.

The embedded application server

This is how I build and deploy my sample application to a new test environment (or to production):

1. mvn install
2. scp someapp-server/target/someapp-1.0.onejar.jar appuser@appserver:/home/appuser/test-env1/
3. ssh appuser@appserver "cd /home/appuser/test-env1/ && java -jar someapp-1.0.onejar.jar&"

This require no prior installed software on the appserver (with the exception of the JVM). It requires no prior configuration. Rolling back is a matter of replacing one jar-file with another. Clustering is a matter of deploying the same application several times.

In order to make this work in a real environment, there are a many details you as a developer need to take care of. As a matter of fact, you will have to take responsibility for your operational environment. The good news is that creating a good operational environment is not more time-consuming than trying to cope with the feed and care of a big-a Application Server.

In this scheme every application comes with its own application server in the form of jetty’s jar-files embedded in the deployed jar-file.

The advantages

Why would you want to do something like this?

• Independent application: If you’ve ever been told that you can’t use Java 1.5 because that would require an upgrade of the application server. And if we upgrade the application server, that could affect someone else adversely. So we need to start a huge undertaking to find out who could possibly be affected.
• Developer managed libraries: Similar problems can occur with libraries. Especially those that come with the application server. For example: Oracle OC4J helpfully places a preview version of JPA 1.0 first in your classpath. If you want to use Hibernate with JPA 1.0-FINAL, it will mostly work. Until you try to use a annotation that was changed after the preview version (@Discriminator, for example). The general rule is: If an API comes with your app server, you’re better served by staying away from it. A rather bizarre state of affairs.
• Deployment, configuration and upgrades: Each version of the application, including all its dependencies is packaged into a single jar-file that can be deployed on several application server, or several times on the same application server (with different ports). The configuration is read from a properties-file in the current working directory. On the minus side, there’s no fancy web UI where you can step through a wizard to deploy the application or change the configuration. On the plus side, there is no fancy web UI …. If you’ve used one such web UI, you know what I mean.
• Continuous deployment: As your maven-repository will contain stand alone applications, creating a continuous deployment scheme is very easy. In my previous environment, a cron job running wget periodically was all that was needed to connect the dots. Having each server environment PULL the latest version gives a bit more flexibility if you want many test environments. (However, if you’re doing automated PUSH deployment, it’s probably just as practical for you).
• Same code in test and production: The fact that you can start Jetty inside a plain old JUnit test means that it is ideal for taking your automated tests one step further. However, if you test with Jetty and deploy on a different Application Server, the difference will occasionally trip you. It’s not a big deal. You have to test in the server environment anyway. But why not eliminate the extra source of pain if you can?
• Licenses: Sure, you can afford to pay a few million $ for an application server. You probably don’t have any better use for that money, anyway, right? However, if you have to pay licenses for each test-server in addition, it will probably mean that you will test less. We don’t want that.
• Operations: In my experience, operations people don’t like to mess around with the internals of an Application Server. An executable jar file plus a script that can be run with [start|status|stop] may be a much better match.

The missing bits

Taking control of the application server takes away a lot of complex technology. This simplifies and makes a lot of stuff cheaper. It also puts you back in control of the environment. However, it forces you to think about some things that might’ve been solved for you before:

• Monitoring: The first step of monitoring is simple: Just make sure you write to a log file that is being monitored by your operations department. The second step requires some work: Create a servlet (or a Jetty Handler) that a monitoring tool can ping to check that everything is okay. Taking control of this means that you can improve it: Check if your data sources can connect, if your file share is visible, if that service answers. Maybe add application-calibrated load reporting. Beyond that, Jetty has good JMX support, but I’ve never needed it myself.
• Load balancing: My setup supports no load balancing or failover out of the box. However, this is normally something that the web server or routers in front of the application server anyway. You might want to look into Jetty’s options for session affinity, if you need that.
• Security: Jetty supports JAAS, of course. Also: In all the environments I’ve been working with (CA SiteMinder, Sun OpenSSO, Oracle SSO), the SSO server sends the user name of the currently logged in user as an HTTP header. You can get far by just using that.
• Consistency: If you deploy more than one application as an embedded application server, the file structure used by an application (if any) should be standardized. As should the commands to start and stop the application. And the location of logs. Beyond that, reuse what you like, recreate what you don’t.

Taking control of your destiny

Using an embedded application server means using the application server as a library instead of a framework. It means taking control of your “main” method. There’s a surprisingly small number of things you need to work out yourself. In exchange, you get the control to do many things that are impossible with a big-A Application Server.

Thanks to Dicksen, Eivind, Terje, Kristian and Kristian for a fun discussion on Jetty as a production app server

Top five shortcuts

There are some keyboard short cuts that everyone who uses Eclipse should know:

• ctrl-1 (quick fix): You hopefully use this shortcut to get quick fix support for compiler errors and warnings. Did you know that you can also use it to assign parameters to new fields, rename variables, and invert if-statements and equals-checks? Learn to think of ctrl-1 as asking Eclipse “what can you do make the code better (or just different)?”
• ctrl-space (complete): Again, you hopefully know that you can use ctrl-space to complete the name of variables and method names. But did you know that you can type “equa<ctrl-space>” in the class body and have Eclipse override the equals-method for you? Or that you can type “getNam<ctrl-space>” and have Eclipse create the whole implementation of a getter for name (if there’s a name-field in the class). Or that you can type “Test<ctrl-space>” and have Eclipse fill in the Test code template. Think of ctrl-space as asking Eclipse “guess what I’m about to write”
• ctrl-f6 (next editor): Use this to cycle between open files. It really should’ve been bound to ctrl-tab, but you can do this yourself.
• f3 (go to definition): Place the cursor on a method call or variable usage and press f3 to go to it’s definition.
• ctrl-shift-t (open type): A nifty dialog to go to any class in your project. Did you know that typing PerCoT will take you to PersonControllerTest?

Top ten runner ups

Here are some eye-openers that people enjoy learning:

• alt-ctrl-down (copy current line): Creates a new copy of the line under the cursor on the next line. Without wiping the clipboard! Try it while selecting several lines, too
• alt-down (move current line): Moves the line under the cursor down one line. Works with alt-up, too. And with a number of lines selected. A quick way to move code around with the keyboard.
• alt-shift-left (extend selection): Progressively selects a larger syntactic element in the editor. Hard to explain. Try it out!
• ctrl-shift-m (static import): Replace a call to Assert.assertEquals with a static import of the org.junit.Assert.assertEquals and a call to assertEquals.
• ctrl-F11 (rerun latest command): To run for example the same test again, you can usually press ctrl-f11. Sadly, a few years back the Eclipse team tried to improve this and failed. Fix it under Windows->Preferences, Run/Debug -> Launching. Change “Launch operation” to “Always launch previous”.
• f12 (activate editor): When you perform an operation where some other pane got the focus, use f12 to return to the editor again.
• ctrl-N (new <something>): Create a new class, XML file or whatever. Be sure to use the filter
• alt-shift-l (extract local variable): My favorite refactoring. Select an expression and press alt-shift-l to assign it to a local variable and replace all uses of the expression with that variable.
• alt-shift-m (extract method): Your bread and butter refactoring to split up complex logic in understandable units.
• alt-shift-i (inline method/inline variable): The inverse of both alt-shift-l and alt-shift-m. Together, these three refactorings let you resculpt your code while being certain that the behavior is unaltered.

Top three properties to change

When I sit down with new programmers, I almost always help them make the following changes in the preferences. Find the preferences under Window->Preferences:

• Use ctrl-tab (and ctrl-shift-tab) to switch between open editors: Go to General->Keys, type in “next editor” in the filter. Select “Copy command” and type “ctrl-tab” in the Binding field. Do the same for “previous editor” and ctrl-shift-tab.
• Type filter: Do you wonder why Eclipse can’t understand that when you say List, you mean java.util.List, not org.hibernate.mapping.List or (ye gods!) java.awt.List. Well, you can make Eclipse understand. Put classes and packages you don’t like under Java->Appearance->Type filters. If your project is like mine, putting org.hibernate.mapping.*, antlr.*, java.awt.List, and com.sun.* in the list makes List unique to java.util.List. Then “organize imports” and completion works as you want.
• Static import favorites: Do you find yourself using static imports with the same few classes again and again? The preference Java->Editor->Content Assist->Favorites lets you list up classes which will have their static methods checked when you press ctrl-space to complete a method call. org.junit.Assert.* is a good first candidate.

Learning your tool

All IDEs are rich and powerful tools. Spending some time to learn a few new tricks is well worth the effort.

But I thought I’d post the video here, to see what you all think. Comments are welcome!

I hope the video will demonstrate how to use refactoring effectively to drive the design of a program.

I chose the FizzBuzz kata – that is, to generate a sequence of numbers where every number divisible by three is replaced by “fizz” and every number divisible by five is replaced by “five”. The music changes to be more aggressive just as I induce a new requirement into the kata: The FizzBuzz generator should be programmable, so, in the kata, numbers divisible by two are replaced by “coconut” and numbers divisible by seven are replaced by “banana”.

Thanks to Emily Bache for the inspiration for the kata.

Enjoy!

Fizz buzz code kata from Johannes Brodwall on Vimeo.

The video was made with IntelliJ IDEA Community Edition on Windows Vista (!) with BB FlashBack Express (free screen recorder), converted to AVI with Windows Media 1 codec and uploaded to Vimeo.

• Java EE Spike: An application that stores People with names to a database and lets me search for them. I’ve repeated this pair programming with several different programmers.
• Programmable Fizz Buzz: Create a sequence of numbers 1,2,fizz,4,buzz,fizz,… you know the one. And the twist: Make it programmable, so that for example numbers divisible by 7 should be replaced with “coconut”.

I’ve learned a lot from repeating these exercises:

• Using test-driven development it takes me longer to get to something that “should work in principle”, but shorter to get to something that works correctly.
• When refactoring to a new data structure, add the new structure while keeping the old one, make switching between them as simple as changing a single line. Delete the old when it works.
• There’s always an automated refactoring you still want to help you out. Extract Parameter Object was my big one.
• Writing for example a method invocation and then using quickfix to have the IDE generate the method is the quickest way of writing code available to you.
• After 8 iterations, the Java EE Spike takes me 80 minutes solo. Pair programming with another programmer who had practiced: 65 minutes. I don’t know why!
• Pair programming a moderately complex kata like the Java EE Spike is fun. It’s also a good chance to discuss different roles of different tests.
• There is a huge difference between a test that takes 3 seconds to run and one that takes 0.5 seconds when you’re test driving. More surprisingly, there’s a big difference between a test that takes 0.5 seconds and one that takes 0.01 seconds
• If you think test-driven development is not for you or that it’s bunk, you probably write really slow tests.

What are your latest coding observations?

Dersom du vil prøve deg på samme øvelse, inneholder denne artikkelen litt informasjon for å komme i gang. Start med koden under og følg feilmeldingene. Send en kommentar dersom du ikke kommer videre fra en feilmelding, så får vi en FAQ.

Oppgaven

Løs et så enkelt som mulig problem som involverer websider og database med så enkel teknologi om mulig.

Oppgaven jeg har laget går ut på å opprette personer med fullt navn og søke etter personer basert på navnet deres. For å gjøre oppgaven så lite som mulig har jeg valgt å la personer kun ha ett informasjonsfelt: Fullt navn. Denne oppgaven tar cirka 2-3 timer uten øvelse og du kan få den ned i 60-90 minutter med trening.

Du kan naturligvis velge en annen oppgave, men uansett hva du velger: Det er mer lærerikt å gjenta den samme oppgaven flere ganger enn å utføre en avansert oppgave.

Når jeg utfører oppgaven er det viktigste jeg lærer meg å forstå feilmeldingene som guider meg gjennom utviklingen. Dersom du trenger hjelp til å komme til de første feilmeldingene kan du se resten av artikkelen.

Steg for steg: Startpunktet

Selv om jeg valgte veldig enkel teknologi for implementasjonen, har jeg valgt et større sett med biblioteker for å skrive testene. Jeg bruker følgende når jeg skriver testene:

• JUnit 4.6
• Jetty 6.1.22
• HSqlDb 1.8.0.10
• WebDriver-HtmlUnit 0.6.1039
• Mockito 1.8.0
• FEST-assert 1.2 (ikke påkrevd, men gjør testene søtere)

Den eneste teknologien jeg har valgt for implementasjonen er Servlet-API 2.5 og Hibernate-Annotations 3.4.0.GA.

For at du skal slippe å plundre så mye med avhengigheter før du kommer i gang har jeg laget en pom.xml-fil som du kan ta utgangspunkt i.

Web-tester

For å starte utviklingen, er det lurt med en test som starter på utsiden av applikasjonen. Noe slikt:

1. Start opp miljøet
2. Legg inn en person
3. Søk etter personen

Slik kommer du i gang med en test som går mot en web applikasjon:

int SERVER_PICKS_PORT = 0;
org.mortbay.jetty.Server server = 
       new org.mortbay.jetty.Server(SERVER_PICKS_PORT);
server.addHandler(
       new org.mortbay.jetty.webapp.WebAppContext("src/main/webapp", "/"));
server.start();
 
int serverPort = server.getConnectors()[0].getLocalPort();
 
org.openqa.selenium.WebDriver browser =
       new org.openqa.selenium.htmlunit.HtmlUnitDriver();
browser.get("http://localhost:" + serverPort + "/");
browser.findElement(By.linkText("Create person"));

Dette oppsettet forventer å finne web.xml-fila på src/main/webapp/WEB-INF/web.xml.

Funksjonell test

En funksjonell test definerer kravene i applikasjonen. Det er lurt å gjøre funksjonelle tester så raske som overhode mulig, samtidig som de går gjennom alle kravene. En funksjonell test trenger ikke være en ende-til-ende test, slik som eksempelet over. Dette er viktig, fordi ende-til-ende tester er ofte veldig trege. Her er noen eksempler på funksjonelle tester:

• Vis en siden for å opprette nye personer
• Opprett en ny person
• Verifiser at personens navn er oppgitt og ikke inneholder ulovlige tegn
• Vis en side for å søke etter personer
• Vis alle personer dersom søkestreng ikke er angitt
• Søk etter angitt søkestreng

En funksjonell test kan se slik ut:

PersonServlet servlet = new PersonServlet();
 
HttpServletRequest req =
    org.mockito.Mockito.mock(HttpServletRequest.class);
HttpServletResponse resp =
    org.mockito.Mockito.mock(HttpServletResponse.class);
 
PersonDao personDao =
    org.mockito.Mockito.mock(PersonDao.class);
servlet.setPersonDao(personDao);
 
org.mockito.Mockito.when(req.getMethod())
    .thenReturn("POST");
org.mockito.Mockito.when(req.getPathInfo())
    .thenReturn("/create.html");
org.mockito.Mockito.when(req.getParameter("full_name"))
    .thenReturn("Johannes Brodwall");
 
StringWriter pageSource = new StringWriter();
org.mockito.Mockito.when(resp.getWriter())
    .thenReturn(new PrintWriter(pageSource));
 
servlet.service(req, resp);
 
org.mockito.Mockito.verify(personDao)
    .create(Person.byName("Johannes Brodwall"));
 
org.fest.assertions.Assertions.assertThat(pageSource.toString())
    .contains("Personen er opprettet");

Data-aksess-test

Hibernate forenkler databasebruken mye. Men Hibernate er selv komplekst og når man bruker det på mer avanserte måter fortjener det egne tester. En typisk test med Hibernate kan være:

1. Legg i tre personer i database
2. Søk etter en del av navnet på en av dem
3. Sjekk at du får tilbake akkurat den du forventet

Når jeg starter med Hibernate, lager jeg en test som dette, og følger feilmeldingene. Pass på å både følge feilmeldinger i loggen og stack tracer.

AnnotationConfiguration conf = new AnnotationConfiguration()
    .setProperty(Environment.URL, "jdbc:hsqldb:mem:persondaotest");
PersonDao dao = new HibernatePersonDao(conf.buildSessionFactory());
 
dao.create(Person.withName("foo"));
 
org.fest.assertions.Assertions.assertThat(dao.find(null))
    .containsExactly(Person.withName("foo"));

Følg feilmeldingene herfra.

Integrasjon

En veldig vanlig måte for web serveren å overlevere spesielt ting som DataSources til applikasjonen er via JNDI. I Jetty kan du gjøre dette i Web-testen på følgende måte:

org.hsqldb.jdbc.jdbcDataSource ds = new org.hsqldb.jdbc.jdbcDataSource();
ds.setDatabase("jdbc:hsqldb:mem:personwebtest");
ds.setUser("sa");
new org.mortbay.jetty.plus.naming.Resource("jdbc/primaryDs", ds);
 
// Oppstart av Jetty som vist over

Konklusjon

Å gjøre en liten øvelse som dette er en god måte å bli bevisst hvilke vaner du har og hvor lang tid det egentlig tar for deg å gjøre oppgavene dine. Du vil oppleve at det å skrive tester før koden føles som om det går saktere enn du tror du er vant til.

Men dersom du er som meg, vil du også oppleve noe annet: Når du tester ut applikasjonen første gang (du kan gjøre dette med Jetty, naturligvis) så er sjansene gode for at den vil være nokså feilfri og at debugging i stor grad er overflødig. Jeg vet ikke med deg, men debugging er en aktivitet jeg gjerne blir kvitt.

1. Effect Enterprise Java architecture in 2009

Since the Effective Enterprise Java book was written, many of the topics regarding transactions, concurrency and shared state have been resolved. Here are the basic guidelines of an enterprise application in Java as of 2009:

1. All processing is triggered by an event, such as an http-request, a timer or an incoming message
2. Each processing event is handled in an isolated scope, never touching the data of another processing event. All coordination of data happens through the data layer. This means that objects are either stateful and short-lived or stateless and immortal.
3. Each processing event is either completed or aborted totally. Very few applications will benefit from trying to automatically recover from most problems.
4. Inconsistent updates are resolved when transactions are committed, usually through optimistic locking.

Some things I told the attendants to consider: First, today most people consider EJBs to be more trouble than value (with the exception of Entity beans 3.0 which is JPA which is really mostly Hibernate, which really isn’t very much EJB). Second, all triggers can be forged. We return to the second issue when we discuss security.

2. Web integration testing

I showed a practical demo using WebDriver and Jetty to perform web integration tests as JUnit test. The remarkable things about this example is that it requires no installation of an app server (Jetty is installed as a Maven dependency), it requires no separate starting of an application server (Jetty can run embedded in the test) and it is very fast (Jetty starts up in about 200 milliseconds).

3. Hibernate integration testing

I showed a practical example of how to test a DAO implemented with Hibernate. The remarkable things about this demonstration was that, again, no installation or startup is required (I use H2 as an in-memory database).

Hibernate is a power tool. I use the following analogy: If you’re building a tunnel and need to mine through a mountain, you want to use dynamite. If you want to remove a rock from you back yard, you may want to use dynamite. But if you don’t know what you’re doing, chances are you may blow your foot off.

Hibernate is like that dynamite. You need knowledge and safety measures to deal with it correctly, but when you do, it can save you a lot of effort. Creating JUnit tests for your Hibernate code is one such safety measure.

4. Security

Almost all the threats an application developer should be concerned with are in the same class, namely that of Injection attacks. An injection attack is when a client tricks another process into treating data as instructions. For example by using SQL meta-characters:

An important source of injection vulnerabilities is HTML injection, also known as Cross-Site Scripting (XSS).

In both situations, and in all others, there’s one important guideline: Data from the outside world should be considered “tainted”. Never use tainted data in unsafe ways. When reading input parameters, validate against malicious characters (but please don’t make poor “O’Reiley” unable to use your system). When writing HTML pages, always escape tainted data. When using tainted data during access to the database or with HSQL or JPAQL, always use PreparedStatement and send in data as parameters.

Another often overlooked exploit is request forgery, often used in combination with phishing attacks. To protect your users from request forgery, supply an authentication token as a hidden field with all forms. Or if you’re lazy: Make sure all operations have confirmation dialogs.

5. Continuous Deployment

Continuous Deployment is the practice of rolling out a deployment to a server after every successful build on your Continuous Integration server. I described two ways of doing Continuous Deployment during the tutorial, but I will restrict this discussion to the more modern one.

Most teams doing continuous deployment use Maven or Ant to invoke the deployment tools of their respective application servers. Many application servers make this pretty hard, but the hardest part of the battle if finding out what command needs to be invoked. The Continuous Integration server can be configured to run this task.

After doing deployment, it is a good idea to run some sort of system level integration tests. Teams use replay of production data, load generators like JMeter and webcrawlers that validate HTML and CSS to do automated non-functional integration tests. If you keep your logs clean, you can actually gain quite a bit of confidence just by looking at the logs after applying simulated load to your system.

Some projects take this even further, by continuously deploying to production. Both IMVU and Flickr are known to practice this.

At any rate, the practice of doing continuous deployment should lead you to consider how to simplify your deployment and runtime configuration, which will result in an easier installation procedure into production, even if it’s not automated.

Summary

Effective Enterprise Java development has progressed a lot since 2004. Much of the emphasis now is on how to improve testing in enterprise Java applications. The way applications usually process data has stabilized as well, with most application preferring each event to be processed in an isolated, transactional context with very little automated recovery.

In the end, Effective Enterprise Java is a lot simpler in 2009 than it was in 2004.

Material

• My slides, including topics that we didn’t discuss as well as code for all the examples
• The complete source code for one iteration of my Enterprise Java Kata, including a pom.xml file with all dependencies needed to get the tests running

But for speed and agility, unit-level tests beat even the fastest integration level tests. Unit tests run much much faster, and lets me pinpoint problems much more accurately. As a general rule: If you introduce a bug, you want a unit test to be the one that sees it first.

For web applications in Java, I’ve found the following pattern useful when testing classes that deal with HttpServletRequest and HttpServletResponse:

@Test
public void shouldDecodeSearchParameters() throws IOException {
    HttpServletRequest req = mock(HttpServletRequest.class);
    HttpServletResponse resp = mock(HttpServletResponse.class);
    Repository repository = mock(Repository.class);
 
    PersonController personController =
            new PersonController(req, resp, repository);
 
    given(req.getParameter("firstName")).willReturn("johannes");
    given(req.getParameter("lastName")).willReturn("brodwall");
    given(req.getParameter("includePaidMembers"))
            .willReturn("true");
    given(req.getParameter("includeUnpaidMembers"))
            .willReturn("false");
    when(resp.getWriter())
            .thenReturn(new PrintWriter(new StringWriter()));
 
    personController.find();
 
    ArgumentCaptor< PersonSpecification> specificationCapture = 
            ArgumentCaptor.forClass(PersonSpecification.class);
    verify(repository).find(specificationCapture.capture());
 
    PersonSpecification personSpec =
            specificationCapture.getValue();
    assertThat(personSpec.getFirstName()).isEqualTo("johannes");
    assertThat(personSpec.getLastName()).isEqualTo("brodwall");
    assertThat(personSpec.isIncludingPaidMembers())
         .isEqualTo(true);
    assertThat(personSpec.isIncludingUnpaidMembers())
         .isEqualTo(false);
}

1: This example uses Mockito to specify what calls to HttpServletRequest should return. A really nice thing about Mockito is that you only have to specify what you care about. If, for example, PersonController ends up calling req.getParameter("nonexistant"), Mockito will simply return null. If PersonContoller asks for “lastName” several times, Mockito will keep returning “brodwall”.

2: Mockito doesn’t verify that a set of methods were called unless you explicitly ask it to. PersonController happens to call several methods on HttpServletResponse that we don’t even mention in the test.

3: Mockito lets you look at what arguments were used to call a mocked-out method after the method was called by using an ArgumentCaptor.

One caveat, though: In the case of HttpServletRequest, the production code can decide to read the request parameters in several different ways: getParameter, getParameterValues or getParameterMap. Mockito will naturally not have any idea of the relationship between these methods, so your test will be sensitive to this sort of implementation change.

After working with EasyMock and JMock for a while, I seriously lost faith in mocking. Mockito has restored my faith again!

Happy testing!

Steel pipes (by monkeyc.net)

The term “Continuous Integration” was first discussed when Extreme Programming was starting to garner interest in the late 90s. From then, it has gone from being a manual process that top-notch team used, to being an automated, nearly ubiquitous process. The tools have gone from being home made through demanding tools like CruiseControl to user friendly tools like Hudson. After Hudson, is there still any radical change in store for us?

Somewhat independent of the evolution of Continuous Integration tools, there have been four trends that have developed in the last few years:

• Continuous testing: Tools like autotest for Ruby and Kent Beck’s JUnit Max for Java execute your tests after every change you make to the source code. Autotest is widely used within the Rails community, and even though JUnit Max did not get the takeoff Kent was hoping for, I think there’s still great potential in this sphere. I’ve used both tools, and they transform the way I work for the better.
• Distributed source control greatly increases our flexibility in terms of multiple sources and stages of source code. Especially Git has seen growing interest in the last two years. Github is quickly becoming one of the large project hosting providers.
• Continuous deployment: Organizations have started pushing the result of their continuous integration process further towards production. In the last three years, I’ve worked on two large projects, both of which deploy every build to a test server. The company IMVU, with it’s large customer base, deploys automatically into production roughly 50 times per day.
• Smaller checkins: In the last issue of The Agile Toolkit George (no last name given in podcast or notes) suggest checking in every time your build is green. I’ve never worked on a project like that, but I’ve experienced a gradual increase in how frequently we check in.

Complexity (by nerovivo)

If we extrapolate from these trends, where do they lead? Here is what I think will be the development experience of advanced teams in the future:

1. Whenever I save a file, my (fast running) tests are run in the background.
2. When all the tests run successfully, my changes are pushed up to my personal clone of the repository.
   A first stage continuous integration server listens to changes from all the developers repositories. When it verifies the tests, it pushes the changes to the integrated repository.
3. Every few minutes, my workspace is updated to reflect new changes from other developers in the integrated repository.
4. After the integrated repository, similar build processes propagate code changes through slower, and possibly even manual tests. The verified result is stored in the staged repository.
5. At the push of a button, I can roll the code from the staged repository into any test or production environment.

Sounds far fetched? Vincent Massol wrote about unbreakable builds five years ago. Distributed version control is being adopted quickly and will greatly simplify the implementation of such processes. Despite Kent Beck’s regretful decision to stop active development of JUnit Max, I believe the time for continuous testing is near. The process I outline can include as enough verification steps to make the organization comfortable. As the trend of improving test quality continues, this process will be gradually more automated.

The strange thing is that we’ve almost made a complete circle: Before the widespread use of revision control, many developers would edit the code directly in their production environment. Extreme Integration will feel almost like this, but with enough non-intrusive verification to make even the most paranoid test manager happy.

Thanks to Martin Eggen for digging up the information on IMVU’s Continuous Deployment. Thanks to Sarah Brodwall, Trond Pedersen and Finn-Robert Kristensen for helpful comments.