<?xml version="1.0" encoding="UTF-8"?><rss version="0.92">
<channel>
	<title>Thinking Inside a Bigger Box</title>
	<link>http://johannesbrodwall.com</link>
	<description>Johannes Brodwall&#039;s Musings on Software Architecture and Programming</description>
	<lastBuildDate>Thu, 02 Sep 2010 07:36:15 +0000</lastBuildDate>
	<docs>http://backend.userland.com/rss092</docs>
	<language>en</language>
	<!-- generator="WordPress/3.0.1" -->

	<item>
		<title>Agile Release Pattern: Database migrations</title>
		<description><![CDATA[As I release more frequently, I start to focus on automating the actual process of deploying a release. One of the most powerful steps of automating deployment is to automatically upgrade the database schema. This technique first saw mainstream use with the Ruby-on-Rails framework. Today, there are several mature tools that will help you organize [...]]]></description>
		<link>http://johannesbrodwall.com/2010/09/02/database-migration/</link>
			</item>
	<item>
		<title>Are you an architect or just a freaking good developer?</title>
		<description><![CDATA[A software architect who doesn&#8217;t care about what his system is supposed to do isn&#8217;t worth his salt. For the term &#8220;software architect&#8221; to hold any meaning at all, it must be to describe someone who understands what the customer needs and designs a system that is fit for this purpose. Sometimes, however, people talk [...]]]></description>
		<link>http://johannesbrodwall.com/2010/08/31/are-you-an-architect-or-just-a-freaking-good-developer/</link>
			</item>
	<item>
		<title>Agile release pattern: Merging configuration</title>
		<description><![CDATA[If you want to release your code frequently, you have to automate the release process. If your software interacts with shared components or other applications, the release script may have to update shared configuration files. In this blog post, I recall when I solved this with a crontab file. The problem My application needs a [...]]]></description>
		<link>http://johannesbrodwall.com/2010/08/26/agile-release-pattern-merging-configuration/</link>
			</item>
	<item>
		<title>Agile release pattern: Feature-on/off-switch</title>
		<description><![CDATA[If you want to release frequently, a problem you may encounter is that some features, even though functionally complete, don't stand well on their own feet, but require other features to be valuable to the user. If you want to release the system in this state, you need a way to hide features. A Feature-on-switch is a simple idea for dealing with this.

A feature-on-switch is some mechanism to hide features from a system. A feature-on-switch must be able to remove links to the feature from menus and other pages and also to prevent adventuresome users from accessing the feature. It may be as crude as commenting out code (not recommended!), to enabling the feature based on a complex set of conditions (also not recommended).

I've encountered features switches triggered by the following mechanisms:
<ul>
  <li>A configuration file or configuration database table tells the system whether to turn the feature on or off.</li>
  <li>The feature is turned on for users that have a specific role (typically something like BETA_TESTER)</li>
  <li>The feature is turned on when the system is deployed as /foo-preview, but not when the system is deployed as /foo</li>
  <li>The feature is turned on after a specific date. This may seem weird, but was a potential solution when we were waiting for a release of another system and operations-freeze during summer was in effect.</li>
</ul>

There are probably many more conditions you may use to trigger a feature-on-switch. Maybe some of my readers have good examples?]]></description>
		<link>http://johannesbrodwall.com/2010/08/19/agile-release-pattern-feature-on-switch/</link>
			</item>
	<item>
		<title>Generalized observation</title>
		<description><![CDATA[As a general observation, it seems that when software architects try to solve general problems, the come up with horrible design; when they solve specific problems, they come up with good designs.

Designs made without reference to a problem often become complex and not very fit for purpose when we're solving specific problems. As a general rule, avoid generalizations.

Some examples:
<ul>
  <li>An Enterprise Service Bus may create a big project and maintainable needs for something that turns out to be only a few simple integration points.</li>
  <li>Splitting a system into generic reusable services may make it harder to understand and maintain</li>
  <li>A generalized security role model may make be hard to understand while the only thing the system needed was an "is_admin?" toggle.</li>
  <li>A complex role based ("can create payment") security model may hide control of authorization from the developers, making it harder to implement the usually more important data based ("can access account number 5") security model</li>
  <li>A general model for workflow transition may make it harder to implement the specific workflow you need for a particular process. And it leads to endless discussions about the relationship between a workflow, a process and a process step.</li>
  <li>Generalized test strategies are often vague and require a large number of test environments. In the end, it doesn't contribute to increased quality.</li>
</ul>

In "No Silver Bullet", Fred Brooks introduces the concepts of "essential complexity" and "accidental complexity." The complexity from generalization is always "accidental" (that is: not inherently necessary). When you focus on solving the essential complexity (that is: the users' problem) as efficiently as possible, you may found the complexity of your problem shrink by half or more.

Have you ever found yourself thinking, "what specific problem was I trying to solve again"? Then you've probably been down the dark road of generic thinking.]]></description>
		<link>http://johannesbrodwall.com/2010/08/17/general-observation/</link>
			</item>
	<item>
		<title>Unified task list: A requirement mirage?</title>
		<description><![CDATA[When developing a system that people use in their day to day work, I often meet the following requirement: &#8220;A user should be able to see all tasks from each functional area on a single screen.&#8221; This requirement requires integration with all parts of the system, making it architecturally costly. Luckily, the requirement might often [...]]]></description>
		<link>http://johannesbrodwall.com/2010/08/14/unified-task-list-a-requirement-mirage/</link>
			</item>
	<item>
		<title>Six ideas that improve your software design</title>
		<description><![CDATA["Design" is a verb, not a noun. If I want to create a good program, studying the process of getting there is much more important than the resulting software. This is why I use coding katas as a form of study. I find an interesting problem problem and then solve the same problem over and over again. In this blog post, I will focus on six principles of software design. I will illustrate each with a screencast from a kata.

One of my favorite problems is that of creating a Java EE application from scratch. I call this kata "The Java EE Spike Kata". In order to understand the role of frameworks, I use no web frameworks in the process of creating this application. I've completed this particular exercise about forty times on my own and more than ten times with various pair-programming partners. The whole exercise takes me about 90 minutes, and I still learn new things.

The total time of the screencasts is around 40 minutes, so you may want to pick and choose. Each section provides a link to the starting point for the source code if you want to follow along.

<em><strong>Please notice</strong>: The videos are accompanied by loud, pounding music. Keep you headphones on or your volume down if you share offices with someone else. Or mute the videos if you dislike the music.</em>


<h3>Idea 1: Build your software from the outside-in</h3>

<embed src="http://blip.tv/play/AYHWhGIA" type="application/x-shockwave-flash" width="480" height="380" allowscriptaccess="always" allowfullscreen="true"></embed>

(10 minutes, <a href="http://github.com/jhannes/java-ee-spike-kata/tree/1-outside-in">github starting point</a>)

I start building the application by writing tests that access the application over HTTP and looks at the resulting HTML. As you might have gathered, when I start this test, there is no web application. Only when the tests require a web application to continue do I start creating it. In this example, I had created a basic sketch of the interaction between the three web pages in the application before I started coding. No further design was necessary.

This particular approach uses <a href="http://code.google.com/p/selenium/wiki/GettingStarted">WebDriver</a> and <a href="http://jetty.codehaus.org/jetty/">Jetty</a> to run. The cute assertion library that you may have noticed at the end of the video is <a href="http://fest.easytesting.org/assert/">FEST-assert</a>.


<h3>Idea 2: Specify behavior rather than implementation</h3>

<embed src="http://blip.tv/play/AYHWhF4A" type="application/x-shockwave-flash" width="480" height="380" allowscriptaccess="always" allowfullscreen="true"></embed>

(6 minutes, <a href="http://github.com/jhannes/java-ee-spike-kata/tree/2-1-servlet-test">github starting point</a>)

I don't make much of a distinction between different types of tests. All good tests try to describe <em>what</em> the software should do at some level, rather than <em>how</em> the software does it. But the <em>how</em> at some level may be the <em>what</em> at another level. My first test specified the interaction between the web browser and the server. In this test, one step may be to fill in the form element of a web page. This second video shows <em>how</em> this form works in terms of actual HTML. But the details of what framework (if any) is used, is not visible in the test.

The second thing you'll notice in the video is that I run the tests more frequently. And each test run is much quicker. As our tests move close to the code, the rate of feedback improves.

This particular test uses <a href="http://mockito.org/">Mockito</a> to mock out the Servlet API. The assertions use <a href="http://fest.easytesting.org/assert/">FEST-assert</a>.


<h3>Idea 3: Increase the rate of feedback</h3>

<embed src="http://blip.tv/play/AYHWhCEA" type="application/x-shockwave-flash" width="480" height="380" allowscriptaccess="always" allowfullscreen="true"></embed>

(5 minutes, <a href="http://github.com/jhannes/java-ee-spike-kata/tree/3-equals-test">github starting point</a>)

This video illustrates the frequency of feedback. The example test-drives creating an <code>equals</code>-method. This task is often not worth test-driving. The resulting method is usually simple and/or generated by your IDE. But it is a good example to of how quick the cycle between test and production code <em>can</em> be when you're writing tests that are close to the problem at hand. When I pair program this part of the kata, we usually use a technique called ping-pong programming: One programmer writes a failing test (or failing assertion) and hands the keyboard to his partner, the other programmer makes the test pass and writes another failing test before passing the keyboard back. On a good run, we will switch who's got the keyboard more often than once per minute.

Notice that I also focus on the <em>behavior</em> of the <code>equals</code>-method in this test.


<h3>Idea 4: Grow the API rather than designing it up front</h3>

<embed src="http://blip.tv/play/AYHWhEwA" type="application/x-shockwave-flash" width="480" height="380" allowscriptaccess="always" allowfullscreen="true"></embed>

(8 minutes, <a href="http://github.com/jhannes/java-ee-spike-kata/tree/4-search-for-people">github starting point</a>)

As the web application grows under my fingers, I discover the need for a Data Access Object (DAO). However, as this represents a major internal interface in my application, I use Mockito to mock the implementation until I'm done with the behavior of my servlet. When this is done, I test-drive the implementation of the DAO in a separate test class.

The video also illustrates another important lesson: The code is getting ripe for a refactoring. But it's important to resist the urge to refactor until the tests are green. If you refactor on red tests, you have much higher chances of running down a dead-end road and you'll have to throw away your progress, wondering what went wrong.

The example uses <a href="http://mockito.org">Mockito</a> to mock out the DAO API.

If you want to see how I implement the DAO with Hibernate, you can see the video on <a href="http://jhannes.blip.tv/file/3487456/">blip.tv</a>.


<h3>Idea 5: Grow the design rather than speculating</h3>

<embed src="http://blip.tv/play/AYHWhEsA" type="application/x-shockwave-flash" width="480" height="380" allowscriptaccess="always" allowfullscreen="true"></embed>

(3 minutes, <a href="http://github.com/jhannes/java-ee-spike-kata/tree/5-2-refactoring">github starting point</a>)

The video is only a partial example of this principle. Throughout the whole application, I've been refactoring, gradually pulling out structure to more well-structured methods and classes. The video illustrates some of the power of IDE's when it comes to refactoring. Using the IDE to massage your code into a better design makes evolutionary design much easier to do in practice. Make sure to learn your IDE's most useful refactorings!

The kata may seem like a non-realistic example at this time, but I've actually grown a very successful architecture on my current project using much the same approach. If you want to explore where to go next, the next step needed for this application is to factor out the views into separate classes and then use either a <a href="http://martinfowler.com/eaaCatalog/templateView.html">View Template</a> language (like Velocity) or a <a href="http://martinfowler.com/eaaCatalog/transformView.html">View Transformer</a> (using, for example dom4j) to generate the HTML. (Let me know if you'd like to see the screencast of this as well).


<h3>Idea 6: It's supposed to work the first time around</h3>

<embed src="http://blip.tv/play/AYHWhF8A" type="application/x-shockwave-flash" width="480" height="380" allowscriptaccess="always" allowfullscreen="true"></embed>

(5 minutes, <a href="http://github.com/jhannes/java-ee-spike-kata/tree/6-getting-it-to-work">github starting point</a>)

In this video, I return to the first test, <code>PersonWebTest</code>, to finish the configuration of the application. I discover a few mistakes I made in the web test as I complete the exercise. Then I try out the code in the browser. And all the scenarios I had planned for work out of the box.

When you try out your code for the first time, it should work. When you master test-driven development, you will probably forget how you programs <em>didn't</em> use to work the first time. Only when you occasionally run into an unexpected error during manual testing it becomes clear how test-driven development changes you life.

If you want to see the whole, uninterrupted 75 minute code kata, the video is available at <a href="http://jhannes.blip.tv/file/3487566/">blip.tv</a>. You can also take a look at the finished source code at <a href="http://github.com/jhannes/java-ee-spike-kata/tree/commit_per_test">github</a>.

Happy programming!


<hr />

<em>A big thanks to Trond, Thomas, Ram and Christian who helped improve this post. Thank you to Finn-Robert, Øistein, Mats, Anders, Siv, Peyman, Ivar, Øystein, Cecilia, Nicolay, and Karianne who have pair-programmed this exercise with me. I especially appreciate how Ivar and Karianne both helped influence the way the application is wired together and showed me that even after 30 iterations, I still had things to learn; and how Øistein showed me how two trained developers could complete the exercise faster with pair-programming than alone. And thank you to Nicolay who graciously brought food to our pair-programming exercise.</em>

The videos were made with the excellent (and free!) <a href="http://www.bbsoftware.co.uk/BBFlashBack_FreePlayer.aspx">BB FlashBack Express</a>. The keyboard echo is courtesy of <a href="http://katastrophos.net/magnus/blog/2008/08/01/keypose-flavour-your-screencasts-with-shortcuts/">KeyPosé</a> by Magnus Jungsbluth.]]></description>
		<link>http://johannesbrodwall.com/2010/05/02/six-ideas-that-improve-your-software-design/</link>
			</item>
	<item>
		<title>Why TDD makes a lot of sense for Sudoko</title>
		<description><![CDATA[My colleague Thomas sent me a <a href="http://devgrind.com/2007/04/25/how-to-not-solve-a-sudoku/">very interesting link</a> about attempts to solve Sudoku using test-driven development. The article, somewhat unfairly, pits Ron Jeffries' explorations of Sudoku using test-driven development against Peter Norvig's "design driven" approach.

I found both attempts lacking. However, while Ron Jeffries freely admitted that he didn't even know the rules of Sudoku when he started, both Norvig himself and his readers fawn over his solution. I didn't find it very understandable.

So I took it upon myself to examine the problem myself. I did some up-front thinking in the shower and on the subway, then attacked the problem with TDD. I ended up with a solution that works in <em>all</em> cases (unlike Norvig). My implementation has readable code, readable tests, and solves the problem reasonably fast.

<h3>Observations and conjectures</h3>

Here are a few things I learned from the exercise:

<ul>
  <li>When you're using TDD to solve a tricky algorithm, you have to think about both the algorithm and the test approach.</li>
  <li>Solving a problem with a known algorithm using TDD gives more readable code than I otherwise would expect.</li>
  <li>When I solved the problem with TDD, running the solution on real problems worked the very first time I tried it.</li>
  <li>The trick to making TDD work is to work from the outside in.</li>
  <li>When creating a Sudoku solver, don't think like a human! Think like a machine! The human algorithm is difficult to understand and likely to not work on all problems. This was the biggest problem with Norvig's code</li>
</ul>

<h3>The journey</h3>

I decided on the following approach:

<ol>
  <li>I had decided upon an initial design with a solver class and a board class. The solver should use a recursive depth first search. The solver asks the board what options exists per cell, but it has no knowledge of the rules of Sudoku (such as no duplicate numbers on the same row).</li>
  <li>The first step was to get the solver ("the outside") correct. For this step, I mocked out the board</li>
  <li>The second step was to implement the interface that the solver needed for the board. Mainly, this is a matter of specifying the rules for what numbers can occur in which cell on a Sudoku board.</li>
  <li>Finally, I wrote some code to read and write the Sudoku board. When trying the solver on real problems, it worked the first time, and solved <a href="http://magictour.free.fr/top95">95 hard problems</a> correct. It was somewhat slow, though.</li>
</ol>

After solving the problem the first time, I practices a few times and recorded a screen cast of the solution:

<embed src="http://blip.tv/play/AYHTrDcA" type="application/x-shockwave-flash" width="480" height="380" allowscriptaccess="always" allowfullscreen="true"></embed>

<h3>The solver</h3>

Testing the solver is a matter of creating a mock board and ensuring that the solver does the correct things. This is the most complex test case:

<pre lang="java">
@Test
public void shouldBacktrackWhenNoMoreOptions() throws Exception {
    SudokuSolver solver = new SudokuSolver();
    SudokuBoard board = mock(SudokuBoard.class);
    when(board.getOptionsForCell(anyInt(), anyInt()))
            .thenReturn(singleOption());

    when(board.getOptionsForCell(8, 7))
            .thenReturn(moreOptions(1, 2));
    when(board.getOptionsForCell(8, 8))
            .thenReturn(noOptions())
            .thenReturn(singleOption());

    assertThat(solver.findSolution(board)).isTrue();
    InOrder order = inOrder(board);
    order.verify(board).setValueInCell(1, 8,7);
    order.verify(board).setValueInCell(2, 8,7);
}
</pre>

It specifies that all cells, except (8,7) and (8,8) return exactly one option. (8,7) returns two options. (8,8) returns no options the first time it is called, and one option the second time. The test verifies that a solution is found, and the solver tries to set both options for (8,7).

This drives a rather simple algorithm. Here's basically the whole algorithm:

<pre lang="java">
public boolean findSolution(Board board, int cell) {
    if (cell == SIZE*SIZE) return true;

    boolean wasEmpty = board.isEmpty(row(cell), col(cell));
    for (Integer value : board.getCellOptions(row(cell), col(cell))) {
        board.setValueInCell(value, row(cell), col(cell));
        if (findSolution(board, cell+1)) return true;
    }
    if (wasEmpty) board.clearValueInCell(row(cell), col(cell));

    return false;
}
</pre>

The algorithm tries all available options for a cell in order. If no solution works for the rest of the board, the algorithm returns false (for "no solution").

The algorithm is not how a human would solve Sudoku. But then again, we're not writing a tutorial on how to solve Sudoku, we're writing a <em>program</em> that solves Sudoku.
The board

As I implemented the solver, the interface for the board started to emerge. At that point in time, I had to create tests for the Sudoku board itself. A typical test verifies that the board doesn't allow duplicate values in a row:

<pre lang="java">
@Test
public void shouldDisallowOptionsInSameRow() throws Exception {
    int row = 4;
    board.setValueInCell(1, row, 5);
    board.setValueInCell(2, row, 8);
    board.setValueInCell(3, row+1, 5);
    assertThat(board.getOptionsForCell(row, 0))
            .excludes(1,2).contains(3);
}
</pre>

The essence of SudokuBoard is finding out what values are legal in an open cell:

<pre lang="java">
public List getOptionsForCell(int row, int col) {
    if (!isEmpty(row,col)) return Arrays.asList(cells[row][col]);
    List result = allOptions();
    removeAllInRow(result, row);
    removeAllInCol(result, col);
    removeAllInBox(result, row, col);
    return result;
}
</pre>

<h3>TDD as a design guide</h3>

I invite you to compare <a href="http://norvig.com/sudoku.html">Peter Norvig's solution</a> to mine (you can find the full source code for my solution in <a href="http://github.com/jhannes/sudoku-kata">my github repository</a>).

It would probably have been possible for me to code the solution faster without tests, but it probably would not have worked the first time I tried it. I also would have much less confidence in the code. Finally, I think the design imposed by the tests made my code easier to understand.
]]></description>
		<link>http://johannesbrodwall.com/2010/04/06/why-tdd-makes-a-lot-of-sense-for-sudoko/</link>
			</item>
	<item>
		<title>Why and how to use Jetty in mission-critical production</title>
		<description><![CDATA[<blockquote>This article is a summary of a seminar I had on the topic. If it seems like it's a continuation of an existing discussion that's because, to some extent, it is. If you haven't been discussing exchanging your app server, this article probably isn't very interesting to you.</blockquote>

By putting the application server inside my application instead of the other way around, I was able to leap tall buildings in a single bound.]]></description>
		<link>http://johannesbrodwall.com/2010/03/08/why-and-how-to-use-jetty-in-mission-critical-production/</link>
			</item>
	<item>
		<title>What is the right iteration length?</title>
		<description><![CDATA[When picking iteration length for an agile project, there are mainly two forces that you have to balance: The rate of learning is proportional with the number of iterations, rather than the length of the project. This means that shorter iterations help you get better faster. But each iteration has some overhead with sprint reviews, retrospectives and planning. You don't want this overhead to dominate the effort spent on the project.

For some reason, most projects I've seen with little experience in iterative development prefer three week iterations. Personally, I prefer two week iterations. Here is the breakdown:

<ul>
  <li><strong>Three week iterations</strong>: After three months, you've spent about 7% of your time on iteration meetings. You've had 4 opportunities to improve.</li>
  <li><strong>Two week iterations</strong>: After three months, you've spent about 10% of your time on iteration meetings. You've had 6 opportunities to improve.</li>
  <li><strong>One week iterations</strong>: After three months, you've spent about 20% of your time on iteration meetings. You've had 12 opportunities to improve.</li>
</ul>

Going from 93% to 90% efficiency for a 50% increase in learning seems like a good deal. Going from 90% to 80% efficiency for a 100% increase in learning, not so much.

These numbers are of course greatly simplified. You might also consider:

<ul>
  <li>With shorter iterations, the planning time may go down. But this takes practice - it doesn't happen automatically.</li>
  <li>With very short iterations, you may not have experienced enough to learn much from the retrospective. However, if you find that you do a timeline, and most of the things people remember happened the last week, it may not be because that's the only time something significant happened.</li>
  <li>You may consider different frequencies for different ceremonies. For example, on my current project we want to have demos with our power users. But they have to travel far to visit us. So we only have a full demo every other four weeks. We plan every two weeks and have an internal review and retrospective every two weeks.</li>
</ul>

What's the right iteration length for your project?]]></description>
		<link>http://johannesbrodwall.com/2010/02/25/what-is-the-right-iteration-length/</link>
			</item>
</channel>
</rss>
