Archive for September, 2010

How pair programming and test-driven development looks in real life

Pair programming and test-driven development are some of the practices that are most often talked about and least often actually understood. So I’ve decided to undertake the task to teach myself to program a simple, yet realistic problem with a pair programming partner. The goal is to create an entertaining and realistic performance that portrays what it feels like to work like this.

I’ve been extremely lucky. I’ve found not one, but two programmers that have been willing to train enough with me to make a smooth performance of a pair programming session. The result is the Java EE Spike Kata. My colleagues in Steria, Anders Karlsen and Ivar Nilsen have been great to work with. Between us, we’ve showed the kata at three conferences, and more are coming.

Here is the performance at JavaZone 2010 by Anders Karlsen and myself.

Make sure to watch the video in full screen mode as there will be LOTS of code.

Some things to watch for when you’re watching a (paired) code kata:

  • How are we working together? How often are we changing who’s at the keyboard? What will the person who’s not at the keyboard be doing?
  • How do we progress from one test to the next? When do we decide to modify an existing test and when do we decide to add a new one? How much of the test do we write before we start modifying the production code?
  • How are we writing the production code? How often are we refactoring existing code?
  • How are we using the IDE? What tricks do we use to take advantage of the code completion facilities? How often do we use the mouse and how often do we use the keyboard?

I also hope to be putting up the video of me and Ivar Nilsen at the TelecomCity conference in Sweden. As we’d both practiced even more for that performance, I think it’s even smoother than the JavaZone performance.

Enjoy!

(Kudos to Cisco Norway, formerly Tandberg, for excellent filming services at JavaZone. And thank you to Robert “Uncle Bob” Martin for turning me on to code katas in the first place)

Comments (9)

Eclipse stenography: Create code faster

According to Wikipedia, stenography or shorthand is “is an abbreviated symbolic writing method that increases speed or brevity of writing as compared to a normal method of writing a language”.

Just as a stenographer learns to take down information really fast, a good programmer can learn to write code really fast by taking advantage of his or her tools. In this post I’ll show you my secret code stenography tricks.

The basic way completion works is: You write something and press ctrl-space. Eclipse will respond with a list of option. Use the arrow keys to select an option and press enter. Eclipse will replace what you typed with the option.

Name completion

The most well-known form of completion: Start writing the name of a class or method, press-ctrl space, and Eclipse will complete the typing for you.

Complete method name

Notice the blue squares around “expected” and “actual”. These are “fields”. Use tab to jump between them.

New to many is the fact that Eclipse support CamelCase syntax when completing names:

Class name completion

When I type “TPH”, Eclipse finds to classes with names that match (“TaggedProfileHelper” and “TaggedProfileHolder”) and offers them to me as a list. When I type “TPHe”, Eclipse finds only one class that matches (“TaggedProfileHelper”), and expands it automatically.

Pro tip: Remove classes you don’t use but that often make completion ambiguous by going to Window->Preferences and adding the classes to Java->Appearance->Type filter.

Expand code template

Another fairly basic one: Type the name of a template (from Window->Preferences, Java->Editor->Templates) and press ctrl-space to insert it.

Expand code template

Notice that Eclipse brings up the type list on the first ctrl-space. Press ctrl-space again to show Template completions.

Pro tip: Edit and remove editor templates to make your coding faster. You may also add template, but I seldom find this as useful.

Override method

Now, we’re getting into new territory for most of you: In the class body, type the name of a method in the super class and press ctrl-space:

Override method

Eclipse offers to override the method for you. Notice that CamelCase completion works here.

Generate getters and setters

Yet another little know super trick: In the class body of a class with a field “name”, type “getN” and press ctrl-space to generate a getter, or “setN” to generate a setter:

Generate getter and setter

Notice that CamelCasing doesn’t work perfectly here; you can’t type “gNa” to create a getter for “name”.

(It’s possible to generate getters and setters for a large number of fields with “Source->Generate getters and setters”. But when I only add one new field, I find code completion to be a faster option)

As I’ve been working on code katas, I’ve found many ways to speed up my programming in my IDE of choice. I hope Eclipse stenography or code completion can help you write code faster as well.

Comments (1)

How to measure quality

Everyone has heard horror stories about pointy-haired-bosses counting lines of source code to track the progress of a project. We roll our eyes and laugh at their stupidity. But before you laugh too much, you might want to find out whether you’re really any better.

Most of what software projects measure are not things we care about. Not things we really care about. Do you really care about the number of coding standard violations in your code? About compiler warnings? About test coverage? About cyclomatic complexity?

No, you only care about the presumed effects of these metrics. What we really care about are things like how much it costs to change the system, and how likely we are to introduce bugs. Hopefully, most of the numbers you collect with your fancy tool will help predict what your really care about. But tools can be fooled.

You can have 100% test coverage without asserting a single thing about your code. And you can have maintainable, bug free code with super-high cyclomatic complexity and not a single line of code comments.

A leading indicator is a term investors use for measurements that change before the economy changes. A company’s stock prices is an example of leading indicators.

On the other hand, the term lagging indicator is used for a measurement that change after the economic reality changes. A company’s reported earning is an example of a lagging indicator.

Leading indicators, like stock prices and code coverage are useful because you can get hold of them early. However, they are not the real thing we’re after. The real thing we’re after is often a lagging indicator, like reported earnings or code maintainability.

Use leading indicators on your project to identify trouble areas early. But don’t be religious about them. Use lagging indicators to prove that you’ve met your commitment to quality. If your commitment to quality is reduced to code coverage, cyclomatic complexity or compiler warnings, you’re no better than the pointy-haired-boss counting lines of code.

Comments (7)

Cleaning up the release process

How many steps do you need to perform to release a new version of your software? Do you even know?

Releasing frequently requires the release process to be as streamlined as possible. A good way to get started is to write a step-by-step instruction that explicitly state everything that needs be performed for an installation or upgrade.

Then get to work cleaning it up:

  • Make sure you can build and package everything that’s needed for the installation with a single command
  • Does the installation or upgrade require several files to be installed? Bundle them in a ZIP-file, MSI-installer or RPM. If you’re installing to an application server, find out how to script deployment to your particular server.
  • Does the installation require files to be copied, moved, linked or otherwise managed? Provide a script that automatically executes the necessary actions
  • Are there dependencies that you assume are already installed in the environment? State these explicitly, or even better, internalize them, to reduce variation
  • Does changing the database schema require a separate step? Why not bundle database changes in the application and execute them as part of the startup-procedure? (see my blogpost on database migrations)

Most projects can get down to upgrading the system safely by executing a single command. But as long as upgrading is a manual and complicated procedure, it is risky to release frequently. Clean up your release process to make it foolproof.

Comments (2)

Agile Release Pattern: Support multiple versions

No computer system is an island. At least not these days. This creates a challenge when you want to change APIs: Do you want to upgrade all affected systems at once, or do you want to support multiple versions?

Both approaches have downsides. Coordinating several upgrades increase the risk that one of the upgrades fail, in which case, you may want to roll everything back. In some situations, rollback may not even be possible. So for everything except the simplest cases, a distributed system must expect to support multiple versions of APIs.

For high-risk upgrades within a system, it may still be useful to support multiple version. A common example is database changes. Many database changes are impossible to rollback once they are done. If these changes are made at the same time as a new version is put into production, the result may be disastrous: Imagine that the new version contains some serious defect that makes it unusable. If the accompanying database change cannot be rolled back, the consequence of this defect will be severe indeed.

A special case of multiple versions of an interface is that of a backwards compatible interface. In SQL, adding a new table or adding non-required columns to an existing table is backwards compatible. With web services, adding a new element may be backwards compatible, depending on the XSD that defines the web service. In Java, adding a method to an existing class is backwards compatible.

Backwards compatible changes require very little coordination, especially if clients can stay with the old changes if they don’t care about the new features.

When your changes are not backwards compatible, having to support clients on multiple versions will add to the maintenance cost of the system. You will want to support as few versions as possible and give clients incentives to upgrade to a new version. Incentives can be anything from asking nicely, to providing a cut-off date (if you can live without your clients) or hike up the service fee (if you have one).

The long and short of it is that you want to avoid interfaces where you have to manage the upgrade and versioning. This is why SOA projects that split up the systems in too fine grained services suddenly discover the need for a bureaucratic service governance effort. Once you’ve published an interface, you’re stuck with it forever.

Comments

Husk å melde deg på til Smidig 2010

Smidig 2010: Norges største smidige konferanse

Har du fått med deg at Smidig 2010 arrangeres 16.-17. November på Radisson BLU, Holberg plass i Oslo? Konferansen har åpnet for foredrag. Early bird prisen på billetter varer KUN TIL TORSDAG, så det gjelder å bestemme seg fort!

Opplev vårt unike format, med over 70 lyntaler og 100 diskusjonsgrupper! Smidigkonferansen er den årlige nasjonale hendelsen som samler mennesker fra alle typer roller i IT-bransjen; prosjektledere, produkteiere, utviklere, bedriftsledere og testere og flere.

Comments

Dynamic subclass APIs make Java seem young again

At JavaZone 2010 I will be giving a lightning talk on APIs that use dynamic subclasses. These APIs make it possible to do things in Java that seem like pure magic. Here are some ideas of what you can get from these APIs and a look under the hood, so you really understand what’s going on.

Mockito – “the best Java library named after a summery drink”

Mocking is a hotly debated subject within testing camps. I think a lot of the arguments come against mocking (beyond overuse…) come down to the fact that the last generation of mocking libraries left a lot to be desired. A desire that has been fulfilled by Mockito.

I will leave the examples at a minimum, but here’s an example from code I actually use a lot. Mocking the servlet API:

HttpServletRequest req = mock(HttpServletRequest.class);
HttpServletResponse resp = mock(HttpServletResponse.class);
 
when(req.getMethod()).thenReturn("GET");
 
servletUnderTest.service(req, resp);
 
verify(resp).setContentType("text/html");

For those of you who’ve struggled with jMock or it’s ilk, you may notice the concise syntax used to specify behavior (“when(…).thenReturn()”) and the fact that you don’t have to set up expectations before exercising the code.

LambdaJ – “making Java beautiful and incomprehensible”

LambdaJ pretty much starts with the realizations that waiting for new language features like closures may prove to be a very long wait indeed. Furthermore, anonymous inner classes may technically be used to simulate closures, but they are so ugly that not even a mother could love them.

Let’s look at an example: Finding the name of every member of the Simpsons family who’s older than 8, in order of age.

In plain Java:

List[Person] tmp = new ArrayList[Person]();
for (Person p : theSimpsons) {
  if (p.getAge() > 8) tmp.add(p);
}
Collections.sort(tmp, new Comparator() {
  public int compare(Person a, Person b) {
     return a.getAge() - b.getAge();
  }
});
List[String] names = new ArrayList[String]();
for (Person p : tmp) {
  names.add(p.getName());
}
System.out.println(names);

More modern languages are way ahead of Java. Here’s Ruby:

puts theSimpsons.select { |p| p.age > 8 }.sort_by(&:age).
     collect { |p| p.name }

The rather simple request is simple to express. Yay!

Java 7 may make this better.

System.out.println(theSimpsons.select(#(Person p) {
  return p.getAge() > 8;
}).sort_by(#(Person p) {
  return p.getAge();
}).collect(#(Person p) {
  return p.getName();
});

All in all, not a bad contender. What about LambdaJ:

System.out.println(with(theSimpsons)
    .retain(having(on(Person.class).getAge(), gt(8)))
    .sort(on(Person.class).getAge())
    .extract(on(Person.class).getName()));

It’s not perfect, but it has one thing to recommend it: This works with standard, everyday Java, today! There’s no special compilation crap. No other language. Nothin. Just plain old Java.

(Thanks to Mario Fusco for improving the example with LamdbaJ 2.3 syntax)

What’s going on

Both LambdaJ and Mockito use a tricky sleight-of-hand: Generate a dynamic subclass which records its method invocations for later use. Creating such an API is in principle easy, although my implementation leave a lot! of details unimplemented.

So: We want invocations on the generated class to be template/mock objects to be saved:

@Test
public void shouldRecordInvocations() throws Exception {
    Person template = DynSubclassDemo.on(Person.class);
    assertNull(DynSubclassDemo.lastMethodCall());
    template.getAge();
    assertEquals("getAge",
        DynSubclassDemo.lastMethodCall().getName());
}

This magic is surprisingly easy to implement with CgLib:

@SuppressWarnings("unchecked")
public static Object on(Class templateClass) {
    return Enhancer.create(templateClass, new InvocationHandler() {
        @Override
        public Object invoke(Object o, Method m, Object[] a) {
            lastMethodCall = m;
            return null;
        }
    });
}

This creates a subclass of the template class (in this case: Person). But every method in Person is overridden by calling the invocation handler. Which simply saves what method was called.

This makes it possible to implement a prototype of LambdaJ’s collect method:

@Test
public void shouldCallRecordedMethodForEveryObject() {
    names = DynSubclassDemo.collect(theSimpsons, 
            DynSubclassDemo.on(Person.class).getName());
    assertEquals(names, asList("Lisa", "Homer", "Marge", ...));
}
 
public static[T ,U] List[U] collect(Iterable[T] c, U property) {
    ArrayList[U] result = new ArrayList[U]();
    for (T t : c) {
        // An insane amount of exception handling removed
        result.add((U) lastMethodCall.invoke(t));
    }
    return result;
}

This is a gross simplification. LambdaJ handles many aspects that my prototype simple ignores: Chained method calls, multithreading, method arguments, and multiple invocations on the same line.

The future of Java

Maybe the future of Java isn’t extending the language. Maybe the future of Java isn’t replacing the language with Scala. Maybe the future of Java is smart people discovering smarter ways of using what’s already there. My hat off to Mario Fusco, Szczepan Faber and everybody who’s contributed to LambdaJ or Mockito.

Come to my talk at JavaZone to learn more!

Comments (5)

Agile Release Pattern: Database migrations

As I release more frequently, I start to focus on automating the actual process of deploying a release. One of the most powerful steps of automating deployment is to automatically upgrade the database schema.

This technique first saw mainstream use with the Ruby-on-Rails framework. Today, there are several mature tools that will help you organize and execute database changes (Scala Migrations, Ruby-on-Rails Migrations, dbdeploy, Liquibase, dbmaintain). And if none fit you perfectly, it’s easy to create your own.

In my current project, we have rolled our own solutions for this:

  • All changes to the database are stored as SQL-files that are packaged into the deployment unit (in our case, a WAR-file). These files will usually contain statements like “ALTER TABLE ADD COLUMN” and “CREATE TABLE”. To get the files executed in the right order, we name the files with an increasing sequence number, like 012-add_payment_type_to_customer.sql.
  • Whenever the application is started, it looks for a table named “MIGRATIONS” in the database and creates it if it doesn’t exist.
  • At startup, the application looks through the list of migration files it has been packaged with and sees which file names don’t have an entry in the MIGRATIONS table.
  • The application executes all the scripts that haven’t been executed already. If any script fails to execute, it makes a note of the error in the MIGRATIONS table and refuses to start the application

We run the migration procedure every time we start up the application, whether it is in test or production. Even the JUnit tests that access the database will run any pending migrations before starting. The result is that any database change that we intent to roll out into production will at the very least be executed once on each developers private copy of the database, as well as once on the continuous integration server. By the time they get executed in a controlled testing environment, we’re pretty confident that they work as intended.

Some migration tools use a more friendly (and portable) syntax than SQL DDL statements. Many allow for rollback of migrations. Most don’t automatically execute the pending migrations on application start, but require a separate command to execute them.

Your first step towards automating database migrations is to make sure that every change in the database is represented by some sort of script and that all these scripts are versioned with the rest of your code. From there, you can improve your process when you notice a step in the process that seems to involve too much work or risk.

Automating the deployment process will reduce the need for documentation and the opportunity for errors during one of the most critical times in the project. It is especially important to reduce the possibility of miscommunication and mistyping if the people responsible for deployment are in a separate organizational unit, which often seems to be the case. Make their job as easy as possible!

Comments (8)

Creative Commons Attribution 3.0 Unported
This work is licensed under a Creative Commons Attribution 3.0 Unported.