The reuse dilemma

The first commandment that any young programmer learns is “Thou Shall Not Duplicate”. Thus instructed, whenever we see something that looks like it may be repeated code, we refactor. We create libraries and frameworks. But removing duplication doesn’t come for free.

If I refactor some code so that instead of duplicating some logic in Class A and Class B, these classes share the logic in Class R (for reuse!). Now Classes A and B are indirectly coupled. This is not necessarily a bad thing, but it comes with some consequences, which are often overlooked.

If Class A require the shared functionality to change, we have a choice: We make the change or we stymie Class A. Making the change comes at a cost of breaking Class B. If these classes are in the same Java package (or .NET Namespace), chances are that we will be able to verify that the change didn’t break anything. If the reused functionality is in a library that is used by another library that is used by a Class B, verifying that the change was good is harder.

This is where Continuous Integration comes in. We check in our change to class R (for reuse, remember). A build is triggered on Jenkins, Bamboo, TFS or what have you. The build causes other projects to get built and eventually, we get a failing build where a unit test for Class B breaks. If we did our job right.

Even with this failing test, we’re not out of the woods. A build system for a large enterprise may take up to an hour or even more to run. This means that by trying to improve Class R, the developers of Class A have made a mess for the developers of Class B at least for a good while.

When organizations see repeated build breaks due to changes in dependencies, the reaction is usually the same: Lets limit changes to the shared code. Perhaps we’ll even version it, so that any change that you need will be released in the next version and dependencies can upgrade at their own pace.

Now we’ve introduced another problem: We are no longer able to change the code. Developers of Class A will have to take Class R as it is, or at the very least go through substantial work to change it. If the reused code is very mature, this is fine. After all, this is what we have to deal with when it comes to language standard libraries and open source projects.

Most reused code inside an organization, however, isn’t so mature. The developers of Class A will often be frustrated by the limitations of Class R. If the reused class is something very domain specific, the limitation is even worse.

So the developers do what any reasonable developer would do: They make a copy of Class R (or they fork the repository where it lives), creating a new duplication of the code.

And so it goes: Removing duplication leads to risk of adverse interactions between reusers. Organizations enforce change control on the reused code to avoid changes made in one context from breaking other parts of the system, resulting in paralysis for the reusers who need the reused code to change. The reusers eventually duplicate the reused code to their own branch where they can evolve it, leading to duplication. Until someone gets the urge to remove the duplication.

The dilemma happens in the small and in the large, but the trade-offs are different.

What we should do less is to reuse immature code with little novel value. The cost of the extra coupling often far outweighs the benefit of reuse.

Posted in English, Software Development | Leave a comment

Dead simple configuration

Whole frameworks have been written with the purpose of handling the configuration of your application. I prefer a simpler way.

If by configuration we mean “everything that is likely to vary between deploys“, it follows that we should try and keep configuration simple. In Java, the simplest option is the humble properties file. The downside of a properties file is that you have to restart your application when you want it to pick up changes. Or do you?

Here’s a simple method I’ve used on several projects:

The AppConfiguration class looks like this:

This reads the configuration file in an efficient way and updates the settings as needed. It supports environment variables and system properties as defaults. And it even gives a pretty good log of what’s going on.

For the full source code and a magic DataSource which updates automatically, see this gist: https://gist.github.com/jhannes/b8b143e0e5b287d73038

Enjoy!

Posted in Code, English, Java | Leave a comment

Turning a new corner

This summer has been the most significant in my life. While I don’t often write about personal matters in this blog, I will make an exception.

On July 12th my son Leon was born. Healthy, happy and obviously, from the perspective a father, perfect.

Going through this fundamental life change has made me decide to re-evaluate a lot of things. Most importantly, I wanted to evaluate my career.

In the last few years, I have spread my focus thin. I have enjoyed working with developers in Norway, Sweden, Sri Lanka and elsewhere tremendously. I have enjoyed working with many different customers in many sectors. I’ve enjoyed new knowledge in C#, Android, as well as continuing to deepen my knowledge in JavaScript and Java. I’m grateful to the whole team at Exilesoft for giving me the opportunities to learn so much.

As my personal life gets more focused, I also want my professional life to be more focused.

As of August, I start a position as Director of Software Development at BrandMaster; a small, but solid Norwegian product company which develops a Marketing Resource Management (MRM) portal in Java and JavaScript.

For years, I have fluttered around to various companies as a consultant, opining and hopefully helped them develop their products. Now I want to be part of building a product where I can feel ownership and long-term commitment.

BrandMaster is a small company that has the energetic feel of a startup, but the established product and customer base of a mature product. The field of MRM is moving from one of a specialized tool to something that more and more companies find themselves investing in, as witnessed by the entry of Microsoft, SAP, IBM and more large players into the field.

I find that the experiences that I have gathered in BBS, Steria and Exilesoft provide a good basis for contributing to BrandMaster. This means that I will focus on a more narrow set of problems in the future: Moving a product towards greater more frequent releases, more effective use of test automation and agile planning and tracking. It means that I will be able to focus more narrowly on my favorite technologies of Java and JavaScript. And it means that I will be able to work more closely together with a great team of developers and creative people.

This does mean that I will not stick my neck out quite as much when it comes to large projects and organizations, C# and mobile development (although there will be some mobility in the future) in this blog.

Stay tuned: Queued up on my plate are instead super-simple project tracking, demo-driven development, effective unit tests, continuous delivery, 12factor Java 8 and product development.

I’m looking forward to a set of challenging problems at the same time as I get the peace of mind to enjoy the company of my son and his lovely, beautiful and sparkling mother.

Posted in English, Personal | Leave a comment

The lepidopterist’s curse: Playing with java.time

Pop quiz: What will be the output of this little program?

The answer is, like with most interesting questions, “it depends”. How can it depend? Well, let try a few examples:

  • getHoursOfDay(LocalDate.of(2014, 7, 15), ZoneId.of("Asia/Colombo")) returns 24. As expected
  • getHoursOfDay(LocalDate.of(2014, 7, 15), ZoneId.of("Europe/Oslo")) also returns 24.
  • But here comes a funny version: getHoursOfDay(LocalDate.of(2014, 3, 30), ZoneId.of("Europe/Oslo")) returns 23! This is daylight saving time.
  • Similarly: getHoursOfDay(LocalDate.of(2014, 10, 26), ZoneId.of("Europe/Oslo")) also returns 25
  • And of course, down under, everything is upside down: getHoursOfDay(LocalDate.of(2014, 10, 5), ZoneId.of("Australia/Melbourne")) gives 23.
  • Except, of course, in Queensland: getHoursOfDay(LocalDate.of(2014, 10, 5), ZoneId.of("Australia/Queensland")) => 24.

Daylight saving hours: The bane of programmers!

Daylight saving hours were instituted with the stated purpose of improving worker productivity by providing more working hours with light. Numerous studies have failed to prove that it works as intended.

Instead, when I examined the history of daylight saving hours in Norway, it turns out that it was lobbied by a golfer and a butterfly collector (“lepidopterist”) so that they could better pursue their hobbies after working hours. Thus the name of this blog post.

Most of the time, you can ignore daylight saving hours. But when you can’t, it can really bite you in the behind. For example: What does the hour by hour production of a power plan look like on the day that changes from daylight saving hours to standard time? Another example given to me by a colleague: TV schedules. It turns out that some TV channels just can’t be bothered to show programming during the extra hour in the fall. Or they will show the same hour of programming twice.

The Joda-Time API and now, the Java 8 time API java.time can help. If you use it correctly. Here is the code to display a table of values per hour:

Given 2014/10/26 and Oslo, this prints:

And on 2014/3/30, it prints:

So, if you ever find yourself writing code like this: for (int hour=0; hour<24; hour++) doSomething(midnight.plusHours(hour)); you may want to reconsider! This code will (probably) break twice a year.

At the face of it, time is an easy concept. When you start looking into the details, there's a reason that the java.time library contains 20 classes (if you don't count the subpackages). When used correctly, time calculations are simple. When used incorrectly, time calculations look simple, but contain subtle bugs.

Next time, perhaps I should ruminate on the finer points of Week Numbers.

Posted in Code, English, Java | Leave a comment

C# tricks: Securing your controllers

This article is dedicated to the ExileOffice team – revolutionizing the way we run our business in Exilesoft.

As applications move more and more of their business logic to the client side, it falls upon us to simplify what’s left on the server to avoid distractions. My previous article shows how I get rid of lot of the boring and noisy code.

There is one thing that the server will always be responsible for: Security. Much functionality can be moved to the client side, but we have no way of stopping users from hacking our client-side code or faking requests.

In this article, I extend upon the generic controllers to handle the one of the most important aspects of security: Authorization.

This is not your data!

The first order of business is to ensure that only logged in users can access our controller. This is trivially done with AuthorizationAttribute:

Here, our custom AuthorizationAttribute sets a property by looking for the Company of the currently logged in user in the database. You probably want to cache this in a session.

The second problem is to ensure that the user can only read and write data.

According to my previous article, we want to move as much as possible of the boilerplate logic into a generic EntityController. Here we add authorization:

Here, we give the controllers a virtual AccessFilter property. Whenever we read data, we need to run it through the access filter, and whenever we write data, we need to check that the data matches the filter.

It’s up to each entity controller to determine how to match the data. Here is an example for PersonData:

The clue here is to ensure that PersonController can only get at Persons through the Entities collection. This collection is always filtered with the Company that the user has access to.

The code will resolve to simply: db.Persons.Where(p => p.Company == Request.Properties["UserCompany"]).Where(p => p.Type == PersonType.Admin && p.City == city).Select(p => new { p.FirstName, p.LastName }). By using the filtered Entities, we can ensure that we never accidentally forget to apply the users company.

Posted in C#, Code, English | Leave a comment

The madness of layered architecture

I once visited a team that had fifteen layers in their code. That is: If you wanted to display some data in the database in a web page, that data passed through 15 classes in the application. What did these layers do? Oh, nothing much. They just copied data from one object to the next. Or sometimes the “access object layer” would perform a check that objects were valid. Or perhaps the check would be done in the “boundary object layer”. It varied, depending on which part of the application you looked.

Puzzled (and somewhat annoyed), I asked the team why they had constructed their application this way. The answer was simple enough: They had been told so by the expensive consultant who had been hired to advice on the organization’s architecture.

I asked the team what rationale the consultant had given. They just shrugged. Who knows?

Today, I often visit teams who have three to five layers in their code. When asked why, the response is usually the same: This is the advice they have been given. From a book, a video or a conference talk. And the rationale remains elusive or muddled at best.

Why do we construct layered applications?

There’s an ancient saying in the field of computing: Any problem in computer science can be solved by adding a layer of indirection.

Famously, this is the guiding principle behind our modern network stack. In web services SOAP performs method calls on top of HTTP. HTTP sends requests and receives responses on top of TCP. TCP streams data in two directions on top of IP. IP routes packets of bits through a network on top of physical protocols like Ethernet. Ethernet broadcasts packets of bits with a destination address to all computers on a bus.

Each layer performs a function that lets the higher layer abstract away the complexities of for example resending lost packets or routing packets through a globally interconnected network.

The analogy is used to argue for layers in enterprise application architecture.

But enterprise applications are not like network protocols. Every layer in most enterprise application operates at the same level of abstraction.

To pick on a popular example: John Papa’s video on Single Page Applications uses the following layers on the server side (and a separate set on the client side): Controllers, UnitOfWork, Repository, Factories and EntityFramework. So for example the AttendanceRepository property in CodeCamperUnitOfWork returns a AttendanceRepository to the AttendanceController, which calls GetBySessionId() method in AttendanceRepository layer, which finally calls DbSet.Where(ps => ps.SessionId == sessionId) on EntityFramework. And then there’s the RepositoryFactories layers. Whee!

And what does it all do? It filters an entity based on a parameter. Wat?!

(A hint that this is going off the rails is that discussion in the video presentation starts with the bottom and builds up to the controllers instead of outside in)

In a similar Java application, I have seen – and feel free to skip these tedious details – the SpeakersController.findByConference calls SpeakersService.findByConference, which calls SpeakersManager.findByConference, which calls SpeakersRepository.findByConference, which constructs a horrific JPAQL query which nobody can understand. JPA returns an @Entity which is mapped to the database, and the Repository, or perhaps the Manager, Service or Controller, or perhaps two or three of these, will transform from Speaker-class to another.

Why is this a problem?

The cost of code: A reasonable conjecture would be that the cost of developing and maintaining an application grows with the size of the application. Adding code without value is waste.

Single responsibility principle: In the above example, the SpeakerService will often contain all functionality associated with speakers. So if adding a speaker requires you to select a conference from a drop-down list, the SpeakerService will often have a findAllConferences method, so that SpeakersController doesn’t need to also have a dependency on ConferenceService. However, this makes the classes into functionality magnets. The symptom is low coherence: the methods of one class can be divided into distinct sets that are never used at the same time.

Dumb services: “Service” is a horrible name for a class – a service is a more or less coherent collection of functions. A more meaningful name would be a “repository” for a service that stores and retrieves objects, a Query is a service that selects objects based on a criteria (actually it’s a command, not a service), a Gateway is a service that communicates with another system, a ReportGenerator is a service that creates a report. Of course, the fact that a controller may have references to a repository, a report generator and a gateway should be quite normal if the controller fetches data from the database to generate a report for another system.

Multiple points of extension: If you have a controller that calls a service that calls a manager that calls a repository and you want to add some validation that the object you are saving is consistent, where would you add it? How much would you be willing to bet that the other developers on the team would give the same answer? How much would you be willing to bet that you would give the same answer in a few months?

Catering to the least common denominator: In the conference application we have been playing with, DaysController creates and returns the days available for a conference. The functionality needed for DaysController is dead simple. On the other hand TalksController has a lot more functionality. Even though these controllers have vastly different needs, they both get the same (boring) set of classes: A Controller, a UnitOfWork, a Repository. There is no reason the DaysController couldn’t use EntityFramework directly, other than the desire for consistency.

Most applications have a few functional verticals that contain the meat of the application and a lot of small supporting verticals. Treating them the same only creates more work and more maintenance effort.

So how can you fix it?

The first thing you must do is to build your application from the outside in. If your job is to return a set of objects, with .NET EntityFramework you can access the DbSet directly – just inject IDbSet in your controller. With Java JPA, you probably want a Repository with a finder method to hide the JPAQL madness. No service, manager, worker, or whatever is needed.

The second thing you must do is to grow your architecture. When you realize that there’s more responsibilities in your controller than deciding what to do with a user request, you must extract new classes. You may for example need a PdfScheduleGenerator to create a printable schedule for your conference. If you’re using .NET entity framework, you many want to create some LINQ extension methods on e.g. IEnumerable (which is extended by IDbSet)

The third and most important thing you must do is to give your classes names that reflect their responsibilities. A service should not just be a place to dump a lot of methods.

Every problem in computer science can be solved by adding a layer of indirection, but most problems in software engineering can be solved by removing a misplaced layer.

Let’s build leaner applications!

Posted in C#, English, Java, Software Development | Tagged | 16 Comments

A canonical web test in NodeJS

Working with web applications in NodeJS is great. Using the same language and libraries on the client and server simplified the thinking. And NodeJS has fast tests and restart for a super quick edit-verify cycle when you’re coding.

I like to write tests to verify the server-side and client-side logic, but do you know that the whole solution really is working? You can of course test your service manually after deploying, but that becomes tedious. By using Selenium, we can test that the solution works end-to-end.

In this article, I show how to write a Mocha test that:

  1. Starts up the Selenium test runner
  2. Creates a browser client that can access the application (using PhantomJS)
  3. Starts up the server and sends the browser to the server (running in ExpressJS)
  4. Clicks a button in the web page that triggers some JavaScript (written in jQuery in this example)
  5. Verifies that the call to the server returns and displays correct data

This test can be run with a simple command after you check out the code and requires no setup on the developers computer!

This test starts a new server and client for each test. When using this for real, you really want to make sure you only start the server and the web driver once as these are expensive operations.

The test case of “it clicks menu item” has a lot of callbacks. There are versions of webdriver APIs for NodeJS which are based on promises that you may enjoy using more.

Implementation details

The full application can be found on Github. To implement it, I used the following NPM modules:

  • phantomjs
  • selenium-standalone
  • webdriverjs
  • expressjs (of course)
  • mocha and chai (of course)

Starting the server looks like this:

And the app.js file looks like this:

I have a server.js file which starts up the express application to run normal (outside the tests).

A simplified version of starting Selenium and WebDriver looks like this:

I found that Selenium and PhantomJS has some issues, at least on Windows, so my final code needed a few hacks to work. See the full details at github.

Conclusion

A web end to end integration test in NodeJS requires a little bit of shaking your fist at the heavens to get to work the first time, not in the least due to the need to work around limitations in Selenium and PhantomJS. But once you got it up and running, you can easily test not only that your logic works, but that your whole application works together.

When making these tests, I allowed for a little flexibility as well: By setting environment variables, the same tests can be run with a manually deployed server, so you can use it to verify that your staging server is up and running (for example). And of course, you can replace PhantomJS as a web browser with Firefox or Chrome and see the tests run for real in your browser.

Automate the end-to-end tests of your NodeJS applications!

Posted in Code, English, JavaScript, Software Development | Leave a comment

Pair programming with Sankalpa

One of my favorite ways to develop software is to do it together with others. Pair programming has always been a motivating and fun activity for me, but some pairings work better than others.

When our team was formed we decided to pair program and rotate partners every day. I had lots of fun programming with Milina, Asanka, Manoj and Chamath, but my favorite session was the one I had with Sankalpa. In this session, we achieved something that we would be helpless to do alone.

The task Sankalpa and I was working on was to include information from a calendar in Confluence in a date picker in a web application. As we sat down, I was dreading the integration part. Integration is often dreadful. More than anything, I wanted to hide from the problem. But with Sankalpa sitting next to me, I didn’t feel that I could give up, so I suggested we took a look at our Confluence calendar to get started. Sankalpa was at the keyboard and he found a “calendar feed” where I hadn’t thought of looking for it.

Looking at the feed, I exclaimed “Oh, I know this – this is vcalendar”. A quick Google later and we had found a library for parsing vCalendar in JavaScript. We quickly finished the code to adapt it to our desired format and moved to the date picker.

I was at the keyboard and I had used a date picker library called jQuery datePicker before. We quickly integrated it and I proudly refreshed the page to show the calendar events in the view! But it turned out that all the functionality that depended on picking a date was now broken.

I started grumbling about how I was much more comfortable with jQuery than with AngularJs. Unfazed, Sankalpa mentioned that Manoj had gotten the date picker to work with AngularJs in an earlier project. “Hey, Manoj, where did we use this date picker?”

Having much more experience with AngularJs than me, Sankalpa integrated the code into our code base and everything was working.

All in all, I had expected this to take 2-3 times as much effort. If we had been alone, we would probably wouldn’t have the courage to start with the integration right away. If he had been alone, Sankalpa probably wouldn’t had known how to parse the vCal feed. If I had been alone, I probably would have searched the Internet for hours to find out how to make AngularJs play well with jQuery date picker.

Together, we did what neither of us could have done alone. (At least not anywhere close to this quick)

Posted in English, Extreme Programming, Non-technical, Pair programming, Software Development | 1 Comment

The economics of reuse

If need the same functionality in two projects, you should reuse code between them, right? Or should you? For as long as there has been a profession of software engineering, we have tried to achieve more reuse. But reuse has both a benefit and a cost. Too often, the cost is forgotten. In this article, I examine the economics of reuse.

True story: One of the earliest projects to embrace object-oriented programming in the 1990s did so with the goal of maximizing reuse. The team responsible for creating the company wide framework used the following formula for calculating the value of their work:

[Value of reuse] = [numbers of uses of framework] * [value of the framework to reusers] – [cost of developing the framework]

This formula is obviously correct, but this is where they went horribly wrong: The organization said [value of framework to reusers] = [cost of developing framework]. In other words: The more expensive it was to create, the more valuable it was to use.

We have clearly progressed beyond this thinking. A more updated formula would say: [value of framework to reusers] = [cost of developing the feature in question]. But even this is too optimistic.

No library comes for free to its users. At the very least, you have to discover the features and learn about the details. The cost of reusing depends on many factors, such as the quality of the framework and the documentation and also upon the type of feature. A complex algorithm with a simple interface is cheap to use, while most domain-specific frameworks require relatively much work to reuse. We can express this as a reuse value factor, likely between 90% and 50%. For most cases, my guess would be at about 75%.

So we have:

[value of reuse] = [number of users] * ([cost of feature] * [reuse value factor]) – [cost of developing the reusable component]

What about the other important factor: [cost of developing the reusable component]?

It’s easy to assume that the cost of developing a feature in a framework is equal to that of developing the feature in an application, but on further analysis shows that this is far from true. A reusable component needs more documentation, it needs to handle more special cases and it has a slower feedback cycle. This cost is actually substantial and may mean that it costs between 150% to 300% or more to develop a feature for reuse. Personally, I think the reusability cost factor lies around 300%. And the lower this number, the higher the cost factor of reuse is likely to be, because that may mean we skimped on documentation etc.

A revised number would be:

[value of reuse] = [number of users] * ([cost of feature] * [reuse value factor]) – [reusability cost factor] * [cost of feature]

Or

[value of reuse] = [cost of feature] * ([number of users] * [reuse value factor] – [reusability cost factor])

The more complex formula actually lets us make a few predictions. Let’s say we assume a reuse value factor of 75 % (meaning that it requires 1/4 of the effort to reuse a library rather than creating the feature from scratch) and a reusability cost factor of 300 % (meaning that it requires three times the effort to create something that’s worth reusing). This means:

[value of reuse] = [cost of feature] * ([number of users] * 75% – 300%)

This equation breaks even when [number of users] = 4. That means that to get any value from your reused component, you better have five or more reusers or you have to find a way to substantially improve the [reuse value factor] or [reusability cost factor]. Very smart people have failed to do this.

Improving the value:

  • Increase the number of reusers: Simple enough, but when you do, you risk that the [reuse value factor] goes down as the framework doesn’t suit everybody equally well.
  • Reduce the cost of reusing the library: This means investing in documentation, improving your design, improving testing to reduce the number of bugs, handle bug reports and feature requests faster from your reusers – all of which increase your cost reusability cost factor.
  • Reduce the extra work in making the library reusable: The most important way to reduce the cost of developing for reuse is to choose the right kind of problem to solve. Problems with a small surface and big volume are best. That means: Easy to describe, hard to implement. Sadly, most of the juiciest fruit was picked years ago by the standard library in your programming language and by open source frameworks.

On a global scale, reuse has saved the software industry tremendous amounts. In an organization, it can be hard to get the same effect. Reuse comes at a cost to the reuser and to the developer of the reusable library. How do you evaluate and improve your [reuse value factor] and your [reusability cost factor]?

Posted in English, Software Development | 17 Comments

Estimation by stuffing things into boxes

I’ve started using an approach for software project estimation that so far is proving to be fairly transparent, quick and reliable. I’ve observed that within a reasonable degree of variation, most teams seems to complete about one “user-relevant task” per developer per calendar week.

There are so many theoretical holes in my argument that there’s no point trying to cover them all. The funny thing is that it seems to work fairly well in practice. And to the degree that it’s dirty, it’s at least quick and dirty. The only thing I will address is that one of these “user relevant tasks” is smaller than a typical application feature.

Most importantly: Most teams never get it right on the first try. Or they spend too long gold-plating everything. Or both.

This article shows an example of estimating a fictive project: The Temporary Staffing System.

The high-level scope

Let’s say that our organization has come up with the following vision:

For a temporary employment agent who wants to match candidates
to client needs, the Temporary Staffing System is an
interactive web application, which lets them register and
match candidates and positions. Unlike competing systems
this lets us share selective information with our clients.

We come up with the following flow through the application:

  1. A new company wants to hire a skilled worker for a temporary position
  2. Administrative user adds the client details to the system
  3. Administrative user adds client logins to the system
    (perhaps we also should let the clients log in with LinkedIn etc?)
  4. Client logs into the application and completes new position
    description, including skill requirements
  5. Temp agency adds a worker to the system
  6. Temp agency proposes the worker to a position registered by a client
    (in the future, the worker may register themselves!)
  7. Client gets notified of new proposals (via email)
  8. Client views status of all open positions in the system
  9. External to the system: Client interviews candidate, request further
    information and makes a decision whether to hire or not
  10. Client accepts or rejects the worker in the system
  11. As worker performs work, they register their time in the system
  12. At the end of a billing period, the system generates billing information
    to accounting system
  13. At the end of a salary period, the system generates salary information
    to the accounting system

Some of these steps may be one user story, some may be many.

The top of the backlog

We choose some of the most central parts of the scope to create the beginning of the backlog. In order to accommodate for the learning as we go along, the first draft of our backlog may look like this:

  1. Experimental create open position
  2. Experimental list positions
  3. Simplified create open position
  4. Simplified list positions
  5. Complete create open positions
  6. Complete list positions

An “experimental” version of a story is a functionality trivial version that touches all parts of the technology. In the case of these two stories, perhaps we have the application leave the logged in client as a hard coded variable. The story may only include writing some of the fields of the positions, maybe only title and description.

The Simplified version may add more complex properties, such as skills from a skill list or it may add filters to the list.

The complete version should be something we’re prepared to put in front of real users.

By revisiting a feature like this, we have the chance to get the feedback to create a good feature without gold-plating.

Continuing the backlog

We add enough of the other stories to the backlog to cover an interesting part of the scope:

  • Basic create client account
  • Complete create client account
  • Basic login admin user
  • Basic login client user
  • Complete login client user
  • Basic add worker
  • Complete add worker
  • Basic propose worker for position
  • Complete propose worker for position
  • Complete confirm worker for position
  • Basic enter timesheet (in this version temp agency enters on behalf of worker)
  • Experimental billing report
  • Basic billing report
  • Basic salary report

This functionality should be enough to have a pilot release where some clients and workers can be supported by the new system. Or we may complete the backlog with complete versions of all functionality, worker login and perhaps a polished version of a feature or two.

Adding the non-functional tasks

There are some tasks that we want to plan some extra time for. I generally find that many of these tasks are tasks that customers understand quite well:

  • Attend training on CSS (the team is rusty in design skills)
  • Basic layout and styling of web pages
  • Complete layout and styling of web pages
  • Polished layout and styling of web pages (they want it really nice)
  • Locate slowest pages and make some performance improvements
  • Deploy solution to target platform
  • Deploy demo version to wider set of stakeholders
  • Deploy pilot version
  • Exploratory test of complete flow

Planning the project

In this example project, we have five team members plus a coach/project manager on half-time. Since our team will be working in pairs, we want to work on three functional areas per week. This way, we can avoid huge merge conflicts. The team agrees to plan for five stories per week, but only three the first week, because things generally go slower. Here is the top of the completed backlog:

  • Week 1: Experimental create open position
  • Week 1: Experimental list positions
  • Week 1: Attend training on CSS
  • Week 2: Simplified create open position
  • Week 2: Simplified list positions
  • Week 2: Basic create client account
  • Week 2: Basic layout and styling of web pages
  • Week 3: Basic login client user
  • Week 3: Deploy solution to target platform
  • Week 3: Basic add worker
  • Week 3: Basic propose worker for position
  • Week 3: Basic enter timesheet (temp agency enters on behalf of worker)
  • Week 4: Experimental salary report
  • Week 4: Complete layout and styling of web pages
  • Week 4: Complete create open positions
  • Week 4: Complete list positions
  • Week 4: Deploy demo version to wider set of stakeholders
  • Week 6: Exploratory test of complete flow
  • Week 7: Deploy pilot version

Presenting the plan

Working through the list gives us a complete timeframe of just over 6 weeks for full feature set for the pilot release. To cover realities of life, we probably want to plan for at least one week of slack or even more, depending on the strength of our commitment and the consequences of being wrong.

This gives a plan indicating 7 weeks times 5 people at 40 hours per week plus a 50% project manager at 20 hours per week or a total of 1540 hours.

I generally find that after a pilot release (or even before it), things change a lot. So I don’t invest much time into planning this.

Tracking the development

The true strength of a plan like this appears when you start running the project. Each week, the team will report on which stories they completed. This allows us to adjust the plan to actual progress.

On the flip side, the weekly planning comes down the team and the customers agreeing on the definition of a story. The vagueness of “basic add worker” is by design! But the team should agree on what they mean by “experimental”, “simplified”, “basic”, “complete” and “polished”.

Conclusions

In this article, I have showed a quick and accurate way of coming up with a project forecast, complete with time and cost estimates. It’s easy to see and react to deviations from the forecast.

A few critical critical observations support this methodology:

  • I never believe a developer estimate other than “by the end of the day” or “by the end of the week”. (Don’t talk to me about hours!)
  • Estimating in hours is a silly way to get to project costs. Any hour-based estimate is always prodded and padded before magically turning into cost. Instead, estimate number of features, feature per week and cost by week.
  • Visiting a feature multiple times lowers total cost due to less gold-plating and investment of in poorly understood areas. It also improves the success of the final feature
  • The ambition of a feature (that is, how many times we will visit it) is a more reliable indication of cost than developer gut feeling

I’ve left many questions on the table, for example: What about architecture? What is meant by a “simplified” user story? How to deal with deviations from the forecast? Feel free to grill me for details in the comments to the article.

“So what will it cost?” Using this simple method to lay out your project forecast week by week, you can give a better answer next time someone asks.

Posted in English, Extreme Programming, Software Development | 1 Comment