Planning software development with a time machine

I have an amazing time machine that lets me think better about projects. This is part 3 in a series of blog posts exploring the use of a time machine.

In order to get a handle on how to build a feature for your next iteration, take a trip with your time machine to the future to watch how the feature will be demonstrated.

It’s often hard for developers to focus on exactly what tasks needs to be performed in order to build a new feature. Instead, we often end up breaking a feature down into nerdy and unfocused tasks such as “create a new table in the database”. Yawn!

Instead, form small groups of 2-3 team members, each working on one new feature at the time. The teams will write down how they would demonstrate that this feature was complete and actually working. Then each group will stand up and actually give the demo, indicating features in a system that’s not yet built.

A typical imagined demo goes something like this:

  1. “Welcome to this milestone of the development of project X. We are very excited to show you a lot of progress this week. I will now show you the exciting new … feature.
  2. “You can access the feature by starting the product and selecting X from the menu. As you can see (indicating to a black screen), you will be given a list of options to X, Y or Z. For the purposes of this demo, I will select X
  3. “….
  4. “This concludes creating a new X. We will now demonstrate how we can see that the X has been created…
  5. “As you can see, the new feature X helps user Y fulfill goal Z. I hope you are as excited about seeing actual uses start to use it as I am.
  6. “Thank you!” (applause)

While planning and giving a demo, a team often starts asking questions like “but where will we get information about X”, “how can we display Y in a sensible way” etc. These are exactly the kind of questions needed in order to plan the work ahead.

Seeing a demo from the future allows the work to be more focused on the goal and helps the team uncover hidden assumptions and missing information.

PS: I want to give kudos to my colleague Lars who has given one of the most amazing imagined demos I’ve ever seen!

Posted in English, Extreme Programming | Leave a comment

Getting excited about your project with a news headline from the future

I have an amazing time machine that lets me think better about projects. This is part 2 in a series of blog posts exploring the use of a time machine.

This is a trick that I learned from my User Experience (UX) friends.

In many projects, the project members have a great feeling about the possibilities of the product they are building, even if they quite know if they will get there or if the road ahead will be bumpy. A great way to focus your mind on the goal is to ask your future self to brag about what success the product was.

This exercise works great in groups: Form groups of 3-4 people (group of diverse people are better). Each group gets 15 minutes to draw the front page of a news paper (or online news paper) that picks up the story of the success of the product you all are building together.

Give each group a thick A3 paper and some colored markers. (Avoid pens – they don’t show up at a distance)

Some things to include:

  • The name (and logo) of the publication
  • A headline for the article
  • A sketch of an article image (including image description)
  • An ingress – a short summary of the news story

When the 15 minutes have expired, each group stands up and presents their article.

The posters make for great decoration of the team area (for a couple of weeks, anyway).

Remember: You will only build something remarkable if you can envision how it will be received.

Posted in English, Non-technical, Software Development | Leave a comment

Replanning your project with a time machine

I have an amazing time machine that lets me think better about projects. This is part 1 in a series of blog posts exploring the use of a time machine.

Let’s say that you have a project that has been running for a couple of months. Looking back at your issue tracker and other artifacts, you notice that it’s hard to see what has been done and especially how much time remains. You really wish that you had a proper product backlog of what has been done, so you could forecast how the future will be.

Pulling out a mental time machine, you can answer this question. This is how I create a plan that I’d like to travel back in time to give to myself.

  1. Look over the actual features that you have build. If the application is a normal application with a set of web pages, list all the screens.
  2. Which of the screens have features that required extra work, such as integration, search, dialogues or complex business rules? Split the screens that required extra work into one item per work
  3. Which of the work items did you have to substantially change or even throw away all together? Add a work item for each of these events
  4. For each work item, list a rough date when it was completed. If you have know the sprint, that’s okay. Just list the end date of the sprint. If you know the week, that’s great!
  5. Look over the list that you have so far: Are there things that are especially big? Can you find a way to split them? Are there things that are so small that they don’t really count? Can you logically merge some of these together?
  6. Now you’re ready to count the number of work items completed per week.

If we’re lucky, you may come up with a list of 2-5 items done every week. Each of the items has a clear demonstrable effect that could be seen by a customer.

Now you can look at the work ahead of you. Can you find a similar-size items? If you have some sort of screen mockups of the rest of the work you’re interested in, this is a great source of information.

To complete a plan, list a planned completion date for each work item:

  • For past work items, the planned date is the same as the completed date.
  • For future work items, put a planned date that gives the same rate of items per week as in a reasonable interval of the past.

There are many sources of uncertainty still left in such a plan, but it is a quick way to get a reasonable idea of the work ahead.

Posted in English, Software Development | Leave a comment

Software 2015 i regi av DND

Jeg sitter i styret i Dataforeningen Østlandet hvor jeg er med å arrangerer Software 2015. Software er en kryssfaglig IT-konferanse hvor vi samler spesialister innenfor forskjellige fagfelt for å få dem til å snakke sammen på tvers av fagområdene.

På konferansen er det dagsaktuelle temaer og trender som står på programmet. Jeg er stolt av alle de spennende temaene faggruppene har tatt fram og samarbeidet vi har fått til på tvers av fagmiljøer.

Vi ser at både kravhåndtering, devops, prosjektledelse og arkitektur fokuserer mer og mer på å bevege seg mot kortere leveransesykluser. Da blir det viktig å forstå verktøyene og teknikkene som gjør dette mulig.

Et annet viktig tema er den stadig mer sentrale rollen IT spiller i samfunnet. Vi har samlet representanter fra store offentlige IT-miljøer som gjennomfører prosjekter som vil påvirke alle i Norge.

Software har også lykkes i å belyse den viktige debatten som skjer mellom sikkerhet, personvern, kriminalitetet og overvåkning i samfunnet.

Jeg synes også det var morsomt å snakke med Frode som er med å arrangere stormaskinsporet. Jeg har aldri satt meg nok inn i stormaskin, men har vært i flere organisasjoner der hjertet av virksomheten har ligget i stormaskinen. Det har som regel vært en målsetning å bli kvitt stormaskinen, men fordi man ikke har satt seg inn i hva den gjør blir dette veldig vanskelig. Frode sa: “De som vil bli kvitt stormaskinen setter seg ikke inn i hva den gjør – så de får det ikke til. Og de som setter seg inn i den vil ikke bli kvitt den”. Jeg er ikke 100% overbevist om at han har rett, men det er spennende tanker.

Meld deg på Software 2015 du også.

Jeg gleder meg, og håper å se deg også på Software 2015!

Posted in Non-technical, Norsk | Leave a comment

Interactive REST hacking with Sublime Text and RESTer

Do you ever miss a command line to a web application? If you’re lucky, you may use a REST API with appropriate tools for the same job.

This Christmas, I bought a Sonos system for our family. It’s a wireless speaker system that uses music streaming services. It supports both Spotify and Deezer, but works better with Deezer and it came with a one year free subscription, so Deezer it is.

First order of business: Importing Spotify playlists into Deezer. I found a free service that does this, but by accidentally refreshing the browser a couple of times, I managed to import five copies of each playlist. Disaster!

Thus began my quest: I had a number of accidentally imported playlists in Deezer, and it turns out that after deleting a playlist, I needed about five clicks, including hitting a small icon, in order to delete the next. This was not going to be fun.

As a programmer, you know there must be a better way. Perhaps that better way requires more time because you have to program something, but dammit, better is better. In the case of Deezer, I found that they had a well-documented REST API. But this time, I didn’t feel like writing a program just to solve this one problem. And so we get to the core of this post: How can you just execute a fairly large number of API calls without writing a program for it? All I needed to do was to find out which set of API calls needed to be made based on my playlist listing and execute these.

My text editor of choice these days is Sublime Text, which has a number of plugins. The RESTer plugin was just what I needed.

So, in order to delete the extra playlists:

  1. I installed SublimeText 3, Package Manager and RESTer
  2. Looking at the Deezer API doc, I created a test application and entered the URL: https://connect.deezer.com/oauth/auth.php?app_id=YOUR_APP_ID&redirect_uri=http://localhost& perms=basic_access,email,delete_library&response_type=token in a web browser. This gave me an authorization dialog with Deezer that ends up redirecting to an invalid URL on localhost. However, the URL will be something like http://localhost/#access_token=xxxxxxxxxxxxxxxxxx&expires=3600. Just copy the access_token part, as this will be needed for later.
  3. Now, let’s list the playlists. In Sublime Text, I write the URL http://api.deezer.com/user/2529/playlists?access_token=xxxxx (replace the id with your own user id and the access_token with the result from last step) and press ctrl-alt-r. This executes the REST call and returns the result in a new buffer as a formatted JSON text.
  4. In Sublime Text, I bring up the search dialog with ctrl-f, click the “.*” button (regular expression), enter "title"|"picture" and press “Find All”. This selects all the relevant lines from the JSON response.
  5. Now, some Sublime Text editing magic. All the lines from the output are selected. Press shift-end to extend the selections to the end of their lines, ctrl-c to copy the texts to the clipboard, ctrl-a to select everything, del to delete and ctrl-v to paste it.
  6. Whoa! That should replace the text with a set of alternating lines like "title": "something" and "picture": "something". We almost have what we want. Double click on the word "title" on any line and press alt-f3. This selects all the places where it says "title" in the buffer. Press end to move to the end of the lines and del to join the line with the next.
  7. Now each playlist should be on a separate line with the title and the picture link. Press F9 to sort the lines. In my example, I can now clearly see all the lines the are duplicates. So I delete the lines I want to keep and keep the lines I want to delete. For all playlists with duplicates, I remove one line in my buffer and keep the rest for deleting.
  8. Now I just have to craft the delete commands. I double click an example of the word picture. I press right-arrow until the cursors (several!) are between the ” and the https-part. Press shift-home to select everything before https://api... and del press to delete this. Press end to go to the end of the line and backspace until you have deleted the ‘/image”,’ part so each URL looks like “https://https://api.deezer.com/playlist/number“.
  9. We now have our targets to delete. At this point in time, I went to the beginning of the buffer and pressed ctrl-alt-down arrow until I have a cursor at the start of all lines. I now type “DELETE “, press end, paste in the access_token from step 2 and press enter. I now have all the requests that I need to execute separated by a blank line!
  10. Finally, I go to the first line, press ctrl-alt-r which executes the HTTP request. I press ctrl-w to close the response, go down two lines and press ctrl-alt-r to do it again.

This is the result of the first request:

This is what my buffer looks like after selecting titles and pictures and joining the lines:

This is what my buffer looked like in the end.

Sometimes, doing an everyday Christmas task like deleting a lot of playlists is hard to do with the programs available. Luckily, with a little bit of programming knowledge and some sharp tools, you can get the job done through an API.

Posted in Code, English | Leave a comment

Promises you can trust

JavaScript Promises provide a strong programming model for the future of JavaScript development.

So here I’m playing with promises.

First I need a bit of a package.json file:

Now I can write my first test (test/promises_test.js):

Notice that the “it” function takes a “done” function parameter to ensure that the test waits until the promise has been resolved. Remove the call to done() or to resolve() and the test will timeout.

This test fails, but because of a timeout. The reason is that done is never called. Let’s improve the test.

Using “done()” instead of “then()” indicates that the promise chain is complete. If we haven’t dealt with errors, done will throw an exception. The test no longer times out, but fails well:

And we can fix it:

Lesson: Always end a promise chain with done().

In order to separate this, you can split the then and the done:

There is another shorthand for this in Mocha as well:

But what is a promise chain?

This is extra cool when we need multiple promises:

Notice that done is only called when ALL of the strings have had their length calculated (asynchronously).

This may seem weird at first, but is extremely helpful when dealing with object graphs:

Here, the save methods on dao.orderDao and orderLineDao both return promises. Our “savePurchaseOrder” function also returns a promise which is resolved when everything is saved. And everything happens asynchronously.

Okay, back to basics about promises.

Here, the second function to “done()” is called. We can use “fail()” as a shortcut:

But this is not so good. If the comparison fail, this test will time out! This is better:

Of course, we need unexpected events to be handled as well:

And of course: Something may fail in the middle of a chain:

The failure is automatically propagated to the first failure handler:

It took me a while to become really comfortable with Promises, but when I did, it simplified my JavaScript code quite a bit.

You can find the whole source code here. Also, be sure to check out Scott Sauyet’s slides on Functional JavaScript for more on promises, curry and other tasty functional stuff.

Thanks to my ex-colleague and fellow Exilee Sanath for the inspiration to write this article.

Posted in Code, English, Java | 1 Comment

The reuse dilemma

The first commandment that any young programmer learns is “Thou Shall Not Duplicate”. Thus instructed, whenever we see something that looks like it may be repeated code, we refactor. We create libraries and frameworks. But removing duplication doesn’t come for free.

If I refactor some code so that instead of duplicating some logic in Class A and Class B, these classes share the logic in Class R (for reuse!). Now Classes A and B are indirectly coupled. This is not necessarily a bad thing, but it comes with some consequences, which are often overlooked.

If Class A require the shared functionality to change, we have a choice: We make the change or we stymie Class A. Making the change comes at a cost of breaking Class B. If these classes are in the same Java package (or .NET Namespace), chances are that we will be able to verify that the change didn’t break anything. If the reused functionality is in a library that is used by another library that is used by a Class B, verifying that the change was good is harder.

This is where Continuous Integration comes in. We check in our change to class R (for reuse, remember). A build is triggered on Jenkins, Bamboo, TFS or what have you. The build causes other projects to get built and eventually, we get a failing build where a unit test for Class B breaks. If we did our job right.

Even with this failing test, we’re not out of the woods. A build system for a large enterprise may take up to an hour or even more to run. This means that by trying to improve Class R, the developers of Class A have made a mess for the developers of Class B at least for a good while.

When organizations see repeated build breaks due to changes in dependencies, the reaction is usually the same: Lets limit changes to the shared code. Perhaps we’ll even version it, so that any change that you need will be released in the next version and dependencies can upgrade at their own pace.

Now we’ve introduced another problem: We are no longer able to change the code. Developers of Class A will have to take Class R as it is, or at the very least go through substantial work to change it. If the reused code is very mature, this is fine. After all, this is what we have to deal with when it comes to language standard libraries and open source projects.

Most reused code inside an organization, however, isn’t so mature. The developers of Class A will often be frustrated by the limitations of Class R. If the reused class is something very domain specific, the limitation is even worse.

So the developers do what any reasonable developer would do: They make a copy of Class R (or they fork the repository where it lives), creating a new duplication of the code.

And so it goes: Removing duplication leads to risk of adverse interactions between reusers. Organizations enforce change control on the reused code to avoid changes made in one context from breaking other parts of the system, resulting in paralysis for the reusers who need the reused code to change. The reusers eventually duplicate the reused code to their own branch where they can evolve it, leading to duplication. Until someone gets the urge to remove the duplication.

The dilemma happens in the small and in the large, but the trade-offs are different.

What we should do less is to reuse immature code with little novel value. The cost of the extra coupling often far outweighs the benefit of reuse.

Posted in English, Software Development | 1 Comment

Dead simple configuration

Whole frameworks have been written with the purpose of handling the configuration of your application. I prefer a simpler way.

If by configuration we mean “everything that is likely to vary between deploys“, it follows that we should try and keep configuration simple. In Java, the simplest option is the humble properties file. The downside of a properties file is that you have to restart your application when you want it to pick up changes. Or do you?

Here’s a simple method I’ve used on several projects:

The AppConfiguration class looks like this:

This reads the configuration file in an efficient way and updates the settings as needed. It supports environment variables and system properties as defaults. And it even gives a pretty good log of what’s going on.

For the full source code and a magic DataSource which updates automatically, see this gist: https://gist.github.com/jhannes/b8b143e0e5b287d73038

Enjoy!

Posted in Code, English, Java | Leave a comment

Turning a new corner

This summer has been the most significant in my life. While I don’t often write about personal matters in this blog, I will make an exception.

On July 12th my son Leon was born. Healthy, happy and obviously, from the perspective a father, perfect.

Going through this fundamental life change has made me decide to re-evaluate a lot of things. Most importantly, I wanted to evaluate my career.

In the last few years, I have spread my focus thin. I have enjoyed working with developers in Norway, Sweden, Sri Lanka and elsewhere tremendously. I have enjoyed working with many different customers in many sectors. I’ve enjoyed new knowledge in C#, Android, as well as continuing to deepen my knowledge in JavaScript and Java. I’m grateful to the whole team at Exilesoft for giving me the opportunities to learn so much.

As my personal life gets more focused, I also want my professional life to be more focused.

As of August, I start a position as Director of Software Development at BrandMaster; a small, but solid Norwegian product company which develops a Marketing Resource Management (MRM) portal in Java and JavaScript.

For years, I have fluttered around to various companies as a consultant, opining and hopefully helped them develop their products. Now I want to be part of building a product where I can feel ownership and long-term commitment.

BrandMaster is a small company that has the energetic feel of a startup, but the established product and customer base of a mature product. The field of MRM is moving from one of a specialized tool to something that more and more companies find themselves investing in, as witnessed by the entry of Microsoft, SAP, IBM and more large players into the field.

I find that the experiences that I have gathered in BBS, Steria and Exilesoft provide a good basis for contributing to BrandMaster. This means that I will focus on a more narrow set of problems in the future: Moving a product towards greater more frequent releases, more effective use of test automation and agile planning and tracking. It means that I will be able to focus more narrowly on my favorite technologies of Java and JavaScript. And it means that I will be able to work more closely together with a great team of developers and creative people.

This does mean that I will not stick my neck out quite as much when it comes to large projects and organizations, C# and mobile development (although there will be some mobility in the future) in this blog.

Stay tuned: Queued up on my plate are instead super-simple project tracking, demo-driven development, effective unit tests, continuous delivery, 12factor Java 8 and product development.

I’m looking forward to a set of challenging problems at the same time as I get the peace of mind to enjoy the company of my son and his lovely, beautiful and sparkling mother.

Posted in English, Personal | Leave a comment

The lepidopterist’s curse: Playing with java.time

Pop quiz: What will be the output of this little program?

The answer is, like with most interesting questions, “it depends”. How can it depend? Well, let try a few examples:

  • getHoursOfDay(LocalDate.of(2014, 7, 15), ZoneId.of("Asia/Colombo")) returns 24. As expected
  • getHoursOfDay(LocalDate.of(2014, 7, 15), ZoneId.of("Europe/Oslo")) also returns 24.
  • But here comes a funny version: getHoursOfDay(LocalDate.of(2014, 3, 30), ZoneId.of("Europe/Oslo")) returns 23! This is daylight saving time.
  • Similarly: getHoursOfDay(LocalDate.of(2014, 10, 26), ZoneId.of("Europe/Oslo")) also returns 25
  • And of course, down under, everything is upside down: getHoursOfDay(LocalDate.of(2014, 10, 5), ZoneId.of("Australia/Melbourne")) gives 23.
  • Except, of course, in Queensland: getHoursOfDay(LocalDate.of(2014, 10, 5), ZoneId.of("Australia/Queensland")) => 24.

Daylight saving hours: The bane of programmers!

Daylight saving hours were instituted with the stated purpose of improving worker productivity by providing more working hours with light. Numerous studies have failed to prove that it works as intended.

Instead, when I examined the history of daylight saving hours in Norway, it turns out that it was lobbied by a golfer and a butterfly collector (“lepidopterist”) so that they could better pursue their hobbies after working hours. Thus the name of this blog post.

Most of the time, you can ignore daylight saving hours. But when you can’t, it can really bite you in the behind. For example: What does the hour by hour production of a power plan look like on the day that changes from daylight saving hours to standard time? Another example given to me by a colleague: TV schedules. It turns out that some TV channels just can’t be bothered to show programming during the extra hour in the fall. Or they will show the same hour of programming twice.

The Joda-Time API and now, the Java 8 time API java.time can help. If you use it correctly. Here is the code to display a table of values per hour:

Given 2014/10/26 and Oslo, this prints:

And on 2014/3/30, it prints:

So, if you ever find yourself writing code like this: for (int hour=0; hour<24; hour++) doSomething(midnight.plusHours(hour)); you may want to reconsider! This code will (probably) break twice a year.

At the face of it, time is an easy concept. When you start looking into the details, there's a reason that the java.time library contains 20 classes (if you don't count the subpackages). When used correctly, time calculations are simple. When used incorrectly, time calculations look simple, but contain subtle bugs.

Next time, perhaps I should ruminate on the finer points of Week Numbers.

Posted in Code, English, Java | Leave a comment