CONTINUOUS INTEGRATION, MAKING THE IMPOSSIBLE POSSIBLE

Our Consultant Leif, a senior Project Manager at Seavus Stockholm, has written about his thoughts on Continuous integration and how he and his team made the impossible possible. 

I often find that the power of the mindset of continuous integration is underestimated. In one of the re-organizations at a large company, we created a vision that we should be able to remove all stabilization/release branches and end up with just one track.

Our challenge

Our challenge was that we had a few hundred of open bug reports to start with and when you corrected one bug, you might have to release the correction in multiple tracks. This caused a lot of extra administration for each bug we corrected.

We already had an embryo to a continuous integration environment, but the number of known faults was too big to handle. We had tried different competitions and incentives to reduce the known faults to zero and we did, occasionally, reach zero faults. But the very next day the number started to grow and after a few weeks we were back at too high levels again.

The big difference this time was that we switched focus from having zero defects to having a fast turnaround time for corrections and, if needed, the ability to back out serious faults.

What we had to do

To support this, we had to improve our test environment so that it was easy to pinpoint the commit that caused a fault. We increased the number of feedback loops, ranging from a few minutes to running the whole weekend and we modernized the programing language for the test cases to Ruby.

As a supporting function, we started a, daily, morning meeting where we reviewed the results from the different loops since the day before and distributed the result back to the teams that had made a delivery. If the result was too bad, the meeting could decide that the track was closed for new deliveries until either a solution for the problem was delivered or the delivery that caused the problem was backed out.

The amazing result

After about one year, we had reduced the number of known faults to a handful and we kept that low number despite that we triple the number of team and increased the number of supported products by a factor of 20 times! In my previous experience, it was almost a law of nature that a growth in number of people come with a growth in number of bugs. But with continuous integration we manage to brake that connection.

Four years later, we had almost reached the goal of doing our releases directly from our one and only track. We had almost achieved what many thought was impossible from the start.

The key to the success

To me, one key element in making this success was that we as leaders of the organization adopted the mindset of continuous integration. This prohibited us to take the easy way out when we were in a tight schedule and we could develop the framework in parallel with the products and features.

Tell us your story

Have you find some great of continues integration strategies of your own? What is your experience —and how are you implementing them?

Let us know in the comments.

Blogpost written by Leif, project manager at Seavus Stockholm.