Agile & PRINCE2: The best of both worlds

Taking the right approach in project and programme management is often half the battle. Wise choices early on can set you on a course to success. However, an inappropriate choice can leave you wasting valuable time. In this article we use a recent project to explore the pros and cons of using agile and waterfall methodologies, and highlight the advantages that can be had from adopting an agile development approach, but supported within an overall PRINCE2 framework.

Background to this Case Study

Aimia is a global leader in loyalty management running loyalty programmes for some of the most visible global brands including Sainsbury’s. 3gamma works with Aimia helping deliver a multi million pound portfolio of projects. This white paper references a development project for the Nectar brand that delivered a cost effective way to integrate systems and bring together data and content — creating value for collectors and partners, promoting innovation and opening new revenue sources. The project used an agile methodology, including Kanban, within a PRINCE2 project management environment to develop Open APIs (Application Programming Interfaces) for the Nectar Loyalty platform.

Waterfall and agile

The traditional approach to managing projects is to use a ‘waterfall’ model — a sequential, document driven process in which progress is seen as flowing steadily downwards like a waterfall through concept, initiation, analysis, design, build, test, implementation and support/maintenance. It originated in the heavy industries producing physical deliverables where there was little scope for change during the process. It depends on a strong initial brief and a detailed understanding of the requirements.

Although some would argue differently, PRINCE2 is generally viewed as a waterfall approach with sequential stage gates that strive to get to a clearly defined, fixed scope of work, with detailed designs and estimates, before embarking on development or implementation. It has a strong emphasis on project controls that support and enforce the methodology.

Agile is an iterative, collaborative process that is responsive to change, allowing for continual adaptation of the brief and response to user experience. It seeks to harness the knowledge, experience and creativity of the users of the solution as well as those developing it — supporting the way that reactive businesses work. Implemented correctly, it delivers quickly, yet robustly, in increments to gain an early return on investment.

Two common approaches to agile are Kanban and scrum. The key differences are summarised in the table on the next page. We used the Kanban approach rather than scrum sprints. The team had previously tried sprints but experienced unstable velocity as we were working within a very dynamic environment with changes requested mid-sprint. Kanban provided us with more flexibility because of its continuous process.

There is much debate on the relative merits of different approaches to managing projects, particularly agile versus waterfall. Many are ardent advocates of one or the other. However, key to success is to understand the pros and cons of each approach and how they ‘fit’ with the project in hand.

The six principles of Kanban

Kanban was developed by Toyota to improve productivity. Essentially there are six principles that need to be applied for an implementation of Kanban.

1. Visualise. We used the software tool Jira to provide a visual representation of the workflow so everybody could see where we were and could understand the impact of change.
2. Limit work in progress. The work was split down into user stories capturing the ‘who’, ‘what’ and ‘when’ of the requirements in a simple concise way. For our project, the capacity
   was an agreed number of stories allowed for each step in the workflow dependent on resources available. With experience the team became adept at breaking down work into stories of consistent, manageable size. This allowed us to take advantage of the pull system — when a later step had capacity, it pulled the next piece of work from the previous step.
3. Manage flow. Each transition between steps in the workflow was monitored, measured and reported using Jira. We monitored cycle time and looked to improve throughput.
4. Make policies explicit. We used Jira to publish standards for how we documented and developed stories, including agreed acceptance criteria. The test team had a set of agreed use cases for functional, performance and security testing. Jira integrated with Bamboo, an automated deployment solution used by Production Support to deploy the new functionality.
5. Implement feedback loops. Through our daily stand-ups (as well as across the desk) the team fed back to each other. In addition, we received input from the business, Production Support and Account Management on behalf of our customers at the meetings.
6. Improve collaboratively, evolve experimentally (using models and scientific method). We ran lessons learnt sessions, tracked improvement through the statistics that Jira provided us and ran other ad hoc workshops. One particular was on the ‘Theory of Constraints’ (the study of bottlenecks). This was a very hands-on exercise that awakened us to vulnerabilities in the team e.g. what would happen if someone was ill for a long time? We followed up this workshop with another one to investigate how we could cover for each other. This in turn led to further training and skills transfer between particular staff that we put into practice over the holiday season. We accepted that there would be a drop in productivity and allowed for this in our plans.

Our benefits from using Kanban

We had significant benefits by using Kanban for the project.

Early return on investment. We provided early return on investment for Nectar, our sponsors and partners by delivering usable functionality on a monthly cycle, with an intermediate fortnightly release if necessary. However, to achieve this required close co-operation with the business to prioritise the work into small manageable pieces of work with clear and agreed acceptance criteria.

We also required the business to be an integral part of the project undertaking an active role. The role of Product Owner was key to this. This was a senior, empowered specialist with appropriate authority and responsibility within the business who set the vision, prioritised the work, arbitrated on requirements and was available for consultation and decisions.

An ability to respond quickly to the business’ changing needs. With a conventional ‘waterfall’ project the business would have had to predict many months in advance exactly what they would need in great detail. With the agile approach we used the concept of “just enough design”. The stories were identified in broad terms initially, prioritised using MoSCoW (Must, Should, Could, Won’t have this time) and a budget and timescale established. The stories were only analysed and specified in detail immediately before the developers were ready for them. The business were able to make early use of the developing solution. This often resulted in them changing their views on later stories or indeed deciding that they would like changes to stories already delivered. This is no problem for an agile project. The buffer of stories feeding the project was just re-prioritised and no time was wasted.

Reduced risk. With agile projects the overall time, cost and quality are fixed and the scope is allowed to to vary. With frequent delivery of relatively small increments of working scope, there was never that much at risk. The business could stop a project and still have working functionality delivered. If a waterfall project is stopped part way through there may only be whatever documentation has been produced and nothing usable to show for the investment.

Increased productivity. The team comprised: Product Owner, Team Leader/Coach, Business Analyst, Technical Architect, Security Analyst, Developers, Production Support, Functional, Performance and Security Testers — all working in a very collaborative way. Most of the team sat together except for one member who was located off-shore. Everybody attended a daily 15 minute stand-up, either personally or via video conference. We had the shared goal of progressing stories through our delivery process. Every day we checked how things were going and if there were any bottlenecks the whole team focused on what could be done to release them.

Sustainable development environment. There are many examples of large software projects going massively over budget and time, or just not delivering what the business expected. The teams can come under pressure to work harder, cut corners etc. With an agile methodolgy and frequent delivery schedule we had all the statistics about the duration, effort and cost for each step in the process that a story went through. This enabled the business architects to size up and cost stories so we became better and better at estimating. Hence when the business came up with a new or changed requirement it was relatively easy to determine the impact and have the business state the priority, allowing us to respond with a realistic overall plan.

Our Kanban process was key to this. Jira helped us manage the process and produced statistics for keeping track of ‘cycle times’ (how long it takes for a story to successfully go from end to end), with a mean and standard deviation so we easily could see how we were doing and take steps to hold a good pace, and also look for opportunities to improve by working smarter, not harder.

Continuous improvement. We worked on small, discrete pieces of work at a time, each of which delivered working functionality. When we came to subsequent pieces of work we had the knowledge of recent deliveries fresh in our minds and the business had up to the minute experience with using it.

We also had continuous visibility of our cycle time statistics. Hence we were in a great position to get together and brainstorm the best way to approach the next bit of work, focusing all of our respective views. These were often only 15 to 30 minute workshops but at the end we could bring our product owner in, go through what we had come up with, and get an instant decision which items we could then progress with. After each release we did a quick lessons learnt to see what went well and what we could improve upon.

“These were often only 15 to 30 minute workshops”.

Trust and relationships. This worked at two levels. Externally, through our frequent communications, regular delivery of working solutions and response to evolving requirements, the business and our management developed trust in us.Internally, the joint ownership and collaboration within the team were key. They challenged each other if opinions differed, but in a very constructive manner, as they also supported each other totally.There was one example where a team member raised a particularly difficult issue which was holding up progress. I’ve seen other projects where the response would have been that it was his problem and other members would have backed away as long as no blame could attach to them. With this team they all looked to either provide advice or see how they could contribute to a solution. It can’t have taken more than 10 minutes and was a joy to behold.

One thing to understand as a project manager working with a well performing team is that the role is to manage the big picture, external dependencies, senior stakeholder engagement, the overall plan and budget. The team leader and project manager were there to observe and support (both worked on multiple projects). Essentially the project manager was focused 80% external (dependencies, budgets, resourcing and overall plans) and 20% internal, whilst the team leader was the reverse (primarily in a coaching role). We focused on dealing with factors that could impact the team’s productivity and made sure the business kept their priorities up to date.

Initially it takes time to train and develop such a team. This is where all involved need appropriate agile training, stakeholders included. There are agile courses available for all levels. If stakeholders take the wrong mental attitude to dealing with an agile team, their expectations could be contrary to the agile approach and this could lead to a breakdown in relationships and success.

Motivated and engaged team. The pride and commitment of the team kept things going effectively. As a project manager of an agile team it is important to recognise that the knowledge workers within the team have the greatest understanding about their work and are best qualified to plan and organise it. Hence, together with the product owner, we set a broad strategy of which stream of stories we should target for each release. We had contractual commitments to get other streams completed by further particular release dates. The team, under the team leader, worked together to confirm that this was viable and how they should organise the work to achieve it.

THE KANBAN BOARD
The work was managed in a series of swim-lanes. From top to bottom they were:

• Expedite/blockers: These items take priority over all others.
• Time sensitive work items: e.g. items with a contractual commitment to a delivery date
• Bugs, tasks and user stories with an external dependency: Where there was an external dependency e.g. infrastructure or a deliverable from another team.
• Release preparation: We delivered a release to production every two weeks. A separate story was created a few days before go-live, gathering up all the stories that were ready i.e. in Publish.
• Bugs, tasks and user stories: Standard work that was wholly within the team’s responsibility.

Principles applied:

• The work flowed from left to right through a number of process steps, each having set Work in Progress limits and was delivered when ready.
• To maintain resource utilisation we had buffer steps (Ready and Development Complete). However, if these started to build up we knew we had a problem and looked to increase capacity downstream. This was part of the collaboration and multi-skilling we instituted e.g. we could switch a developer into test as long as they were not testing their own code.
• The board was in Jira; clicking on a story opened up the detail. The whole team had access to the stories which included sub-tasks for each skill. As they progressed there was the ability to comment so it encouraged collaborative working.

Agile and PRINCE2

The standard PRINCE2 processes were used to provide the overall framework for managing the project. The Startup and Initiation stages were still used to give the project its overall structure and governance. Likewise the Closing A Project stage was there to complete the loop and the check back to the initial goals and benefits outlined at the start. The agile approach operated within the Managing Product Delivery stage, as indicated in the diagram below. Some of the elements of PRINCE2 were applied in a “light” way as there was only a need for “just enough” planning and design. However, this framework and its documents gave the business the high level visibility and control it wanted in order to have confidence in the governance. In addition, the infrastructure elements of the project were delivered using a more traditional waterfall approach and also sat within the Managing Product Delivery stage. Agile and waterfall approaches were therefore used within the project as the different needs played to their respective strengths.

Managing an agile project within PRINCE2

Conclusion

The nature of our business is that it is constantly evolving. In such an environment we can’t spend time going through an extended specification, high level design, detail design, build, test and then deliver on a large scale. What we would deliver might have been fine if it had gone live 6 months ago but may be useless now. Having said that, there is still a place for the waterfall approach where it is appropriate or necessary as described earlier. Using an agile approach enabled us to be very responsive to business needs whilst the PRINCE2 elements meant we did so without losing control and discipline. However the agile approach only works if everyone understands and is committed to fulfilling their respective roles in an agile way.

Download the whitepaper as PDF

About the author

Tony Davis has over 20 years experience as a senior IT Project Manager managing IT Infrastructure, Network Security and Systems & Business Transformation projects.

Tony Davis
Senior Consultant and Project Manager
tony.davis@3gamma.com