The state of ITSM in agile organizations: delivering agile innovation while supporting stable IT operations

Swedish companies are struggling with the balance between governance, control and stability, and the need for greater innovation and flexibility. Conflicting forces are driving complexity. Regulatory requirements are increasing the demand for greater control and governance while customer demands are accelerating and changing. IT is caught in the middle – trying to balance stability, control and maintaining operations – and at the same time deliver innovation.

3gamma has surveyed 100 IT leaders across a range of industries in Sweden and interviewed eight high-profile companies on the topic of the state of IT service management in agile organisations. The purpose of the study was to quantify the perceived issues in the interface between agile development methodologies and IT service management frameworks. And also to better understand how companies maintain stable and controlled IT operations while at the same time delivering agile development and business development.

Within this study the term agile development methodologies refers to iterative and incremental software development methodologies. Agile IT organisations are IT organisations that are quick to respond to changing external or internal circumstances and have made the process of change a routine part of their organisational culture.

The study has shown that Swedish companies are facing a set of well-known IT-related issues. These issues are connected to both internal, external challenges and requirements. However, it seems that these issues are unrelated to any specific framework or methodology. Success is not tied to any specific methodology, it’s about execution.

The way to an efficient and value-creating IT delivery is based on a disciplined approach to whatever method or framework that is used. There is no correlation between any specific set of methods and the identified or perceived issues.

Successful IT organisations focus on the customers’ needs. They create a truly service-oriented organisation that is supported by the right technical pre-requisites, a well-tailored delivery model, and stringent fit-for-purpose delivery methods. Most importantly, it is their communication across teams and with their customers and stakeholders that truly sets them apart.

The successful IT organizations are focused, base their delivery methods on fundamental capabilities within requirements management, communication and planning, and have the right technical and cultural pre-requisites. There is no quick fix or silver-bullet solution. They work hard and diligently with an unwavering focus, they do not jump ship to ‘the next buzzword’-method.

Time to market and the ability to quickly adapt to changing customer requirements have become a competitive necessity

Companies today are pressured to deliver new products and services at an increasing pace as the technology share of each new product and service increases. IT is struggling to keep up. IT has become an integral part of new business development initiatives; 84 % of the respondents in 3gamma’s survey state that time to market is the key focus. In face of this change, IT organizations have deployed three key strategies: legacy modernization, outsourcing and new ways of working.

Legacy modernization

Modernize the underpinning infrastructure and applications services to enable a more flexible development and delivery. Key initiatives include transformation to a service oriented architecture, cloud-based services (both infrastructure and application services) and application rationalization.

A major telco in 3gamma’s study is implementing an internal cloud to support rapid deployment of new business applications. This new internal cloud is seen as an enabler of agility, as it improves the IT department’s ability to quickly respond to the development teams, and ultimately the business’ changing requirements.

It also reduces the amount of hardware specification and ‘techno jargon’ needed to set up a new service to their business. Simply put, through adoption of a service-oriented delivery model based on a cloud-style infrastructure, time to market can be reduced and business orientation is improved.

Figure 1: 84 % of the respondents somewhat agree or agrees that the demands for shorter time to market have increased in the last five years.

Outsourcing

Right-source IT capabilities to ensure the right mix of internal and external competencies, access to a flexible resource pool, cost efficiency and scalability. Many second and third generation outsourcers are transitioning to a multi-sourcing model based on selective right-sourcing.

The length of each contract has been shortened to support a more flexible service portfolio. In some cases the purchasing pattern of IT services closely resembles a consumer pattern with a monthly subscription/pay as you go-model.

A fast moving consumer goods company in 3gamma’s study uses several cloud-based services to improve time to market, reduce cost and improve flexibility within marketing. There is no traditional contract, just a simple subscription paid with a credit card.

Figure 2: Over 90 % of the respondents somewhat agree or agrees that IT has become more important to the business in the last five years.

Figure 3: Over 80 % of the respondents somewhat agrees or agrees that the demands on high availability and continuous delivery have increased in the last five years.

New ways of working

Adopt an agile methodology to improve time to market and internal efficiency. The adoption of these methodologies mark a shift from the waterfall model where a product or service could or would be detailed upfront to a more evolutionary approach. 73 % of respondents in 3gamma’s survey work at companies that have adopted an agile method. The adoption is widespread within development organizations. Agile principles are also applied to IT service delivery organizations to ‘cut through red tape’ and improve service quality.

It is notable that IT operations have typically been left out of the agile transformations and are seldom represented within the agile teams, and are therefore often expecting a waterfall-oriented delivery from the development teams.

As such, an operations perspective is typically taken during the hand-over from development to operations. The effect of this is often a considerable delay for the business. IT service management on the other hand has come further, where service introduction and integrations are widespread.

The need for an agile and flexible development organization has not offset the need for operational stability. IT is expected to deliver high-quality services at all times as the stability of the services is key, the lack of communication between development and operations is accentuated. Changes are forced into production with little or no control.

77 % of respondents within 3gamma’s survey highlight ITIL as an enabler of quality delivery of IT services. Their primary focus is on the basic core processes: Incident management, problem management and change management. The top performers supplement this with an exceptional demand management and extensive collaboration and communication. There are indications of increased bureaucracy when IT service management frameworks are extended to a wider scope of processes.

Agile methods are being adopted to improve time to market but confidence is low

The key driver behind the adoption of agile methods is time to market. However, only 27 % of respondents in 3gamma’s survey believe their implementation of an agile method has led to shorter time to market. This exceptional lack of confidence in the agile delivery methods is attributed to a set of classic short-comings:

• Poor requirement’s management,
• Poor cooperation with the business
• Too many parallel activities

Several respondents in the survey have experienced methodologies that were being implemented from the top-down. However, agile methodologies are highly oriented towards empowerment of the individuals and teams. Agile is built around self-organizing teams. However, implementing agile through a top-down approach is not easily done. Support and commitment from IT development, and likewise, IT operations teams are a pre-requisite.

It is clear from 3gamma’s survey that there is little to be gained from a top-down framework implementation. Changing the way of working needs to be fine-tuned to the organization’s specific pre-requisites and context, while being implemented with the utmost discipline. A company in the financial sector clearly highlighted that a key obstacle to efficient IT delivery was the fact that each unit within IT was busy implementing a new way of working, all in isolation from each other. Implementing a new way of working became more important than the actual delivery of the IT services to the end users and customers.

The ‘new way of working’ cure to delivery issues can be outright detrimental to quality, cost and time. The organization focuses internally on its own processes and capabilities rather than on customer needs. The tendency for this is higher with IT service management than agile methodologies. Agile methodologies seem, based on the survey, to be originating from business requirements and not from within IT.

Companies that are suffering from issues between agile and IT service management are often new to agile.

They typically have a fairly mature IT service management organization with a tendency for bureaucracy. When they implement agile it is often driven from the top-down by management with the purpose of reducing bureaucracy. It is often done as a one-off project, leaving the delivery teams to sort out the details. The effect: agile is used as an excuse to become undisciplined and unstructured.

A key finding within the study is that successful companies keep focus through their implementation projects making sure to operationalize the entire value chain, from operations via service development to end customer and user. They often start small with a single team and gradually expand and fine-tune.

Swedish companies are facing challenges in combining agile methods and IT service management

IT service management frameworks like ITIL are wide spread through Swedish IT organizations today. 70 % of respondents in 3gamma ́s survey state that their organization is working with IT service management.

As noted, increased demands for shorter time-to-market and better responsiveness to customer requirements are driving implementation of agile methods. 62 % of respondents in 3gamma’s survey have stated that they face challenges in combining agile methods and IT service management. The challenges are primarily related to the interface between development and operations and are accentuated in areas where the business requires a higher release frequency.

Change management is one area where organizations face many challenges in the interface between agile and IT service management. The change management process is often developed and adjusted for ‘traditional’ waterfall development methods using a ‘command and control’ approach, requiring the change requestors to plan well in advance in order to get approval. Agile teams releasing new code frequently have difficulties fitting into the traditional change management process.

For some of the companies in 3gamma’s study there is a four week notification period for changes. This is actual longer than the average sprint in an agile method. One company within the study used daily change advisory board meetings to offset this and to improve communication and collaboration with the business stakeholders.

Figure 5: 62 % of the respondents somewhat agree or agrees that there are challenges in combining agile methodologies and IT service management frameworks

The result was improved commitment from the business, improved quality in the change requests and better understanding for the overall process.

Request management is another area where the survey respondents have identified challenges. Due to bureaucratic request management processes, slowed down by delivery organizations working in silos and complex financial set ups, the delivery time of new infrastructure to development projects becomes too long. This is often accentuated through an internal focus on methods rather than service delivery.

What underlines is that IT service management processes can become too complex and bureaucratic. They sometimes become too task-oriented and misaligned with the business’ requirements. There is a disconnect between IT and the business.

This is visible in 3gamma’s survey, especially amongst respondents working with system development. 80% think that IT service management processes prevent shorter time-to-market and 50% experience them as being bureaucratic and time consuming.

Successful companies within 3gamma’s study focus on communication and collaboration. They have created a culture of collaboration across teams and departments and they readily share information across the organizational boundaries, often supported by easy-to-use tools. They include their outsourcing providers in their processes, as well as their business stakeholders. Additionally, they strive for transparency. This is often exemplified through the use of whiteboards showing the progress of an activity.

They do not focus on forms for changes and/or requests to support multiple hand-overs. In a sense they have adopted the principle of self-organizing teams where they make sure that all competence required to solve an issue is available within the process. In short: ‘customer collaboration over contract negotiation’, as stated in the manifesto for agile software development.

For operations, a corresponding agile operations manifesto has been articulated:

• Customer satisfaction over service level agreement compliance
• Attitude and collaboration over certification
• Focus on results over focus and on activities
• Adaptivity over procedures

The principles articulated within the agile operations manifesto are clearly visible in the successful organizations within 3gamma’s study. These are also underpinned by the notion of self-organizing teams containing all resources, knowledge and competencies for the task at hand.

There’s a disconnect between IT operations and the business: IT operations have no insight into business demands

IT in general, and IT operations in particular believe that there is a gap between their organization and the business. The level of understanding within IT operations of the business need for the services is poor. The customers do not see IT as a service provider but rather a hardware and application provider. This gap is larger in organizations that have adapted an agile development methodology.

IT development and IT operations professionals that participated in the survey highlight:

• A lack of collaboration with business customers as a key reason for not being able to reduce time to market for new development/IT services.
• The lack of a proper hand-over to IT operations as the number one reason for lack of quality in the IT delivery.

The disconnect between the business stakeholders and IT operations creates a bullwhip effect: the lack of an operational perspective in the development phase creates an unsustainable delivery situation. IT does not know what is expected and it is too late and too expensive to do anything about it.

3gamma’s study shows that 89 % of the respondents feel the need to make IT operations more attentive to changed business needs or changed end-customer needs. IT operations needs to be included within the development/design phase. This, in turn, means that IT operations need to become a business partner. IT operations must focus on “delivery as is” as well as on business value and results.

Forrester notes that “[…] most Agile adoption in the past decade has seemed to focus on optimizing workflow within a development team rather than across the entire value chain”[3]. 3gamma’s study shows that Swedish companies are suffering from the lack of a holistic value chain-approach to agile. IT operations have been left out and cross-team synergies are not being realized, as noted above in the example with the company within the financial sector.

At this time, DevOps has not been adopted on a broad scale by companies participating in the 3gamma survey; as noted above, there are signs that there is a need to increase agility and time to market for IT operations.

Successful companies within 3gamma’s study are using a holistic approach, to some extent based on DevOps ‘principles’. They are empowering teams to deliver end-to-end services across their entire life-cycle. Key approaches are:

• Inclusion of IT operations into the agile development teams
• Inclusion of operational requirements in the definition of done
• Clear, communicated and agreed ‘rhythm’ for delivery frequencies across both development and operations
• Visualization of operations’ activities through the use of e.g. Kanban boards
• Using disciplined feedback processes and retrospectives to support learning

Several successful companies are also actively striving to minimize the amount of parallel activities to create a focused organization. Using a set frequency for delivery and visualization on e.g. Kanban boards support this. The companies that have deployed these methods are experiencing an improved efficiency and an increase in customer satisfaction as the delivery becomes aligned with business priorities.

Legacy technology is an obstacle to agile — Agile is what agile does

Several of the companies within 3gamma’s study have pin-pointed legacy technology, monolithic solutions and applications/infrastructure strategies as an obstacle to agile:

• One size fits all enterprise resource planning systems are not easily developed, especially as they are often delivered under a contract not supporting incremental deliveries.
• Centrally defined top-down technology stacks are locking down innovation and driving cost as they are less ideal to the solutions that are being developed and driving shadow IT.
• Centrally defined hosting strategies are locking down delivery options for new IT services, often accentuated by a sourcing strategy based on a prime vendor.

Legacy systems of record, often containing massive amounts of data and transactions, are often designed for stability and reliability. System of engagement; the systems used to interact with end customers, are changing rapidly, due to the advent of mobile and increased self-service. These systems are often tightly connected to the legacy system records, creating the need for an increased frequency of change in these systems.

To facilitate and support agile and technology, contractual obstacles need to be removed. Systems of record need to be every bit as agile as the systems of engagement, supported by an agile delivery model. The first step is this journey is often to adopt a cloud-like model for infrastructure services and to start breaking up the legacy monoliths.

The advent of Agile operations — think big, empower and start small

An agile IT organization is not created through the adoption of an agile development methodology. There is no quick fix. Becoming a high-performing IT organization cannot be attributed to a single factor. It is done through focused execution of a business-driven IT strategy encompassing people, processes and technology.

3gamma’s survey shows that the issues experienced within the participating organizations can’t be attributed to any specific methodology or framework. They are generic and wide-spread.

From the financial sector, through retail, to media, the pre-requisites vary. The companies and organizations within 3gamma’s study that are successful in meeting their businesses’ requirements, regardless of size or industry, share a set of common traits:

Customer Centricity

They are customer and service oriented. They invest heavily in creating and maintaining a well-defined, disciplined and transparent demand process. Often with the explicit purpose of creating focus and reducing the number of parallel activities while managing stakeholder expectations.

Communication

They communicate in abundance with all stakeholders, both orally and through the use of physical and digital tools. They understand their business’ needs and have the ability, processes and culture to meet these requirements.

Continous Improvement

They are disciplined in their approach to change. They avoid framework implementations and have a structured approach for continuous improvement in their ways of working. These processes are often supported by robust feedback loops and retrospectives making the IT organizations aligned with the needs of their customers.

Removing Obstacles

They create the right pre-requisites by removing obstacles to agile, primarily legacy technology and applications. They are avoiding “the monolith” approach to technology and are avoiding one size-fits all solutions for system of records to support rapid development of their systems of engagement. They are revising static contracts with outsourcing providers to cater for agile approaches.

To become a successful, agile IT organization, it is important to think big, empower and start small.

• Articulate and communicate a clear vision across the organization, both within and outside of IT. Answer the ‘why’ over and over again.
• Empower a team or a part of the organization to start the change journey. Use an incremental approach to change. Include business stakeholders in the change project to continuously adapt to their needs and requirements. Make sure to maintain the customer perspective. Answer the ‘how’ using a bottom-up approach.
• Execute with discipline. Focus on people, culture, processes and technology. Do not jump ship to test the latest buzzword methodology. Stay focused and continue with incremental change. Avoid over-formalization and over-design. Remove technical obstacles to agile through modernization and modularization, enabling self-organizing teams with low dependencies to other teams. Think ‘spot on’ and ‘good enough’.
• Steal with pride from new methodologies and include them as appropriate continuous improvement initiatives. Apply discipline in feedback processes and retrospectives to create a learning organization. Do not stop changing. Expand, evolve and adapt!

About the author

Jens Ekberg
Head of Group Insights
jens.ekberg@3gamma.com

Jens Ekberg is a Director at 3gamma and Head of Solutions at 3gamma Sweden. 3gamma is an international IT consultancy organization, covering IT strategy through sourcing and operations.