The combination of DevOps and agile application development is rapidly gaining in popularity. In the current market, most organizations cannot afford to deliver software that customers do not like, is buggy or is delivered late. DevOps is shaking things up by breaking through the traditional divide between development and operations teams. And agile development methods are used to develop and deliver new functionality in multiple short cycles (sprints) on new technical platforms, such as containers and hyper-converged infrastructure. This combination makes it possible to add more value to the business processes of organizations in terms of higher quality, short times-to-market and optimization of the total cost of ownership.
To benefit from these, managers must be prepared to challenge their current organizational form, even if this means that their ‘domain’ will shrink or drastically change. DevOps is a portmanteau of Development and IT Operations. The emphasis is on communication, collaboration (between customers, software developers and IT operations) and integration, drawing on the thinking of Agile and Lean IT. It will come as no surprise that the emergence of DevOps in combination with agile application development (for the sake of ease, jointly referred to as DevOps in this article) will also have an impact on outsourcing. Executives would be wise to pause and take stock of the consequences that this will have on outsourcing. Strategic principles relating to the scoping process and contracting method of infrastructure services and application development, for instance, are different. And this has an impact on both outsourcers and vendors alike. This article takes a more in-depth look at the impact and offers some concrete guidelines.
The impact of DevOps on the scoping process
DevOps has a major impact on the scoping process, which is determined during decision-making on the sourcing strategy. In qualitative and financial terms, this sourcing strategy will only be future-proof if IT services, applications and infrastructure are insourced and outsourced in accordance with a well thought-out scoping process, with enterprise architecture as a benchmark. Many large organizations and government bodies have traditionally chosen a stovepipe approach to scoping, in search, as it were, of benefits of scale and clearly defined delimitation of responsibilities. In this horizontal form of scoping, application development, application management and infrastructure services (hosting) are often split into three lots and awarded to different vendors, and ultimately the service desk, work station and security & connectivity form separate lots. Technical application management was often allotted to the infrastructure service provider in order to contract for end-to-end application availability with that vendor. The transfer of a new application release from the application provider to the infrastructure provider then had formal gateways added to it. This was necessary if the infrastructure provider wanted to be able to guarantee application availability and did not want their bonus to be affected. And this sometimes left the customer waiting for months for the latest release.
The impact of DevOps on teams
DevOps and Agile break through this traditional stovepipe demarcation between development and operations, applications and infrastructure. DevOps is primarily used in innovative environments where short-cycle change is desirable in the business to effectively deal with high levels of competition, a growth market, high visibility or high levels of business dynamics. In stable legacy environments, the waterfall approach is often used. A large organization will probably have traditional waterfall teams as well as DevOps teams. A modern catalog of products and services should facilitate deployment of both alongside each other. DevOps introduces a vertical form of scoping (‘value stream’) which has consequences for the deployment of DevOps teams, discussed in more detail below.
Business System teams
DevOps tackles application development and management, databases and middleware through an integrated approach per functionally oriented domain (‘value stream’). Outsourcing in these areas is delivered by self-organizing, multi-disciplinary teams working in ‘scrums’. The members of these teams share responsibility for delivery of the results. Each team has a product owner who holds lead responsibility for the functional result. The scrum master is responsible for the process, the speed of delivery, and for providing and maintaining oversight and reflection. The structure of the competencies of the team can be visually represented as a T, in other words the team is experienced and has team members with many competencies. Members are able to look beyond their own discipline. In this process, architects aim to pay down the technical debt. This often arises from organically occurring designs and construction under intense pressure (fix now, pay later), which caused deviations from architectural standards, interfaces, etc. Technical debt can have a crippling effect, so that the required speed of the sprints is not achieved
You built it, you run it
Technical application management (database management, middleware), traditionally the responsibility of the infrastructure provider, ideally comes under the umbrella of the Business System team under a motto of ‘you built it, you run it’. While in practice, we also observe a variant in which this activity is still managed by the platform team, which then provides lifecycle management of the entire technical stack including the middleware, these days teams achieve this by deploying containers and hyper-converged infrastructure. Of course, the Business System team also uses highly standardized infrastructural platform services (the technical stack). These are selected based on their value for the team. Traditionally, most business applications are hosted within a private cloud service, while development and testing environments are more commonly hosted within a public cloud service. A shift is clearly visible: public cloud services are increasingly being used even for production environments.
Principles of DevOps teams
When DevOps teams, often comprised of internal and external employees, start working closely together with customers to continually deliver functionality through agile application development, this constitutes a completely different way of sourcing than, say, offshoring large waterfall projects. DevOps teams espouse the following principles:
- Customer-centric action: short feedback loops with real customers and end users; all development activities are centered around the customer.
- Create with the end in mind: avoid waterfall and process-oriented models where work takes place with your own role or function in mind, losing sight of the destination.
- End-to-end responsibility: DevOps teams are vertically organized so that they are responsible from concept to the end of the lifecycle, from concept to grave. IT products and services are built and delivered by the team, and remain under management of the team. ‘You built it, you run it’ is the motto.
- Cross-functional autonomous teams: if teams are organized vertically and are responsible for the entire lifecycle of a product or service, they must also be autonomous.
- Continuous improvement: the focus is on ongoing improvement of products and services. It goes without saying that the aim is to raise the quality of these as far as possible, but also to minimize waste – here we see Lean thinking – and to optimize speed, costs and ease of delivery.
- Automate everything you can: automation whenever and wherever possible is a consequence of the desire to continually innovate and improve the delivery of services.
The platform team offers a standardized environment. This is preferably comprised of PaaS containers, although IaaS is a viable alternative. This environment can be complemented by options for orchestration of the cloud management platform when public cloud services are used. Monitoring of standardization is an important task for this team. Exceptions lead to mistakes, require separate management and cost money. For the same reasons, ‘automation wherever possible’ is always on the agenda. Accurate cost allocation is indispensable because the Business System team works with infrastructure environments based on self-service. This promotes the cleanliness of the environment after use, so that the meter does not continue to run. To make good collaboration possible, the platform team has a representative in the Business System team, entirely in the spirit of DevOps. This also aims to guarantee accurate coordination with the sprint pace of the Business System team so that final delivery of the new functionality comes within reach without waiting times (single piece flow). With the adoption of DevOps, the service catalog needs some fresh ingredients, such as new tools to make the life of the DevOps team easier. This may include a self-service portal, tools for ongoing integration of systems, automated testing, automated implementation and automated delivery. This serves to enforce standards for DTAP streets in order to limit the technical debt from the platform team as far as possible. This includes patch management, for instance.
The impact of DevOps on contracting
The impact on the scoping process and the team focus is clear. How will this affect the contracting? If organizations wish to do justice to DevOps and Agile, they will have to change the way contracting works. The most important aspect is the shift from closed-scope contracting to open-scope contracting, where innovation and operations have to find a space in the same partnership. DevOps teams cannot easily handle usual methods of contracting at all. DevOps in combination with agile application development does not come with a fixed scope, but uses a different approach to scoping aimed at value generation and moves in line with changing insights of the customer. Compare this to the traditional scope definitions and comprehensive descriptions of functional and technical designs, result-oriented service descriptions, responsibilities and tools. The working method is based on a more or less static situation and a controlled process, not on a continually changing market that the customer wishes to respond to today. If you want to be the world champion in Formula 1, it is impossible to plan your race in advance, and your car will need modification between races. This is a fundamental point. We offer some guidelines.
Figure 1. Cone of the uncertainty
Business System teams
If you want to be successful in sourcing partnerships, you must define the relationship based on the cone of uncertainty (see Figure 1). Basically, the customer and the vendor ultimately adopt their own position when a service is contracted. This position is determined, among other things, by the degree of uncertainty that you perceive. The degree of uncertainty can be determined based on the answers to four questions:
- Who carries which risks (business operations, social, financial, time to market, technical)?
- What scope can be contracted with assurance of the agile business case?
- What is a reasonable contract price given the amount of work?
- When and how will we approve changes in the scope?
We propose that agile contracting is performed in phases based on the cone of uncertainty.
- If the level of uncertainty is high or if mutual trust is low, it is advisable for the parties to commit to small initiatives in, say, the first three sprints. KPIs are agreed and customer satisfaction is measured. However, final payment is based on time and materials. Cooperation is given the chance to grow and a clear product vision often emerges in the initial sprints, and the initial development priorities will become clear (visually represented in the first product backlog). The pace of delivery of the Business System team will also improve in the initial sprints. N.B.: sometimes the customer will proceed with this initial phase with more than one vendor as part of a preferred vendor selection process. This sometimes takes place in the form of a pressure cooker: the hackathon.
- After this courtship period, the time comes to formalize the relationship: agile contracting. The uncertainty has now diminished and the cooperation has been tested. In this interim period – between, say, sprints 4 and 12 – a performance bonus is introduced. 70 percent of the team’s financing comes from time and materials, while the remaining 30 percent comes from performance-based measures based on customer satisfaction, technical debt and team velocity. Additional bonus measures can be optionally considered, with the vendor ultimately being able to earn 110 percent. The objective is to gain experience with KPIs and the fine-tuning that goes hand in hand with this.
- Finally, it will be time for the experienced, mature DevOps team to tie the knot and engage in DevOps contracting. After 12 to 15 sprints, contracting can be fully (100 percent) based on performance targets, such as customer satisfaction, technical debt, availability, lead time for a change, number of incidents per release (the norm is zero) and productivity/burndown measures. However, a team happiness index should be included too.
DevOps contracting is a form of contracting which the customer and vendor grow into together. What’s more, KPIs are directly related to future business as a direct translation of business value. Examples from practical experience include the shortening of the lead times of a business process, an increase in conversion ratio (from lead to customer) and an increase in the net promotor score.
In the infrastructure domain, there are not many changes in terms of contracting, apart from an expansion of the service catalog with all kinds of tooling and metering. This becomes a part of an integrated dashboard that is also fully visible for the Business System teams. For utility infrastructure, pricing is per unit, entirely dependent on use, the technology and the grade of the corresponding utility. This allows for maximum scalability (up or down) based on pricing per unit, without applying pricing bandwidths or guaranteed purchase volumes. For infrastructure projects, a tariff sheet per profile/man-hour is also contracted, making it possible to calculate fixed prices for infrastructure projects. Fees for tooling are separate, sometimes per corporate license, sometimes based on numbers of users. Sometimes these costs are already included in the utility being paid for. It goes without saying here that the financial drivers are in place to enforce the required standardization. In the transition and transformation phase, we still often see fixed prices per sub-project, for migrations, transformations, transfers of personnel, harmonization costs, etc.
The impact of DevOps on the SLA
How will this all affect the SLA? Well, it will certainly look a lot different. There will be a move towards Business System team SLAs, based on entirely different principles. DevOps is about creating enduring customer value based on an open, redefinable scope in contrast to the closed scope of traditional SLAs. The focus in DevOps is much more on minimizing the time-to-market of new features, the contribution to end-to-end business KPIs, minimizing the mean time to repair & recover, and the technical debt incurred. Monthly reporting is already outmoded, with real-time monitoring now the norm. Where reporting is still done, these reports are much lighter and more frequent than previous monthly reviews common in traditional SLAs. With a sprint cycle of two weeks, a monthly report is already two iterations behind. The SLA and corresponding KPIs are in line with the cone of uncertainty, but not merely with the aim of imposing restrictions or penalties. KPIs also contribute to the three basic goals: optimizing the result (quality, time-to-market, business value), optimizing the financial aspects (costs and added value) and reducing the risks.
Finally, the customer and vendor find themselves in the same boat. The strong combination of DevOps and agile development is concerned with managing the partnership, not about controlling a vendor. From this perspective, a KPI for the customer is indispensable: lead time to decide. The flow in the teams must be guaranteed: a dithering customer will immediately be a drag on the entire team.
DevOps and Agile have a considerable impact on the scoping process in outsourcing, how we approach contracting and the collaboration models. Underestimating this will result in a mismatch between the efforts made in outsourcing as well as the added value of DevOps and Agile. These teams cannot handle the way in which the world of sourcing traditionally approaches contracting. Organizations would do well to anticipate what DevOps and agile sourcing will need in order to prepare for the world of … today.