Why it is Time to Abandon your Change Advisory Board

The Change Advisory Board (CAB) is a significant component of Change Management in IT departments. Its promise? Minimize risk, speed up incident resolution and improve communication and collaboration. But in practice, the CAB significantly slows down Time-to-Market and is costly in terms of team mobilization and financial resources. It can also generate a false sense of security and lower the quality of delivery. 15 years of experience in Devops have taught me one thing: it's time to abandon your CAB in favor of automation.

‍

The last decades have witnessed profound changes in the way applications are designed and managed. The two most significant ones are probably the generalisation of Agile and Devops approaches. At the crossroads of these two major transformations is Change Management, or how to ensure that the evolutions expected by the business are actually in production, in a secure way and without impact for the users.

One of the most visible aspects of Change Management is the CAB, which stands for Change Advisory Board, and at Alenia we work with many players, including the main investment banks. In these structures, the IT departments are made up of thousands of people, spread over the main financial centers (NY, London, Paris, Hong-Kong...) and are in charge of hundreds of critical applications. As in many large companies, the CAB culture is still very present. I have observed up to 4 instances of CAB for the same change request before it was authorized to go into production:

• The CAB of the tribe located in Paris, because the production manager of the tribe wants to make sure that everything is well done in order to "save time"
• The CAB of the region, for example for a delivery by the New York teams to a bank whose headquarters are in Paris
• An ad-hoc CAB, because the application is part of a sensitive business process that has experienced several major incidents in the past
• The global CAB of the IT department

Each of these meetings often takes place on a weekly basis, lasting at least one hour and bringing together a significant number of participants (the business and their representatives, developers, business analysts, project managers, testers, integrators, application support, infrastructure professionals, change managers, release managers, managers, etc.)

Remember that CAB meetings are just the tip of the iceberg, and that it can take an hour or more of work upstream to draft and validate the workflow for each change request. On top of that, there is the CAB preparation and validation of each of these requests, often several hundred per week.

‍

Why are CABs so prevalent?

We understand quite quickly that the process of making a new feature available to users is not going to be the easiest, but more importantly that it is going to take time! So why have most companies chosen to implement a CAB?

Once you get past human behavioral biases, which should not be underestimated, such as "we've always done it this way" or "if we don't have a CAB we don't have serious change management" or a corporate culture that generates more distrust than trust, the following reasons are mainly to be found:

1. Minimize the risks associated with changes that lead to disruptions or breakdowns
2. Accelerate the resolution of incidents with clear traceability in order to quickly identify the changes that caused the disruption
3. Ensure that changes are properly reviewed and approved to comply with regulatory or internal audit requirements
4. Improve communication and collaboration between different teams in the organization

‍

In reality, what do we observe?

The CAB will mainly allow us to verify that each actor has correctly completed the change request. Given the volume, diversity and complexity of the changes to be processed, no one will be able to ensure that the actions have really been carried out correctly, or even to ensure that one change will have an impact on another. Worse, the CAB body, because of its position, often lacks the authority to really prevent a change. All of this creates a false sense of security and dilution of responsibility, where everyone thinks that what they do is validated by someone else, which contributes to a lower quality.

As far as traceability is concerned, there is often room for improvement. The slowness of the process means that the teams who are asked to go into production, generally under pressure from the business or management, prefer not to track or to make changes in a hurry.

Finally, regarding communication between teams, such problems are so deep that the CAB will generally become a battlefield, which is not the ideal place for collaboration.

The objectives of the CAB are therefore at best very partially achieved, and are in fact very costly:

• A huge slowdown in Time to Market, from a few days to several weeks, which leads to consequences on the business, but also on the motivation of employees.
• The number of people mobilized before and during the CAB represents a significant financial cost‍
• Although there are too few serious scientific studies on the subject, the research that led to Accelerate, written by leading authors in the field, shows a negative correlation between delivery quality and the existence of a CAB. This correlation remains even when the CAB is only used for high-risk changes. Even worse, teams without a CAB have higher delivery quality.

Finally, the first reaction to a major incident will usually be to increase the control criteria of the CAB, or even to add an instance, which will accentuate this vicious circle.

‍

In this case, why continue?

This observation is often shared by the different actors of Change Management, but it is still too rare to see CABs disappear. I think the main reason is that abandoning the CAB can be perceived as a reckless risk-taking and a signal that everyone can do what they want because the police will no longer be there.

In reality, the best answer to combine speed and quality can only be automation:

• First of all, invest in software quality rather than in procedures. Spread the use of BDD, TDD, peer programming, code coverage management, writing acceptance tests...
• Build a robust, fully automated CI/CD pipeline that ensures that tests are systematically played as early as possible in the development cycle (shift left) and that only allows delivery with a high level of confidence.
• This same pipeline is responsible for creating change requests in the ITSM tool and generating mandatory documents to satisfy regulator and audit requests. This automation will also have the direct consequence of improving data quality and therefore the resolution of delivery-related incidents.
• As for collaboration between teams, the implementation of concepts related to Agility or Lean management is often the most effective. The construction of the pipeline and its continuous improvement by the development and production teams are often perfect exercises for talking and understanding each other.

‍

Can we really do without a CAB?

After more than 15 years in the delivery business, I think the only good reason to keep a CAB is to do so only when you have a clear strategy to get out of it. Here are 3 of the most common examples:

1. I have a mainly "infra" CAB. In this case I work with my IT teams to make my applications resilient to failure, at the extreme this can lead to Chaos Monkey. This way, the adherence between the infrastructure and the application is reduced to a minimum, or even disappears completely, which makes the CAB useless.
2. I have a CAB for some problematic applications, very transversal, monolithic and often vendor. In this case, the solution would be to set up a go/no go procedure specific to these applications, the time to work on the implementation of deployment without service interruption and to make the application more modular.
3. My teams are not mature, it will be a disaster if I stop the CAB. It is preferable to set up indicators to define precisely the level of maturity that I expect to authorize deliveries outside the CAB. This can be done by setting up an Error Budget Policy based on business SLAs, or by switching back to CAB mode when the application has had more than X release incidents in the past month.

In the end, what should we remember? Change management and a structure to manage it are essential today. But to think that the CAB is necessary for the smooth running of the company is clearly a mistake, given its very relative efficiency and cost. The quality of the production process comes above all from automation. Automation that must be supported by a test factory that ensures with a high level of reliability that what goes into production will not cause regression or unexpected behavior. In addition to this, there are all the resilience levers, in order to limit the failure, or to make sure to be able to relaunch quickly if necessary.

If you are interested in this topic and want to extend the discussion, please contact me!

‍