Mature Enough? Using Maturity Models to Assess and Improve Software Projects

It is not rare that projects approach us not as a consultant firm but more like a medical first-aid station. In those cases, the patient represents a software or engineering project that is hanging on by a bare thread. So what do we do? We act like a first responder: (1) we check the symptoms, (2) the capabilities of the team, (3) create a treatment plan. Since more and more people are asking how we conduct our audits, I want to explain these steps in this blogpost.

Step 1: Check the Symptoms

If your project is struggling, the first step we alwaysdo, is asking you to provide us with all information that you have: Requirements, test cases, user stories, acceptance criteria, process descriptions, guidelines, … Let us do our job, trust us, we know what is relevant and what is not. Based on these documents, we will ask you a couple of direct questions. These questionsenable us to summarize your status quo on a single page. How do we do this? It’s simple: Maturity Models! Based on the work of Marlies van Steenbergen, we developed a Maturity Model with a focus on requirements engineering and test engineering.

The model has a simple structure:

• An Engineering Area is a high-level domain, such as a department or a group within a company. An example of such a domain: Requirements Engineering.
• A Focus Area describes one type of content that is documented or a theoretical concept that is implemented by the team. An example for a focus are could be the question whether a team documents system interaction scenarios.
• Capabilities denote the various levels of maturity for a focus area, the capability within the matrix is represented by a letter. For instance: Within the engineering area Requirements Engineering we defined that:
  • “Scenerio: Capability A” means: the team documents the system’s main scenarios.
  • “Scenerio: Capability B” means: the team documents alternative scenarios.

The maturity model we currently use, consists of four engineering areas, 40 focus areas, and 107 capabilities. When we join a team, we assess each of the capabilities into one of four values: Capability fulfilled, partly fulfilled, not fulfilled, or capability is irrelevant in this concrete case.

And boom, based on your provided documents and a few discussions with your team, we get a wonderful first picture of where you stand. For one of our customers, it looked like this:

Step 2: Checking in on Your Team.

Now, we have checked the facts to get an objective view on the status quo. Just like your doctor always first checks your temperature, breathing etc before even starting to diagnose you. We check your status in three ways:

• Short, 30-minutes 1:1 interviews with key stakeholders. These interviews reveal what is working well and where the main challenges lay from the teams point of view.
• Scenario-based interviews simulate common scenarios, such as requirements changes or what happens when a core person leaves the team. We take a closer look at how the team responds and how well the team is prepared for these scenarios.
• Root-Cause-Analysis takes one specific challenge (i.e. production bugs, failed reviews or test cases) and analyses the reasons for this problem.

These methods drive our understanding of what the actual problem is. In particular, they help us understand if your requirements engineering, your test engineering and your overall software engineering is dysfunctional. And, most importantly, they help us design a treatment plan!

Step 3: Creating Treatment Plan

Now, In Step 3 we are done with data gathering and observing. Based the maturity model and the more qualitative data from Step 2 and our long-term experience from consulting many different domains and projects, we can now add our assessment for each focus area: Do we think the way the team approaches this focus area needs to be changed? If so, how urgently? Depending on the case, all this evidence allows us to make a concrete treatment plan. A treatment plan consists of a number of action items. The action items can be as simple as modifying or changing a tool or process. And sometimes it’s important to first train people and expand their skillset.

Summary and Onward Journey

So in three simple steps we were able to objectively assess the status of the project, based on hard evidence and the root cause of the problem. We have included people’s opinion into the process, too, because, guess what, often your team members are aware of the problems. We add our expertise and show you a way out of your problems. And if you’re like me, having a map already solves part of the problem, which is the feeling of being completely overwhelmed by the sum of challenges.

This is the end of the assessment. But this is where our journey together only starts. Action item after action item, we turn the capabilities green. Some action items are quick wins and easily solved. Some need a year or even longer of experimenting. Some of our customers are using the maturity model in their improvement process for multiple years now, and they always have a big picture to transparently communicate progress to everyone involved.

Isn’t that awesome? If you would like to know more about maturity models or need some urgent treatment for your projects let me know via LinkedIn, in the comments or via mail.