Requirements quality is quality-in-use
I’ve worked quite some time on understanding and detecting quality defects in requirements documents and requirements quality in general. All the time, I was very dissatisfied with the current state in both research and practice on this topic. I think, the problem behind this is that there is no guidance: In times of rapid change and delivery, where every project looks different, we still have no good rule of what a good requirements document is.
A few years ago, we came up with such a rule, and tried it in various applications. And – so far – it seems to work! We’ve collected this experience and are now ready to tell you about it, because we really believe this should change how you view requirements engineering, and this should change what you consider good requirements documents.
The current views on requirements quality are incomplete, inadequate and imprecise
Current approaches are usually lists of quality characteristics, such as the ones described in the current RE standard ISO-29148 or the criteria described by the requirements engineering board IREB. When we compare these characteristics, we see that only quite a small subset of characteristics are shared (blue boxes). And both ISO and IREB can’t event completely agree on these: For example, the ISO considers cloning (copy-and-paste reuse) of requirements a violation of consistency, but IREB doesn’t.
Requirements Quality Characteristics of ISO-29148 and IREB in Comparison. They only agree on the blue characteristics. And there only partially.
Why? Because the truth is: Someone came of with the list, and this is what they could think of. Then someone else came with another list. Is one better then another? We don’t know! There is no systematic approach, and that’s why there is also no way to compare.
To make things short, the problem with these lists is that they are incomplete, inadequate and imprecise: As one can easily argue, each of the standards only got half of the characteristics. Not all characteristics are relevant in all project contexts. And finally, for many characteristics it remains unclear, first, what they mean and, second, what the consequences are if I violate the characteristics as a project.
I think this approach is fundamentally flawed. It assumes that there is a definition of a perfect requirements document. It assumes that I can break these down into a set of factors. And it assumes that these factors are the same for all projects.
Requirements Engineering is a service to its stakeholders!
The core idea that we take as the basis for all our work, is that requirements are rarely the goal of a project. Usually, you want to deliver great software. Requirements are a great tool to help you do that. This changes the perspective, since now there is no golden standard for requirements quality anymore. What matters is what helps your stakeholders. In other words, requirements engineering is a service to it’s stakeholders.
Who are the stakeholders? There will be another blog post on this, but I’ll give you the summary: First and foremost – there are the readers. Here, a first insight is that readers differ: Take, for example, software development in the insurance domain. Stakeholders here include, on the one side, qualified readers with lots of technical know-how. For example, think about the established programmer, who has worked developing software for 20+ years, but outside the insurance domain. And there might be rather non-technical readers. For one example, consider a customer or end user, who is a domain expert, but has no idea about software systems. If you want to write a good requirements document, you must consider both of these readers in your writing.
But besides readers, there are also invisible stakeholders, who read the requirements only in specific cases, such as legislation. Lastly, there is also future readers, who might need the document in a few years time. Is your requirements document prepared for this?
The basic assumption here is that RE is a service to its stakeholders – no more, no less.
The golden rule for quality-in-use or activity-based requirements document quality
This now enables us to understand what bad requirements documents are: Low quality requirements documents are those that are inefficient (i.e. cause mistakes) or ineffective (waste time) to use for its stakeholders. And this explains why quality is so specific to each context and each project: Because the stakeholders are different and the usages (or activities) are different.
In order to understand in a concrete project how good the requirements document is, we thus first need to understand two things: First, who are the stakeholders of the document (i.e. who are the potential readers)? Then second, how are they using it? What do they do with the document? Once we have understood this, everything else falls into place: Quality is then expressed through quality factors, which are properties of an RE artifacts (e.g. a requirements document). If these factors are present, they make certain activities of certain stakeholders or roles more or less efficient and effective.
Activity-based Requirements Engineering Quality Meta Model. An artifacts has certain properties (quality factors), which have an impact onto activities, which are conducted by stakeholders.
Examples for quality factors
To go back to the previous example of the established programmer and the user with no IT background: The established programmer needs the requirements document to understand what he must implement. He is interested in a requirements document that is precise and easy to read. This guy might not know the terms of this particular domain, so it helps to use domain terms consistently and define them in a glossary. He would also like a more formal description, e.g. in a UML diagram, because that makes programming easier.
On the other hand, the user, a domain expert with no IT background, just wants to make sure that the IT expert understands what he wants. By reviewing the requirements document, he wants to ensure that the IT expert builds the right system. But he might not be able to understand the UML diagram. In consequence, using UML will create more confusion instead of clarifying things. This shows that formalization can help, but that it really depends on the readers what is good requirements document quality.
Activity-based quality in an example: Usages of glossaries and formalizations in UML diagrams. The ‘+’ and ‘-‘ indicates a positive or negative impact of a quality factor onto an activity.
To sum it up: What is requirements document quality again?
This is obviously just the top of the iceberg. Once we see requirements quality from a quality-in-use or activity-based perspective, it completely changes how we do research, how we write guidelines, even how we do quality assurance in general. But I will discuss this, along with some more examples, in a future post. For now, all that I wanted to explain was:
High quality requirements documents are those that enable efficient and effective usage by requirements document stakeholders.
* this is a rough summary of a paper of papers I’ve written and discussions we’ve had with many colleagues, including Manfred Broy, Daniel Mendez Fernandez, Jakob Mund, Jonas Eckhardt, Andreas Vogelsang, and many more (tell me, if I forgot your name). Checkout also our full paper, which we presented at RET workshop in Florence, Italy: It’s the Activities, Stupid! A New Perspective on RE Quality
