Requirements engineering
Many problems in the software world arise from shortcomings in the ways that people learn about, document, agree upon and modify the product's requirements. Common problem areas are informal information gathering, implied functionality, miscommunicated assumptions, poorly specified requirements, and a casual change process. Nowhere more than in the requirements do the interests of all the stakeholders in a project intersect. Handled well, this intersection can lead to delighted users and fulfilled engineers. Handled poorly, it is the source of misunderstanding and friction that undermine the product's quality and value.
Our organization already excels at the specification of solutions and the verification of their correctnes, so this document proposes a process to bring a similar level of formalization to the requirements step. No matter how well specificied or verified a solution is, if it is not the solution that the stakeholders or the ecosystem at large needed, then time and resources would be wasted and contributors would be left only with frustration.
Despite decades of industry experience, many software organizations struggle to understand, document, and manage their product requirements. Inadequate user input, incomplete requirements, changing requirements, and misunderstood business objectives are major reasons why so many technology projects are less than fully successful.
The majority of the content of this document has been extracted from "Software requirements" by Karl Wiegers and Joy Beatty. This document serves in fact basically as a summary of the book (which is 500+ pages long), incorporating the aspects that I considered most relevant and applicable to our situation. I have also made some slight adaptations where I saw fit to the context of our organization and the nature of our projects.
Requirements engineering is the process of discovering, negotiating, specifying, and tracking requirements. It is usually split into two separate disciplines: requirements development and requirements management. During requirements development you iteratively:
- explore requirements and identify user classes;
- gain a richer understanding of the different types of requirements (business, user, functional, etc) and identify unnecessary requirements;
- represent and store the requirements in a well organized fashion;
- review the documented requirements and confirm correctness.
When requirements are fleshed out and the engineering team is ready to start development you do requirements management by:
- defining the requirements baseline (i.e. the set of requirements agreed-upon for a specific release);
- evaluate the impact of proposed requirements changes and incorporate approved changed in a controller way;
- trace individual requirements to their corresponding, designs, source code and tests.
Let's begin by defining what requirements are:
Requirements are a specification of what should be implemented. They are descriptions of how the system shoud behave, or of a system property or attribute. They may be a constraint on the development process of the system.
The major consequence of problems in the requirements is rework (i.e. doing again something that you thought was already done) late in development or after release.
Shortcomings in requirements practices pose many risks to project success, where success means delivering a product that satisfies the user's functional and quality expectations at the agreed-upon cost and schedule. Some of the most common requirements risks are:
- Insufficient user involvement: Customers often don't understand why it is so essential to work hard on eliciting requirements and assuring their quality. Engineers might not emphasize user involvement, perhaps because they think they already understand what the users need. In some cases it's difficult to gain access to people who will actually use the product, and user surrogates don't always understand what users really need. Insufficient user involvement leads to late-breaking requirements that generate rework and delay completion.
- Inaccurate planning: Vague, poorly understood requirements lead to overly optimistic estimates, which come back to haunt you when the inevitable overruns occur.
- Creeping user requirements: As requirements evolve during development, projects often exceed their planned schedules and budgets (which are nearly always too optimistic anyway). To manage scope creep, begin with a clear statement of the project's business objectives, strategic vision, scope, limitations, and success criteria.
- Ambiguous requirements: Ambiguity leads to different expectations on the part of various stakeholders. Some of them are then surprised at whatever is delivered. Ambiguous requirements cause wasted time when engineers implement a solution for the wrong problem. Testers who expect the product to behave differently from what the engineers built waste time resolving the differences.
- Gold plating: Engineers should strive for leanness and simplicity, not going beyond what stakeholders request without their approval.
- Overlooked stakeholders: Most products have several groups of users who might use different subsets of features, have different frequencies of use, or have varying levels of experience. If you don't identify the important user classes for your product early on, some user needs won't be met.
It comprises of:
- Elicitation: the activities to discover requirements (interviews, workshops, focus groups, prototyping, etc) and identifying user classes and other stakeholders.
- Analysis: reaching a more precise understanding of each requirement and decomposing high-level requirements into an appropriatel level of detail.
- Specification: the representation and storage of the collected requirements.
- Validation: reviewing and confirming that you have the correct set of requirements.
Progressive refinement of detail is a key principle for effective requirements development. It is very important to plan for multiple cycles of exploring requirements, progressively refining high-level requirements into more prevision and detail, and confirming correctness with users.
 |
|---|
| Requirements development is an iterative process |
The following picture shows an example of a process framework for requirements development:
The heart of requirements development is elicitation, the process of identifying the needs and constraints of the various stakeholders for a software system. Elicitation is not the same as gathering requirements. Nor is it a simple matter of transcribing exactly what users say. Elicitation is a collaborative and analytical process that includes activities to collect, discover, extract, and define requirements. Elicitation is used to discover business, user, functional, and non-functional requirements, along with other types of information. Requirements elicitation is perhaps the most challenging, critical, error-prone, and communication-intensive aspect of software development.
The output of requirements development is a common understanding of the needs held by the diverse project stakeholders. When the engineers understand those needs, they can explore alternative solutions to address them. Elicitation participants should resist the temptation to design the system until they understand the problem. Otherwise, they can expect to do considerable design rework as the requirements become better defined. Emphasizing user tasks rather than user interfaces, and focusing on true needs more than on expressed desires, help keep the team from being sidetracked by prematurely specifying design details.
Validating requirements to ensure that they have all the desired properties of high-quality requirements is also an essential activity; it allows teams to build a correct solution that meets the stated business objectives. The validation activities attempt to ensure that:
- The software requirements accurately describe the intended system capabilities and properties that will satisfy the various stakeholders' needs.
- The software requirements are correctly derived from the business requirements, system requirements, business rules, and other sources.
- The requirements are complete, feasible, and verifiable.
- All requirements are necessary, and the entire set is sufficient to meet the business objectives.
- All requirements representations are consistent with each other.
- The requirements provide an adequate basis to proceed with design and construction.
The following figure shows a defect checklist to help reviewers look for typical kinds of errors in requirements:
The checklist is obviously very long and no one will be able to remember all the items, so it might be better to prune the lists to meet our organization's needs and select the items that reflect the problems that people encounter most often with our own requirements.
Acceptance criteria (and hence acceptance testing) should evaluate whether the product satisfies its documented requirements and whether it is fit for use in the intended operating environment. Think about and specify a set of acceptance tests that will validate that the solution does what it is supposed to. Focus on testing the normal flows of the use cases and their corresponding exceptions, devoting less attention to the less frequently used alternative flows. The earlier that acceptance tests are written, the sooner they can help the team filter out defects in the requirements and, ultimately, in the implemented software.
There are multiple levels of requirements that come from different sources at different times during a project, have different audiences and purposes, and need to be documented in different ways.
Not all of these may be needed for every single product or feature, so judge yourself which ones to use in your particular situation.
An example of how different stakeholders participate in requirements development:
They describe why the organization is implementing the product or feature (the benefits the organization hopes to achieve). They provide a reference for making decisions about proposed requirement changes and enhancements (i.e. decide if a proposed requirement is in or out of scope). Business requirements directly influence which user or functional requirements to implement and in what sequence. The sections below describe the components that may be present in a business requirements document.
Not all these might be needed for every single product or feature, so use your own judgement to decide what sections of the template pertain to your product and skip the rest. Regardless of whether you follow the proposed template or not, the main purpose of the business requirements document is to define the problem that the organization is trying to solve and the outcomes that it will achieve by solving it.
This section describes the problem that needs to be solved or the process that needs to be improved, as well as the environment in which the system will be used. This section could include a comparative evaluation of existing products, indicating why the proposed product is attractive and the advantages it provides. Describe the problems that cannot currently be solved without the envisioned solution. Show how it aligns with market trends, technology evolution, or strategic directions. List any other technologies, processes, or resources required to provide a complete solution.
Describe the needs of typical users or of the target market. Provide examples of how users would use the product. Define any known critical interface or quality requirements, but omit design or implementation specifics.
Summarize the important benefits the product will provide in a quantitative and measurable way. Platitudes (become recognized as a world-class <whatever>) and vaguely stated improvements (provide a more rewarding customer experience) are neither helpful nor verifiable.
Many projects suffer from scope creep (rampant growth as more and more functionality gets stuffed into the product). The first step to controlling scope creep is to define the project's scope. The scope and limitations help to establish realistic stakeholder expectations.
List the product's major features or capabilities. Think about how users will use the features, to ensure that the list is complete and that it does not include unnecessary features that sound interesting but don't provide value. Optionally, give each feature a unique and persistent label to permit tracing it to other system elements.
Focus on those features that will provide the most value, at the most acceptable cost, to the broadest community, in the earliest time frame.
If you envision a staged evolution of the product, or if you are following an iterative or incremental life cycle, build a release roadmap that indicates which functionality chunks will be deferred and the desired timing of later releases. Subsequent releases let you implement additional use cases and features, as well as enriching the capabilities of the initial ones. The farther out you look, the fuzzier these future scope statements will be and the more they will change over time. Expect to shift functionality from one planned release to another and to add unanticipated capabilities. Short release cycles provide frequent opportunities for learning based on feedback.
List any product capabilities or characteristics that a stakeholder might expect but that are not planned for inclusion in the product or in a specific release. List items that were cut from scope, so the scope decision is not forgotten.
For an example of business requirements, check out the corresponding section of the requirements document for Interchain Accounts.
A necessary prerequisite to designing software that meets user needs is to understand what the users intend to do with it. Some teams take a product-centric approach. They focus on defining the features to implement in the software, with the hope that those features will appeal to prospective customers. In most cases, though, you're better off taking a user-centric and usage-centric approach to requirements elicitation. Focusing on users and their anticipated usage helps reveal the necessary functionality, avoids implementing features that no one will use, and assists with prioritization.
User requirements describe goals or tasks the users must be able to perform with the product that will provide value to someone. User requirements describe what the user will be able to do with the system. The most commonly used techniques for exploring user requirements are use cases and user stories. Focusing on users and their anticipated usage helps reveal the necessary functionality, avoids implementing features that no one will use, and assists with prioritization.
The intent of this approach is to describe tasks that users will need to perform with the system, or user-system interactions that will result in a valuable outcome for some stakeholder. That understanding leads to deriving the necessary functionality that must be implemented to enable those usage scenarios. It also leads to tests to verify whether the functionality was implemented correctly. Usage-centric elicitation strategies will bring you closer to understanding the user's requirements on many classes of projects than any other technique.
Use cases and user stories work well for exploring the requirements for business applications, websites, kiosks, and systems that let a user control a piece of hardware. However, they might not be adequate for understanding the requirements of certain types of applications. Applications that have just a few use cases or applications where the complexity doesn't lie in the user-system interactions might not benefit much.
For an example of user requirements, check out the corresponding section of the requirements document for Interchain Accounts.
A use case describes a sequence of interactions between a system and an external actor that results in the actor being able to achieve some outcome of value. An actor is a person (or sometimes another software system or a hardware device) that interacts with the system to perform a use case.
A possible template for use cases could contain the following elements:
- A unique identifier and a succint name that states the user goal.
- A brief textual description that describes the purpose of the user case.
- A trigger condition that initiates execution of the use case.
- Zero or more preconditions that must be satisfied before the use case can begin.
- One or more postconditions that describe the state of the system after the use case is succesfully completed.
- A numbered list of steps that shows the sequence of interactions between the actor and the system (a dialog) that leads from the preconditions to the postconditions.
The list above is pretty extensive and makes for through use case descriptions. It is also possible to write fairly simple use cases and document the functional requirements derived from each one. This is is the approach we followed in the use cases for Interchain Accounts.
A user story is a short, simple description of a feature told from the perspective of the person who desires the new capability, usually a user of customer of the system. User stories are often written according to the following template:
As a [type of user], I want [some goal] so that [some reason].
The user story also identifies the user class and the rationale behind the request for that system capability.
User stories provide a concise statement of a user's needs. Use cases dive further into describing how the user imagines interacting with the system to accomplish his objective.
They should describe as completely as necessary the system's behaviors under various conditions. They describe what the engineers must implement to enable users to accomplish their tasks (user requirements), thereby satisfying the business requirements.
Software engineers don't implement business requirements or user requirements. They implement functional requirements, specific bits of system behavior. User requirements describe the user's perspective, looking at the externally visible behavior of the system. They don't contain all the information that an engineer needs to write the software.
Functional requirements describe the observable behaviors the system will exhibit under certain conditions and the actions the system will let users take.
For an example of functional requirements, check out the corresponding section of the requirements document for Interchain Accounts.
They describe the interfaces to other software systems, hardware components, and users. Ideally they should state the purpose, format and content of messages used for input and output.
For an example of external interface requirements, check out the corresponding section of the requirements document for Interchain Accounts.
Other-than-functional requirements that do not specify what the system does, but rather how well it does those things. Among others:
They describe the product's characteristics in various dimensions that are important either to users or to engineers and maintainers, such as performance, safety, availability and portability. One way to classify attributes distinguishes those characteristics that discernible through execution of the software (external quality) from those that are not (internal quality). External quality factors are primarily important to users, whereas internal qualities are more significant to development and maintenance staff.
For an example of non-functional requirements, check out the corresponding section of the requirements document for Interchain Accounts.
They place restrictions on the design or implementation choices available to the engineer. Constraints can be imposed by external stakeholders, by other systems that interact with the one you're building or maintaining, or by other life cycle activities for your system, such as transition and maintenance.
A popular software requirements template is the IEEE Software Requirements Specification template.
Stakeholder requirements are often written by individuals who are not experts in requirements definition. And most commonly, stakeholder requirements are written in unstructured natural language. Some of the problems that can appear in unstructured natural language requirement documents are:
- Ambiguity - a word or phrase has two or more different meanings.
- Vagueness - lack of precision, structure and/or detail.
- Complexity - compound requirements containing complex sub-clauses and/or several interrelated statements.
- Omission - missing requirements, particularly requirements to handle unwanted behaviour.
- Duplication - repetition of requirements that are defining the same need.
- Wordiness - use of an unnecessary number of words.
- Inappropriate implementation - statements of how the system should be built, rather than what it should do.
- Untestability - requirements that cannot be proven true or false when the system is implemented.
Requirement statements should be instead:
- Complete: each requirement must contain all the information necessary for the reader to understand it.
- Correct: each requirement must accurately describe a capability that will meet some stakeholder's need and must clearly describe the functionality to be built.
- Feasible: it must be possible possible to implement each requirement within the known capabilities and limitations of the system and its operating environment.
- Consistent: consistent requirements do not conflict with other software requirements or with higher level (system or business) requirements.
- Unambiguous: there should only be one way to interpret the requirement for any reader.
- Prioritized: they should be prioritized according to which are most important to achieving the desired value and indicating how essential they are to a particular product release.
- Traceable: it should be possible to link each requirement to its source, which could be a higher-level system requirement or a use case. Also, link each software requirement to the design elements, source code, and test cases that implement and verify it. Traceable requirements are uniquely labeled and written in a structured, fine-grained way.
- Verifiable: it should be possible to devise tests or other verfication approaches to determine whether each requirement is properly implemented. If a requirement is not verifiable, deciding whether it was correctly implemented becomes a matter of opinion.
Requirements can be written from the perspective of either something the system does or something the user can do. Because effective communication is the overarching goal, it's fine to intermingle these styles, phrasing each requirement in whichever style is clearer. State requirements in a consistent fashion, such as
The system shallorThe user shall, followed by an action verb, followed by the observable result.
EARS provides different requirement template constructs:
-
Ubiquitous requirements:
- They are not invoked by an event detected at the system boundary or in response to a defined system state, but are always active.
- Syntax:
The [system name] shall [system response].
-
Event-driven requirements:
- They are initiated only when a triggering event is detected at the system boundary.
- Syntax:
When [optional preconditions] [trigger event] the [system name] shall [system response].
-
Unwanted behaviour:
- They are used to cover all situations that are undesirable. This includes failures, disturbances, deviations from desired user behaviour and any unexpected behaviour of interacting systems.
- Syntax:
If [optional preconditions] [trigger], then the [system name] shall [system response].
-
State-driven requirements:
- A requirement that is active while the system is in a defined state.
- Syntax:
While [in a specific state] the [system name] shall [system response.
-
Optional requirements:
- A requirement that is applicable only in systems that include a particular feature.
- Syntax:
Where <feature is included> the <system name> shall <system response>.
For requirements with complex conditional clauses, combinations of the keywords When, While and Where may be required. They can also be used within If-Then statements to handle unwanted behaviour with more complex conditional clauses.
It encompasses practices that help you deal with requirements after you have them in hand. These practices include version control and baselining, change control, tracking requirements status, and tracing requirements to other system elements. Requirements management will take place throughout the project's duration at a low level of intensity.
A requirements baseline is a set of requirements that stakeholders have agreed to, often defining the contents of a specific planned release or development iteration. At the time a set of requirements is baselined, the requirements are placed under configuration (or change) management. Subsequent changes can be made only through the project's defined change control procedure.
In addition to textual description, each requirement should have supporting pieces of information or attributes associated with it. These attributes establish a context and background for each requirement.
Following is a list of potential requirement attributes to consider:
- Date the requirement was created.
- Current version number of the requirement.
- Author who wrote the requirement.
- Priority.
- Status. See table below for a list of possible requirement statuses.
- Origin or source of the requirement.
- Rationale behind the requirement.
- Release number or iteration to which the requirement is allocated.
- Stakeholder to contact with questions or to make decision about proposed changes.
- Validation method to be used or acceptance criteria.
It's valuable to keep a record of rejected requirements and the reasons they were rejected. Rejected requirements have a way of resurfacing later during development or on a future project. You don't need to use all of the possible statuses from the table; choose the ones that add value to your requirements activities.
Trace links allow you to follow the life of a requirement both forward and backward, from origin through implementation and testing. Trace information documents the dependencies and logical links between individual requirements and other system elements. These elements include other requirements (i.e. business, user, etc), architecture and other design components, source code modules, tests, etc. Trace information facilitates impact analysis by helping you identify all the items you might have to modify to implement a proposed requirement change. For requirements to be traceable, each one must be uniquely and persistently labeled so that you can refer to it unambiguously throught the project.
Customer needs are traced forward to requirements, so you can tell which requirements will be affected if those needs change during or after development. Customer needs could be articulated in the form of user requirements. A complete set of forward traces also gives you confidence that the requirements set has addressed all stated customer needs. Conversely, you can trace backward from requirements to customer needs to identify the origin of each software requirement.
As requirements flow into downstream deliverables during development, you can trace forward from requirements by defining links between individual functional and non-functional requirements and specific system elements. This type of link allows you to determine that you've satisfied every requirement because you know which design components and code elements address each one. The fourth type of link traces specific product elements backward to requirements so that you know why each element was created.
Many kinds of traceability relationships can be defined on a project. Of course, you don't need to define and manage all these trace link types. On many projects, you can gain most of the traceability benefits you want for just a fraction of the potential effort. Maybe you only need to trace system tests back to functional requirements or user requirements. Perform an analysis to decide which links will contribute to the success of your project, both in terms of development and long-term maintenance effort.
This is a list of practices that can be used in each stage of the requirements engineering process; it doesn't mean that all of them need to used in every situation. So use good judgment, common sense, and experience to decide what suits.
