As the Product Owner of ScrumWorks, I'm often asked why ScrumWorks does not support hierarchical nesting of backlog items (i.e., user stories, PBIs, etc.). The answer is simple but highlights some profound differences in the way requirements are managed in Scrum and Agile as opposed to other methods.
The short answer is prioritization. Scrum is predicated on the fact that all work is listed in a linear, one-dimensional list and prioritized relative to other items on that list (see Agile Software Development with Scrum for more on the Product Backlog). The relative priority of items is always clear and there are no confusing situations like two backlog items with the same priority (e.g., #1!) that both need to be done (e.g., now!). Rather, the Product Owner has the hard job of distinguishing which items are more important than others.
So why can't backlog items be organized into tree-like hierarchies? You can easily prioritize the top level item, but often the top-level of the hierarchy represents a large amount of effort. So much so that the item itself may span sprints or even several releases. So how does one prioritize such a large item that will be done over several sprints and possibility as part of several releases? The problem is that your hierarchy actually contains individual backlog items with differing priorities. Some might be done in the first few sprints, while others are not as important (gold-plating) and might be done in a later sprint or even a later version of the product. The hierarchy is obscuring the true priority of the individual items.
There is a second reason why taking a hierarchical approach to requirements is less desirable in Scrum: it encourages product owners/managers to prematurely define detailed requirements. With a hierarchical system, the Product Owner is encouraged to break work down into very granular detail far before the feature defined in the requirement is needed in the product. This is counter to the lean principle that suggests a just-in-time approach to requirements that encourages product owners to define the real requirements when they are needed just in time for a sprint.
Scrum and Agile propose a new paradigm: use large high-level backlog items (often called epics) and peel off smaller items from that large Epic as the need arises and priority demands. Pretty soon, your epic dissolves (in terms of effort points) because it is split off into many smaller items. This is why CSM courses teach that you not split your backlog items into very granular items until they become a priority for the product. You will have an unwieldy backlog. Not to mention, this is not very agile/lean: you're spending a lot of time and effort detailing requirements you may never actually need or build. Instead, keep distant, low priority items as large "epics" so that your backlog remains manageable.
How does ScrumWorks handle categorization? Themes represent a free tagging system that allows users to apply keywords, or Themes, to individual backlog items. Themes are meant to keep the association between split off items and the original epic. Simply theme all split off items to keep them related. In the future, we're considering building a split-history/traceability function that shows how a particular epic was split over time, the relationship of the splits, when items were created, finished, deleted, etc. I've attached a mock-up (login required) of what the report might look like. But in essence this report provides a roll up of how a requirement evolved and it's current status.
The short answer is prioritization. Scrum is predicated on the fact that all work is listed in a linear, one-dimensional list and prioritized relative to other items on that list (see Agile Software Development with Scrum for more on the Product Backlog). The relative priority of items is always clear and there are no confusing situations like two backlog items with the same priority (e.g., #1!) that both need to be done (e.g., now!). Rather, the Product Owner has the hard job of distinguishing which items are more important than others.
So why can't backlog items be organized into tree-like hierarchies? You can easily prioritize the top level item, but often the top-level of the hierarchy represents a large amount of effort. So much so that the item itself may span sprints or even several releases. So how does one prioritize such a large item that will be done over several sprints and possibility as part of several releases? The problem is that your hierarchy actually contains individual backlog items with differing priorities. Some might be done in the first few sprints, while others are not as important (gold-plating) and might be done in a later sprint or even a later version of the product. The hierarchy is obscuring the true priority of the individual items.
There is a second reason why taking a hierarchical approach to requirements is less desirable in Scrum: it encourages product owners/managers to prematurely define detailed requirements. With a hierarchical system, the Product Owner is encouraged to break work down into very granular detail far before the feature defined in the requirement is needed in the product. This is counter to the lean principle that suggests a just-in-time approach to requirements that encourages product owners to define the real requirements when they are needed just in time for a sprint.
Scrum and Agile propose a new paradigm: use large high-level backlog items (often called epics) and peel off smaller items from that large Epic as the need arises and priority demands. Pretty soon, your epic dissolves (in terms of effort points) because it is split off into many smaller items. This is why CSM courses teach that you not split your backlog items into very granular items until they become a priority for the product. You will have an unwieldy backlog. Not to mention, this is not very agile/lean: you're spending a lot of time and effort detailing requirements you may never actually need or build. Instead, keep distant, low priority items as large "epics" so that your backlog remains manageable.
How does ScrumWorks handle categorization? Themes represent a free tagging system that allows users to apply keywords, or Themes, to individual backlog items. Themes are meant to keep the association between split off items and the original epic. Simply theme all split off items to keep them related. In the future, we're considering building a split-history/traceability function that shows how a particular epic was split over time, the relationship of the splits, when items were created, finished, deleted, etc. I've attached a mock-up (login required) of what the report might look like. But in essence this report provides a roll up of how a requirement evolved and it's current status.
| Attachment | Size |
|---|---|
| Traceability_SH_1.png | 36.27 KB |
| Hierarchical Requirements and Scrum blog.pdf | 152.84 KB |
Moving a large organization with 275 developers spread around the world who are maintaining and improving an 11.5MLoC embedded real-time application to Scrum requires levels of coordination that we think we are creating from scratch as no books, consultants (we've hired Ken Schwaber and others) or tools we've reviewed go into detail about the issues we face. Yes, we use hierarchial requirements: Goals, User Stories and Action Requests.
We have Goals that are received from many sources ranging from new features actual paying customers want to infrastructure repair work. With many customers world wide using our equipment in ways we never dreamed possible we have corporate departments working to find the nuggets in the barrage of feature requests that will give the most value to our best paying customers. Our Engineering System Managers add infrastructure improvement Goals to both spackle over past sins and move us to where a system can be built and at least 'smoke' tested in less than two to three days. Our Management and Marketing teams priorize Goals. Our Product Owner Proxies work those issues to prioritize User Stories with the teams. The contention for prioritization is a constant on many levels. The twenty two Scrum teams have to work from one database that holds the Goals as well as the User Stories so we can meet both internal (to the tools team) requirements of giving a single view of all work being accomplished looking down and smaller more discrete views to those working at prioritizing long lists during each team's sprint planning meetings and using those lists during the sprint. ARs are units of work (changes) added to the code base. We use Themes on top of this hierarchy to group work that takes many goals, user stories and sprints to accomplish such as adding a new family of devices.
We are migrating our tools from our current set because we can find nothing that meets the complex needs of a large organization trying to do Scrum. We are creating new processes around the Scrum of Scrums because we can find no literature or help from anyone who has faced the same issues of coordinating so much work before.
Is the Scrum of Scrums its own team? Are the members the ScrumMasters or representatives from each team? Does it have a ScrumMaster? Are the POPs pigs or chickens in the SoS? Is its product backlog the cross-team barriers and coordination issues (we think it is, right now)? Who is the SoS's Product Owner (currently a wide open question)? Our SoS's product backlog varies constantly and requires a significant amount of work by a large number of people; how is this to be factored in at the sprint planning meetings?
Nick Nicoll
DocuSP Problem System Admin