Measuring Product Ownership – What does “Good” look like?

Measuring Product Ownership – What does “Good” look like?

You are currently viewing Measuring Product Ownership – What does “Good” look like?

In 2009 I first published Scrum Product Ownership. In 2013, I followed it up with a second edition. The book has been a popular read for those who are looking for a solid overview of what it takes to be a competent and craft-focused Product Owner.

Here’s what a new Product Owner from Spotify had to say about the book:

“I was recommended your book “Scrum Product Ownership – Balancing Value from the Inside Out” by senior colleagues at Spotify as the one book to read when new to product owning. After recently finishing reading it, I fully agree and will keep recommending your book to anyone getting started as a product owner.

I just wanted to say thank you for making the start of the ride less bumpy and for great advice that I will keep returning to as I gain experience.”

I share this because it helps to set the stage for this article and where my inspiration lies.

When I published the second edition, I also released an assessment framework for the role of Product Owner. The framework is loosely based on Bill Kreb’s Agile Journey Index, which I wrote about in more detail here.

In this article, I want to explore the categories of focus for the assessment tool. I’m sharing as a way of either:

  1. Inspiring you to use the tool in assessing your Product Owner organizational maturity OR
  2. Leveraging it to construct your own tools for measuring you Product Ownership maturity

In either case, my hope is to give you a framework to get you thinking about the dimensions of what good looks like, when it comes to Agile/Scrum Product Ownership.


So that you have a baseline of practices, techniques, or approaches that are important for a well-rounded and balanced Product Ownership practice. Not for grading or comparing one PO against another. Please don’t do that.

But simply as a tool for establishing what good might look like in your organizational context and then using it to drive continuous learning and continuous improvement across your Product Owners.

There are four areas where I have gathered what I consider to be crucial Product Ownership practices:

  1. Table Stakes
  2. Basic Practices
  3. Communications
  4. Steering

Next we’re going to explore each area in turn – examining the practices within each in high-level details so that you get a feel for the overall assessment framework. If at the end of the article, you’re intrigued and want to learn more, then I’ll share how to get more details on the framework.

Let’s get going…

Table stakes are just that – things that you should be investing in right at the beginning. First is the investment in Product Ownership, or more specifically, Product Owners in your organization. Do you have a PO per team and is it their primary responsibility? Often organizations struggle with the move to Scrum because of the initial “investment” in Product Owners and Scrum Masters. But it’s this very investment that’s a table takes item.

Next we’ll explore one of the most prevalent requirement artifacts being used in agile teams – the ubiquitous user story. They have become an incredibly popular way to craft agile requirements, but beyond the stories, they illustrate the elements of agile requirements in general. That is that they’re essentially just-in-time and ambiguous in nature. They are essentially intended to evolve until they are delivered and the Product Owner is a shepherd of the stories.

And the final table stakes element is the Product Backlog. There’s an incredible amount of confusion surrounding this simple list of “things to do”. Is it requirements/stories or something else entirely? For example, is it the beginnings of a plan for your release? The short answer is that it is – what it needs to be in order to effectively guide the teams’ deliverables. And the Product Owner is the head guide.

Product Owner

  • Do you have one? Do they have the training, skill and understanding necessary to do the job?
  • What about domain experience? And organizational experience?
  • Do they have sufficient time to invest in their role? Both the inward and outward components.
  • Are they empowered to make business decisions and effectively guide their team?

User Stories

  • Are you even using lightweight user stories or do you continue to hold to traditional, heavy weight requirements?
  • If you’re using stories, do you understand the “three parts” of a user story and how to write them well?
  • Do you have the notion of entry or readiness criteria?
  • And are you heavily investing in acceptance criteria as a means of communicating the business “what” and “why”?

Product Backlog

  • Multi-faceted prioritization between the business, team, and technology needs?
  • Does the Product Owner ‘own’ the backlog or does the team?
  • Is backlog refinement actively occurring and are stories being examined 3-4x as they evolve towards being executable?
  • Does the Product Owner do sufficient longer term look-ahead so that the team understands “where they’re going”?

Clearly the Product Owner doesn’t play a direct role in team estimation. However, they foster an environment via backlog refinement, where the team estimates backlog items. But beyond the teams’ estimates, their participation can give them all sorts of insights into level of complexity and level of effort for the types of features they’ll be asking the team to deliver. This information can easily be factored into product strategies and workflow.

Valuation (prioritization) is one of those areas that I’ve found to be really hard. Most Product Owners are intelligently guessing, with their stakeholders, around what the customer really needs. More deterministic approaches for feature valuation, and importantly, measuring usage post-release, are often non-existent. I’d separate these challenges from trying to take sufficient time in effective prioritization from multiple perspectives – customer, team, and technology.

And finally, goal setting is one of those under focused activities within many agile teams. Which is unfortunate, because it’s ultimately how you “drive results” from your team. Not by telling, but by painting a vision and setting goals for them to envision and achieve. In traditional organizations, goals typically come from “management”. In the agile organization, this responsibility lies with product ownership. And these are expansive or inclusive sorts of goals. For example: yearly roadmaps, quarterly release objectives, release feature lists, and down to the level of individual sprint goals.

Whether you acknowledge it or not, product ownership is largely a factor of setting goals for your teams to achieve and then measuring goal achievement at the end of iterations.


  • Avoiding non-team based or management/leadership based estimation of any sort. Instead, looking to the team for lower level estimation (stories sized for sprints) and higher-level forecasts.
  • Leveraging one of the whole-team estimation practices (for example: Planning Poker) as a means of collaborative estimation. Consistently fostering an “all voices” style in estimation.
  • Allocating time and space for sufficient story spikes to determine true size and scope instead of the work. This could be for technically and/or business functionally complex items.


  • Have a balanced strategy for valuating (prioritizing) the Product Backlog that includes business, customer, and technical drivers. Involving stakeholders directly in the process (value poker).
  • Unafraid to break down elements, minimizing scope to increase experimentation, speculation, and fact-finding to increase understanding of value.
  • Release criteria contain ROI measures that have pre-release goals and post-release actuals. Learning and adjustments are made based on real team results and retrospective feedback.


  • Firstly, do you consistently have goals both at a sprint and a release level? Are the clear and well understood? Have you engaged the team in making them feasible and reasonable?
  • I consider the Definition-of-Done to be a unifying goal of sorts connecting the team’s work to stakeholder expectations.
  • Is goal attainment something that’s kept hidden or obfuscated or is it full transparent. And how much “wiggle room” have you built into the goals so the teams have flexibility to be innovative and creative?

I consider communication to be one of the prime directive elements of solid product ownership. Now an important part of tat communication is leveraging the natural Scrum ceremonies/meetings to their full effect.  One that I think is particularly important is the sprint demo or review. This is THE event for a Scrum team to show their results and speak to progress and future plans. Sometimes I call that “connecting the dots” for their stakeholders.

A good Project Manager usually pulls together something called a communications plan. This plan highlights the communication channels, responsibilities and people for the project. It’s usually too heavy weight for most agile projects, but the intent of the thing is very sound. In Scrum, most if not all of this communication falls to the Product Owner. Here we explore aspects of effective communication.

While it might strike you as odd, I include listening as a crucial part of communication. Perhaps unfairly because I’m using my own style as a factor – meaning: I talk too much. So I want to remind Product Owners that they need to continuously sharpen their listening skills. And an important part of listening is observing “body language” and what isn’t being said as well.

Sprint Review

  • Are we continuously demonstrating working software, while absolutely adhering to our Definition of Done?
  • Has there been sign-off and acceptance of all of the work. And did the team meet their sprint goals?
  • Are key stakeholders always in attendance for the sprint review? And not just in attendance, but full engaged, asking question and providing feedback?
  • Are you tying the results to your release plan? Connecting the dots if you will to previous sprint results and forward to what you’re planning to do next?


  • Are effective information radiators in place and is the organization “listening” to them? One way to measure this is if you’re getting questions around your teams’ artifacts.
  • Does the team share congruent, open, and honest feedback in the sprint retrospectives? One of the key challenges is for the Product Owner to not be too “pushy” regarding business needs, which stifles communication.
  • Is the Product Owner an active “publicist” for the team down, around, and up within the organization? Do senior leaders and key stakeholders always come to the PO to find out where things stand?


  • Engagement levels in retrospective. Bringing personal, business, and value views into the teams purview for improvement consideration
  • Actively incorporating team feedback into the Product Backlog via stories, ordering, strategy, and efforts to address technical debt.
  • Attend sprint and release planning meetings. Listen attentively to team discussions around technical design, trade-offs, architecture, and level of complexity. Both from a development and testing perspective.

The steering practices are just that – focused towards steering the efforts of the team towards business and customer-centric goals.

The first area of consideration is product mentoring and establishing a product owner centric Community of Practice. This is where standards are kept and lessons learned are shared. Where product owners mentor each other towards consistency in execution, but also in guiding and leading the entire organization.

Another critical steering area I call envisioning. This is where projects are essentially chartered and the project level goals, constraints, and conditions of acceptance are identified and explored. If you are familiar with the practice of Story-Mapping, it would be an envisioning practice for the Product Owner and their team(s).

Finally, release planning towards establishing and committing to road-maps is the final envisioning practice. Not only is this for the team to establish a shared view towards their strategies, but it’s necessary for the Product Owner to communicate the game plan and commitments to stakeholders and customers.

Far too often, agile teams “dive in” to projects and initiatives without truly plotting where they are going. The steering practices focus on avoiding that common anti-pattern.

Product Mentoring and Community of Practice

  • A CoP is established to guide consistency across Product Ownership artifacts (backlogs, stories, road-maps, etc.) and activities (story writing standards, grooming / refinement meetings, release planning, etc.)
  • Product Owners collaborate as a team. To use Spotify terms, they form a “Chapter” that is intended to guide the practice of Scrum product ownership. In other words, they actively pair and mentor one another.
  • There is a tendency within agile contexts to reduce or remove standards and templates. Here the product organization established healthy and balanced standards that aren’t too restrictive to team performance.


  • Mission and vision for all project-level activity is clearly communicated by leadership and then activated by the product organization. The teams need to be able to “see” the mission and vision.
  • Some form of project chartering is used to begin or instantiate each project initiative. Often this is a prerequisite for release planning initiatives. (in SAFe, there is a clear notion of chartering within the PSI Planning activity. This is a good example of the point…)
  • Notions of Minimal Marketable and Minimal Viable are regularly used to communicate the focus and goals of the project. Included with this effort is Story-Mapping to envision how the functionality will unfold and solve the customers’ challenges.

Release Planning & Road-mapping

  • Here we move from looking at the Product Backlog as a “series of requirements or features” to a dynamic flow of work directed towards achieving the businesses’ objectives.
  • Having a balance between features, technical investment, and technical debt buy-down activity. Another part of this is committing to fight “feature-it is” as the only way to load the backlog.
  • Release planning is a whole team activity that leads to road-mapping and commitments to the business regarding deliverables. Point being, the team is engaged in the envisioning, delivery planning and commitments. And the business realized that there is variability in the commitments.

There you have it. We explored four areas of product owner maturity and three central practices within each. The intent of this article was to expose you to my AJI-extensions for Product Owners. Remember it’s a framework and intended for healthy improvement of your practices.

Not only am I hopeful that I made you think about Product Ownership maturity, but I hope I inspired you to download the assessment tool.

And not only to download it, but also to read through it and to possible use it in your organization.

And please remember, this is an iterative framework and I’m constantly learning and adjusting it. If you use it and have feedback towards improvement, ANY FEEDBACK, please take the time to send it to me. I promise to carefully consider it and incorporate it in the next edition of the framework.

Stay agile my friends,


Leave a Reply