Reporting on product performance has three powers:
Explanatory power is the ability of reporting to explain what has previous happened.
Predictive power is the ability of reporting to generate testable predictions about what may happen.
Narrative power is the ability of reporting to tell a story that helps people understand what happened and what might happen.
I used to start the day with a short planning session to pull together a list of the things I wanted to do that day. Sometimes I’d get everything done, most times I wouldn’t. New things would pop up, older things would be become irrelevant. But the problem wasn’t whether or not I got stuff done, it was that I had no feedback loop to tell me whether I was doing the right things. So I redesigned my entire approach.
We humans are pretty terrible at predicting the future and estimating how long a task will take, so this approach focuses on recording what I actually did and then trying to correlate it back to whether I’m achieving what I want to. From this I can learn how to set better goals and make sure I do the work to achieve them.
This is how it works.
The whole point of this system is to answer three questions:
I try to answer those questions over three different timescales; daily, weekly and monthly. I hope this gives a more balanced view of the answers to those questions and doesn’t favour one over another.
At the start of the month, I write a goal for each of the projects I’m working on.
At the end of the month, I review whether I achieved the goals or not and colour code them to make it easy to tell. I use green for completed, orange for progressed, and red for didn’t achieve.
This is the longest of the loops for the biggest goals. It’s the hardest to connect with the daily work.
At the start of a week, I write a short description of what I want to do over the week for each project. In fact, I started with doing this for five projects, partly because I know they are moving quickly and I have clearer idea about their direction, and partly to start small.
At the end of the week, I review whether I achieved what I set out to or not and colour-code in the same way as the monthly goals.
This is the middle-sized loop. It’s easy to see whether the work done this week has achieved the weekly goal. I use this to help me write the right goals next week.
Each morning I check my tasks list and yesterday’s done list for things I should do today. Some are things that have to be done, other’s are more flexible and I’ll get to them if I get time. This gives me a bit of short-term guidance and helps me not miss things.
Throughout the day, I make a note of each thing I work on for each project. Sometimes it’s a small thing like a quick chat, sometimes it’s a meeting, sometimes it’s a bigger thing like spending a few hours writing a document.
A formula in my spreadsheet counts these and displays them on a heatmap of tasks done for each project each day.
At the end of the day I do a quick check on which projects I worked on, how many things I did, and how it compares to the average. This is the shortest feedback loop and helps me answer my three questions.
I’ve been doing this for a month, or 22 working days.
I averaged 8 and a half tasks a day. 17 on my busiest day and 3 on my least busy days.
It’s easy to see which projects haven’t had much done on them – they are the long lines of red and low total. You can also see where there have been spikes of activity on a project with a few things getting done in one day. This helps me understand whether I’m focusing my efforts on the right things.
Some projects show constant activity and others a weekly pulse of activity. This helps me understand how different projects require focus at different times.
There’s also a weekly pattern of Monday’s being busiest and tapering off over the week. This helps me think about when to do certain types of work – Monday’s for syncing type work, Friday’s for deep work.
I would say yes, I am working on the right things. The average number of tasks per project is 13.8 and the 50% of projects that are above that are the ones that are higher priority (each for different reasons, but higher priority still) than the bottom half. This seems like a good, although pretty blunt, measure to show.
Thinking about what the table tells me about whether I spent the right amount of time on the right projects, I’d say that it broadly does but only because I have additional knowledge about the projects such as which are higher priority, which have dependencies, and which I could become a blocker on. Maybe I should think about including that contextual information.
Over the past two weeks I completed 1 thing, progressed 7 things, and didn’t complete 3 things. Given that I only managed a 10% success rate in the last fortnight, I’d say there isn’t a strong connection between the tasks I’m doing each day and the goals I set at the beginning of the week.
This goes back to the point above about not being very good at predicting the future, not having enough time for five projects (let alone 15). There’s also an important point about some projects being more dependent on other people who are also busy on other things, and doing work that unblocks them but doesn’t achieve my goals.
Lots more thinking required to properly connect the daily work to the bigger goals.
Using the average number of tasks across all projects as the middle benchmark and then ranking projects by how far above or below average they are might give a quick way of seeing if the number of tasks correlates with the priority of the project.
I started and ended the month with the same projects, so nothing new was added. That is a bit of a WIP control, even if the overall number is too high. That some projects don’t get much done but stay on the list shows that there are too many projects to make meaningful, regular progress.
Maybe the table needs a way to differentiate between projects that are in progress but not having any work done and projects that are complete, as at the moment they both show as red.
I’ve decided not to do this so far because I think it’ll drive the wrong behaviour. It’ll make the work more quantifiable and that’s too easy to mistake for value. Sometimes a couple of chat messages that take three minutes is more valuable than three hours spent analysing data. So, if I do start tracking my time I’ll have to think carefully about the right way to do it.
There’s no shortage of frameworks and processes, models and methods in product management. And they are all good thinking tools, conversation starters, and learning opportunities for teams to develop their practices. But as teams develop those practices it’s good for them to hold onto the lessons they learned from using the frameworks but be freed from following a fixed process.
Good products teams learn to play jazz.
They don’t need orchestration. They don’t have a central controlling role. No one leads the team or assigns the work. The team does this themselves. They decide together.
They don’t need a plan. They have the confidence to develop the next step based on what they doing together now. They bounce off each other, develop new ideas, try things, drop things that fail, get out of sync and back in sync. They adapt their work as things change.
They don’t work in isolation. They understand each other’s skills and perspective, share what they’re working on and change it as others change what they’re doing. Their ability to actively respond to each, to adapt, comes from a mutual respect and encouraging interplay. They jam together.
They don’t ever stop practicing. They are constantly learning, improving their skills and their understanding of each other’s skills. They revisit foundational knowledge and introduce new opportunities to learn. They turn to guiding principles rather than strict rules. Their knowledge becomes intuition.
It’s this ability to improv that shows the maturity of a team better than any output metric. Because the ability to improv only evolves with psychological safety, team cohesion, and all the other things that are necessary for good teams, it is the best indicator of a mature and successful product team.
Why is the Give Blood app so bad? I don’t know, but here are some improvements I’d make.
Make the phone number a tel link so I can call with a click.
Is this the most important information I need to see every time I open the app? Don’t think so. Move it to the About you section.
Earliest donation date should be replaced with a ‘You can donate now’ message.
What’s the primary call to action here? It should be to ‘Book an appointment’ but ‘See why you’re a First Responder’ is more prominent. And I read it, so why do I need to see it on the home screen every time I log in?
Hide the ‘Your upcoming appointments’ if I don’t have any upcoming appointments. It’s a distraction from getting me to book an appointment.
Minor detail I know, but the search box and button should be the same height.
Venues can be filtered by dates but even when ordered by nearest there is no way to change the default distance for the search (which seems to be 10 miles).
Every venue has a ‘View on a map’ link which opens Google maps in a browser and uses the coordinates to find the place. This is where the link from the venue ‘Aylesbury Methodist Church’ goes, to Elsinore House. That’s confusing.
Another example of how text behaves badly when the device text size is zoomed.
Even though a venue is shown in the search results, when going to pick pick an appointment time it shows as having no dates available. Don’t show it in the search results.
This screen introduces a new user interaction – the plus/minus reveal that doesn’t look like something to interact with. All the other screens with blue text links open another screen, this is the only one that expands on the same screen, so it’s a bit inconsistent.
With the dates set from 29 August 2023 to 1 December 2023 there are 19 venues with appointments, but…
… if I expand the date to 31 August 2024, there are only 7 venues with available appointments. How can there be fewer venues with appointments for a shorter time period?
Always test how text behaves when the device has font size zoom set high.
And always allow your text to line break.
No previous donations are shown, which is fine but hide the heading.
Stop work on the thing that is dependent until the dependency is removed. Deliver no value during this time. Learn nothing during this time. Start the work again when the dependency is removed.
This way is efficient as it only does work that will be kept, nothing gets thrown away.
Separate the work from thing it is dependent on. Do the work in another way. Deliver value sooner. Learn more. Redo the work when the dependency is removed.
This way is effective as it enables users to get value and the team to learn.
I like to know what’s going on with people I respect, what they’re thinking about, and stuff they’ve learned. But social media is an unreliable way of finding what they’ve written recently. The algorithms make it impossible to know what you’ve missed. So, I wanted to create a reliable, timely and free way of finding out what these people have published.
Lots of personal website and blogging platforms, along with sites like YouTube have RSS feeds, which is a great way to get links to newly published content. Unfortunately, not all do, so I guess there’s still some people’s writing that I’ll miss. No solution is perfect.
I looked at using automation products like Zapier and IFTTT, but there are two drawbacks. The free plans allow a very small number of feeds, and until I’ve proved the value for myself I don’t want to spend too much on this. And I’d still need somewhere else to display the feeds that could notify me and organise the feeds, and as I’m still validating whether the idea works I want to keep the system as simple as possible. It’s good to know your constraints.
Slack, as it turns out, is a great solution. It has an RSS app that can post into channels, different channels let you organise messages, and message features help you
I set up a workspace just for myself, and created channels for ‘product management’, ‘agile’, ‘ai’, design’, strategy’ and ‘weeknotes’. As my interests change, I can add and remove channels without disrupting the whole solution. And if there’s someone who’s writing doesn’t fit a single topic, I add their feed to the ‘general’ channel.
Slack has other features that help with the content discovery experience too. Messages can be bookmarked or have a reminder set for later. I also use different emoji’s to help me. A tick means I’ve read the blog post. Thumbs up means I really liked it and might want to come back to it later.
Slack is one of the few apps that I enable notifications on my phone, but it means I’ve got belt and braces for not missing new content. I get a notification when someone posts a new article or video, and if I’m too busy to read it then, I’ve got unread messages next time I go into the Slack app so I can read them then.
What could be better? The main thing would be a way to tidy/select the contents of RSS message before it’s posted. Substack is one of the cleanest, but there’s still messy duplication. Ideally, I’d see just the name of the publication and the title of the article which links to it. That would mean less unhelpful, unfinished text and less scrolling.
I’ll keep adding to this list as I add more feeds to my Slack.
I’d like to find a way to decouple ‘conversation’ from social media. I don’t converse a lot on social media or any other platforms, but I’d like to know that I have a reliable way of doing so if I wanted to.
It would be great if social media platforms introduced RSS feeds so, for example, if someone posts on LinkedIn or Threads I could add replies, but I don’t think that will ever happen. And other platforms like Substack and Medium aren’t well designed for conversation. Another problem for another day.
Roadmaps are best created using deductive reasoning, starting with the end in mind, and with the outcomes and opportunities for achieving them. There’s lots written about outcome-based roadmaps and how to create good ones.
But, often, instead of outcome-based roadmaps, which as good as are for product managers require a certain amount of familiarity to make sense of, many organisations have feature-based roadmaps or project delivery plans that are called roadmaps.
Of course, product managers can help others to understand the benefits of outcome-based roadmap but that might take some time.
But maybe there’s another way.
Maybe there’s a way to take the outputs expressed in those plans and translate them into outcomes.
Let’s start with a website, like the one you’re on now. And a random list of changes that I could do:
These are all outputs. Not an outcome in sight. And no prioritisation either. This could be any website, any product or project. So, what’s a product manager to do?
To get to outcomes, we need to group them by what they seem like they are trying to achieve. As we’ve said, if we were starting with what we know we want to achieve, we’d be creating an outcomes-based roadmap, but instead we need to make some assumptions about those outputs which we can test later.
It seems to me that most of the things on that list are about organising stuff on the website; changing the menu, reorganising content, providing a different way of interacting with info on the site. So, we’ll want an outcome that includes all of those. The one that doesn’t quite fit is writing blog posts. That isn’t about organising stuff. Maybe it’s about getting people to the website.
Now, as we’re talking about outcomes, we’re talking about changing user behaviour, so let’s express our outcomes as:
Then I can group the outputs together by which outcome they seem most likely to achieve.
| Outcomes | Outputs |
|---|---|
| Get more people to visit the website. | Write more blog posts. |
| Make it easier for people to find information on the website | Add a chatbot trained on all the content on the website. Redesign the home page. Change the menu, e.g., include weeknotes and resources. Reorganise the Projects section. Add a wiki on a subdomain. |
So, we’ve taken a list of random ideas, made some assumptions about what those things are trying to achieve and grouped them, and phrased an outcome for them. Now we’re ready to test our assumptions. This is important. The difference between a guess and an assumption is that assumptions are called out and tested to try to figure out if they are true. Guesses aren’t, they stay as guesses. Product managers, in their attempts to bring order to chaos and empirical rigor to the messy real world, test their assumptions.
We can ask the people that contributed to the list of outputs whether they agree that those outcomes are really what they are trying to achieve. If they say yes, then our assumptions are validated and hopefully we’ve been able to have a helpful conversation about outcomes. If not, perhaps those people can help create new assumptions about what to achieve and how to group them, and we still get to talk about outcomes.
You’ll notice that one of our outcome has only one output, and the other has lots (almost like I planned it). So, to get more people to visit the website we can get on with writing more blog posts, that’s obvious. But to make it easier for people to find information on the website, there are lots of thongs that we could do, so we need to choose between them.
Ideally, we’d be able to work on both outcomes at the same time, but if we have to choose between them then we should choose by which provides the most value to the user. In this case, that’s probably writing more blog posts to get more people to the website. How do we know this? Analytics are a product manager’s best friend, and our analytics tell us that the majority of traffic comes from search engines and goes directly to blog posts. Therefore, more blog posts helps more people get to the website.
We also know that the average page view is 1.3, which tells us people aren’t navigating around the site, which back-ups the need for the second outcome and gives us a measure.
For a product manager, just delivering those outputs isn’t enough. That’s fine for project managers but product managers need to know if those outputs are achieving the outcomes.
Measuring whether writing more blog posts correlates with getting more people to visit the website (the outcome) is a far more useful indictor of success than just how many blog posts were written (the output).
This is where we get to the ‘but why’ of outcomes. If one output doesn’t achieve the outcome, then maybe the next one will. The outcome is still worth pursuing because it is closely tied to value. If writing more blog posts doesn’t achieve the outcome of getting more people to the website, we can think of other things that might. Maybe posting on social media or starting a newsletter. Outcomes are more long-standing, and open to new opportunities of outputs being added over time to try to achieve it.
If, tomorrow, someone comes up with another great idea, it can be tested against the existing outcomes to see where it fits. If the output they suggest doesn’t help to achieve the current outcomes, then it’s a far easier conversation to explain why it shouldn’t be worked on. And if it doesn’t fit, then we’ve made it easier for people to bring ideas without having to do the product thinking of starting with outcomes.
Product management is a discipline. It is responsible for the entire lifecycle of a product.
Product development is the process of creating a new product. It is one part of the product lifecycle.
Product managers might be responsible for developing a new product but most manage existing products where the goals are to introduce and grow the product as quickly as possible and then maintain the product in the maturity phase for as long as possible.
Developing new products and managing existing products require very different approaches.
For new product development, validating the riskiest assumptions first is the surest way to create a product that is valuable, usable, feasible and viable.
The riskiest assumptions are about things you know will be important for the success of the product, but which you know little about if/how that thing is going to work in practice. Michael Birchall talks about mapping these things on a two-by-two grid by importance and uncertainty. This is a useful approach as it helps to prioritise the assumptions that should be validated first.
For example, getting the target audience to find out about and try the product is a riskiest assumption that needs validating early because there is no point doing any more development work on a product unless people will use it. This assumption is often validated with an experiment involving a simple landing page with a waiting list and promotional activities to drive people to the page. If no one signs up, the conclusion could be that the value proposition didn’t communicate the product’s value, in which case it can be changed and the experiment run again. But if multiple experiments fail to get people interested, then the assumption is proved wrong and no time has been wasted building a product that would fail.
For existing product improvement, removing biggest barriers first will have the greatest effect on the success of the product.
This approach comes from the theory of constraints which says that “Every system has a limiting factor or constraint. Focusing improvement efforts to better utilize this constraint is normally the fastest and most effective way to improve profitability.” There are probably lots of constraints, or barriers, getting in the way of the product being success but only one of them is the biggest. Removing that will have the most positive impact. Then the second biggest barrier becomes the biggest so that’s next to be removed. Eventually, removing barriers provides only marginal gains and this is pretty good signal to stop investing in improvements.
For example, if lots of people are dropping off at the sign-up and onboarding stage, that that is the biggest barrier to users getting value from the product. Fixing that problem will make the product more successful than adding new features for the users that do make it through onboarding (assuming of course that increasing the number of users is the goal).
Full loop product management is an approach and a skillset that allows product managers to be involved throughout the entire lifecycle of an opportunity. It starts with understanding potential opportunities, brings them into the organisation to align with strategy, creates a means of leveraging the opportunity, and takes it out into the world and measures it’s impact.
Full loop applies to any size or stage of organisation and for new products or for improving existing features. It’s a way of joining up the parts of the value stream that creates products, of giving the whole process more coherence, and so creating more valuable products that leverage the right opportunities.
There are four parts to the loop.
From horizon-scanning across an industry to competitor analysis to interviewing customers, external discovery is all about finding opportunities for the organisation and the products.
Internal discovery involves bringing an opportunity into the organisation and aligning it with organisational and product strategy. In some organisations this involves writing business cases and managing a roadmap.
Building the new product or feature, and especially ensuring the rationale for how it will leverage the opportunity, is part of internal delivery. This can involve working in agile ways with developers and testers, writing user stories, and communicating progress. This is often where product managers spend (too much of) their time.
Delivering the solutions for customers to use includes go-to-market planning, marketing and promotion, and customer support. Monitoring the effect the solution has in the external world completes the loop and informs the ongoing external discovery to understand whether the product or feature is leveraging the opportunity.
When product managers are involved in all sections throughout the loop they are more able to reduce the disjointed lack of connection between an opportunity, a solution and its impact. The more connected and coherent the product development process is, the more value it will deliver.
Many organisations focus their product managers on only internal delivery, in the bottom right quadrant. Whenever a product manager job description talks about tools like Jira and agile but doesn’t mention understanding customer needs or measuring outcomes, that’s a sign of an organisation that constrains it’s product managers to only doing internal delivery work. This is the source of the criticism of product managers being used as project managers. If they are not involved in understanding the problem or opportunity, and not involved to launching and measuring the effectiveness of the solution, then all they can do is manage the small part of the loop that builds the software.
A organisation that expects its product managers to only work across the internal quadrants, usually means a product manager taking ideas about often unvalidated opportunities from internal stakeholders and liaising with developers to build something that the stakeholder specifies and thinks will deliver some unspecified value. In job descriptions this often shows in phrases such as “liaising with stakeholders”, “communication skills” and “writing requirements”.
It’s rare for a product manager to be only focused on the external delivery of a product, things like go-to-market plans, marketing, customer support and success, but it’s worth being aware of as there’s an emerging trend of product management shifting more towards marketing in some tech firms.
So, how do product managers constrained within one part of the loop start to get involved in the other parts? Those other parts of the loop happen within an organisation. They may not happen in a coherent and robust way, but they do happen. Find out where and how, speak to those people and ask to learn about what they do. Get involved. Find ways to make their lives easier. Move upstream and downstream, and connect people and ideas along the way.
Coherence doesn’t happen by accident, and is so easily lost in the handover between teams and across different parts of an organisation. Full loop product management fixes that.
