Weeknotes 538

I did:

Momentum

Three day week so I was super-focused on moving the important stuff forward.

  • Discussed conversion rate forecasting as we get ready to scale an MVP.
  • Chatted about agile governance and why stakeholder engagement (and education) is so important.
  • Ran a team expectations workshop.
  • Chatted to content designers about automated communications.
  • Updated our roadmap.
  • Talked through a really good example of a data-driven product process that identifies user needs and validates whether we can solve them.
  • Reviewed a product requirements document.
  • Started figuring out how to approach bringing lots of analytics data together in a way that make sense.
  • Briefed myself on some new work. Writing stuff down helps make things clearer.

I read:

Roadmapping exemplar

Josh Hunt and the team at DHCW set a new standard in roadmapping. If you’re thinking about how to improve your roadmap, you should read their blog post and take a look at their public roadmap. Jealous much?

Product management resources

Seyifunmi Olafioye put together this super useful resource list.

AI product system

Rian van der Merwe describes his AI product system. Two things really interested me about how this works. First, the loop of updating context keeps information fresh, second, the commits graph is an indicator of how quickly the work is progressing. It’s basically flow rate, because every time someone from the team does something (has a meeting, diagrams some thoughts, writes some code) it’s put into context and shows in the commit graph. Very cool.

I thought:

Theory of product management

I’ve been working on some theory building for product management. My core claim is that product management is the organisational function that runs the scientific method on the value chain. It operates by forming hypotheses about where value comes from explicit, testing them cheaply, and reallocating resources based on the evidence. The value of product management comes from fewer wrong bets and faster learning for course correction, not the usage of the shipped product (it should generate value too but that’s separate from the value the product function provides).

It means that the success of the product function is down to the intensity to which an organisation applies the scientific method. The more consistently value hypotheses are explicitly stated, evidence that would falsify them is defined, proportionate tests are run, and decisions are changed when evidence disconfirms, then the more value an organisation will get from product management.

If I had the time, I’d love to research this and create a proper theory for product management.

Observe / change / measure

In my ongoing attempts to understand product management, and communicate what we do and why it matters (and why it isn’t project management), my latest thing is that what we do is observe, change, measure. It’s about as simple a version of the scientific method as I can get.

Observe and measure can be hard enough, but it’s the change part that is most challenging. It’s the hard bit because organisations are built to maintain the status quo. If that’s what most product managers spend most their time on, and we don’t make the observe and measure parts visible and important enough, then is it a surprise that we are seen as project managers?

Defining value

The MBA module I’m studying at the moment is about value so it’s got me thinking about how to define value.

Value is a surplus that has been co-created through exchange, most often where the things being exchanged are of different types and the surplus of the types isn’t clearly distinguishable, but continues to exists after the exchange is complete, and diminishes over time.

For example, you and Netflix co-create value through exchanging things you have that they want for things they have that you want. You have money, time and attention which you give to Netflix and they have videos, games and mobile apps which they give to you.

Weeknotes 537

I did:

Steps and leaps

Change happens in small gradual steps and quick dramatic leaps, and at the time you don’t know which is which.

  • Watched a really exciting demo of agentic software development.
  • Demo’d some work and got stakeholder feedback about what to do differently next time.
  • Figured out a solution to a new problem by joining together two older solutions.
  • Planned a workshop for next week.
  • Discussed what we might do at our next product community of practice.
  • Chatted about product managers defining their product process.
  • We’ve got a new product launch next week so got everyone involved lined up.

I read:

Conversational design

Dipped in and out of Erika Hall’s brilliant book Conversational Design to help me define some upcoming work.

The Web We Know Is Going to Disappear

Yep. Anyone disagree? Anyone? Bueller?

The Omnichannel Playbook

Read Commercetools’ whitepaper on composable commerce for omnichannel businesses.

The AI-Shaped Organization Reimagines Work, Not Jobs

Organizations that focus on value creation rather than workforce reduction are better positioned to unlock the benefits of AI.

Dreaming new dreams

Teams should spend time developing new ideas.

I thought:

Problem space questions

  • What problem are people facing?
  • How many people does it affect?
  • How much does it affect them?
  • How do they solve it now?
  • How satisfied are they with the current solution?

You don’t need more questions. You need better answers.

High WIP = DDOSing yourself

I read somewhere about a product team that had so many AI generated pull requests they couldn’t ship anything because they had no way to tell which requests are worth it. That’s not an AI problem, that’s an organisational problem. Controlling work-in-progress is one of the most important things we can do to deliver value effectively.

Shared AI

When I chat to my copilot it knows my context from my emails, messages and documents. When someone else chats to their copilot it knows their context. Which means we get different answers to the same questions and, more importantly, it means we can’t collaborate with copilot effectively. So, we need shared agents.

Working together or on your own

I was wondering how much of my work is with other people and how much do I do on my own. Luckily, my kanban tells me. 57% of items on the board are meetings, so that’s working with others. And 43% are tasks that I do on my own. Is that the right balance? Who knows?

On having a winnable heart and mind

We talk a lot about product managers influencing others, winning their hearts and minds. I think it’s equally important to being open to being influenced by others, to have an open mind and a welcoming heart.

Weeknotes 536

I did:

You’re only as strong as your network

I was chatting to a couple of colleagues about how knowing who knows what you need to know is the secret to success, which made me think about the theory of weak ties again. Anyway, this stuff happened too:

  • Talked about which markets we compete in and how they are changing.
  • Lots of mixing service vision, product strategy, user experience design.
  • Prototyping with copilot (I was pleasantly surprised).
  • Ranted about “Individuals and interactions over process and tools”.
  • Added new prompts to my AI tracker to see how the answers change over time and whether we’ve been able to affect them.
  • Set up a group for some reporting. Small communities around shared challenges is where it’s at.
  • Talked different types of work as requirements-driven, evidence-driven and value-driven. Which made me think about push and pull systems and where the ‘No Signal, No Work’ mantra fits.
  • Did some roadmapping to test out ideas for communicating for connections between oragnisational priorities, user outcomes and business impact.

I read:

Design Engineering with Maggie Appleton

Your Strategy Is Decaying… Here’s How to Save It

Interesting episode of the One Knight In Product podcast with Evan Campbell. He talks about how, “strategy naturally decays unless you actively maintain it. Organisational inertia, competing priorities and everyday distractions gradually erode strategic clarity. Leaders need to continually reinforce strategic intent and check whether people are still pulling in the same direction.”

The 7 Moats That Still Hold in the Age of AI

Hayder Schneider uses Hamilton Helmer’s 7 Powers to talk about most for AI products. A moat, he says, “is a sustainable competitive advantage that protects a business from competition over time. It makes it difficult or costly for competitors to replicate the business’s performance or take its market share.” I’ve not given it much thought but there’s something interesting how organisations might mix the powers to the create unique moats.

I thought:

Product is sharing

On my more idealistic days, I see products as an ultimate act of sharing. They bring together people’s knowledge, expertise and lived experience, codify it in software, and then share it with lots of other people for them to benefit from.

Think about any product you use, and think about all the people who contributed stuff they’ve learned to make that product possible. I don’t mean the designers and developers who built it, I mean all the people who’s knowledge they built upon. For example, I use kanban boards a lot. And there are lots of kanban-esque products (Trello, Jira, Azure DevOps, etc., etc.). All of those products are only possible because people like Taiichi Ohno and thousands of others contributed their knowledge to the idea, practice and popularity of kanban, which is built into those products.

Three outcomes

When you shift to being outcome-focused, success becomes far more ambiguous. Outputs are easier, you either shipped or you didn’t. With outcomes, there are three possible results:

  1. The outcome is achieved (user behaviour changed and it lead to business results) and it was because of the work you shipped. This is the one we all hope for but without robust measurement and analysis it’s easy to mistake correlation for causation.
  2. The outcome is achieved (user behaviour changed and it lead to business results) but it would have happened regardless of whether you shipped your work or not.
  3. The outcome isn’t achieved (user behaviour didn’t change, or it did but it didn’t lead to business results). This is where lots of product teams find themselves more often than not. Shipping work that changes . It’s the learning space. There’s lots to figure out about why the outcome wasn’t achieved and what to do about it. Do you iterate or abandon?

I reckon there’s some Bayesian calculations that could help figure out the probability of each.

Cowork

Co-work, not cow-ork!

A cow ork standing in a bleak lanscape

Weeknotes 535

I did:

Generalist FTW

One of the things I like about being a product manager is how I get to do lots of different things. This week I worked on data protection, cost analysis, planning and resourcing, adoption, preference management, supplier relations, analytics and evaluation, accessibility, reporting… and this stuff too:

  • Worked with some of our brilliant user-centred design and research people on the vision for a service.
  • Talked about product strategies, including direct distribution, which means users have to use your products to do what they want to do, and indirect distribution, which allows users to do things via third parties.
  • Wrote up some thoughts on the direction I want one of our products to go in. The challenge is in getting the right balance of specific actions that need to happen and creating space for things I haven’t thought of yet.
  • Joined in with a planning exercise our fantastic lead delivery manager organised to prepare for some big pieces of work coming up.
  • Chatted about long-term roadmaps and engaging with stakeholders to shape it.
  • Thought about how to formalise and codify my process for assessing new work. And how to get AI to help because I’m just not getting it done quickly enough.
  • Helped a few teams prepare for a product launch in a couple of weeks.

One month of tracking against my objectives

I totalled 216 tasks and meetings over 22 working days in October, which is pretty much sticking to my 10-a-day goal. Of them, 37% contributed to my six objectives. Within that, 20.8% was for the objective I worked on most and 0.9% was for the one I worked on least, with the other four in between.

Some of my objectives are broad and others quite specific. Some have end dates and others are open-ended. I’m interested to see how those differences are reflected in the data.

Study kanban

The next module of my MBA starts next week so I’ve been setting up my kanban to manage everything I’ll need to do between now and April. There are over two hundred items that I could never keep track of without my own kanban. Thank you Taiichi Ohno.

I read:

Bizarre love triangle

Thinking and writing like this from Simon Morgan-Wilson is absolutely brilliant. Real answers based on real experience. Not buzzwords, not regurgitated, not what AI mashed together. We need more of it in product management to help us all learn, adjust and refine our thinking about what it is we do.

The feature that made you ordinary

Interesting lesson on the risk of productising away the emotional needs of users. Starbucks’ app focused on convenience, removed pain points for customer, made lots of money… and still failed because customers didn’t just want coffee, they wanted to feel seen and part of a culture. Nothing to do with this, but I still think the TV show Friends was the cultural soft power behind the rise of the coffee shop and the success of Starbucks and Costa.

I thought:

Responsibility = judgment + information + action + control

I thought about responsibility a bit this week, and what it takes to actually be responsible for something. I think it takes four things, and if any of them is missing on incomplete (and they usually are), taking responsibility gets really difficult.

  • Without judgement, someone can’t take responsibility for something because they don’t have the capacity for making decisions about it. It’s why mentoring, coaching and getting the reps in is so important for new product managers, it builds judgement.
  • Without information, someone can’t responsibility because they don’t know enough . It’s why speaking to people, understanding context and building up mental models is so important for every product manager, it maintains the flow of information.
  • Without the ability to take action, someone can’t take responsibility because they won’t be able to make things happen. It’s why influence and relationships are so important for product managers, because they almost always need other people to do things.
  • Without control of those actions, someone can’t take responsibility because they won’t be able to correct things that go the wrong way or change things based on new information and learning. It’s why data, feedback loops, reflection and learning are so important for product managers, because they have to be able to detect and understand what is and isn’t working.

(Aside, but personally, I think responsibility is taken, not given which is why the concept of RACI never really works in autonomous product teams (Aside, aside, but I saw a video about autistic understanding that said we don’t get that others know things to be untrue but accept them anyway, so things like RACI’s MECE approach is an accepted fiction that people go along with even though it doesn’t make sense).)

Weeknotes 534

I did:

Workshoptastic

Spent 60% of my time at in-person workshops this week, which has me looking at the flow efficiency of all my other work with raised eyebrows, but still did this stuff:

  • Helped out at two delivery playbook workshops and learned a lot from a consultant who’s been helping/challenging us.
  • Went to a fantastic workshop about shaping up the vision for one of our services.
  • Talked about how we might improve how we manage split tests and build a knowledgebase that helps us understand where we are more able to change user behaviour in ways that gets business results.
  • Did some work with other product managers on a group-level roadmap. The really interesting stuff of talking to about a hundred stakeholders is still to come.
  • Analysed some new ideas and thought about when and how is the right time to get the team involved.
  • Thought about product strategy workshops using AI to create the tools and templates specifically for developing the strategy that you’re working on. Our usual tools, e.g., Lean UX Canvas are generic and fixed because they have to apply to a wide range of circumstances, but that’s no longer the case when AI can create a canvas for a specific situation.
  • Chatted about where we want to go next with the user-behaviour-triggered communications capability we’re working on.

I read:

Agentic web evolution

“The interfaces mediating all of this (ChatGPT, Claude, Gemini, etc.) want to keep users inside their own surface. They’re deeply invested in building a web where agents can find the information and do the tasks a user requires without ever sending them elsewhere. To me, it seems natural that these interfaces will start rewarding sites pre-structured for agent operability.”

Where AI capability meets customer need

Started reading Jeff Gothelf’s new book ‘Product management and AI: turning capability into customer outcomes’. “Product management done well in the age of AI takes the raw capability of AI and points it at problems real customers have, in ways that change what those customers do hopefully for their benefit and then turns that into a measurable business impact. Without that discipline, AI makes teams faster at producing things nobody asked for. With it, AI becomes the most powerful accelerant we have ever had for learning what customers need and delivering it.”

AI Doesn’t Change Product Management Fundamentals

Jason Knight talks to Jeff Patton about why product management fundamentals are still so difficult to put into practice, how AI can help teams succeed or fail faster, and why understanding what customers actually do remains more important than ever.

AI week

I haven’t watched all of them yet, but there are some really interesting talks about AI particularly from universities.

Build the Agentic Enterprise

I thought:

Problems and solutions

It’s much easier it is to think in solutions than problems. We all do it and see it. Start talking about a problem and before you even realise it, before you’ve really understood the problem, you’re already thinking about how to solve it. Maybe, when we’re building products, it’s because the solution is almost always known and the same, it’s build software. But it’s much harder to think in problems because problems can be process problems, people problems, user problems, organisational problems, big problems, little problems. Problems are messy, unclear, and difficult to reach a single agreed definition. That’s why product managers have to be really comfortable with ambiguity.

Culture is how we behave

In a workshop this week, one of our senior leaders said, “culture is how we behave”, and it stuck with me. Sure, there are lots of theories about organisational culture, and they have their place, but as a product manager whose job is to change user behaviour in ways that get business results it makes me question whether a product approach is a good fit for changing culture. Previously I’d had the opinion that product thinking gets organisational change wrong because it believes that once a change is made it stays made (because that’s how software behaves), but when changes happen to people, those people can change back or change in different and uncontrollable ways, which means change doesn’t stay made. But I’m not so sure anymore. Maybe all this AI product stuff about nondeterministic tech making changes to user behaviour in unpredictable ways is getting to me.

Marginal gains, diminishing returns

A lot of product management focuses on the exciting stuff of introducing new products, but eventually, if you’re lucky, your new products become old products and then you need a different approach to managing them. Managing mature products is a marginal gains, diminishing returns game.

Marginal gains means recognising that you aren’t going to make big leaps forward on any of your goals. The only progress to be had is in lots of small changes all over the product to get the last few percent increases. Diminishing returns means understanding that those gains require more and more investment, or to put it another way, for the same investment the returns will get smaller and smaller. That point on the lifecycle where maturity crosses into decline is where you have to make some hard decisions about what to do with the product; keep investing, just do the essential maintenance work to keep it running, or retire it.

Weeknotes 533

I did:

Mental models

How we individually understand how things work so we can change them is a constant fascination and theme to my work. It always seems to come back to whether enough people understand something in a sufficiently common way to do something with it. Some examples from this week:

  • Finished some idea analysis. It was a idea good but not for our team.
  • Got our platform product managers together to work on our long-term roadmap.
  • Planned a workshop for introducing a new delivery playbook.
  • Chatted about operationalising data-driven decision-making and what the whole loop looks like.
  • Facilitated a workshop to categorise products.
  • Worked on an evaluation model for split tests.
  • Thought about how a product strategy co-writing session might work.

I read:

The return of the feature factory

John Cutler reflects on his feature factory post; the one that coined the term, divided the factions, led to notoriety. We need terms like this to group our understanding around and prompt discussion, and move our practice forward. But John makes the point about the problems with x vs y narratives. I guess, as his post shows, we never know where the narrative might go.

Online learning developments in UK higher education

Neil Mosley’s quarterly reviews are always interesting, with this one reflecting a number of shifts in UK higher education including school leavers choosing to study online and AI courses. Higher education trends often seem like lagging indicators for societal change. The strategic challenge for universities is which societal change to lean into.

Start Before Product

Scott Colfer talks about the product work outside the product in complex service organisations, and getting oriented before starting, which means understanding where you’re working, understanding the change you’re trying to make, understanding who needs to be involved. All good advice.

I thought:

Merge

Most product development processes are linear. I don’t blame Cooper but he has a lot to answer for. It looks nice in diagrams but it’s not the reality.

I have recently come to the realisation that product development processes are really about merging things, or maybe merging the right things at the right time.

When is the right time to bring together technology and use cases? When should compliance and data architecture come together? When does it make sense to bring evaluation in with user experience design?

A merge approach avoids artificial phases and allows for lots of adaptation depending on the product, teams, etc.

WIP x CIP

The calculation for organisational complexity is work-in-progress multiplied by change-in-progress. Work-in-progress is all the work going on, and I mean all the work, not just the stuff on the roadmap but all the side projects, new ideas people took initiative on, leftovers that never really got finished. Change-in-progress is all the change going on, stuff like people leaving and joining, new organising logic, management methods, and all those reorgs and transformations that never completely changed what they were meant to but leftover things that didn’t get fully replaced when the next change came along. Those two multiplied is how complex an organisation is.

Visioning

A vision isn’t just a statement of what the world looks like if your vision becomes a reality. That’s part of it, but a really good vision provides a sharply refined description of all the aspects of that future world. It gives you a sense of what the market might look like, what size and shape the team needs to be.

Four stages to get to a product vision:

  1. Setting the boundaries. You need to know what you’re going to try to change and what you aren’t.
  2. Diverging on lots of things within the boundaries. User research, new technologies, market trends, operational processes, architecture, budget, teams, etc., etc., gives a sense of all the changeable things.
  3. Converging on the things that matter. Narrow in on all those things until you understand which you would have to change to create your new world.
  4. Asking what would have to be true. If you want to have any chance of turning the vision into reality you have to be able to articulate the changes you need to make for each thing you converged on. If your vision demands a different architecture, whole bunch of data you don’t collect and partners who don’t even know you exist, you should be fully aware of that before you commit to a vision.

A product vision isn’t just a statement, it’s a mental model. The visioning work is to make that a shared mental model.

Effort-to-value

Effort doesn’t tell you how long the work will take, it tells you if the work is worth doing. It’s a return on investment calculation that helps you understand opportunity cost because if the time spent working on this can’t be spent on that. It’s different to time-to-value because it doesn’t include wait time, it’s the total time everyone will spend/spent on this work to get it to the point where it is returning the required value.

Weeknotes 532

Weeknotes 532

I did:

All of us are smarter than any of us

One of my fundamental beliefs about modern approaches to tackling complex problems is that it needs people to work together. Sometimes that’s easier said than done, but it’s always worth it. And…

  • Got product and delivery people together to talk about how we intake and triage stakeholder requests. It was a really helpful honest conversation with no agenda.
  • Planned this month’s work with the team (well, actually, our fantastic delivery manager did all the hard work).
  • Wrote a proposal for some product practice development.
  • Designed a workshop to categorise our products.
  • Researched training I might do.
  • Wrote up some ideas on how to respond to AI intermediaries.
  • Started thinking about product vision, ways of choosing it, and more importantly, what has to be true to achieve it.
  • Chatted to our new lead content designer.

I read:

Knowing the right things to say

This article helps explain how empathy is culturally shaped and some of us just don’t get it. Knowing what is the “right” thing to say in every situation depends on being able understand how others are feeling and how what we say might affect that and make judgement calls on a culturally-specific implicit understanding.

Personal Agent Protocol

How agents and organisations might interact fascinates me, so Sierra’s blog about developing a Personal Agent Protocol is interesting reading.

The protocol is an open standard that defines how personal agents interact with businesses, including authentication, observability, etc. It’ll be interesting to see how well adopted it is.

Good neighbours

Really interesting thinking from Jeffrey Miller, product lead on Ask the NHS, about choosing where across the NHS’s digital estate a new service should be used first.

I particularly like the point about products being good neighbours and making each other better. Sometimes products have competing objectives that never really get resolved and make both products worse, so it’s great to see the thinking through of this kind of problem.

I thought:

The law of requisite complexity

After a discussion about strategy, I was thinking about The Law of Requisite Complexity which says for organisations to effectively adapt and respond to the system they are part of, they have to have at least as much internal complexity as the external environment. It explains why universities are as complex as they are, because the environment they operate is as complex.

So, strategically speaking, there are two ways universities can handle the complex environment; try to decrease the complexity in the environment so it matches internal complexity, or increase internal complexity to match the environment.

Actually, there are other strategies, an organisation can reduce internal complexity regardless of the external complexity but that will almost certainly fail so let’s stick to what might succeed.

In practice, a good strategy should try to reduce external complexity whilst increasing internal complexity to create a better chance of the two matching. Choices about which markets to operate in and what to do to affect the market, alongside building capabilities that can adapt quickly to change, affect complexity externally and internally.

The thing that’s sometimes hard to see about building flexible internal capabilities is that they can look simple, look like they are reducing internal complexity rather than increasing it, but really they enable far more variation that does increase internal complexity.

Shadow collaboration

If shadow IT is when people use their own devices and products to do their work despite organisational technology and policy, then shadow collaboration is when people find ways to work together despite organisational silos.

Dashboards are dangerous

In their efforts to make complicated metrics easy to digest, dashboards risk presenting information in ways that make it too easy to over-generalise and reach the wrong conclusion. I saw a dashboard that reported metrics for a website. The metric looked bad, but worse than that, it was meaningless. It was an average from across the website that didn’t recognise the very different use cases for the website, it just presented a number. Good dashboard design is a real skill.

To be outcome-focused you need to know what your users value

People can value many things at the same time. They can value their time spent doing something, money they made or saved, the feelings they get from doing something, the feelings they avoid, what others think of them, theirs and others safety and wellbeing. They can value things now and value things they might get the future. The promise of future value has its own value right now. And so many more things.

So, being outcome-focused (if we agree that an outcome is a change in user behaviour that gets business results) means recognising that you are dealing with complex human beings who have a lot going on when they use your product. It’s far too easy to build a product that delivers one type of value but takes away another. It might save time but reduce trust. It might increase belonging but increase cost. If you don’t know what your users value it’s really hard to achieve the right outcomes for them.

The problem with models

I accidentally found myself in Stewartby this week. Stewartby, in case you’ve never heard of it, is a model village built by the London Brick Company in 1920s and 30s.

Most villages were built organically over time because people settled there a long time ago. Model villages are designed upfront and built according to some agreed vision and principles. All modern housing developments take this approach too, they have certain standards to meet such as how much green space per number of houses is provided.

You can see where I’m going with this, right? It’s a metaphor for product development. And for thinking about what makes great products. How much does big upfront design versus iterative change affects the success of a product? Of course, often that choice is less about what might make a successful product and more about how organisations invest in products (thanks Conway) but it’s still an interesting question.

My bias (and I fully recognise it is a bias) is for small design, iterative development, fast feedback loops, frequent change. But I like to challenge that bias when I find myself in Stewartby.

Weeknotes 531

I did:

Artificial constructs

My team is on a bit of a leading edge with AI. We’re the first to be doing things like using an MCP server and figuring out how to navigate proprietary AI systems. It’s very cool and I want us to keep it up and show the value of what we’re doing. Also did this stuff:

  • Talked about AI, EU AI act, and how a product approach can derisk AI features.
  • Ran a workshop with an operational team to help design new processes.
  • Talked about roadmaps (the artefacts) as an expression of the shared understanding of the roadmap we all hold in our heads.
  • Prepared roadmaps and backlogs for a prioritisation session.
  • Started building a little AI product operating system for a piece of work.
  • Started my new annual objectives and set up tracking in my kanban so I can see how much focus each objectives gets.
  • Chatted about shared practices and processes for product teams.
  • Played with a new analytics tool.

I read:

The AI-native SDLC playbook

Claude’s AI-native SDLC playbook is a really useful guide to thinking about where and how to use AI in the software development process. It describes how each stage ends by writing to files AI can access, with the next stage beginning reading them. Fundamentally, it’s the same as a human approach to software development, with each stage building on the information generated in the previous in a cause-and-effect way. The difference is in the type of knowledge used. Software development relies on a lot of tacit knowledge, implied understanding and assumptions that only humans have. They can’t be codified for AI to use. So, the thing to figure out is what parts humans play and how we use AI to do the things we’re not so good at.

Why are software development task estimations regularly off by a factor of 2-3?

This quora post was mentioned by Elisabeth Hendrickson and Joel Tosi in Signals and levers. It uses a hiking trip as a metaphor for software development. It wrote about this kind of measurement problem a few years ago too. We all know that estimations of uncertain work are always wrong, but it’s important to understand why.

The chipping and the craft

Really nice post from Simon Morgan-Wilson on the difference between how we do product work and how we do product work.

He says, “Chipping is easy to see and easy to count. Things shipped, deadlines met, boxes ticked. Craft mostly shows up as things that didn’t happen: the feature not built, the wrong direction not taken.”

I thought:

Flow efficiency

Thought about classifying work by its flow efficiency as a way of showing actually focus/time spent, which indicates whether the work is being treated as a priority or not.

Imagine a priority piece of work that 10 people are involved in. To make the numbers easy we’ll say they each have 30 working hours a week which is a total of 300 hours a week. Counting up how many hours they spend on this work in one week, we find together they’ve spent 40 hours on it. That’s a flow efficiency score of 15% Is that acceptable for a priority piece of work or should they spend more time on it?

Now imagine that all work has a flow score and is in buckets of >80%, 60-79%, 40-59%, 20-39%, 0-19%. We’d easily see whether work we consider to be a priority is getting the necessary time spent on it.

Rise or fall

James Clear said, “You do not rise to the level of your goals. You fall to the level of your systems.” He was talking about personal habits could have been talking about work in organisations. I’ve been thinking about objectives this week, which inevitably means thinking about creating the systems that make achieving objectives more likely.

Part of my system is tracking the work that contributes to my objectives in my kanban. That makes it easy for me to see how many tasks or meetings are about an objective, and plan next week accordingly. Feedback loops are important in any system.

Storming, norming, forming, performing

Tuckman’s stages of group development are a really useful lens for understanding how work happens (particularly in matrix organisations), because of course, all work is done by groups of people.

When a new piece of work starts there is a flurry of activity, different people getting involved, no one is clear what the work will involve. That’s the storming stage. Then things settle down, the work becomes clearer, some people figure out they don’t need to be involved. That’s norming. When the people who are left get together and figure out the work, they are forming. Eventually, they get into a rhythm of working together and get to the performing stage (although sometimes the work is finished before the group achieves this stage).

Those stages have to happen. There is no way to avoid them. Organisations can choose how and with who they happen, but they can’t choose for them not to happen.

The benefit of stable teams working on standardised things in consistent ways is that they aren’t involved in the storming and norming. It still happens, but it happens with another group of people, often leaders, figuring out what the work should be before handing it to a team that is already performing. There are pluses and minuses to this. It can make delivery more efficient, but sometimes the solutions don’t achieve the results because there’s too much disconnect between people involved at each stage.

The other way is for leaders to involve teams at the storming stage, have them help norm the work. There are pluses and minuses to this way too. It’s often more time-consuming (see flow efficiency above) but it can result in better solutions and more effective outcomes.

Product vision

Every product vision implicitly ends with “…because we believe this will make the world a better place”. If adding that to your vision statement doesn’t make sense, then you haven’t got a good vision. Try again.

The weeknote machine

Matt Ballatine (weeknote extraordinaire) built an app for insights from his 800 weeknotes. Makes we wish I had time for doing cool stuff like this. So many ideas, so little time…

Here’s one for the content designers…

The best frequently asked question ever.

Screenshot showing FAQs for a website with the question, "Can I bring my pet iguana to a gig?"

Weeknotes 530

I did:

Evolution

A few things from this week seemed to be around the theme of evolution, which is an interesting way to think about organisational change because it has no centralised oversight of change, it happens in response to the environment, but also not entirely without coordination.

  • Went to a fantastic product group session where teams presented their visions and progress towards them. I think there’s some useful things me and the other product managers can do to help these teams be more successful.
  • Talked about when a product approach (you know, understand your users and their problems, try out ways of solving them, get feedback to know if you’ve been successful, go round again with a better understanding of your users and their problems) does and doesn’t apply. For example, it probably works for communicating to large groups of people but might not work for organisational change.
  • Prepared presentations for next week. One on the compatibility of proprietary AI models with organisational processes and another on integrating with a third party systems. How cool am I?
  • Did some more work on platform product practices, particularly on understanding developer experience and sharing insights.
  • Set up a new planning system to try out for a couple of months.
  • Chatted about backlogs, vision and strategy, which was really a chat about pluralism (see below).

I read:

Making data-driven decisions with a small sample size

How to Test Product Hypotheses When Your User Base is Tiny

Signals & Levers: Systems Thinking Tools to Unblock Software Delivery

Written in the vein of The Phoenix Project and The Unicorn Project, Signals & Levers talks about metrics and measures, activity and progress, in software delivery. I’m always alert to the fact that software delivery is not the totality of product work, but I’m interested in what I can learn, especially as Phoenix and Unicorn got me interested in Lean again.

I bought it as a kindle book, which I don’t usually do, but given I have five big boxes of books that I don’t know what to do with, I’m beginning to question my life choices.

Marty Cagan’s 10 regrets from the past 25 years

I thought:

Pluralism

If you work in a complex organisation, particularly public sector, pluralism is a useful explanation for how organisations operate. It’s a term used in many different contexts but with the same underlying idea:

  • a condition or system in which two or more states, groups, principles, sources of authority, etc., coexist,
  • a political theory or system of power-sharing among a number of political parties,
  • a theory or system of devolution and autonomy for individual bodies in preference to monolithic state control,
  • a form of society in which the members of minority groups maintain their independent cultural traditions,
  • a theory or system that recognizes more than one ultimate principle.

Why it matters so much in product work is because it shows us that the popular narrative of one vision, alignment, single purpose, etc., isn’t the only way things can be. It tells us that multiple goals, ideas, values, etc., can coexist, can be in conflict and yet be equally right. It tells us, as product people, that we are in a constantly negotiation, that multiple visions, strategies and priorities can be in play at the same time. It requires us to avoid extreme positions and engage in good faith dialogue that reflects and balances competing principles. It’s a very different kind of product work than you see in the YouTube videos, but it’s just as important, and I think, more interesting.

Coherence

I’ve been (very slowly) writing a blog post about coherence in products. It’s an interesting concept that, to me at least, explains great products. Coherence is, “in general, a state or situation in which all the parts or ideas fit together well so that they form a united whole” and in “scrum and agile methodologies, coherence is defined as a measure of the relationships between backlog items which make them worthy of consideration as a whole.”

We all know that product vision and strategy are our tools for achieving coherent products, but I’ve been wondering if there’s a way to empirically test coherence in a product. And I’m not talking only about the features users interact with, I also mean things like budget, team size and skills, goals, measures, reporting, compliance, decisions, etc., all the organisational parts that make up a product. The logic test I’m playing around with at the moment is whether one thing conflicts with multiple things. An example might be ambitious goals for a well-used product with strong decision-making (all things that seem to have a coherent relationship with each other), but budget that makes the ambition impossible to achieve. The budget is in conflict with the other three things and reduces the overall coherence of the product.

Context ≠ process ≠ results

Those three things are largely independent of each other. The context within an organisation can change without changing the product development processes used by an organisation. And similarly, the product development process can be changed without requiring any changes to the organisational context. Results, and this is the hardest pill to swallow, are mostly unaffected by the organisation’s context or the product development process it uses. We like to think the three are intimately interconnected, that by making changes to processes for example, we can get better results, but it’s just not true.

Weeknotes 529

I did:

Past, present, future

Even though all work is work, I think about my work as either being about the past, e.g., evaluating a feature we launched, present, or future, e.g., preparing for upcoming. The split of time spent on these changes each week, and this week was mostly future.

  • Did lots of workshop planning. I’ve got very limited time with a team so I want to make the most of it.
  • Chatted about trust-building, working in the open and using feedback and iteration.
  • Went to our Engineering practice all-hands day. It was very cool to see some of the things they’ve been doing around observability and resilience.
  • Talked about platform product management practices and how we might evolve them together.
  • Finalised a theory of change for a product evaluation.

I read:

The University after AI

This paper develops an analytic framework to offer strategic guidance to rebuild or reform universities through the AI era. It’s amazing work and I could happily spend a lot of time understanding and contemplating. Its representation as universities as multi-sided market platforms that sells matching (of “at least fifteen distinct groups”) and verification (”a credible warrant of quality over a good whose quality nobody can directly observe”) is really insightful and useful.

AI Evals

I’ve been reading more about evals as I start to build them into my agents. I haven’t got it working yet, but I think an agent should be able to follow instructions and then assess it’s output against evaluation criteria.

Get the reps in

“… the one trait consistent across all great products is that the team behind them iterated intelligently, over and over, until they found product market fit. In short, the best predictor of product quality is the # of reps – and not just random reps, rather targeted improvements that compound over time. If you iterate in a disciplined (vs random) way, quality will emerge over time as you find the optimum solution path and experience.”

Grading OKRs is just resulting with better branding

“Resulting is judging a decision by how it turned out. A good decision can have a bad outcome. A bad decision can have a good outcome. On the one hand, the result tells you almost nothing about the quality of the call.” Done well, OKR’s can help us improve the quality of the decisions we make, but maybe that thinking needs to go all the way back to opportunity analysis, before implementation starts, so there is a clearer separation between the decision to pursue an opportunity and how that opportunity pans out.

Organizations are perfectly designed for the results they get

“If you want different results, you must be willing to redesign the system that generates them. Everything else is noise.”

Aging (work, not people)

“Aging is the leading indicator and root cause of all that overplanning” Aging of work items is about how long things hang around in backlogs. It’s ‘overplanning’ because of all the work that goes into planning items that either never get delivered or need replanning again and again before they do.

(Thanks to Emily Webber for the link)

I thought:

Being outcome-focused demands higher quality outputs

I agree with the idea of ‘outcomes over outputs’, but the more deeply I think about what it takes, the harder it gets to separate them. Sure, they are definable in ways that make them clearly different. Outcomes are changes in user behaviour that gets results and outputs are tangible changes in products and processes. But, not only are they causally connected by outcomes only being affected by outputs, but the quality of the output affects the outcome too.

Standard of shared understanding

Did a bit more thinking on my standard of shared understanding, partly due to an interesting conversation about how hard it is to achieve. I need to make sure the standard never implies that the same shared understanding is possible. Everyone has their own unique individual understanding from their perspective, and that’s a good thing, the tough part is in creating enough common ground. That’s where really good user stories as boundary objects (not detailed requirements) are particularly useful. We need more boundary objects.

It’s who you know, not what you know

Heard an interesting point about enterprise AI adoption; it’s a context problem. It’s a data problem too but the path to a solution is more knowable, whereas organisations with multiple complex contexts that require human judgment about what applies in which situation are just not available to AI. That’s a big blocker to AI being anything more than a productivity tool, it doesn’t lead to transformation.

As an example, one of the conversations I had this week was about how to find the one right person who knows how to do one very specific niche thing. I replied that the theory of weak ties gives us the answer and we should spend time getting to know people so that we are as few steps as possible away from that person because of the other people we know. Knowledge about who to speak to and how to keep that info up-to-date when people move on or responsibilities change, is tacit knowledge, it isn’t codifiable in a way AI could understand.