Weeknotes 530

I did:

Evolution

A few things from this week seemed to be around the theme of evolution, which is an interesting way to think about organisational change because it has no centralised oversight of change, it happens in response to the environment, but also not entirely without coordination.

  • Went to a fantastic product group session where teams presented their visions and progress towards them. I think there’s some useful things me and the other product managers can do to help these teams be more successful.
  • Talked about when a product approach (you know, understand your users and their problems, try out ways of solving them, get feedback to know if you’ve been successful, go round again with a better understanding of your users and their problems) does and doesn’t apply. For example, it probably works for communicating to large groups of people but might not work for organisational change.
  • Prepared presentations for next week. One on the compatibility of proprietary AI models with organisational processes and another on integrating with a third party systems. How cool am I?
  • Did some more work on platform product practices, particularly on understanding developer experience and sharing insights.
  • Set up a new planning system to try out for a couple of months.
  • Chatted about backlogs, vision and strategy, which was really a chat about pluralism (see below).

I read:

Making data-driven decisions with a small sample size

How to Test Product Hypotheses When Your User Base is Tiny

Signals & Levers: Systems Thinking Tools to Unblock Software Delivery

Written in the vein of The Phoenix Project and The Unicorn Project, Signals & Levers talks about metrics and measures, activity and progress, in software delivery. I’m always alert to the fact that software delivery is not the totality of product work, but I’m interested in what I can learn, especially as Phoenix and Unicorn got me interested in Lean again.

I bought it as a kindle book, which I don’t usually do, but given I have five big boxes of books that I don’t know what to do with, I’m beginning to question my life choices.

Marty Cagan’s 10 regrets from the past 25 years

I thought:

Pluralism

If you work in a complex organisation, particularly public sector, pluralism is a useful explanation for how organisations operate. It’s a term used in many different contexts but with the same underlying idea:

  • a condition or system in which two or more states, groups, principles, sources of authority, etc., coexist,
  • a political theory or system of power-sharing among a number of political parties,
  • a theory or system of devolution and autonomy for individual bodies in preference to monolithic state control,
  • a form of society in which the members of minority groups maintain their independent cultural traditions,
  • a theory or system that recognizes more than one ultimate principle.

Why it matters so much in product work is because it shows us that the popular narrative of one vision, alignment, single purpose, etc., isn’t the only way things can be. It tells us that multiple goals, ideas, values, etc., can coexist, can be in conflict and yet be equally right. It tells us, as product people, that we are in a constantly negotiation, that multiple visions, strategies and priorities can be in play at the same time. It requires us to avoid extreme positions and engage in good faith dialogue that reflects and balances competing principles. It’s a very different kind of product work than you see in the YouTube videos, but it’s just as important, and I think, more interesting.

Coherence

I’ve been (very slowly) writing a blog post about coherence in products. It’s an interesting concept that, to me at least, explains great products. Coherence is, “in general, a state or situation in which all the parts or ideas fit together well so that they form a united whole” and in “scrum and agile methodologies, coherence is defined as a measure of the relationships between backlog items which make them worthy of consideration as a whole.”

We all know that product vision and strategy are our tools for achieving coherent products, but I’ve been wondering if there’s a way to empirically test coherence in a product. And I’m not talking only about the features users interact with, I also mean things like budget, team size and skills, goals, measures, reporting, compliance, decisions, etc., all the organisational parts that make up a product. The logic test I’m playing around with at the moment is whether one thing conflicts with multiple things. An example might be ambitious goals for a well-used product with strong decision-making (all things that seem to have a coherent relationship with each other), but budget that makes the ambition impossible to achieve. The budget is in conflict with the other three things and reduces the overall coherence of the product.

Context ≠ process ≠ results

Those three things are largely independent of each other. The context within an organisation can change without changing the product development processes used by an organisation. And similarly, the product development process can be changed without requiring any changes to the organisational context. Results, and this is the hardest pill to swallow, are mostly unaffected by the organisation’s context or the product development process it uses. We like to think the three are intimately interconnected, that by making changes to processes for example, we can get better results, but it’s just not true.

Weeknotes 529

I did:

Past, present, future

Even though all work is work, I think about my work as either being about the past, e.g., evaluating a feature we launched, present, or future, e.g., preparing for upcoming. The split of time spent on these changes each week, and this week was mostly future.

  • Did lots of workshop planning. I’ve got very limited time with a team so I want to make the most of it.
  • Chatted about trust-building, working in the open and using feedback and iteration.
  • Went to our Engineering practice all-hands day. It was very cool to see some of the things they’ve been doing around observability and resilience.
  • Talked about platform product management practices and how we might evolve them together.
  • Finalised a theory of change for a product evaluation.

I read:

The University after AI

This paper develops an analytic framework to offer strategic guidance to rebuild or reform universities through the AI era. It’s amazing work and I could happily spend a lot of time understanding and contemplating. Its representation as universities as multi-sided market platforms that sells matching (of “at least fifteen distinct groups”) and verification (”a credible warrant of quality over a good whose quality nobody can directly observe”) is really insightful and useful.

AI Evals

I’ve been reading more about evals as I start to build them into my agents. I haven’t got it working yet, but I think an agent should be able to follow instructions and then assess it’s output against evaluation criteria.

Get the reps in

“… the one trait consistent across all great products is that the team behind them iterated intelligently, over and over, until they found product market fit. In short, the best predictor of product quality is the # of reps – and not just random reps, rather targeted improvements that compound over time. If you iterate in a disciplined (vs random) way, quality will emerge over time as you find the optimum solution path and experience.”

Grading OKRs is just resulting with better branding

“Resulting is judging a decision by how it turned out. A good decision can have a bad outcome. A bad decision can have a good outcome. On the one hand, the result tells you almost nothing about the quality of the call.” Done well, OKR’s can help us improve the quality of the decisions we make, but maybe that thinking needs to go all the way back to opportunity analysis, before implementation starts, so there is a clearer separation between the decision to pursue an opportunity and how that opportunity pans out.

Organizations are perfectly designed for the results they get

“If you want different results, you must be willing to redesign the system that generates them. Everything else is noise.”

Aging (work, not people)

“Aging is the leading indicator and root cause of all that overplanning” Aging of work items is about how long things hang around in backlogs. It’s ‘overplanning’ because of all the work that goes into planning items that either never get delivered or need replanning again and again before they do.

(Thanks to Emily Webber for the link)

I thought:

Being outcome-focused demands higher quality outputs

I agree with the idea of ‘outcomes over outputs’, but the more deeply I think about what it takes, the harder it gets to separate them. Sure, they are definable in ways that make them clearly different. Outcomes are changes in user behaviour that gets results and outputs are tangible changes in products and processes. But, not only are they causally connected by outcomes only being affected by outputs, but the quality of the output affects the outcome too.

Standard of shared understanding

Did a bit more thinking on my standard of shared understanding, partly due to an interesting conversation about how hard it is to achieve. I need to make sure the standard never implies that the same shared understanding is possible. Everyone has their own unique individual understanding from their perspective, and that’s a good thing, the tough part is in creating enough common ground. That’s where really good user stories as boundary objects (not detailed requirements) are particularly useful. We need more boundary objects.

It’s who you know, not what you know

Heard an interesting point about enterprise AI adoption; it’s a context problem. It’s a data problem too but the path to a solution is more knowable, whereas organisations with multiple complex contexts that require human judgment about what applies in which situation are just not available to AI. That’s a big blocker to AI being anything more than a productivity tool, it doesn’t lead to transformation.

As an example, one of the conversations I had this week was about how to find the one right person who knows how to do one very specific niche thing. I replied that the theory of weak ties gives us the answer and we should spend time getting to know people so that we are as few steps as possible away from that person because of the other people we know. Knowledge about who to speak to and how to keep that info up-to-date when people move on or responsibilities change, is tacit knowledge, it isn’t codifiable in a way AI could understand.

Weeknotes 528

I did:

Sharper axes

Two day week so only did this stuff:

  • Led a co-writing workshop with bunch of product managers to help us develop a shared tone-of-voice for how we talk about our products. I found it really useful, and I hope some of them did too. I think we should do more group work like that.
  • Did a surprising amount of planning a) for things I’m working on and b) with others to help them with things they are working on. And, as always, facing the question of how much time you spend sharpening the axe versus chopping down the tree.
  • Talked about how we might prioritise work that affects three teams when each team has different objectives, in ways that are fair to everyone. It’s one of the interesting things about matrix organisations.
  • Started chatting to product manager’s about where our platform product practices might go. I think more of our teams might be platform teams than we realise it so it’s essential stuff to get right.
  • Connected a few different pieces of work that might all have the same need and same solution. It made me think more about how we look out for zeitgeist signals about emerging work, and also, how urgency, recency and frequency bias affects prioritisation thinking.

I read:

Intent-driven design

Really smart and thoughtful article by Carrie Webster about UX design shifting from visible interfaces to guiding transparent, intent-driven AI experiences.

Roadmap essentials

James Higgot’s excellent advice on roadmaps. Like I always say, the roadmap is all in your heads, how it gets expressed depends on the situation.

Golden Era of Product Management

YouTube’s algorithm sent me this video, which got me thinking about the ‘preaching to the choir’ problem of product managers who believe in empowered teams watching videos like this but it never shows up to the people who really need to see it. It’s an interesting organisational change problem.

I thought:

The bluntness of first order thinking

First order thinking tells us there is a single cause to an effect. It makes us believe that we can create an effect through a simple, single cause, and it makes us believe that the effects we see came about because one cause. When you put it like that it should be obvious that no effect ever has a single cause, but still it’s too easy for our brains to jump to that as a explanation.

It might be a natural human behaviour but it has no place in rational, logical product thinking. As product managers, we have to figure out the complex, multiple step causes that create effects. To me, that’s the core, fundamental purpose of product thinking. We use rationality to overcome the naturally occurring bias in human thinking. How we do that is what makes it infinitely interesting.

Analyse, Prioritise, Sequence

The thing about being outcome-focused is that when you set out to change user behaviour in ways that get business results you don’t know what will actually achieve that.

That means you need a different approach to prioritising work and managing backlogs. The output-focused approach takes everything on the backlog as in need of being delivered regardless of whether it might affect the outcome. Lining up work in an outcome-focused way means having lots of ready-to-go ideas, many of which will never get delivered because they don’t help achieve the outcomes that matter at the time.

Maybe the process looks something like this:

  1. Have lots of ideas and analyse them to understand things like audience, value, step in user journey, etc.
  2. When an outcome becomes important, pick from the list of ideas and prioritise them based which is most likely to achieve the outcome. Prioritisation logic is ‘this, not that’ so it’s about filtering out those ideas that won’t affect the outcome.
  3. Sequence those ideas based on constraints such as resourcing, capacity, dependencies, etc.,. Sequencing logic is ‘this, then that’, so it’s about ordering the work to be delivered.
  4. Take the first idea into the backlog, break it down, deliver and measure the work.
  5. If that delivered work achieves the outcome, stop working on that outcome and move to the next. If it doesn’t, take the next prioritised and sequenced idea into the backlog.
  6. Repeat step 5 until the outcome is acheived.

Queuing

Queues, like switches, are an entirely human invention. Based on all kinds of human ideas like scarcity, resource management, growth and fairness, queues allow us to organise our world in a way that makes sense to us. Sure, fire, electricity, AI, etc., are all great inventions, but they’d be nothing without the mental inventions humanity has created, and queues are one of the most important.