Weeknotes 529
I did:
Past, present, future
Even though all work is work, I think about my work as either being about the past, e.g., evaluating a feature we launched, present, or future, e.g., preparing for upcoming. The split of time spent on these changes each week, and this week was mostly future.
- Did lots of workshop planning. I’ve got very limited time with a team so I want to make the most of it.
- Chatted about trust-building, working in the open and using feedback and iteration.
- Went to our Engineering practice all-hands day. It was very cool to see some of the things they’ve been doing around observability and resilience.
- Talked about platform product management practices and how we might evolve them together.
- Finalised a theory of change for a product evaluation.
I read:
The University after AI
This paper develops an analytic framework to offer strategic guidance to rebuild or reform universities through the AI era. It’s amazing work and I could happily spend a lot of time understanding and contemplating. Its representation as universities as multi-sided market platforms that sells matching (of “at least fifteen distinct groups”) and verification (”a credible warrant of quality over a good whose quality nobody can directly observe”) is really insightful and useful.
AI Evals
I’ve been reading more about evals as I start to build them into my agents. I haven’t got it working yet, but I think an agent should be able to follow instructions and then assess it’s output against evaluation criteria.
Get the reps in
“… the one trait consistent across all great products is that the team behind them iterated intelligently, over and over, until they found product market fit. In short, the best predictor of product quality is the # of reps – and not just random reps, rather targeted improvements that compound over time. If you iterate in a disciplined (vs random) way, quality will emerge over time as you find the optimum solution path and experience.”
Grading OKRs is just resulting with better branding
“Resulting is judging a decision by how it turned out. A good decision can have a bad outcome. A bad decision can have a good outcome. On the one hand, the result tells you almost nothing about the quality of the call.” Done well, OKR’s can help us improve the quality of the decisions we make, but maybe that thinking needs to go all the way back to opportunity analysis, before implementation starts, so there is a clearer separation between the decision to pursue an opportunity and how that opportunity pans out.
Organizations are perfectly designed for the results they get
“If you want different results, you must be willing to redesign the system that generates them. Everything else is noise.”
Aging (work, not people)
“Aging is the leading indicator and root cause of all that overplanning” Aging of work items is about how long things hang around in backlogs. It’s ‘overplanning’ because of all the work that goes into planning items that either never get delivered or need replanning again and again before they do.
(Thanks to Emily Webber for the link)
I thought:
Being outcome-focused demands higher quality outputs
I agree with the idea of ‘outcomes over outputs’, but the more deeply I think about what it takes, the harder it gets to separate them. Sure, they are definable in ways that make them clearly different. Outcomes are changes in user behaviour that gets results and outputs are tangible changes in products and processes. But, not only are they causally connected by outcomes only being affected by outputs, but the quality of the output affects the outcome too.
Standard of shared understanding
Did a bit more thinking on my standard of shared understanding, partly due to an interesting conversation about how hard it is to achieve. I need to make sure the standard never implies that the same shared understanding is possible. Everyone has their own unique individual understanding from their perspective, and that’s a good thing, the tough part is in creating enough common ground. That’s where really good user stories as boundary objects (not detailed requirements) are particularly useful. We need more boundary objects.
It’s who you know, not what you know
Heard an interesting point about enterprise AI adoption; it’s a context problem. It’s a data problem too but the path to a solution is more knowable, whereas organisations with multiple complex contexts that require human judgment about what applies in which situation are just not available to AI. That’s a big blocker to AI being anything more than a productivity tool, it doesn’t lead to transformation.
As an example, one of the conversations I had this week was about how to find the one right person who knows how to do one very specific niche thing. I replied that the theory of weak ties gives us the answer and we should spend time getting to know people so that we are as few steps as possible away from that person because of the other people we know. Knowledge about who to speak to and how to keep that info up-to-date when people move on or responsibilities change, is tacit knowledge, it isn’t codifiable in a way AI could understand.