Part IV. Value, measurement and more than one team

One ladder of goals

Product Goal, Sprint Goal, EBM, OKRs and the logic model

Retaining members

At the October strategy review, Begoña set a single goal for the following year: retain members. GymTonic was losing one in three in the first year, and half of those in the first three months. Everything else, she said, had to serve that.

At the next Sprint Planning, Olga wrote the company goal on the whiteboard and, below it, the Sprint Goal the team was proposing: "integrate payments with the new provider". They stood looking at the two sentences. There was no way to connect them. The change of provider was necessary, but it was not going to retain anyone.

A lot of what does not matter

AI produces a lot. Without a ladder of goals that connects what the team does with what the company wants, it produces a lot of what does not matter. And it produces it fast, so the problem shows up late: when there are already months of deliveries that have not moved anything.

The rungs

The Evidence-Based Management (EBM) framework, for managing based on evidence, arranges goals in three levels:

The Strategic Goal: what the organisation wants to achieve, years ahead. Retain members.
The Intermediate Goals: the steps that bring you closer to the strategic one, months ahead. For example, new members finding their place in the first month.
The Immediate Tactical Goals: what you do now to get closer to the intermediate one. A Sprint Goal.

In Scrum, the Product Goal and the Sprint Goal are rungs on that ladder. The Product Goal is an intermediate goal for the product; the Sprint Goal is the immediate tactical one. If a rung does not rest on the one above, the ladder leads nowhere.

What to look at on each rung

EBM proposes measuring four Key Value Areas:

Current Value: what the product delivers today to the people who use it. Satisfaction, usage, retention.
Unrealized Value: what it could deliver and does not yet. Members who do not use the app, needs not yet met.
Time-to-Market: how long it takes for an idea to reach users and bring back learning.
Ability to Innovate: how much time the team can spend on new things, rather than fighting fires or maintaining old ones.

With AI, the temptation is to look only at Time-to-Market, because that is what visibly improves. The other three tell you whether it is worth anything.

OKRs for the middle rung

OKRs are a good format for the intermediate goal: an ambitious, qualitative objective that says where to go, and some key results (quantitative) that say whether we are getting closer. With one condition: a key result is an outcome for customers or a benefit for the company, not a deliverable.

"Launch the waiting list" is a deliverable. "40% of new members book at least one class a week in their first month" is a good key result. The deliverable may or may not help achieve it; the key result tells you.

The logic model, outside software

For AI initiatives that are not software (training staff, rolling out an assistant in member services, changing an administrative process), the logic model works well. It is a tool that comes from the evaluation of public and social programmes. It arranges an initiative in five links:

Resources: budget, people, licences.
Activities: what you do. Train, configure, pilot.
Outputs: what comes out. Trained people, a working assistant.
Outcomes: what changes in behaviour. Member services resolves more queries without waiting times.
Impact: what changes in the organisation. More members who stay.

Diagram of the logic model: five links in a row. What you do: resources (budget, people, licences), activities (train, configure, pilot) and outputs (trained people, a working assistant). What changes: outcomes (more queries resolved without waiting times) and impact (more members who stay). Many AI adoption reports stop at the outputs. Adapted from the W. K. Kellogg Foundation logic model, based on a diagram by Josh Seiden.

Its value is that it separates what you do from what changes. Many AI adoption reports stop at the third link: so many people trained, so many active licences. The model forces you to ask about the fourth and the fifth.

The ladder test

If the Sprint Goal or the Product Goal does not touch any company goal, the problem may well be in how work reaches the team. They may be tactical needs, but not strategic ones.

At GymApp, the change of payment provider went ahead, because it had to be done. But Héctor wrote, for the first time, a Product Goal connected to Begoña's goal ("new members find their place in the first month"), and since then every Sprint Goal is written beneath it, with its measure.

In short

Without a ladder of goals, AI produces a lot of what does not matter. Strategic, intermediate and tactical goals: the Product Goal and the Sprint Goal are rungs. Measure with the four Key Value Areas and use OKRs with key results that are behaviours. Outside software, the logic model separates what you do from what changes.

Try it this week

Write on one page the chain of three goals for your product: the strategic one, the intermediate one and the current Sprint Goal. If a rung is missing or they do not connect, that is where the work is.

For the middle rung, the OKR tracking template (in Spanish) helps you write key results and track them during the quarter. And if you want more on EBM, it is in the article What is Evidence-Based Management? (in Spanish).