Where these mechanisms meet a real room

When my kids ask what bank work looks like, a live demo beats any job description I've tried. Year 8 and 9 students got the same treatment. I started with sneakers, marked down in front of them, because that demo rarely fails.

From the actual session: the envelopes, the handout, and the room.

For a teenager, money runs less like a calculator and more like a calculator spliced onto Instagram.

By lunch the word “anchoring” will be gone, and that is fine. Thirty hands went up for “$80 in savings,” then heard, immediately, that they'd fallen for it. That beat worked because they owned the mistake in public. The envelope bit was weaker: I talked through it from the front instead of putting coins in their hands. Afterwards they argued about whatever they had physically handled. Next time, every pair of hands gets coins for that part.

South Park came first. In the underpants episode the gnomes write a plan on a whiteboard: Phase 1, collect underpants. Phase 2, a question mark. Phase 3, profit. Ask about Phase 2 and they freeze. I showed that slide so people could laugh, then pointed out how close it sits to our roadmap decks.

The example that landed hardest: “just increase engagement”

I asked them to map a familiar case: a bank wants more app use. The chain almost writes itself: Customers → Engagement → Loyalty → Retention → Products per Customer → Profit. It looks tidy. Then you check the data.

Drawing the arrow from Initiative to Profit was easy. Defending it was not.

Kohavi's number and Sowell's line were scaffolding. The work happened when people mapped Phase 2 for something they had shipped and hit the gnomes' wall themselves. Nobody likes learning their success metric was a stand-in for something else. Order mattered: people who mapped after the engagement twist already knew the punchline, so their maps were softer than the ones who went first. Next time I run the exercise before the reveal.

For about a decade, banking-app design has treated more money visibility as an unalloyed good. Show the live balance, show what is “safe to spend” after bills, and people will spend more carefully. I opened with a finding that cuts against that story.

When paying hurts less, people buy more. A “safe to spend” figure is credit's old trick dressed as help.

Most people in the room had shipped a smoother payment flow by stripping friction out. Few had checked whether that friction had been protecting anyone. Nobody plans to bake the credit-card effect into a savings feature, yet that is what an inflated balance does. I audited a stranger's balance screen live, and the pattern still showed. If the room audited its own screens instead of watching mine, they would catch more, and care more.

John List keeps returning to an unglamorous question: why do ideas that look strong in a small pilot collapse once you try them on everyone? The Voltage Effect names that pattern a “voltage drop.” I had already skimmed it in the outcome-logic session. This talk went deeper, using one of List's field experiments and the story that best shows the “very best people” trap product teams walk into.

A pilot staffed with your best people proves your best people work. It does not prove the idea does.

Ask who staffed their last pilot with the A-team and most hands go up, on purpose, to give the idea its best shot. CHECC's design pushes against that instinct, and that story landed harder than Uber's because it argues with habit rather than just showing a neat result. Once you sit with List's point it is plain: stacking a pilot with your strongest people answers a different question than the one you thought you asked, and most teams never notice the swap.

I booked a routine check-up and a filling. Between the X-ray and the drill I mentioned what I work on, then spent the rest of the visit spotting behavioural economics in the room. None of it looked designed, and it did not need to. The mechanisms were already running, and I would bet they already show up in that practice's revenue whether anyone there can name them.

None of this was designed as behavioural economics, and it did not need to be. It was already working, and it is probably already in that practice's revenue line whether anyone there can name a mechanism.

No single moment stuck. The stack did: one ordinary appointment, unplanned, unnamed, each piece doing real work. A dental practice is not a fintech or a subscription product, and nobody there runs an experimentation programme. That does not matter. The mechanisms do not ask permission. If I ran a version of this for clinicians or small-business owners, I would open with the same question I ask any product team: which of these are you already doing by accident, and which one, done on purpose, would move the number you care about?

I open every version of this talk the same way, because most rooms arrive with one of two wrong pictures. Either behavioural economics is “common sense with a fancy name,” or it is “psychology with extra steps.” Both miss the claim. If I do not fix that in the first five minutes, every later example lands as a trick instead of a discipline. So the chart goes up first.

Economics studies incentives and decisions, which is another way of saying behaviour. Classical economics sits at one end: simple and scalable, because it treats people like spreadsheets, rational, consistent, always maximising. Psychology sits at the other: high on explainability, because it studies the real mechanism, but a full account of one person's decision will not scale into a rule you can ship. Behavioural economics sits between them on purpose, informed by psychology, still asking what to predict and when. Part spreadsheet, part Instagram: mostly rational, with specific, predictable biases on top.

The concrete example: classical economics says a free product and a one-cent product should behave about the same, because a cent is nearly nothing. They do not. Zero Price Effect research finds that genuine free does more than lower cost. It changes the category of the decision. Demand for a free item jumps far more than a one-cent gap predicts. That is the line under the chart: classical economics misses it, behavioural economics catches it, and the reason is a specific, citable, replicated finding.

Why any of this matters for product and design work

Once the definition lands, the room asks why it matters here. Product and experience work already has a one-line job most teams never say out loud: influence human behaviour, on purpose, toward an outcome. Jeff Patton's chain puts it plainly: find it → try it → use it → keep using it → say good things. That sits under the acquisition, activation, retention, referral, and revenue funnel a growth team already reports every week. If product work is behaviour change, a real model of how people decide is not a nice-to-have on top of the roadmap. It is what the roadmap is for.

“So it's just marketing with a science degree?”

Fair pushback, and I hear it most times I give this talk. The honest answer is falsifiability. Marketing wants something to work and will keep the story if it does. Behavioural economics wants to know when a mechanism stops working, and for whom. That is why every principle on this site carries a named study, a sample, and a methodology critique, not only a claim. I used to leave that distinction for Q&A. Now it sits right after the chart, because the room trusts what follows once they have heard it.

In almost every room, usually from the most senior person, someone asks: “why can't we just make the change and monitor a metric?” Fair question. A controlled test costs time, traffic, and engineering a simple launch does not. Instead of arguing, I show a real chart and let the room talk itself into the wrong answer first. It is a live demo, not my own controlled study, and it is the fastest way I have found to make the risk felt rather than merely stated.

The real source, and what it actually found

The example is not hypothetical, and it is not mine. It comes from Airbnb's own engineering blog, and the whole talk hangs on it.

“So basically, I can't trust anything I haven't tested?”

Someone asks a version of this most times I run the reveal, usually sounding a little defeated. That is not quite the lesson, and I say so faster than I used to. A metric alone cannot tell you why it moved, and a good story about why is what makes people stop checking. The Airbnb example sticks because the story was so plausible: faster to scan, better on mobile, matches how people search. Nobody would have doubted it without the A/B result underneath. I used to close with the confounder list. Now I close with the reveal line again, because that is the sentence people actually carry out of the room.

Agile Australia put 200+ product and agile leaders in front of me for about 45 minutes. I had given a version of this talk before. The Uber apology finding shows up here, and so does the Airbnb filter reveal. This was the first time I built the whole thing around one frame: a “10x” way of judging product work, and why most organisations never get there. I opened the way I open most rooms: with a number, not a definition.

From the actual talk: AgileAus23, August 2023.

It's powerful, like a chainsaw, which means it can cut trees with ease, but also limbs.

That line closed the talk on purpose. The two heavyweight case studies I lean on most, the Uber apology finding and the Airbnb filter reveal, already have full write-ups on this site. I did not re-tell either one here just to pad the session. Both are linked below, the same way I would point a room to them afterwards. What this talk added was the frame: a “10x” way of judging whether applied behavioural economics actually worked. Confidence of impact, speed to learning, scalable learning, probability of working, and the size of the measurable outcome. When I asked which of the five they were weakest on, most of the room said measurement. Not the theory. Not the idea. The part where you find out if you were right.