Every one starts from a mechanism already covered on this site, a real field session or a real cited study. Browse by product area, or jump straight to any blueprint by name.
These are blueprints, not results. Each one starts from a mechanism already on this site, a field session that was run, or a cited study. The write-up then spells out what a real test would need: a theory, a baseline to beat, control versus treatment, a sample large enough to trust, and the outcomes that would settle it. Search by name, or narrow by product type.
How the pieces fit. Under Mental Accounting, money is not one pool in someone's head. It gets sorted into labelled buckets, and a labelled bucket gets protected in ways a generic balance does not. Salience is what does the labelling work in practice. A name people can picture (“Save for Phone”) stands out against the interchangeable rows around it (“Savings Pocket 2,” “Everyday Account”), the way one bold figure jumps off an otherwise plain statement.
What already ran.The high school field session put the labelling half in front of a room of Year 8/9 students with paper envelopes. Roughly $30 piled up in the “save” envelopes without anyone consciously deciding to save that amount.
What this isolates. The digital version asks whether a name that stands out in an account list protects a balance the same way a hand-labelled envelope did in that room.
Hypothesis
Ask someone to name a new savings pocket when they open it, instead of leaving the auto-generated label in place, and the pocket should behave differently afterward. A smaller share of that balance should hit zero within 90 days. The naming step should not add enough friction to cut onboarding completion in a meaningful way.
Assumed baseline
Illustrative, not real product data. Assume 30% of customers who open a second savings pocket withdraw its full balance within 90 days. The generically-labelled pocket behaves less like “savings” and more like a second everyday-spend account with extra taps.
Control vs. treatment
The only thing that changes between the two arms is who chooses the label. Everything else about opening a pocket stays identical.
Control · auto-named pocket
Savings Pocket 2
A new pocket, automatically named. Nothing to fill in, one tap and it's created.
Treatment · named at creation
What's this pocket for?
Give it a name you'll picture later, it's the easiest way to keep track of what this money's already spoken for.
PhoneHolidayEmergency
Sample size to actually trust this
Baseline30%90-day full withdrawal
Target24%−6pp, ~20% relative
Significanceα = 0.05two-sided
Power80%1−β
Sample per arm856customers
Total sample1,7122 arms
Assumed volume800/wknew second pockets
Est. run time~3 wksincl. buffer
Two-proportion test, equal allocation: n = (zα/2 + zβ)² × [p₁(1−p₁) + p₂(1−p₂)] / (p₁−p₂)². The volume figure is an assumption to make the timeline concrete, not a real product number. Swap in your own weekly pocket-creation rate and the run time follows.
Outcome variables
✓Primary: % of pocket balance withdrawn to $0 within 90 days (lower is the win)
✓Secondary: average days-to-first-withdrawal: does naming delay the first dip, not just reduce the total
✓Guardrail: pocket-creation completion rate: the naming step must not become a reason people abandon the flow
✓Guardrail: time-to-complete onboarding, and the % who skip the naming prompt where a skip option exists
The mechanisms behind this:Mental Accounting & Salience, the full study-and-citation write-up for each is on the Principles page.
Experiment blueprintDigital & Product Experiment
Retail Banking · Credit & LendingAlso applies to: BNPL, personal loans, any business selling more than one way to finance the same purchaseNot yet run
Does tradeoff transparency work between products, not just within one?
CBA showed the effect when people compared credit cards to each other. Does the same logic hold when credit is set beside other ways to pay?
Extends Tradeoff Transparency, from the Buell & Choi (2025) field experiment inside Commonwealth Bank of Australia's credit card funnel
Theory
What CBA already proved. Buell and Choi ran a live field experiment in CBA's credit card acquisition funnel. Putting each card's drawbacks on equal footing with its perks, a “Good and the Bad” page instead of benefits-only marketing, left overall take-up statistically flat while improving which card people chose. Customers who saw the downsides later spent 9.9% more, cancelled 20.5% less, and missed fewer payments. The full teardown, with the real numbers, sits on this site's Tradeoff Transparency page.
What still sits untested. That work stayed inside one product category: card versus card. It never asked the next-layer question: credit card against the other ways the same person could finance the same purchase, a personal loan or a Buy Now, Pay Later plan. Those options still live on separate pages, owned by separate teams, with no shared drawbacks shown against each other.
Hypothesis
Put a customer's real financing options, credit card, personal loan, BNPL, on one screen, with each option's drawbacks named as plainly as its benefits. Some people should move toward the fit that matches how they plan to use the money. Overall take-up should stay roughly flat. CBA's within-category version changed which card people picked without changing how many applied, and this is that mechanism one product-category higher.
Assumed baseline
Illustrative, not disclosed CBA data. Assume 8% of applicants who land in today's credit-card-only funnel end up choosing a personal loan or BNPL plan instead, entirely on their own initiative. The funnel never surfaces either as an alternative, so this only happens when someone already knew to go looking for one.
Ethical guardrail
Where the line sits. Making a financing alternative visible for the first time changes the decision environment, so the constraint belongs up front. The design is not there to talk people out of revolving credit. For many customers a credit card is still the right tool.
What the design is allowed to do. It surfaces the other real options before commitment: inform the choice without steering it into a preferred product, the same restraint the original CBA study held. See Not Testing Is Still a Bet for the longer case for that line.
Control vs. treatment
Control · one product at a time
Low Rate Card
12.99% p.a., no annual fee. No rewards, no interest-free days on cash advances.
Treatment · compare how you'd actually pay
Three ways to finance this
Credit Card: goodReusable, ongoing credit line
Credit Card: badInterest compounds if not paid in full
Personal Loan: goodFixed end date, usually lower rate
Personal Loan: badCan't redraw once it's repaid
BNPL: goodNo interest if paid on time
BNPL: badLate fees stack fast, builds no credit history
The card stays the first, largest option in both arms. The treatment only adds the comparison, it doesn't hide or de-rank anything.
Sample size to actually trust this
Baseline8%choose loan/BNPL unprompted
Target14%+6pp, ~75% relative
Significanceα = 0.05two-sided
Power80%1−β
Sample per arm423applicants
Total sample8462 arms
Assumed volume2,400/monew financing applicants
Est. run time~3 wksincl. buffer
Two-proportion test, equal allocation: n = (zα/2 + zβ)² × [p₁(1−p₁) + p₂(1−p₂)] / (p₁−p₂)². The volume figure is an assumption: swap in a real monthly applicant count and the timeline follows.
Outcome variables
✓Primary: among applicants who switch to a loan or BNPL plan, missed-payment or default rate in the first 6 months, benchmarked against that product's normal book. The actual fit test: does an informed switch perform better, or just happen more often
✓Secondary: % of applicants selecting a personal loan or BNPL plan instead of a credit card. The volume shift the primary outcome needs to be interpreted against
✓Secondary: among applicants who still choose the card, average interest paid in the first 6 months: does staying on-card get cheaper too when the choice was more informed, mirroring CBA's own within-category result
✓Guardrail: overall financing conversion rate across all three products combined. Must not drop as a side effect of showing the comparison
✓Guardrail: support-contact rate referencing the comparison screen, as an early read on confusion
The mechanism behind this:Tradeoff Transparency, the full CBA study-and-citation write-up, including the real result table, is on the Principles page.
Experiment blueprintDigital & Product Experiment
Financial Services · Service & SupportAlso applies to: telco, airlines, delivery/logistics, any service business handling failures and complaintsNot yet run
Does a costly, specific apology beat a generic one after a service failure?
List showed a costly apology paid for itself the first time, then backfired when the same coupon repeated. How does a design hold both findings at once?
What the Uber field test showed. Halperin, Ho, List and Muir ran this at scale on 1.5 million Uber riders after a late ride. A $5 coupon apology paid for itself by restoring future spend that a generic “sorry” alone did not restore. A costly signal can do work a free apology cannot. That mechanism is documented here as Reciprocity.
Why the design needs a second branch. The same paper found that sending the identical coupon again after a second bad ride made riders spend less than if Uber had said nothing. The authors' reading: an apology reads as a promise, and recycling the exact same one a second time reads as that promise already broken. Full numbers sit in the Voltage Effect field session.
Hypothesis
After a service failure, a costly and specific apology, a fee credit plus a plain account of what went wrong, should beat a generic scripted apology on post-incident satisfaction. Because reusing an apology verbatim can reverse the gain on a repeat failure, the design also needs a secondary branch on the same sample. Does changing the apology's content and framing keep the advantage when a fixed script would lose it?
Assumed baseline
Illustrative, not real support-desk data. Assume post-incident CSAT (1–5) currently averages 3.1 after a generic apology message, standard deviation 1.1, typical for a scripted, one-size response with no real cost attached.
Control vs. treatment
Both arms fire at the same moment, right after a qualifying failure is logged. Only the cost and the specificity of the response change.
Control · generic script
Sorry about that
We're sorry for the inconvenience. Thanks for your patience.
Treatment · costly & specific
Your transfer arrived 40 minutes late
A processing delay on our end. We've refunded the $3 fee and added a $5 credit. Sorry for the wait.
Sample size to actually trust this
Baseline mean3.1CSAT, 1–5 scale
Target mean3.4+0.3, ~10% relative
Assumed SD1.1illustrative
Significanceα = 0.05two-sided
Sample per arm212incidents
Total sample4242 arms
Assumed volume150/wkqualifying incidents
Est. run time~3 wksincl. buffer
Two-sample means test, equal allocation: n = 2 × (zα/2 + zβ)² × σ² / (μ₁−μ₂)², power 80%. This covers the headline test only: the repeat-failure sub-branch below draws from a smaller slice of this same sample, not an additional recruited group.
Outcome variables
✓Primary: post-incident CSAT (1–5), collected immediately after resolution
✓Secondary: 90-day retention for affected customers, matching the real spend-based outcome List's team used
✓Guardrail: cost per incident (average credit issued) weighed against retained-revenue estimate. The whole point of List's finding is that this pays for itself
✓Guardrail: support re-contact rate: the test of whether a specific explanation actually resolves the question, rather than only sounding reassuring
Sub-branch: if the failure repeats
Whichever customers in the headline sample have a second qualifying failure within the test window fall into this branch automatically. It isn't a separately recruited group, and it isn't the result the experiment is powered to confirm. Because who gets a second failure isn't random, treat this cut as exploratory: a read worth having, not a policy to ship on its own.
Sub-branch: same script again vs. varied
Control · same script again
Sorry about that
We're sorry for the inconvenience. Thanks for your patience.
Treatment · varied, not repeated
This shouldn't have happened twice
We've escalated the underlying issue to our transfers team, and changed how it's handled going forward.
The mechanism under test here isn't whether to apologise again. It's whether the repeat apology reads as a new response to a new problem, or the same promise, broken again.
Outcome for this branch: the same post-incident CSAT measure as the headline test, read only within this subset. Plus, as a directional check rather than a powered comparison, whether “varied” avoids the spend-drop the Uber study found when the identical coupon repeated.
Retail Banking · Credit & LendingAlso applies to: insurance, wealth management, business banking, any multi-channel business launching one standout digital capability for the first timeNot yet run
Does a great digital home loan lift home loan trust everywhere else, too?
A customer who never opens the new digital home loan still looks more ready to sign one at a branch. Why would awareness alone move that?
Applies Halo Effect · Thorndike (1920), Journal of Applied Psychology
Theory
Thorndike's claim, compressed. A rater who forms a strong impression on one visible trait spreads that same judgment to other traits of the same target they have not actually scored. See Halo Effect for the full study. Thorndike, and later Nisbett & Wilson, measured that at the level of one person's traits: an instructor's warmth colouring ratings of his unchanged accent and mannerisms.
Moving the same claim onto channels. Here the target is a brand's channels rather than a person's traits. The bank in this scenario has never offered a digital home loan; every application has gone through a broker or a branch appointment. It is about to launch a fully digital application: pre-approval in an afternoon, no paperwork mailed anywhere. Launch day changes nothing else: branch process, broker relationships, underwriting, and call centre stay identical to the day before.
If the halo travels. Awareness alone should matter. Knowing the bank now has one excellent digital channel should lift trust in the channels that did not change, and application intent in those same unchanged channels should rise with it.
Hypothesis
People who see messaging about the bank's new digital home loan capability, whether or not they ever open it, should show higher intent to take a home loan with that bank. The lift should show through any channel, digital or otherwise, versus a matched group with no exposure. That is the headline claim. Awareness of one strong channel lifts trust across the relationship.
Held out of the headline. Whether people who later use the digital channel show a still larger lift is a secondary, non-randomised read (see Sub-branch). It stays out of the primary comparison.
Assumed baseline
Illustrative, not real/disclosed data. Assume 6% of existing customers who are in the market for a home loan in a given quarter say they'd seriously consider getting it with this bank, through any channel. That is before the digital product launches or any awareness messaging goes out.
Ethical guardrail
The failure mode to watch. A halo is trust extended to something nobody actually checked. If digital-channel awareness lifts intent to apply through a branch or broker, and those channels have not been resourced to match the promise the digital product makes, the bank has a problem. It converts people on trust it has not earned in the channel they actually use.
What this test is not permission for. It is not cover to under-invest in non-digital application support because the digital halo is doing the persuading; see Not Testing Is Still a Bet. The outcome guardrails below exist specifically to catch that failure mode before a customer hits it.
Control vs. treatment
Both arms see the same everyday banking app. Only the presence of a banner about the new digital home loan capability changes, who taps it, or applies through any channel afterwards, is observed, not assigned.
Control · no exposure
Accounts
Everyday Account · $2,140.50 Savings · $8,760.00
Treatment · aware of digital home loan
Get pre-approved this afternoon
Our new home loan application: entirely digital, no paperwork mailed anywhere.
Sample size to actually trust this
Baseline6%channel-agnostic consideration
Target9%+3pp, ~50% relative
Significanceα = 0.05two-sided
Power80%1−β
Sample per arm1,205customers
Total sample2,4102 arms
Assumed volume500/wkeligible customers entering test
Est. run time~5 wksincl. buffer
Two-proportion test, equal allocation: n = (zα/2 + zβ)² × [p₁(1−p₁) + p₂(1−p₂)] / (p₁−p₂)². This covers the headline aware-vs-not comparison only: the sub-branch below draws from a smaller, self-selected slice of the treatment arm and isn't independently powered.
Outcome variables
✓Primary: % who apply for, or seriously consider, a home loan with the bank via any channel within 6 months of the awareness window, channel-agnostic conversion, the actual halo test
✓Secondary: the same metric split by channel of application (digital vs. branch/broker/phone), the lift showing up in channels that never changed is the halo signature; a lift confined to the digital channel alone is not
✓Secondary: unprompted, survey-measured trust/consideration score for a home loan specifically with this bank, independent of whether an application was ever started
✓Guardrail: branch/broker application abandonment rate and time-to-decision for the treatment group: a customer drawn in by the digital halo must not land in a slower, un-improved process than the impression promised
✓Guardrail: complaint rate and post-application CSAT for non-digital applicants in the treatment group, compared to control, catches the halo luring people into a channel that then under-delivers
Sub-branch: does actually using it deepen the halo further?
Whichever treatment-arm customers go on to actually start or complete a digital home loan application fall into this branch automatically. They aren't randomly assigned to “use it,” they choose to, so this comparison is self-selected and exploratory, not a third randomised arm. Treat it as a directional read on dose-response, not a number to plan a rollout around.
Sub-branch: saw the banner vs. actually applied digitally
Aware · saw the banner, didn't apply
Get pre-approved this afternoon
Seen, not opened. The customer never starts the digital application.
Used · started or completed a digital application
Pre-approval in progress
Direct experience of the excellent trait, not just awareness of it.
The mechanism under test here is the same one Thorndike's raters showed for a single trait: exposure alone produced some halo, but direct, first-hand experience of the excellent trait produced more.
Outcome for this branch: the same channel-agnostic home loan intent measure as the headline test, read only within the treatment arm. Compare those who engaged with the digital product against those who were merely exposed to it.
The mechanism behind this:Halo Effect, Thorndike (1920) and Nisbett & Wilson (1977), full study and citations on the Principles page.
Experiment blueprintDigital & Product Experiment
Retail Banking · Credit & LendingAlso applies to: personal loans, credit cards, any product priced with a percentage rate customers compare at a glanceNot yet run
Does a home loan rate of 5.99% get remembered as a 5% loan, not almost 6%?
A 5.99% home loan rate sits almost at 6%. Does comparison behaviour treat it like a 5% offer instead?
Mechanism: Left Digit Bias · Lacetera, Pope & Sydnor (2012), American Economic Review
Theory
Evidence from cars and price tags. Lacetera, Pope and Sydnor showed that a number's leftmost digit dominates how people judge its size. Crossing a leading-digit boundary, 39,999 miles to 40,000, or $2.99 to $3.00, shifts perceived value far more than the tiny real gap justifies. Their evidence is 22 million real used-car sales; Thomas & Morwitz found the same pattern in ordinary retail prices.
Onto an interest rate. Neither study used a lending rate. This blueprint puts Left Digit Bias on a home loan's headline figure. A rate advertised at 5.99% p.a. and one at 6.00% p.a. cost a borrower almost the same in real repayments. The leading digit still flips: 5.99% reads as “5% something,” while 6.00% reads as “6%” outright.
Expected behaviour. If the mechanism travels the way it does for an odometer reading, 5.99% should be judged, and chosen, as meaningfully cheaper than 6.00%, out of proportion to the real repayment difference.
Hypothesis
A home loan advertised at 5.99% p.a. should look cheaper than an otherwise identical loan advertised at 6.00% p.a. It should also convert to more started applications, even when the two rates sit close enough that the monthly repayment gap is trivial.
Assumed baseline
Illustrative, not real/disclosed data. Assume 4% of visitors to the rate page who are shown a 6.00% p.a. headline rate go on to start an application within the same session.
Ethical guardrail
Already industry practice. This experiment does not invent a new advertising trick. .99 rate framing is already standard, legal advertising across lending, and the real cost stays disclosed via the comparison rate whichever headline number is shown.
The measurement question. How far that convention alone moves behaviour. With real numbers, a bank can decide whether to keep leaning on it, limit how far it leans, or counter it by surfacing the comparison rate more prominently. See Not Testing Is Still a Bet for why measuring this beats assuming it is fine because everyone already does it.
Control vs. treatment
The comparison rate, the standardised, all-costs-included figure regulators require, is identical in both arms. Only the headline rate's leading digit changes.
Control · 6.00% p.a.
6.00% p.a.
Variable rate home loan. Comparison rate 6.04% p.a.
Treatment · 5.99% p.a.
5.99% p.a.
Variable rate home loan. Comparison rate 6.04% p.a.
Same loan, same real cost, same comparison rate. The only thing that moves is which side of the 6% boundary the headline number sits on.
Sample size to actually trust this
Baseline4%started an application
Target5.5%+1.5pp, ~38% relative
Significanceα = 0.05two-sided
Power80%1−β
Sample per arm3,150rate-page visitors
Total sample6,3002 arms
Assumed volume900/wkeligible visitors entering test
✓Primary: % of rate-page visitors who start a home loan application within the same session
✓Secondary: intercept survey asking visitors to rate the offer as cheap / about right / expensive, checking whether the perception itself shifts, not just the click
✓Guardrail: application-to-approval drop-off once the comparison rate is unavoidably shown. The headline number must not be pulling in applicants who bail once they see the real cost
✓Guardrail: complaint or dispute rate referencing the rate not matching expectations, a spike here would mean the framing misled people beyond the ordinary digit effect
The mechanism behind this:Left Digit Bias, Lacetera, Pope & Sydnor (2012) and Thomas & Morwitz (2005), full study and citations on the Principles page.
Experiment blueprintDigital & Product Experiment
Retail Banking · Credit & LendingAlso applies to: personal loans, car loans, any credit product weighing a prize-linked promotion on top of an already competitive rateNot yet run
Does a chance at 0% increase home loan applications more than the guaranteed discount alone?
The guaranteed rate is already competitive. Does adding a small, real chance at 0% for a year pull in more applicants than that guaranteed rate on its own?
Grounded in Probability Weighting · Kahneman & Tversky (1979), Econometrica · Rottenstreich & Hsee (2001), Psychological Science · Burger, Hendriks, Pleeging & van Ours (2020), Experimental Economics
Theory
Where the mechanism already shows up in product.Bankwest's real Interesting Rates promotion pays every Easy Saver customer a genuinely competitive 5.75% p.a. everyday rate, then draws 50 winners a month for an 11.50% p.a. rate on top. Probability Weighting is what makes that structure bite: a small, real chance of the better rate gets weighted in people's minds far past its true odds.
Two findings that sharpen the weight. Rottenstreich and Hsee (2001) found people overweight a small chance of something emotionally vivid even more than an equally small chance of plain cash of the same value. That is why a fast-food deal sells “free food for life” rather than its cash equivalent. A year of 0% interest on a real mortgage is at least as vivid: it converts into a large, concrete, easy-to-picture sum. Burger et al. (2020) found a separate effect: simply holding a lottery-style entry lifts happiness before the draw resolves, win or lose. A home loan application is normally a stressful, multi-week wait. A live entry gives an applicant something hopeful to hold during it, a different benefit from the conversion lift Probability Weighting alone would predict.
Why this is not a prize-linked savings swap.Classic prize-linked savings research, including Filiz-Ozbay, Guryan, Hyndman, Kearney and Ozbay's 2015 study, tests a real substitution: part of a saver's own guaranteed return is swapped for lottery expected value. Bankwest's promotion does not do that; its 5.75% rate exists whether or not anyone enters the draw. A home loan version that pairs an already-competitive guaranteed discount with a separate chance at 0% sits even further from that substitution: none of the applicant's real rate funds the prize. The test isolates Probability Weighting's own pull.
What gets compared. Advertising a genuinely competitive guaranteed discount alongside a separate, real chance at 0% p.a. for 12 months, versus advertising that same guaranteed discount alone, with the guaranteed rate held constant between arms. A second prediction runs in parallel: simply holding a live entry should change how applicants feel during the underwriting wait, beyond whether they start applying.
Hypothesis
Visitors who see the guaranteed discount plus a chance at 0% p.a. for 12 months should start more applications within 7 days than visitors who see the identical guaranteed discount with no draw. The guaranteed rate never changes between arms. Separate from that conversion lift, applicants in the treatment arm should report less stress and higher satisfaction during underwriting than control applicants. That matches the anticipatory-happiness effect Burger et al. found in lottery ticket-holders, whether or not they win the draw.
Assumed baseline
Illustrative, not real/disclosed data. Assume 8% of visitors to the new home loan rate page start an application within 7 days when the page advertises the guaranteed discount alone.
Ethical guardrail
Hard constraints on the offer. The guaranteed discount has to be genuinely competitive against the rate the bank would otherwise offer that same applicant. It cannot be a rate held back elsewhere to help fund the draw. It must stay identical in both arms so the draw is the only variable under test. The odds, the number of winners, and the exact prize, 0% p.a. for 12 months on a capped loan amount, need disclosing as plainly as the guaranteed rate itself.
What success is allowed to mean. The question is whether a real, cited effect helps a genuinely competitive offer reach more of the people it is meant for. It is not whether draw excitement can be tuned to pull applicants into borrowing more than they can service. Loan size and serviceability assessment run identically in both arms, and a guardrail metric below checks that directly. See Not Testing Is Still a Bet.
Control vs. treatment
Both arms are new visitors to the home loan rate page, randomly assigned. The guaranteed discount rate is identical in both arms; only whether the page also advertises a chance at 0% differs.
Control · guaranteed discount only
6.00% p.a. fixed, first year
A genuinely competitive fixed rate for your first 12 months, reverting to our standard variable rate after.
Treatment · guaranteed discount + prize draw
6.00% p.a. fixed, first year
The same fixed rate as always. Every eligible application this month is entered into a draw for 0% p.a. on that first year instead.
Same guaranteed rate either way: the draw is a separate chance at 0%, not a discount taken off the 6.00% on offer.
Same guaranteed rate, same eligibility, same application form in both arms. Only whether the page also advertises the draw differs.
Sample size to actually trust this
Baseline8%started an application, 7 days
Target10.5%+2.5pp assumed lift
Significanceα = 0.05two-sided
Power80%1−β
Sample per arm2,102rate-page visitors
Total sample4,2042 arms
Assumed volume3,000/moeligible rate-page visitors
Est. run time~6 wksto enrol both arms
Two-proportion test, equal allocation: n = (zα/2 + zβ)² × [p₁(1−p₁) + p₂(1−p₂)] / (p₁−p₂)². The volume figure is an assumption to make the timeline concrete, not a real product number.
Outcome variables
✓Primary: application-start rate within 7 days of viewing the home loan rate page, treatment versus control
✓Secondary: application completion rate among those who start, confirming the draw pulls in applicants who finish, not just curious clicks
✓Secondary: applicant-reported stress and satisfaction during the underwriting wait, a short post-decision survey testing whether simply holding a live entry lifts mood independent of the draw's outcome
✓Guardrail: share of started applications later reduced or declined at the serviceability-assessment stage must not rise in the treatment arm
✓Guardrail: support or complaint contacts referencing confusion about the draw's odds or eligibility must stay within the same range as ordinary rate-page queries
The mechanism behind this:Probability Weighting, Kahneman & Tversky (1979), together with the affect-rich-prize and anticipatory-happiness findings cited on that same page (Rottenstreich & Hsee, 2001; Burger et al., 2020). See also The Lottery You Can't Lose for the classic prize-linked-savings trade-off this experiment deliberately doesn't reproduce, and Why does Bankwest say you're “still a winner” even when you lose the draw? for the real campaign this concept extends into lending.
Experiment blueprintDigital & Product Experiment
Retail Banking · Fees & PricingAlso applies to: insurance, telecom, subscription and utility pricing, any recurring price a real business has to raiseNot yet run
Does explaining why a fee went up change whether customers think it's fair?
Kahneman, Knetsch and Thaler showed this with a hypothetical snow shovel. Does the same fairness logic hold for a real bank fee notice?
Dual entitlement in one line. The Perceived Fairness finding from Kahneman, Knetsch and Thaler is simple enough to state outright. An identical price rise looks fair when framed as protecting the seller's existing profit from a real cost increase. It looks unfair when it looks like exploiting a shift in demand or bargaining power. The number alone does not decide the verdict; the story behind it does. Their evidence is a telephone survey of Toronto and Vancouver residents judging realistic vignettes. One was a hardware store that raised snow-shovel prices the morning after a storm (rated unfair by roughly 82% of respondents). Another was a landlord charging existing versus new tenants differently for the same unit.
What the vignettes never covered. Those surveys measured stated opinions about hypothetical price changes. They did not measure real behaviour after a real change. None of them used a financial-services fee or rate notice, where the cost being passed on (funding costs, compliance costs) is far less visible to a customer than a snowstorm is.
What a live fee notice can show. If the same dual-entitlement logic holds, a bank that names the real cost driver behind a fee or rate increase should see the change land as fairer. Unlike KKT's survey, this design can check whether that shows up in behaviour: fewer support contacts, fewer customers leaving.
Hypothesis
Customers who get a fee or interest-rate increase notice that names a real, verifiable cost driver should be less likely to contact support to question or dispute the change. They should also be more likely to rate it as fair than customers who get an otherwise identical notice stating only the new number. The actual new fee or rate stays the same between groups.
Assumed baseline
Illustrative, not real/disclosed data. Assume 18% of customers who receive a bare fee-increase notice, new amount only, no explanation, contact support to question or dispute the change within 30 days.
Ethical guardrail
Honesty constraint. The mechanism only works cleanly when the disclosed cost driver is real and verifiable, a genuine rise in funding costs, compliance costs, or similar that the bank could substantiate if asked. Citing a vague or invented cost story to manufacture fairness would be deceptive framing, and sits outside this experiment's scope.
Scope of the claim. The test asks whether naming a true cost driver changes how a genuine, already-decided fee change lands. It does not ask whether that framing can justify an increase that would not otherwise be defensible on its own. See Not Testing Is Still a Bet for why measuring this matters more than assuming good framing is harmless.
Control vs. treatment
The new fee and its effective date are identical in both arms. Only the presence of a real, named cost driver changes.
Control · number only
Your account fee is changing
From 1 October, your monthly account fee will be $12, up from $8.
Treatment · number + real reason
Your account fee is changing
From 1 October, your monthly account fee will be $12, up from $8.
Why this is changing: our cost of providing fee-free access across the shared ATM network has risen. This new fee reflects that increase, nothing else about your account is changing.
Same new fee, same effective date. The only difference is whether a real, named cost sits behind the number or not.
Sample size to actually trust this
Baseline18%contacted support about the change
Target13%−5pp, ~28% relative
Significanceα = 0.05two-sided
Power80%1−β
Sample per arm820customers receiving the notice
Total sample1,6402 arms
Assumed volume300/wkcustomers receiving a fee or rate-change notice
✓Primary: % of affected customers who contact support to question or dispute the fee change within 30 days of the notice
✓Secondary: a one-question follow-up survey to a subsample, asking how fair the change felt (1–5), checking whether the perception itself shifts, not just the support-contact behaviour
✓Secondary: 60-day account or product retention rate among affected customers, versus a matched unaffected group
✓Guardrail: rate of complaints escalated to a formal dispute or ombudsman referral, naming a reason must not itself invite more scrutiny than a bare notice would have
The mechanism behind this:Perceived Fairness (Dual Entitlement), Kahneman, Knetsch & Thaler (1986), full study and citations on the Principles page.
Experiment blueprintDigital & Product Experiment
Retail Banking · Savings & DepositsAlso applies to: budgeting apps, neobanks, any card issuer with an onboarding flow and in-app messagingNot yet run
Does telling customers the payment-transparency heuristic change what they actually spend?
Prelec and Simester showed credit blunts the felt cost of paying. Does telling a customer that, at the right moment, put some of that cost feeling back?
Why the payment method changes the felt bill.Payment Transparency says a purchase's felt cost depends on whether the payment method makes someone rehearse the amount and whether the money leaves immediately. The Credit Card Premium is the sharpest documented case. Bidders told they would pay by credit bid close to double what cash-instructed bidders bid for the identical tickets, because credit strips out both cues.
Suggested in the principles, never isolated. Both principle articles' “Where it can help” sections, and the synthesis in How You Pay Changes What You Spend, argue those two cues can be put back on purpose. One version is a running balance at the moment of tap. Another is a receipt that totals spend by category. Neither study tested whether simply telling someone the mechanism, in plain language, changes what they actually spend. That is a weaker intervention than redesigning the interface. It relies on a customer remembering and acting on a fact rather than on a changed screen.
What the arms isolate. Whether education alone, with no interface change and no restricted product, moves real spending, and whether timing matters: repeated messages close to the moment of use, versus a one-time explanation before any spending has happened.
Hypothesis
Customers told the payment-transparency heuristic in plain language should spend less on their lowest-transparency linked payment method (a stored or linked credit card) over the following months than customers told nothing. The cut should be larger and last longer when the heuristic arrives repeatedly as a contextual in-app message near spending. It should be smaller when it arrives once as onboarding content before any transaction. Rehearsal and immediacy operate on what is salient while paying; a single abstract explanation earlier is a thinner version of the same idea.
Assumed baseline
Illustrative, not real/disclosed data. Assume customers in the control arm spend an average of $780 a month on their lowest-transparency linked payment method in the 90 days after opening a new debit account. Use a standard deviation of $360, reflecting how unevenly card spending is actually distributed across a customer base.
Ethical guardrail
How the message has to read. It must state the real mechanism honestly, something like “paying by credit doesn't feel like spending right away, which is part of why people spend more on it.” Shame-based or fear-based framing is out. The same wording has to apply regardless of income or existing balance, so it reads as financial literacy rather than a judgment about who is trusted to spend.
What the test is for. Whether informing someone of a real, cited mechanism changes their own behaviour once they know it. The question is not whether the copy can be tuned to push spending up or down on demand. See Not Testing Is Still a Bet for why measuring this matters more than assuming a well-meaning message is automatically harmless.
Control vs. two treatments
All three arms are new debit account holders in their first 90 days, randomly assigned at account opening. Only the presence, content, and timing of the heuristic message differ.
Control · no messaging
Your account
Standard account home screen. No spend-heuristic messaging is shown at any point.
Treatment A · in-app, contextual
You spent $64 on your credit card today
Paying by credit doesn't feel like spending right away, that's part of why credit purchases add up faster than debit or cash.
Try this: check your running balance before your next purchase.
Treatment B · onboarding, once
Before you start: how you pay changes what you spend
Research shows people tend to spend least with cash, more with debit, and most with credit, because credit doesn't feel like spending right away.
One-time note: you'll see this once, during setup. It won't be repeated.
Same account, same product, same starting balance. Only whether the heuristic is ever shown, and how often, differs between arms.
Sample size to actually trust this
Baseline$780avg. monthly credit spend, control
Target$730−$50, the smaller expected effect (Treatment B)
Significanceα = 0.05two-sided
Power80%1−β
Sample per arm815new debit account holders
Total sample2,4453 arms
Assumed volume900/monew debit account sign-ups
Est. run time~3 moto enrol, plus each customer's own 90-day window
Two-sample continuous-outcome test, equal allocation: n = 2 × (zα/2 + zβ)² × σ² / (μ₁−μ₂)². Sized against Treatment B's smaller expected effect, so Treatment A, expected to move spending further, is over-powered by comparison.
Outcome variables
✓Primary: average monthly spend on the customer's lowest-transparency linked payment method over the 90 days after enrolment, each treatment versus control
✓Secondary: share of total spend that shifts from credit to debit or cash over the same 90 days, whether the mix moves, not just the credit total
✓Secondary: message engagement: tap-through rate on Treatment A's in-app card, and completion rate on Treatment B's onboarding screen, confirming both arms actually saw the message before comparing outcomes
✓Guardrail: rate of missed or late credit card repayments among treated customers must not rise, the goal is awareness, not anxiety that backfires into worse account management
✓Guardrail: 90-day app engagement and account retention must not fall relative to control, the messaging should inform, not annoy people into disengaging
Retail Banking · Savings & DepositsAlso applies to: telco, insurance, and subscription apps with a multi-step onboarding or setup flowNot yet run
Does framing account setup as one set to complete finish more setups?
An arbitrary cluster of tasks can feel unfinished and pull someone to close it. Put that same visual frame around a bank's own setup checklist: do more customers actually finish setting up?
Builds on Pseudo-Set Framing · Barasz, John, Keenan & Norton (2017), Journal of Experimental Psychology: General
Theory
What Barasz and co-authors found.Pseudo-Set Framing groups otherwise arbitrary items into a visible set, and people push harder to finish that set. The pull survives when the reward stays fixed, when finishing costs effort, and even when participants are told the grouping is invented. Five studies covered gambling, effort tasks, giving, and purchasing.
Where product onboarding sits. Those studies never used a live product's own multi-step setup. Banks, telcos, and insurers already ask for several small, useful tasks on day one: verify ID, link a funding source, name a beneficiary, set a goal. Those tasks usually appear as a flat checklist with no shared visual frame.
What the arms compare. The same required steps, pictured either as ordinary checklist rows or as wedges of one circle to complete. Nothing new is required, and no extra step is added. The only change is whether unfinished work looks like an incomplete set. That is a different lever from the Goal Gradient Effect, which pulls people who are objectively closer to a finish line; here the group simply looks unfinished.
Hypothesis
New customers who see remaining setup steps as wedges of one “complete your profile” circle should finish every required step inside 14 days more often than customers who see those same steps as a plain checklist. Step count, effort per step, and the finishing reward stay fixed across arms.
Assumed baseline
Illustrative, not real/disclosed data. Assume 41% of new customers complete all 4 required setup steps (ID verification, linked funding source, a named beneficiary, a savings goal) within 14 days of opening an account under the existing plain-checklist flow.
Ethical guardrail
What each wedge is allowed to be. Every slice of the circle has to map to a step that is already required or already useful to the customer. No filler step exists only to round out the shape. Completion pressure cannot exceed what the checklist arm already uses: no delayed access to core banking, no extra interstitial nags, no dark-pattern blocking.
What the measurement is for. Whether a cited completion mechanism helps people finish setup they already need. The visual is not there to hurry someone through a step they had good reason to postpone. See Not Testing Is Still a Bet.
Control vs. treatment
Both arms are new account holders in their first setup session, randomly assigned at account opening. Only how the same 4 remaining steps are pictured differs; the steps, their order, and the reward for finishing are identical.
Control · plain checklist
Finish setting up
1 of 4 steps done: ID verified. Link a funding source, add a beneficiary, and set a savings goal to finish.
Treatment · one circle, four wedges
Your profile: 1 of 4 pieces in place
The same 4 steps, pictured as wedges of one circle. ID verified. Link a funding source, add a beneficiary, and set a savings goal to complete it.
Same steps, same reward: nothing required here changes, only how it's pictured.
Same account, same 4 requirements, same completion reward. Only whether the remaining steps are pictured as a checklist or as wedges of one circle differs between arms.
Sample size to actually trust this
Baseline41%complete all 4 steps, control
Target51%+10pt assumed lift, treatment
Significanceα = 0.05two-sided
Power80%1−β
Sample per arm386new account holders
Total sample7722 arms
Assumed volume1,200/monew account sign-ups
Est. run time~1 moto enrol, plus each customer's 14-day window
Two-sample proportion test, equal allocation: n = (zα/2 + zβ)² × [p₁(1−p₁) + p₂(1−p₂)] / (p₁−p₂)². At 1,200 new accounts a month split evenly, 772 total is reached inside roughly a month of enrolment.
Outcome variables
✓Primary: share of new customers who complete all 4 required setup steps within 14 days of account opening, treatment versus control
✓Secondary: median time to completion among customers who do finish, whether the pull to close the set also speeds up completion, not just raises the rate
✓Secondary: view rate on the profile/checklist screen itself, confirming customers in both arms actually saw their remaining steps before comparing completion
✓Guardrail: rate of steps completed with invalid or placeholder data (a fake beneficiary, an unreachable funding source) must not rise in the treatment arm, the goal is real completion, not gaming the shape
✓Guardrail: setup-related support contacts must not increase relative to control, the visual should clarify what's left, not create confusion or pressure
The mechanism behind this:Pseudo-Set Framing, full study and citation on the Principles page. See also Goal Gradient Effect, the related but distinct pull of getting objectively closer to a finish line, versus this experiment's test of a group simply looking unfinished.
Experiment blueprintDigital & Product Experiment
Retail Banking · Savings & DepositsAlso applies to: neobanks, fintech savings apps, any product paying a small variable bonus rateNot yet run
Does a prize-linked feature beat an equal-value rate bonus for new deposits?
Lab participants deferred more for a lottery-style payout than for a guaranteed one of identical expected value. Swap the same structures inside a live savings feature: do real deposits move?
Builds on The Lottery You Can't Lose · Filiz-Ozbay, Guryan, Hyndman, Kearney & Ozbay (2015), Journal of Public Economics; Kearney, Tufano, Guryan & Hurst (2010), NBER
Theory
Two pieces of evidence already on the table. In a controlled lab, people offered a lottery-style bonus deferred payment more often than people offered a guaranteed bonus of the same expected value. Low-balance participants showed the strongest pull. Michigan's Save to Win programme, covered in The Lottery You Can't Lose, moved $8.56 million into new accounts in its first year on that same structure.
What neither source isolates. The lab paid a one-off laboratory bonus. Save to Win is an entire account type. Neither one layers a payout-structure swap onto an existing digital savings feature while holding the rest of the product fixed.
The swap under test. Take part of an existing feature's guaranteed bonus rate and replace it with a prize draw of equal expected value. Same app, same customers, same expected payout. Only the shape of that payout changes.
Hypothesis
Customers offered a prize-draw bonus, funded from the same pool that would otherwise pay a guaranteed rate bump, should deposit more net new money into the dedicated savings feature over 60 days than customers offered the rate bump directly. Expected total payout matches across arms. The largest gap should sit among customers with the lowest starting balances, matching the lab pattern.
Assumed baseline
Illustrative, not real/disclosed data. Assume customers in the guaranteed-rate control arm deposit an average of $310 in net new funds into the dedicated savings feature over 60 days. Use a standard deviation of $410, reflecting how unevenly savings deposits are actually distributed.
Ethical guardrail
How the prize pool has to be funded. It comes from the exact dollars that would otherwise pay guaranteed interest. The product cannot become worse in expectation. Deposits stay whole win or lose, which is how this structure avoids triggering Loss Aversion on the principal itself. Odds and expected value need the same plain disclosure an interest rate would get, not excitement copy that buries them.
What success is allowed to mean. A cited payout-structure effect helping people save more of their own money. Framing aimed at low-balance customers who feel the lottery pull most cannot be tuned to push deposits past what they can spare. See Not Testing Is Still a Bet.
Control vs. treatment
Both arms are existing customers newly opting into a dedicated savings feature, randomly assigned at opt-in. The expected payout is identical in both arms; only whether it's paid as a guaranteed rate or a prize draw differs.
Control · guaranteed rate bonus
Savings Boost: +0.4% bonus rate
Deposits into this feature earn an extra 0.4% on top of your standard rate, paid monthly, guaranteed.
Treatment · prize-draw bonus
Savings Draw: monthly prize entries
Every $25 in this feature earns one entry into a monthly prize draw. Your deposit is never at risk, win or not.
Same value, different shape: the prize pool pays out the same total, on average, as the +0.4% rate.
Same underlying account, same expected payout, same funding pool. Only whether that payout arrives as a guaranteed rate or a prize draw differs between arms.
Sample size to actually trust this
Baseline$310avg. net new deposit, 60 days, control
Target$370+$60 assumed lift, treatment
Significanceα = 0.05two-sided
Power80%1−β
Sample per arm733customers opting into the feature
Total sample1,4662 arms
Assumed volume1,000/monew feature opt-ins
Est. run time~2 moto enrol, plus each customer's 60-day window
Two-sample continuous-outcome test, equal allocation: n = 2 × (zα/2 + zβ)² × σ² / (μ₁−μ₂)². At 1,000 opt-ins a month split evenly, 1,466 total is reached in roughly 3 months of enrolment.
Outcome variables
✓Primary: net new deposits into the dedicated savings feature over 60 days, treatment versus control
✓Secondary: feature opt-in rate itself, confirming both arms activate the feature at similar rates before comparing what happens after
✓Secondary: balance retention in the 30 days after the first prize draw resolves, whether engagement fades for customers who didn't win
✓Guardrail: rate of overdrafts or missed payments elsewhere in the account must not rise among treatment customers, the feature shouldn't pull in money people can't actually spare
✓Guardrail: support contacts about how the draw works or whether a customer won must stay within the same range as ordinary account queries, confirming the mechanic was clearly understood
The research behind this:The Lottery You Can't Lose, full citations and the Save to Win field data. See also Loss Aversion, the asymmetry a no-loss lottery structure is specifically built to avoid triggering.
Experiment blueprintDigital & Product Experiment
Retail Banking · Credit & LendingAlso applies to: insurance quotes, account switching, any lead form long enough that people abandon partway throughNot yet run
Does showing progress through a home loan appointment booking increase completion?
A pre-filled loyalty card sped people up with the identical distance left to go. In a long booking flow, does a visible progress cue do the same work, and does finer chunking of the remaining steps add anything on top?
Builds on Goal Gradient Effect · Kivetz, Urminsky & Zheng (2006), Journal of Marketing Research
Theory
Kivetz, Urminsky and Zheng's lever. Effort and motivation rise as someone feels nearer a finish line. Felt closeness does the work even when real remaining distance stays fixed. A 12-stamp card with 2 stamps already filled accelerated purchasing the same way a genuinely shorter 10-stamp card did, though both still needed exactly 10 more purchases. The Goal Gradient Effect page's own “where it can help” case is an honest progress bar on a form.
Still untested on a live booking flow. Home loan appointment booking is a long multi-step form people abandon mid-way: contact details, loan and property details, meeting type, time slot. No published test has held that flow fixed and varied only the completion cue.
Two ways to apply the same mechanism. One arm shows an honest step count at the flow's natural, coarse chunking. The other re-chunks the identical real work into more, still-genuine steps so a customer sits further through the count at the same real point. That is the 12-stamp card's lever against a 10-stamp one, applied to a booking form. It is distinct from Pseudo-Set Framing, which pulls people because a group of steps looks unfinished rather than because remaining distance has shrunk.
Hypothesis
Visitors shown a real step-progress indicator, at either chunking, should complete and confirm a booking more often than visitors shown the identical flow with no progress indicator. Required fields and appointment options stay the same in every arm. Because the mechanism runs on felt closeness, splitting the same real work into more granular genuine steps should produce a larger lift. At an identical real point, the customer sits further through the count than under an honest step count at the flow's natural, coarser chunking.
Assumed baseline
Illustrative, not real/disclosed data. Assume 34% of visitors who start the appointment booking flow complete all 4 steps and confirm a booking under the existing no-progress-indicator flow.
Ethical guardrail
Honesty on every screen. Step count and progress bar have to match the real number of steps and the real position at every screen. Compressing or re-ordering to shrink remaining effort is out. Total step count belongs on the first screen, so nobody commits to step one without knowing three or five more sit behind it.
Where re-chunking stays legitimate. Finer sub-steps are allowed only when each resulting step is still genuinely distinct required content. Contact details and identity verification are two real things; splitting one field in half to manufacture an extra step is not. No step at either chunking may be fabricated, reordered to hide what is left, or padded only to inflate the count.
What the cue is for. Helping people finish a booking they already want to make. The count is not there to disguise how long the form takes. See Not Testing Is Still a Bet.
Control vs. two treatments
All three arms are visitors starting a home loan appointment booking flow, randomly assigned at the first screen. Every field and every appointment option offered is identical across all three; only how, or whether, progress is shown differs. Control and Treatment A are shown at the flow's natural step 2 of 4; Treatment B shows the same real work re-chunked into 6 genuine steps, generic and unbranded.
Control · no progress shown
Loan & property details
Estimated property value, deposit amount, and the purpose of the loan.
Treatment A · honest step count
Step 2 of 4About 3 min left
Loan & property details
Estimated property value, deposit amount, and the purpose of the loan.
Treatment B · re-chunked, 6 steps
Step 4 of 6About 2 min left
Loan amount & purpose
Deposit amount and the purpose of the loan. Property details are already confirmed.
All three arms require the exact same real information: contact details, identity verification, property value, deposit amount, loan purpose, meeting preference, and a chosen time. Only whether progress is shown, and how finely the same steps are chunked, differs.
Why Treatment B looks different. It doesn't change how much work is left, real or perceived honestly, it changes how many genuine pieces that work is split into. “Loan & property details” becomes its own property step and its own loan-amount step; no field is added, removed, or hidden. A customer who has done the identical real work now sits at step 4 of 6 instead of step 2 of 4. That's the exact lever the original study used. The same 10 purchases left to go read as 0% done on a blank 10-stamp card, or already under way on a 12-stamp card with 2 stamps pre-filled: identical real distance, different perceived proximity.
Sample size to actually trust this
Baseline34%completed booking, control
Target42%+8pp, the smaller effect (Treatment A)
Significanceα = 0.05two-sided
Power80%1−β
Sample per arm574visitors starting the flow
Total sample1,7223 arms
Assumed volume900/wkvisitors entering the flow
Est. run time~3 wksincl. buffer
Two-proportion test, equal allocation: n = (zα/2 + zβ)² × [p₁(1−p₁) + p₂(1−p₂)] / (p₁−p₂)². Sized against Treatment A's smaller expected effect, so Treatment B, assumed to reach 48% (+14pp) if the re-chunking mechanism holds, is over-powered by comparison.
Outcome variables
✓Primary: % of visitors who start the flow and confirm an appointment, each treatment versus control
✓Secondary: the difference between Treatment A and Treatment B specifically, isolating whether re-chunking the same real work into more granular steps adds anything beyond simply showing an honest count at the flow's natural chunking
✓Secondary: step-by-step drop-off in each arm, whether the lift is concentrated near the end (the goal-gradient signature) or spread evenly across the flow
✓Secondary: median time to complete the flow, whether visible progress also speeds people up, not just raises who finishes
✓Guardrail: show-up rate for the booked appointment itself must not fall in either treatment arm, a rushed booking made just to finish a bar is worse than no booking at all
✓Guardrail: rate of visitors who abandon after seeing the total step count on screen one must not spike in either treatment, confirming upfront honesty about the length, 4 steps or 6, doesn't itself deter people
The mechanism behind this:Goal Gradient Effect, full study and citation on the Principles page. See also Pseudo-Set Framing, a related completion pull driven by how a group of steps is framed rather than by shrinking distance to a goal.
Experiment blueprintDigital & Product Experiment
Retail Banking · Savings & DepositsAlso applies to: super/retirement contribution apps, budgeting apps with an auto-save or round-up featureNot yet run
Does timing a savings increase to a raise beat asking for it right now?
Save More Tomorrow tied the increase to an employer's known pay schedule. A banking app only sees the raise after it lands in a deposit. Does the same advance commitment still hold?
Builds on Smart Defaults · Thaler & Benartzi (2004), Journal of Political Economy
Theory
Save More Tomorrow in one paragraph. Employees committed today to a contribution-rate increase that only took effect at their next scheduled pay raise, so take-home pay never shrank on the day it changed. Of 162 employees who had just declined an immediate increase, 78% accepted the deferred version. Average savings rates rose from 3.5% to 13.6% of pay across four raises (about 40 months), and 78% were still enrolled at the end. Full write-up: Smart Defaults.
The employer schedule that banking apps lack. Every tested version of that design starts from an employer who already knows the date and size of the next raise. A savings or banking app has no such calendar. At best it notices, after the fact, that a regular salary deposit just got larger.
If timing against a gain is the real lever. Whether the raise is scheduled or detected should not matter. What should matter is committing to the increase in advance, and only applying it once the extra money has already arrived.
Hypothesis
Customers offered a one-tap commitment to raise their auto-save rate at their next detected pay rise should accept some increase more often than customers asked to raise it immediately. The size of the increase on offer is identical in both arms.
Assumed baseline
Illustrative, not real product data. Assume 9% of customers shown an in-app prompt to increase their auto-save rate by 2% right now accept it.
Ethical guardrail
Deposits that look like raises. A jump in a bank deposit is often something else: a one-off bonus, a tax refund, a temporary higher-paying gig. Treating any of those as a permanent rise would auto-escalate savings against income that is about to disappear.
How detection has to behave. A detected increase should hold across at least two consecutive pay cycles before the commitment fires. Any triggered increase must reverse with one tap and refund in full for a short window afterwards, not only cancel going forward. A wrong detection should cost the customer nothing.
What the arms isolate. Whether advance commitment against a gain, rather than a felt cut today, drives acceptance. Raise-detection cannot be tuned to fire more often than the evidence supports. See Not Testing Is Still a Bet.
Control vs. treatment
Both arms are customers shown a prompt to increase their auto-save rate by the same 2%. The only thing that changes is when the increase takes effect and what it's measured against.
Control · increase now
Increase your auto-save
Move your auto-save rate from 5% to 7% of each pay, starting with your very next pay.
Treatment · increase at next raise
Increase it at your next raise
We'll watch for a rise in your regular pay and move your auto-save from 5% to 7% only once it lands. Your take-home stays the same until then.
The 2% increase on offer, and the account it applies to, is identical in both arms. Only the timing, and what the increase is measured against, a felt cut today or a gain that hasn't arrived yet, differs.
Sample size to actually trust this
Baseline9%accepted, increase now
Target15%+6pp, ~67% relative
Significanceα = 0.05two-sided
Power80%1−β
Sample per arm456customers prompted
Total sample9122 arms
Assumed volume550/wkeligible customers prompted
Est. run time~2 wksincl. buffer
Two-proportion test, equal allocation: n = (zα/2 + zβ)² × [p₁(1−p₁) + p₂(1−p₂)] / (p₁−p₂)². The volume figure is an assumption to make the timeline concrete, not a real product number. Swap in your own weekly eligible-customer count and the run time follows.
Outcome variables
✓Primary: % of prompted customers who accept some increase to their auto-save rate, control's immediate accept versus treatment's deferred commitment
✓Secondary: of treatment customers who commit, % whose increase actually triggers within 90 days, whether the commitment survives long enough for a real raise to be detected
✓Guardrail: % of triggered increases reversed or refunded within 14 days, catching a raise the detector got wrong
✓Guardrail: overall auto-save opt-out rate in the 30 days after the prompt, the ask itself must not drive people to switch saving off altogether
The mechanism behind this:Smart Defaults, full study and citation on the Principles page.
Three fronts, one mechanism
Reactance shows up at three different moments
When a choice gets taken away, when someone's asked to make one, and when access to one gets restricted. Three separate blueprints test each moment on its own, all built on the same enriched Not Enough Choice principle. Pick one below, or scroll through all three in order.
Removing a choice
Does explaining why a feature was removed reduce the backlash?
Not Enough Choice already claims transparency defuses reactance. This tests that claim directly, for the first time, against a real product change.
Does framing a tier as restricted increase upgrade interest?
The founding reactance study proved restriction increases desire. The sharpest of the three to get right, since it only stays honest if the restriction named is real.
Retail Banking · Savings & DepositsAlso applies to: any product retiring a feature, plan, or account type customers currently useNot yet run
Does explaining why a feature was removed reduce the backlash?
Restriction without a reason is what Not Enough Choice says triggers reactance. On a real retirement notice, does one added sentence change what customers do next?
When reactance fires.Not Enough Choice ties the backlash to an active removal. An option that was never available does not trigger the same response. The principle's “where it can help” section states plainly that naming the reason should defuse it.
Never isolated on a real notice. That claim has sat untested against an actual feature-retirement message. No published design has held the removal, timeline, and balance migration fixed and varied only whether a reason appears.
What both arms share. Identical removal, identical timeline, identical balance migration. The only difference is whether the notice states why the feature is going.
Hypothesis
Customers told why a feature is being removed should contact support or lodge a complaint about it less often than customers told only that it is being removed. What happens to their money is the same in both arms.
Assumed baseline
Illustrative, not real product data. Assume 22% of customers who receive a plain “this feature is going away” notice contact support or complain about it within 30 days.
Ethical guardrail
Money has to move the same way. Balance migration stays identical, immediate, and lossless in both arms. Which notice a customer receives cannot change anything about their funds.
The reason has to be true. Treatment copy states the actual reason the feature is being retired. A plausible cover story invented for messaging tests is out. See Not Testing Is Still a Bet.
Control vs. treatment
Both arms are customers affected by the same real feature retirement. The only thing that changes is whether the notice explains why.
Control · no reason given
This feature is going away
Sub-accounts will no longer be available from 1 October. Your balance will move to your main savings account automatically.
Treatment · reason given
This feature is going away
Sub-accounts will no longer be available from 1 October. We're retiring them to keep pricing and features consistent across every savings account. Your balance will move automatically.
Sample size to actually trust this
Baseline22%complained, no reason given
Target14%−8pp with a reason
Significanceα = 0.05two-sided
Power80%1−β
Sample per arm358affected customers
Total sample7162 arms
Assumed volume500/wkcustomers affected by the removal
Est. run time~2 wksincl. buffer
Two-proportion test, equal allocation: n = (zα/2 + zβ)² × [p₁(1−p₁) + p₂(1−p₂)] / (p₁−p₂)². The volume figure is an assumption to make the timeline concrete, not a real product number.
Outcome variables
✓Primary: % of affected customers who contact support or lodge a complaint within 30 days
✓Secondary: account closure rate in the 30 days after the notice
✓Guardrail: support contact-handling time and volume, the explanation itself must not generate confused follow-up questions that offset the drop in complaints
Retail Banking · Savings & DepositsAlso applies to: any optional feature opt-in prompt, budgeting apps, super/retirement contribution appsNot yet run
Does naming the freedom to decline raise opt-in on a real ask?
A street request for bus fare nearly quintupled once people heard they were free to refuse. Put that same line on a repeatable in-app financial ask: does opt-in move?
Guéguen and Pascual on the street. Adding “but you are free to accept or to refuse” to an otherwise identical request for bus fare raised compliance from about 10% to about 47.5%. That finding now sits inside this site's Not Enough Choice write-up.
From one cash ask to a digital setting. The original test was a single, low-stakes, in-person request. Whether the same freedom-naming line moves a digital, repeatable financial decision is still open.
The ask under test. Turning on round-up savings: a genuinely optional feature that is easy to feel nagged into. Same offer in both arms; only whether the prompt names the freedom to decline differs.
Hypothesis
Customers shown an in-app prompt to turn on round-up savings that explicitly names their freedom to decline should accept it more often than customers shown the identical prompt without that line.
Assumed baseline
Illustrative, not real product data. Assume 11% of customers shown a plain round-up savings prompt turn it on.
Control vs. treatment
Both arms see the identical round-up savings offer. Only whether the prompt names the freedom to decline differs.
Control · plain ask
Turn on round-up savings
Every purchase gets rounded up, the extra goes straight into savings.
Treatment · freedom named
Turn on round-up savings
Every purchase gets rounded up, the extra goes straight into savings. You're free to turn this on or leave it off, whichever suits you.
Sample size to actually trust this
Baseline11%opted in, plain ask
Target17%+6pp with freedom named
Significanceα = 0.05two-sided
Power80%1−β
Sample per arm521customers prompted
Total sample1,0422 arms
Assumed volume600/wkeligible customers prompted
Est. run time~2 wksincl. buffer
Two-proportion test, equal allocation: n = (zα/2 + zβ)² × [p₁(1−p₁) + p₂(1−p₂)] / (p₁−p₂)². The volume figure is an assumption to make the timeline concrete, not a real product number.
Outcome variables
✓Primary: % of prompted customers who turn on round-up savings
✓Secondary: 30-day retention of the setting, whether freedom-framing attracts people who then turn it back off
✓Guardrail: opt-out rate within 7 days of turning it on
Retail Banking · Fees & PricingAlso applies to: any tiered product upsell with a real, checkable eligibility gateNot yet run
Does framing a tier as restricted increase upgrade interest?
Hammock and Brehm showed restriction increases desire. Does that still hold when the restriction is a real, checkable eligibility gate rather than manufactured urgency?
The founding reactance finding. Hammock and Brehm showed that actively restricting an option makes it more wanted, beyond simply making it harder to get. Marketing already leans on that lever informally. The mechanism sits inside Not Enough Choice.
Honest upsell versus invented scarcity. This site has not yet tested that lever as an upsell where the restriction named is real and checkable. Manufactured urgency belongs to a different, dishonest category.
What the message is allowed to name. A real, existing eligibility gate. Interest in upgrading is the outcome. No invented scarcity, no fake headcount, no countdown without a deadline behind it.
Hypothesis
Standard-tier customers shown a premium-tier message that names a genuine eligibility restriction should click through to check their eligibility more often than customers shown the identical benefits with no restriction framing.
Assumed baseline
Illustrative, not real product data. Assume 6% of standard-tier customers shown a plain premium-benefits message click through to “see if you qualify.”
Ethical guardrail
Where this goes wrong. A restriction that reads as real but is not is exactly the manufactured urgency this site's Scarcity and Signpost Effect articles warn against.
What the copy may state. Only a restriction that is true at the moment it is shown: a current eligibility gate or a genuinely limited rollout. Fabricated headcounts and countdowns with no real deadline stay off the screen.
What the arms measure. Whether naming a real restriction changes interest. Inventing a restriction to manufacture demand is outside the design. See Not Testing Is Still a Bet.
Control vs. treatment
Both arms see the identical premium benefits and face the identical real eligibility gate. Only whether the message names that restriction differs.
Control · plain benefits
Premium benefits
Lower fees, priority support, and higher transfer limits.
Treatment · restriction named
Premium benefits
Premium is currently offered to a limited group of customers. Lower fees, priority support, and higher transfer limits.
Sample size to actually trust this
Baseline6%clicked through, plain benefits
Target9%+3pp, restriction named
Significanceα = 0.05two-sided
Power80%1−β
Sample per arm1,205standard-tier customers shown the message
Total sample2,4102 arms
Assumed volume900/wkeligible customers shown the message
Est. run time~3 wksincl. buffer
Two-proportion test, equal allocation: n = (zα/2 + zβ)² × [p₁(1−p₁) + p₂(1−p₂)] / (p₁−p₂)². The volume figure is an assumption to make the timeline concrete, not a real product number.
Outcome variables
✓Primary: click-through rate to “see if you qualify”
✓Secondary: completed upgrades among those who click through
✓Guardrail: complaint rate mentioning the exclusivity claim specifically, catching the message reading as manipulative even when it's technically true
Retail Banking · Credit & LendingAlso applies to: any rewards or loyalty programme with both a practical and an indulgent redemption optionNot yet run
Does completing a responsible credit action change how someone redeems reward points right after?
Same points balance, same redemption screen. After someone has just done the responsible thing, do they pick the indulgent option more often than the practical one?
Builds on Licensing Effect · Khan & Dhar (2006), Journal of Marketing Research
Theory
Khan and Dhar's lab sequence. People first imagined committing to volunteer work, then chose between a pair of jeans and a vacuum cleaner at the same price. Imagining the virtuous act pushed choice toward the jeans (the indulgent option) compared with people who imagined no such act. A second study found the same shift toward luxury sunglasses over practical ones. Full write-up: Licensing Effect.
Imagined virtue versus a live product action. Both studies used a hypothetical virtuous act in a lab. Nothing on this site has tested whether a real, completed responsible action inside a live product changes a real choice made minutes later.
Why a loan repayment should transfer. If the mechanism is a temporary boost in feeling like a responsible person, the virtuous act here being financial (setting up an extra loan repayment) rather than social (volunteering) should not block the effect. A completed action should shift a real redemption choice the way an imagined one shifted a hypothetical purchase.
Hypothesis
Customers who have just set up an automatic extra repayment on their credit card then reach a reward-points redemption screen. That screen offers a practical option (a statement credit) and an indulgent option (a dining or travel voucher) of equal point value. They should choose the indulgent option more often than customers who reach that same screen without having just taken the repayment action.
Assumed baseline
Illustrative, not real product data. Assume 35% of customers reaching the redemption screen without a prior prompt choose the indulgent voucher over the statement credit.
Ethical guardrail
How the finding could be misused. This measures a real effect on real spending choices. Read carelessly, the same result could be taken as licence to stage a virtuous moment purely to sell more discretionary redemptions, which works against the customer's own financial interest.
What has to stay true of the setup. The extra repayment opt-in already exists as a useful feature offered on its own merits. It is not invented or timed to manufacture the licensing effect. Both redemption options stay equally visible and equally easy to pick in every arm. A guardrail metric tracks whether the extra repayment is cancelled soon after, which would signal it was ticked as a box rather than adopted.
What the arms isolate. Whether completing a responsible action shifts a later, unrelated choice. The responsible-action prompt itself is not there to appear more often than product usefulness justifies. See Not Testing Is Still a Bet.
Control vs. treatment
Both arms see the identical redemption screen with the same two options at the same point value. The only difference is what happened in the screen immediately before it.
Control · straight to redemption
Redeem your points
12,000 points ready. Choose: $120 statement credit, or a $120 dining voucher.
Treatment · after the responsible action
Extra repayment set up
You'll pay an extra $50 off your card balance each month from now on. Nice work.
Tapping "Continue to rewards" in the treatment arm leads to the exact same redemption screen shown in the control arm. Only what the customer just did, and saw confirmed, differs.
Sample size to actually trust this
Baseline35%chose indulgent, control
Target45%+10pp, treatment
Significanceα = 0.05two-sided
Power80%1−β
Sample per arm375customers reaching redemption
Total sample7502 arms
Assumed volume100/dayeligible redemptions
Est. run time~8 daysincl. buffer
Two-proportion test, equal allocation: n = (zα/2 + zβ)² × [p₁(1−p₁) + p₂(1−p₂)] / (p₁−p₂)². The volume figure is an assumption to make the timeline concrete, not a real product number. Swap in your own daily eligible-redemption count and the run time follows.
Outcome variables
✓Primary: % of redeeming customers who choose the indulgent voucher over the statement credit, control versus treatment
✓Secondary: overall redemption completion rate, checking the extra step in the treatment arm doesn't itself suppress redemption
✓Guardrail: % of treatment customers who cancel the extra repayment within 30 days, catching a responsible action taken only to unlock the redemption screen
✓Guardrail: average outstanding card balance 60 days after redemption, checking the shift towards indulgent redemption isn't paired with a rise in carried balance
The mechanism behind this:Licensing Effect, full study and citation on the Principles page.
Experiment blueprintDigital & Product Experiment
Retail Banking · Savings & DepositsAlso applies to: any product asking someone to commit a windfall, bonus, or refund to a future-focused optionNot yet run
Does a savings commitment made a week in advance survive contact with the actual moment?
One customer, one windfall, two moments a week apart. Does deciding in advance lock in a higher share than deciding once the money's actually there?
Builds on Present Bias · Read & van Leeuwen (1998), Organizational Behavior and Human Decision Processes
Theory
Read and van Leeuwen's snack choice. Office workers picked fruit or junk food for the following week, either seven days early or on the day itself. A week out, hunger barely moved the choice. On the day, hungry workers took junk food far more often than workers who had just eaten, even though both groups faced the same two snacks. That timing reversal is the core of Present Bias.
Food versus cash on hand. Their reward was a snack. This site has not yet checked whether the same advance-versus-in-the-moment flip appears when the prize is money that could be spent today instead of saved.
Why a savings share should travel. If value collapses once “now” is on the table, swapping a snack for a savings percentage should not erase the pattern. Naming a share a week before a bonus or refund arrives should favour saving more than asking the identical question once the cash is already in the account.
Hypothesis
Customers asked a week before an upcoming bonus or tax refund how much of it to move to savings should commit a higher average percentage than customers asked the identical question on the day the money arrives.
Assumed baseline
Illustrative, not real product data. Assume customers choosing on the day the money lands commit an average of 15% of it to savings.
Ethical guardrail
Where an early lock-in hurts. A savings share chosen a week early can be wrong by the time the money lands if circumstances change. A surprise bill, a job loss, or an emergency can make the earlier number a bad fit.
What stays reversible. Only customers who already opted into a windfall-savings feature on its own merits enter the test. The advance commitment stays editable or fully cancellable until the money lands, one tap, no fee. A mistimed ask should cost the customer nothing.
What the arms isolate. Whether ask timing changes the committed share. Hardening advance commitments so they are harder to reverse is outside the design. See Not Testing Is Still a Bet.
Control vs. treatment
Both arms make the same choice about the same upcoming payment. The only difference is when they're asked.
Control · asked on the day
Your refund just landed
$800 arrived today. How much would you like to move to savings?
Treatment · asked a week ahead
Your refund lands in 7 days
Expecting around $800 on the 14th. How much would you like to move to savings when it arrives?
The estimated amount and the destination account are identical in both arms. Only the date on which the customer sets the percentage, and whether the money has already landed, differs.
Sample size to actually trust this
Baseline15%avg. committed, on the day
Target25%avg. committed, week ahead
Significanceα = 0.05two-sided
Power80%1−β
Sample per arm100customers making a choice
Total sample2002 arms
Assumed volume25/dayeligible bonus/refund events
Est. run time~8 daysincl. buffer
Two-sample t-test for a continuous outcome, equal allocation: n = 2 × (zα/2 + zβ)² × σ² / (μ₁−μ₂)², assuming a standard deviation of 25 percentage points around each arm's average. The volume figure is an assumption to make the timeline concrete, not a real product number. Swap in your own daily eligible-event count and the run time follows.
Outcome variables
✓Primary: average % of the windfall committed to savings, control's on-the-day choice versus treatment's week-ahead choice
✓Secondary: of treatment customers, % who reduce or cancel their commitment before the money lands, the reversal Present Bias predicts
✓Guardrail: overdraft or missed-payment incidents in the 30 days after, checking the committed amount isn't leaving customers short
✓Guardrail: windfall-feature opt-out rate, treatment versus control, catching reactance to being asked to commit early rather than a genuine timing effect
Subscriptions & Telecom · Fees & PricingAlso applies to: any recurring plan with a real, checkable cheaper alternativeNot yet run
Does showing what you've already paid reduce comparison shopping at renewal?
Theatergoers who paid full price kept attending more than people given the same season cheaper. Put that logic on a renewal screen: does showing someone their own year-to-date spend suppress comparison shopping when a cheaper plan is one tap away?
Arkes and Blumer's theater season. People who paid full price for a season of plays attended more of it than people handed the identical season at a random discount. Future attendance tracked what had already been paid, not what the remaining shows were worth going forward. That is the mechanism behind Sunk Cost Fallacy.
Attendance after the fact versus a renewal prompt. Their field study measured shows attended once the ticket was already bought. This site has not yet tested whether naming a customer's own sunk cost, at the moment they would otherwise shop around, changes the next tap.
What the renewal screen adds. Put the customer's real year-to-date spend on the plan next to the renew button. The question is whether that figure alone cuts how often they open a cheaper alternative before renewing.
Hypothesis
Customers shown their own cumulative spend on the current plan at renewal should click “compare other plans” less often than customers shown an identical renewal screen with no spend figure. The same cheaper plan stays available in both arms.
Assumed baseline
Illustrative, not real product data. Assume 22% of customers shown a plain renewal screen click "compare other plans" before renewing.
Ethical guardrail
How the finding could be misused. The same lever could trap customers on a worse deal by leaning on the fallacy the Sunk Cost Fallacy article already flags. Measuring the effect is not the same as recommending it as growth copy.
What the screen is allowed to show. Only the customer's own real, checkable spend, never rounded up or estimated in the company's favour. “Compare other plans” stays equally visible and fully working in both arms. Switching never gets harder; only the accompanying figure changes.
What a positive result means. The arms ask whether a true sunk-cost figure moves behaviour by itself. They do not ask whether comparison shopping should be buried. Confirmation is a reason to avoid shipping the pattern, not a green light to ship it. See Not Testing Is Still a Bet.
Control vs. treatment
Both arms see the identical renewal price and the identical "compare other plans" link. Only whether the screen states the customer's own year-to-date spend differs.
Control · plain renewal
Your plan renews
$69.99/month, unchanged.
Treatment · sunk cost shown
Your plan renews
$69.99/month, unchanged. You've paid $840 into this plan this year.
Sample size to actually trust this
Baseline22%clicked “compare,” plain screen
Target16%−6pp, sunk cost shown
Significanceα = 0.05two-sided
Power80%1−β
Sample per arm667renewing customers shown the screen
Total sample1,3342 arms
Assumed volume800/wkrenewals reaching this screen
Est. run time~3 wksincl. buffer
Two-proportion test, equal allocation: n = (zα/2 + zβ)² × [p₁(1−p₁) + p₂(1−p₂)] / (p₁−p₂)². The volume figure is an assumption to make the timeline concrete, not a real product number.
Outcome variables
✓Primary: click-through rate to “compare other plans”
✓Secondary: overall renewal completion rate
✓Guardrail: complaint and negative-sentiment rate mentioning feeling pressured, plus churn rate 90 days later, catching a suppressed decision that just resurfaces as regret
The mechanism behind this:Sunk Cost Fallacy, full study and citation on the Principles page.
Experiment blueprintDigital & Product Experiment
Subscriptions · Fees & PricingAlso applies to: any checkout offering a monthly vs. annual choiceNot yet run
Does a truthful daily-equivalent price increase annual-plan uptake, even with the total shown right next to it?
Gourville's labs pitted pennies-a-day against the aggregate total. Most real checkouts show both. Does the lift still hold when the real annual total sits right next to the daily figure?
Gourville's pennies-a-day finding. Restating a cost as a small daily amount, rather than one lump total, shifted which other expenses people compared it against and raised willingness to pay. That comparison shift is the heart of Temporal Reframing.
Daily framing alone versus full disclosure. Those lab arms put the daily figure in place of the total, not beside it. Legal and trust reasons usually force real checkouts to keep the true total on screen, so whether the effect survives that disclosure is still open.
What the checkout arms change. Keep the real annual total visible in both arms. Add a truthful daily-equivalent figure next to it in one arm only. Nothing else about disclosure moves. The question is whether that addition still pulls more people toward the annual plan.
Hypothesis
Customers shown “33¢/day, billed annually at $120” should pick the annual plan over the identical monthly alternative more often than customers shown “$120/year” alone. The true total stays visible in both arms.
Assumed baseline
Illustrative, not real product data. Assume 18% of customers choose the annual plan over the monthly plan when shown the plain annual total.
Ethical guardrail
Where the reframe turns dishonest. A daily-equivalent that is not a true, correctly rounded division of the real total misleads. That is the failure mode the Temporal Reframing article already names as the dishonest use of this lever.
What the copy must compute. The daily figure is the exact annual total divided by 365, rounded normally. The annual total appears in the same type size, never as fine print. The monthly plan's price and visibility stay fixed across arms.
What the arms isolate. Whether a fully disclosed daily reframe still moves plan choice. Shrinking disclosure to make the reframe hit harder is outside the design. See Not Testing Is Still a Bet.
Control vs. treatment
Both arms show the identical monthly plan as the alternative and the identical real annual total. Only whether a daily-equivalent figure appears alongside that total differs.
Control · total only
Annual plan
$120/year, billed annually.
Monthly: $12.99/mo
Treatment · daily equivalent added
Annual plan
33¢/day, billed annually at $120/year.
Monthly: $12.99/mo
Sample size to actually trust this
Baseline18%chose annual, total only
Target24%+6pp, daily equivalent added
Significanceα = 0.05two-sided
Power80%1−β
Sample per arm719checkouts reaching the plan choice
Total sample1,4382 arms
Assumed volume500/wkcheckouts reaching the plan choice
Est. run time~4 wksincl. buffer
Two-proportion test, equal allocation: n = (zα/2 + zβ)² × [p₁(1−p₁) + p₂(1−p₂)] / (p₁−p₂)². The volume figure is an assumption to make the timeline concrete, not a real product number.
Outcome variables
✓Primary: share choosing the annual plan over monthly
✓Secondary: overall checkout completion rate, either plan
✓Guardrail: refund and support-contact rate within 30 days mentioning the price, catching a choice that felt smaller at checkout than it does on the first bill
The mechanism behind this:Temporal Reframing, full study and citation on the Principles page.
Experiment blueprintDigital & Product Experiment
Add-on Products · Fees & PricingAlso applies to: any cross-sell offering two or more add-ons with real, equal-sized discountsNot yet run
Does stating one discount on a bundle beat splitting the identical discount across its parts?
Yadav and Monroe weighed a saving stated on the bundle against the same dollars split across items. Move that framing fight onto financial add-ons instead of retail goods: does one bundle line still win?
Yadav and Monroe on transaction value. An extra discount stated once on a combined bundle price lifted perceived transaction value more than the identical dollar amount split across the bundle's separate items. That framing difference is the load-bearing claim in Bundling.
Described goods versus a live cross-sell. Their scenarios used physical products people imagined buying. This site has not yet tested whether the same framing shift changes real take-up when the “items” are financial add-on services.
What the arms hold constant. Two add-ons, one total $2 saving. State that saving once on the bundle in treatment, or $1 on each line in control. The question is whether the single bundle statement lifts how often customers take both.
Hypothesis
Customers offered “save $2 when you bundle ID theft protection and device insurance together” should take both add-ons more often than customers offered “save $1 off ID theft protection” and “save $1 off device insurance” as separate line items. The total saving stays $2 in both arms.
Assumed baseline
Illustrative, not real product data. Assume 11% of customers take both add-ons when the two discounts are shown separately, one per item.
Ethical guardrail
Where framing oversells. A bundle line that makes a saving feel larger than it is can push add-ons a customer would otherwise skip, then cancel once the bill arrives and the framing has worn off.
What the discount math must match. The bundle discount equals, never exceeds, the sum of the two item-level discounts. No inflated “bundle-only” saving may be invented to stack the framing test.
What the arms isolate. Whether framing alone changes take-up for an identical real saving. Pricing a bundle to look better than it is sits outside the design. See Not Testing Is Still a Bet.
Control vs. treatment
Both arms offer the identical two add-ons at the identical final prices. Only whether the $2 saving is stated once, on the bundle, or twice, one per item, differs.
Control · per-item discounts
Two add-ons
ID theft protection: $4/mo, save $1. Device insurance: $4/mo, save $1.
Treatment · bundle discount
Bundle both, save $2
ID theft protection + device insurance, together, save $2/mo when bundled.
Sample size to actually trust this
Baseline11%took both, per-item discounts
Target16%+5pp, bundle discount
Significanceα = 0.05two-sided
Power80%1−β
Sample per arm729customers shown the cross-sell
Total sample1,4582 arms
Assumed volume350/wkcustomers reaching the cross-sell
Est. run time~6 wksincl. buffer
Two-proportion test, equal allocation: n = (zα/2 + zβ)² × [p₁(1−p₁) + p₂(1−p₂)] / (p₁−p₂)². The volume figure is an assumption to make the timeline concrete, not a real product number.
Outcome variables
✓Primary: share of customers taking both add-ons
✓Secondary: share taking exactly one add-on, checking whether the bundle framing cannibalises single-item take-up
✓Guardrail: cancellation rate of the bundled add-ons within 60 days, catching a deal that felt bigger at checkout than the add-ons turned out to be worth
The mechanism behind this:Bundling, full study and citation on the Principles page.
Experiment blueprintDigital & Product Experiment
BNPL Provider · Credit & LendingAlso applies to: payday lenders, overdraft products, any short-term credit where fees compound on a missed paymentNot yet run
Does naming the real dollar cost of repeat late fees cut how much someone borrows next?
Payday borrowers who saw dollar fees over time borrowed about 11% less afterward. Drop that same format into a BNPL confirm screen: does the next purchase and repayment behaviour move?
Builds on Bertrand & Morse (2011)'s real field experiment on payday loan disclosure formats, covered in full in The Fine Print Nobody Reads. Also builds on Disclosure Backfire for what a badly-designed version of this could do instead.
Theory
Bertrand and Morse at the payday counter. Across 77 stores, 1,441 real borrowers saw either ordinary terms or one of several disclosure formats at the counter. APR figures, and comparisons to credit-card APRs, barely moved borrowing. Naming the real dollar fees a borrower would rack up if the loan renewed repeatedly cut borrowing over the next four months by about 11%. Full teardown: The Fine Print Nobody Reads.
Paper at a counter versus a digital confirm screen. That disclosure arrived in person, on paper, for a payday loan. Whether the same dollar-fees-over-time format, delivered inside a digital checkout instead of by a clerk, moves Buy Now, Pay Later behaviour the same way is untested. BNPL is built around the repeat, compounding late fee the original study measured.
Which decision the arms target. Like Bertrand and Morse, this is about an existing high-cost-credit borrower's next borrowing decision, not whether to finish the purchase sitting in front of them. The dollar figure is a projection from the current order's own real late-fee schedule, mirroring the original format rather than a personal history lookup.
Hypothesis
Existing BNPL users with at least one prior late payment who see the real dollar late-fee cost of missing payments on their current order at confirm should miss fewer payments and take out fewer new BNPL purchases over the following months than users shown only the standard terms link.
Assumed baseline
Illustrative, not real BNPL provider data. Assume 24% of users with a prior late payment miss a payment again on their next order when shown only the standard terms link.
Ethical guardrail
Why the disclosure is not a neutral add-on. Showing a real dollar-cost projection to someone who has already missed a payment changes the emotional temperature of the checkout. The design is not built to talk anyone out of BNPL when they intend to pay on time. Plenty of BNPL purchases clear without extra cost.
What the figure is allowed to be. A real projection tied to the current order's own instalment terms, never an invented worst case. It sits beside the purchase, never in place of it. Checkout speed and tap count stay the same across arms.
The backfire this specifically watches for. Honesty alone does not guarantee help. That is why Disclosure Backfire sits in the design: a late-fee warning that reads as an accusation, or that shoves a borrower toward a less transparent product instead of more careful repayment, would be the intervention failing.
Control vs. treatment
Both arms see the identical order and the identical instalment schedule. Only whether the late-fee cost is named in real dollars differs.
If you miss 2 of these 4 payments, the way an average repeat late payer does, this order costs $30 more in late fees: $180 total.
The $30 figure is this order's own real late-fee schedule applied twice, not an invented or worst-case number, and confirming the order takes the identical number of taps in both arms.
Two-proportion test, equal allocation: n = (zα/2 + zβ)² × [p₁(1−p₁) + p₂(1−p₂)] / (p₁−p₂)². This covers the primary missed-payment outcome only. The volume figure is an assumption to make the timeline concrete, not a real provider number.
Outcome variables
✓Primary: share of the current order's instalments paid late or missed
✓Secondary: total BNPL amount borrowed over the following 4 months, the same window and outcome type Bertrand & Morse's own result was measured on
✓Guardrail: checkout abandonment rate on the current order, catching a warning that just kills otherwise-affordable purchases instead of changing repayment behaviour
✓Guardrail: rate of switching to a less transparent, higher-cost credit product for a purchase the user would otherwise have made with BNPL, checking the disclosure didn't just push the same borrowing somewhere worse
The mechanism behind this:The Fine Print Nobody Reads, the full Bertrand & Morse teardown and the wider case for why most disclosure goes unread or backfires. See also Disclosure Backfire on the Principles page.
Experiment blueprintDigital & Product Experiment
Financial Services · Credit CardsAlso applies to: membership organisations, insurers, and any subscription business with an annual renewal decisionNot yet run
Does a printed annual member guide increase renewal more than a digital-only version?
Touch raises how much people value a physical object. Whether that lift shows up in an actual renewal decision a year later is still an open product question.
Peck and Shu, then Atasoy and Morewedge. Brief physical contact with an object raised how much people valued it and what they would pay. Atasoy and Morewedge ran the mechanism the other way: identical content lost value and willingness to pay once it became a purely digital good. A digital copy generated a weaker sense that it was “mine,” the same ownership pull that sits under the Endowment Effect. Broader case: Printing Costs More Than Digital. The Paper Copy Still Wins.
Lab objects versus a dated renewal mailer. Those studies used everyday consumer goods and generic digital content: mugs, sunglasses, souvenir photos, books, films. Neither tied a recurring benefit mailer to a real, dated renewal decision for a paid product. Whether physical ownership still shows up in renewal months later remains untested.
What the card programme arms compare. A premium card's annual member guide, either as a printed booklet or as the digital-only equivalent already in the app. Renewal at the next annual fee date is the outcome that matters, and the same premium-card context as Credit Card Premium.
Hypothesis
Cardholders who receive a printed annual member guide on top of the digital version every cardholder already gets in the app should report higher perceived value of the card at 90 days and renew at a higher rate at their next annual fee date than cardholders who receive the digital-only version.
Assumed baseline
Illustrative, not real card-programme data. Assume the premium card's current annual renewal rate is 85%, with the digital-only guide as the existing standard.
Ethical guardrail
Cost is not an excuse to thin the control. A physical mailer costs materially more per cardholder than a push notification. Testing it against a cheaper default is fair only when the digital-only arm still gets the full, real content, not a deliberately thinner version built to make print look better.
What both arms receive. Identical content. The only difference is the medium the guide arrives in, and every cardholder still gets the in-app digital version regardless of arm.
The fairness read this watches for. A cardholder who notices peers got a printed booklet and they did not could read the difference as being treated as a lesser member, the unfairness pattern covered under Perceived Fairness. The complaint-rate guardrail below catches that directly.
Control vs. treatment
Every cardholder in both arms sees the identical digital guide in the app. Only the treatment arm also receives a printed booklet in the mail.
Control · digital-only
Your Member Guide
This year's benefits, travel perks, and partner offers, all in the app.
Treatment · printed booklet + digital
Your Member Guide, printed
The same content, bound as a booklet and mailed to the cardholder's address on file.
Arrives by mail
The printed booklet supplements the in-app guide; it never replaces it. Randomisation happens at the cardholder level, not the household level, to avoid one printed copy being shared across two cardholders at the same address.
Two-proportion test, equal allocation: n = (zα/2 + zβ)² × [p₁(1−p₁) + p₂(1−p₂)] / (p₁−p₂)². The long run time is structural, not padding. Enrolling the full sample takes about 7–8 months at the assumed volume. But the outcome itself only fires on each cardholder's own annual renewal date, so the last cardholders enrolled don't produce a result until roughly a year after they join.
Outcome variables
✓Primary: renewed or cancelled at the cardholder's own next annual fee date
✓Secondary: self-reported perceived value of the card, 1–7 scale, surveyed at 90 days
✓Guardrail: complaint or support-contact rate about not receiving a printed guide, among digital-only cardholders
✓Guardrail: print and postage cost per incremental renewal, weighed against the annual fee retained, so a real effect can still be judged not worth the cost
Retail Banking · Savings & DepositsAlso applies to: any investing app, super fund, or robo-advisor letting users hand-pick from a flat list of portfolios or fundsNot yet run
Does trimming a 10-option investing menu to 3 curated ones change how evenly people spread their money?
Retirement savers split contributions evenly across however many funds a plan listed. Does a micro-investing app's own flat menu do the same thing to a new user's first portfolio?
Benartzi and Thaler's 1/n pattern. Retirement savers roughly split contributions evenly across however many funds their employer's plan listed. The resulting stock/bond mix tracked the mix of funds on offer far more closely than any stated risk preference. That is the core of Diversification Heuristic.
Employer 401(k) menus versus an editable app list. Their evidence came from employer-chosen plan menus, not a screen a product team can redesign and re-test inside its own app. It also predates today's single-screen, pick-as-many-as-you-like investing menus like CommSec Pocket's. Whether trimming a live investing app's option list changes how evenly a new user's first contribution spreads is still open. A shorter default list also brushes against Choice Overload, the companion finding that more options can make choosing itself harder before the 1/n split kicks in.
What the signup arms compare. Replace a flat 10-option “Invest in” menu with three pre-grouped options, each already diversified inside itself. Track how many portfolios a new user selects, and whether the resulting split still tracks pure equal-weighting by count, inside the same signup flow for the same app.
Hypothesis
New users shown the trimmed, 3-option menu should end up with a genuinely diversified first portfolio more often than users shown the original 10-option menu. Both groups should still tend to split money close to evenly across however many options they pick. The 10-option group should be more likely to overweight several overlapping single-country or single-theme funds by picking, for example, 4 of the 10 and splitting evenly across them.
Assumed baseline
Illustrative, not real/disclosed data. Assume 45% of new signups shown the existing 10-option menu end up with a first-session portfolio classified as genuinely diversified. That means spanning at least two distinct, non-overlapping asset classes rather than several near-duplicate single-country or single-theme picks split evenly.
Ethical guardrail
What the menu redesign may not do. Arms may regroup options and change how many appear by default. They may not push a user to invest more than intended or hide a legitimate option's existence. Every option from the original 10-item menu stays reachable through the curated screen, just organised differently. Any “already diversified” curated option must actually be diversified, not marketed that way while concentrated.
What the arms isolate. Whether restructuring the menu reduces the accidental overlap naive equal-weighting produces. Funneling new users into house-preferred, higher-fee products via a shorter, more persuasive menu sits outside the design. See Not Testing Is Still a Bet.
Control vs. treatment
Both arms are new sign-ups reaching the "Invest in" screen for the first time, randomly assigned at account creation. Every option available in the control's flat list is still reachable in the treatment, just organised differently.
Control · flat 10-option menu
Invest in
Aussie Top 200, Aussie Dividends, Aussie Sustainability, Aussie Corporate Bonds, Global 100, Diversified Equities, Emerging Markets, Health Wise, Sustainability Leaders, Tech Savvy.
Treatment · 3 curated options
Invest in
Balanced Starter: already spread across shares, property and bonds. Aussie Shares: broad exposure to the ASX. Go further: see all 10 individual options.
Nothing removed: every original option is still one tap away behind “Go further.”
Same account type, same underlying ten options, same signup flow. Only how many choices are shown by default, and how they're grouped, differs between arms.
Sample size to actually trust this
Baseline45%diversified first portfolio, control
Target60%+15pp assumed lift, treatment
Significanceα = 0.05two-sided
Power80%1−β
Sample per arm171new signups reaching the menu
Total sample3422 arms
Assumed volume1,500/monew app signups
Est. run time<1 moto reach 342 at 1,500/mo split evenly
Two-proportion test, equal allocation: n = (zα/2 + zβ)² × [p₁(1−p₁) + p₂(1−p₂)] / (p₁−p₂)². The volume figure is an assumption to make the timeline concrete, not a real product number. Swap in an actual monthly signup count and the timeline follows.
Outcome variables
✓Primary: proportion of new accounts whose first-session portfolio is classified as genuinely diversified, spanning at least two distinct, non-overlapping asset classes, treatment versus control
✓Secondary: average number of options selected per new account, confirming the trimmed menu changes allocation quality rather than just suppressing engagement
✓Secondary: total dollar amount invested in the first session, confirming a decluttered menu doesn't reduce how much people are willing to commit
✓Guardrail: click-through rate on "Go further" must not collapse to near zero, confirming the other 7 options stay genuinely reachable, not just nominally present
✓Guardrail: support contacts about missing or hidden options must stay within the normal range for new signups
The research behind this:Diversification Heuristic, full citation and the real CommSec Pocket menu behind this test. See also Choice Overload, the companion finding that more options can make choosing itself harder even before the 1/n split kicks in.
Experiment blueprintDigital & Product Experiment
Retail Banking · Credit & LendingAlso applies to: any windfall, bonus, or lump sum an app can detect and apply toward an existing debtNot yet run
Does directing a tax refund to pay down a credit card increase spending right after?
The refund and the balance cut stay the same. Does pausing to confirm the paydown happened lift discretionary spend in the weeks that follow?
Grounded in Licensing Effect · Khan & Dhar (2006), Journal of Marketing Research
Theory
Khan and Dhar's imagined volunteer act. People first imagined committing to volunteer work, then chose between two equally priced products, one practical, one indulgent. Imagining the virtuous act shifted choice toward the indulgent option compared with people given no such prompt. A completed (or even merely imagined) responsible act can license a later indulgence. Full write-up: Licensing Effect.
Earned points versus new money. The reward-redemption experiment already on this site tests the mechanism on a small, already-earned resource: points, choosing between two redemptions of equal value. Nothing here yet tests it on new money entering someone's life, a real lump sum with no assignment yet, and a real choice about whether it goes toward debt.
Why confirmation should matter. If making a virtuous act visible is what does the licensing, two customers whose refund hits their card balance identically should still spend differently afterward. The only difference is whether the app told them so as its own moment.
Hypothesis
Customers already enrolled in Refund Auto-Payoff who get an explicit confirmation that their tax refund hit their card balance should show a larger rise in discretionary card spending over the next 30 days. Enrolled customers whose identical refund lands with no distinct confirmation, only a line in ordinary transaction history, should show the smaller rise.
Assumed baseline
Illustrative, not real product data. Assume 40% of customers whose refund is applied without a distinct confirmation show a rise in discretionary card spend above their own prior 30-day average in the month that follows. That is ordinary post-windfall spending, refund or no confirmation.
Ethical guardrail
Where the bank's incentive warps the find. A confirmation that lifts discretionary spending after a debt paydown can win back, in interest and fees, some of what the customer just saved. That incentive tempts redesigning the confirmation to maximise the effect rather than to inform honestly.
What stays identical across arms. Every enrolled customer's refund hits their balance the same way regardless of arm. Only whether they are told about it, not whether it happens, its size, or its timing, is under test. The confirmation states only true, specific facts: the dollar amount applied and the new balance. No encouragement to spend, celebrate, or reward the moment.
What the arms isolate. Whether making a real debt reduction visible changes later spending. Redesigning the confirmation to push that spending further sits outside the design. See Not Testing Is Still a Bet.
Control vs. treatment
Both arms are already enrolled in Refund Auto-Payoff, and both refunds are applied to the card balance identically, in full, the moment the refund lands. The only difference is whether that fact is confirmed to the customer as a distinct moment.
Control · no confirmation moment
Transaction history
ATO refund applied · $1,200. Card ending 4821. Balance: $2,340.
Treatment · explicit confirmation
Refund applied
Your $1,200 tax refund paid down your card. Nice work, your new balance is $2,340.
Both customers' $1,200 refund reduces their balance by exactly $1,200. Only whether the app pauses to tell them so, as its own screen rather than a line in a list, differs.
Sample size to actually trust this
Baseline40%rose above own baseline, control
Target52%+12pp, treatment
Significanceα = 0.05two-sided
Power80%1−β
Sample per arm270enrolled customers with a detected refund
Total sample5402 arms
Assumed volume45/dayduring tax-refund season
Est. run time~14 daysincl. buffer
Two-proportion test, equal allocation: n = (zα/2 + zβ)² × [p₁(1−p₁) + p₂(1−p₂)] / (p₁−p₂)². The volume figure is an assumption to make the timeline concrete, not a real product number, and tax-refund season is itself seasonal, so a real run would need to fit inside that window.
Outcome variables
✓Primary: % of customers whose discretionary card spend in the 30 days after the refund exceeds their own prior 30-day average, control versus treatment
✓Secondary: average card balance at day 30, checking whether a confirmed paydown gets re-spent back up
✓Guardrail: % of treatment customers who opt out of refund confirmation messaging, a signal the moment itself is unwanted
✓Guardrail: average interest paid over the following 90 days, checking whether any licensed spending costs the customer more than the paydown just saved them
The mechanism behind this:Licensing Effect, full study and citation on the Principles page.
Cards & Rewards · Fees & PricingAlso applies to: interest-rate offers, loyalty tier upgrades, or any two-option choice where a peer benchmark can be shown truthfullyNot yet run
Does showing a customer where a cashback rate ranks against similar spenders pull them toward the lower-paying card?
Two real rates, same dollars on the table. Does seeing that one card would leave you ahead of similar customers pull people off the card that simply pays more?
Drawn from Positional Concern · Solnick & Hemenway (1998), Journal of Economic Behavior & Organization
Theory
Solnick and Hemenway's two worlds. People chose between earning $50,000 while others earned $25,000, or earning $100,000 while others earned $200,000. Most picked the poorer absolute income as long as it left them ahead of the people around them. That relative ranking is the load-bearing claim behind Positional Concern.
Survey worlds versus real cards. Their design was a hypothetical questionnaire: no real dollars changing hands, no product anyone could walk away with. This site has not yet asked whether the same ranking pull appears when both options are real cashback rates on a card a customer can actually choose.
Why a peer line should move the choice. If relative standing weighs on a live purchase, adding a truthful peer benchmark beside two unchanged rates should shift some people off the higher-paying card and onto the one that merely ranks better against similar spenders. That is a ranking question, distinct from Social Norm, which tracks what most people do rather than whether someone ends up ahead of them.
Hypothesis
When two real cashback rates sit side by side on a card-choice screen, adding a truthful benchmark of what similar-spend customers typically earn should raise the share who pick the lower-rate card. The same two rates with no benchmark should leave that share lower.
Assumed baseline
Illustrative, not real product data. Assume 6% of customers pick the lower-rate card when shown only the two rates, most likely people choosing on a card feature other than cashback, rather than misreading the numbers.
Ethical guardrail
Where the issuer's margin sits. The lower-cashback card earns the issuer more. A rank benchmark that steers people toward it, packaged as helpful context, would cost real customers real dollars for the issuer's gain.
What stays fixed across arms. Both rates are real, unchanged products either arm could choose without this test. The benchmark is an independently computed figure, identical in method for both arms; only whether it appears on screen differs. It is never tuned to manufacture the result.
What the arms isolate. Whether visible relative standing pulls a real choice away from the higher-paying option. Building that pull into how the cards are sold sits outside this design. See Not Testing Is Still a Bet.
Control vs. treatment
Both arms see the same two card offers side by side, Card A paying a higher cashback rate and Card B paying a lower one. The treatment arm adds one extra line to each card: how that rate compares with what similar-spend customers typically earn.
Control · rates only
Choose your card
Card A: 2.2% cashback on everyday spend. Card B: 1.8% cashback on everyday spend.
Treatment · rates plus benchmark
Choose your card
Card A: 2.2% cashback. Similar spenders typically earn 2.4% on this card. Card B: 1.8% cashback. Similar spenders typically earn 1.3% on this card.
Card A still pays more than Card B in both arms, in real dollars, on identical spend. Only whether each rate's standing against similar customers is shown changes between them.
Sample size to actually trust this
Baseline6%chose Card B, control
Target15%+9pp, treatment
Significanceα = 0.05two-sided
Power80%1−β
Sample per arm180customers mid card-choice flow
Total sample3602 arms
Assumed volume25/daythrough the card-upgrade flow
Est. run time~15 daysincl. buffer
Two-proportion test, equal allocation: n = (zα/2 + zβ)² × [p₁(1−p₁) + p₂(1−p₂)] / (p₁−p₂)². The volume figure is an assumption to make the timeline concrete, not a real product number.
Outcome variables
✓Primary: % of customers who choose Card B, the lower-cashback-rate card, control versus treatment
✓Secondary: average annual cashback forgone per customer who switches to Card B under the benchmark, the real dollar cost of the effect if it appears
✓Guardrail: comprehension check score on a short follow-up question about which card pays more, ruling out simple confusion as the cause of any shift
✓Guardrail: complaint or support-contact rate referencing the card comparison screen
The mechanism behind this:Positional Concern, full study and citation on the Principles page.
Distinct from:Social Norm, which tests what most people do; this tests whether a customer ends up ahead of or behind others, regardless of what the group's own behaviour is.
Experiment blueprintDigital & Product Experiment
Retail Banking · Credit & LendingAlso applies to: any automated decline, denial, or rejection notice a customer receives with no person actually delivering itNot yet run
Does a benevolence-framed decline reduce how harshly customers judge the bank after a loan rejection?
The decline and the reason codes stay put. Does opening with one sentence of genuine care change how a customer feels about the bank that sent it?
Applies Shooting the Messenger · John, Blunden & Liu (2019), Journal of Experimental Psychology: General
Theory
John, Blunden and Liu on messengers. People who deliver bad news get rated less likeable, even when a random process produced the outcome rather than the messenger. Signalling a benevolent motive while delivering the news shrank that penalty. The full mechanism write-up is under Shooting the Messenger.
A person versus a template. Their finding came from a human delivering news to another human. A loan decline is usually the reverse: a templated, automated letter with no person in the room. Whether benevolence-framing still softens the hit when the “messenger” is a system-generated notice remains open on this site.
What should transfer to automation. If an honest statement of care softens the penalty for a live messenger, the same idea should travel to an automated decline. Opening an otherwise identical decline with that statement should soften how a declined applicant feels about the bank, while the underlying decision stays identical.
Hypothesis
Applicants who receive an identical loan decline, same decision and same reason codes, should report less negative sentiment toward the bank on a short post-decision survey when the decline opens with an explicit, genuine statement of care. Applicants on the current generic decline message should score worse on the same survey.
Assumed baseline
Illustrative, not real product data. Assume 34% of applicants who receive the current generic decline rate their experience 1 or 2 stars out of 5 on a short post-decision survey.
Ethical guardrail
Where warmer copy can paper over a bad call. Banks gain from fewer complaints and better sentiment scores whether or not the underlying decision was fair. Softer language can make a weak or biased decision feel more acceptable, or discourage someone with a real grievance from escalating it.
What stays identical. The credit decision, the reason codes shown, and the applicant's appeal rights match in both arms. Only the opening framing sentence changes. That sentence states true next steps and real resources, and never implies the decision will change just because someone asks.
What the arms isolate. Whether framing changes how a customer feels about an unchanged decision. Using framing to dampen complaints about decisions that are actually wrong sits outside the design. See Not Testing Is Still a Bet.
Control vs. treatment
Both arms receive the identical decision, reason code, and reapplication timeline. The only difference is the opening of the decline message itself.
Control · current generic decline
Application update
Your personal loan application has been declined based on our lending criteria. Reason: insufficient serviceable income. You may reapply after 90 days.
Treatment · benevolence-framed opening
Application update
We know this isn't the outcome you were hoping for, and we want to be upfront about why. Your application was declined due to insufficient serviceable income. You may reapply after 90 days, and we've included what could change next time.
The decision, the reason code, and the reapplication window are identical in both arms. Only the opening sentence and its framing differ.
Sample size to actually trust this
Baseline34%1–2 star rating, control
Target24%−10pp, treatment
Significanceα = 0.05two-sided
Power80%1−β
Sample per arm320declined applicants surveyed
Total sample6402 arms
Assumed volume60/daydeclined applications
Est. run time~12 daysincl. buffer
Two-proportion test, equal allocation: n = (zα/2 + zβ)² × [p₁(1−p₁) + p₂(1−p₂)] / (p₁−p₂)². The volume figure is an assumption to make the timeline concrete, not a real product number.
Outcome variables
✓Primary: % of declined applicants rating their experience 1 or 2 stars out of 5 on a short post-decision survey, control versus treatment
✓Secondary: complaint or escalation rate within 14 days of the decline
✓Guardrail: rate of customers who use the formal appeals process, checking that warmer framing doesn't discourage a legitimate appeal
✓Guardrail: reapplication rate within 90 days, checking that warmer framing doesn't discourage applicants who could genuinely qualify next time
The mechanism behind this:Shooting the Messenger, full study and citation on the Principles page.
Experiment blueprintDigital & Product Experiment
Retail Banking · Credit & LendingAlso applies to: any booked call or appointment where the customer's own follow-through, not the reminder itself, determines whether they show upNot yet run
Does asking a customer to state their own plan reduce no-shows on a home loan appointment?
Answering a phone appointment is one kind of follow-through. Getting to a branch on time is another. Does asking customers to say what they'll do, or how they'll get there, change no-show rates in either case?
Tests Implementation Intentions · Gollwitzer & Brandstätter (1997), Journal of Personality and Social Psychology
Theory
Gollwitzer and Brandstätter's if-then plans. Naming exactly when and where you will act on a goal raises follow-through far above holding the same goal with no specific plan. That is the core claim of Implementation Intentions.
What CommBank's flow already says. The live appointment booking path (commbank.com.au/appointment) lets a customer book a home loan appointment by phone or in-branch. Both paths already tell the customer the confirmed time and what to have ready, everything an implementation intention needs, but they state it to the customer rather than asking it back. An in-branch slot also adds a barrier a phone call never has: getting to a specific address on time.
Four prompts, two appointment types. The confirmation screen tries four ways of asking for a plan: a vivid, specific moment; the whole plan restated in the customer's own words; naming what might get in the way and planning around it; and a light, optional suggestion with nothing to fill in. Running the same four against phone and in-branch appointments separates which approach works at all from whether the ranking holds for a remembering problem and a travelling problem alike.
Hypothesis
Customers who book a home loan appointment, by phone or in-branch, and then see any of the four confirmation-screen prompts should no-show less often than customers who are only ever told the details. How the four prompts rank against each other may differ by appointment type. A travelling problem could favour a different prompt than a remembering problem, and that ranking difference is part of what the run would reveal.
Assumed baseline
Illustrative, not real CommBank data. Assume 22% of booked home loan phone appointments end in a no-show under the current reminder-call script.
Illustrative, not real CommBank data. Assume 28% of booked in-branch home loan appointments end in a no-show, higher than the phone rate. Travelling somewhere by a set time is a real added barrier a phone call never has.
Ethical guardrail
Where a plan prompt overreaches. A prompt that assumes a specific time, travel mode, or personal obstacle can feel intrusive if it asks for more than a customer wants to give before they have even arrived. A prompt with no real payoff to the customer can also read as the bank directing their thinking rather than helping them plan something they would benefit from planning.
What stays optional and identical. The appointment, the specialist or branch, and the information required match across all five arms for whichever appointment type is shown. Every confirmation-screen prompt is optional and skippable. Options offered are neutral and non-exhaustive, and none presume which obstacle, travel mode, or moment matters most for a given customer.
What the arms isolate. Whether asking the customer to state their own plan changes follow-through on an identical appointment. Whether the confirmation screen or reminder call should exist at all sits outside this design. See Not Testing Is Still a Bet.
The reminder-fatigue risk. The reminder call itself carries a separate cost this experiment does not measure: repeated contact can lift today's response while raising how many customers opt out of future contact altogether. See Reminder Fatigue.
The real flow today
Spotted in the wild
Real screenshots from Commonwealth Bank's live home loan appointment booking flow (commbank.com.au/digital/homeloanappointment), captured booking a real phone appointment slot. The same flow also offers an in-branch option; no real screenshots of that specific path were captured for this page.
The confirmation screen: a date, time and specialist, plus “Add to Calendar.” The specialist's name is blurred here for their privacy.
Further down the same confirmation: a reminder call is already promised, “to confirm the time and location and to ask you a couple of quick questions,” followed by a real document checklist.
This is a real, current product flow, not a demonstration that any change to it works. The reminder call and checklist already exist; nothing here shows the proposed prompts actually being tested.
Control vs. four treatments
All five arms book the identical appointment, with the identical specialist or branch and identical information required, whichever appointment type is shown below. Use the toggle to see how the same four prompts adapt to a phone appointment versus an in-branch one.
Control · today's screen
Appointment confirmed
Thursday 10:30am with your Home Lending Specialist. We'll call a few days before to confirm the time and go through what to bring.
Treatment A · vivid cue
Appointment confirmed
Thursday 10:30am with your Home Lending Specialist.
Where will you be when the call comes in?
Customers tell us this makes the call feel less rushed, one less thing to think about on the day.
Thursday 10:30am with your Home Lending Specialist.
Entirely optional: some customers find it helps to have a rough mental picture of 10:30am ahead of time. Picture where they'll be when the call comes in, and what they'd do if it catches them at a bad time.
Nothing to fill in here, just something worth having in the back of your mind.
Control · today's screen
Appointment confirmed
Thursday 10:30am with your Home Lending Specialist, CommBank George Street. Bring photo ID and your last two payslips.
Treatment A · vivid cue
Appointment confirmed
Thursday 10:30am, CommBank George Street.
What will you be doing right before you leave?
Customers tell us this makes the trip feel less rushed, one less thing to think about on the day.
Finishing breakfastAt work alreadyDropping the kids off
Entirely optional: some customers find it helps to have a rough mental picture of 10:30am ahead of time, when they'll leave and how they'll get there.
Nothing to fill in here, just something worth having in the back of your mind.
All five arms get the same appointment, the same specialist or branch and the same information requirements. Only whether the customer is asked to state a plan back differs. These are illustrative recreations of an app screen, not real screenshots.
Why four treatments, not one. Each one tests a different psychological lever from the same underlying theory. Treatment A makes the moment itself more vivid and easier to notice when it arrives. Treatment B has the customer build the whole if-then plan themselves, the mechanism the site's own principle article centres. Treatment C names a specific obstacle in advance and plans a response to it, the goal-shielding effect from the same research tradition. Treatment D tests whether any interaction is actually necessary at all, offering the identical idea as something to simply read. If Treatment D performs close to Control, active response is doing the real work. If it performs close to A, B or C, a lighter, optional suggestion might be doing most of that work on its own.
Sample size to actually trust this
Baseline22%no-show rate, control
Target, Treatment A18%−4pp, vivid cue
Target, Treatment B18%−4pp, self-stated plan
Target, Treatment C19%−3pp, name the obstacle
Target, Treatment D19.5%−2.5pp, text only
Significanceα = 0.05two-sided
Power80%1−β
Sample per arm4,127booked appointments
Total sample20,6355 arms
Assumed volume40/daybookings through this flow
Est. run time~74 wksincl. buffer
Two-proportion test, equal allocation: n = (zα/2 + zβ)² × [p₁(1−p₁) + p₂(1−p₂)] / (p₁−p₂)². Sized against Treatment D's smaller expected effect, the weakest of the four since it asks nothing of the customer beyond reading; Treatments A, B and C are correspondingly over-powered at this sample. If Treatment D's real effect turns out even smaller than assumed here, running it as a smaller, separate exploratory test rather than a full arm of this trial would be the more honest call. The volume figure is an assumption to make the timeline concrete, not a real product number.
Same test, same logic, sized against Treatment D again. A lower real-world volume and a higher baseline no-show rate both push this well past a year. At this real volume, a full-powered test of the weakest arm alone would take close to three years. That's a strong signal Treatment D belongs in a higher-volume channel first, or as a much smaller confirmatory follow-up once A, B and C have already run. The volume figure is an assumption to make the timeline concrete, not a real product number.
Outcome variables
✓Primary: % of booked appointments ending in a no-show, each treatment versus control, for whichever appointment type is shown
✓Secondary: pairwise comparisons among Treatments A through D, isolating which mechanism, a vivid cue, a self-generated plan, a named obstacle, or a passive suggestion, produces the larger effect
✓Secondary: whether the ranking among the four treatments differs between the phone and branch versions, testing whether a remembering problem and a travelling problem call for different tools
✓Guardrail: confirmation-screen abandonment rate must not rise in any treatment, every prompt has to stay skippable in practice, not just on paper
✓Guardrail: customer complaints or negative feedback specifically referencing any of the four prompts
The mechanism behind this:Implementation Intentions, full study and citation on the Principles page, including the goal-shielding mechanism Treatment C is built on.
Also worth citing: Nickerson, D. W., & Rogers, T. (2010). “Do You Have a Voting Plan? Implementation Intentions, Voter Turnout, and Organic Plan Making.” Psychological Science, 21(2), 194–199. A real field experiment, 287,228 voters ahead of the 2008 US presidential election: a call that helped people form a specific plan (what time, coming from where, doing what beforehand) raised turnout by 4.1 percentage points overall, and by 9.1 points in single-voter households. The branch version of Treatment B adapts that exact three-part structure directly.
Also worth citing: Martin, S. J., Bassi, S., & Dunbar-Rees, R. (2012). “Commitments, Norms and Custard Creams: A Social Influence Approach to Reducing Did Not Attends (DNAs).” Journal of the Royal Society of Medicine, 105(3), 101–104. A real NHS trial found asking patients to state their own appointment time back to staff, rather than simply being told it, measurably reduced missed appointments, the same self-stated-plan mechanism behind Treatment B here.