Part III, The Craft · Chapter 15

The Scoreboard

Marketing ROI for sales promotions: how to measure sales uplift, and what entries cannot tell you

The company that stopped promoting

In the early 1990s Procter & Gamble did something unusual for a company of its size. It decided that its brands were spending too much on deals and coupons, and it pulled back. Under a policy it called value pricing, P&G cut the frequency of its deals by about 16 per cent and of its coupons by more than half, and put about a fifth more into advertising instead. Three marketing professors, Kusum Ailawadi, Donald Lehmann and Scott Neslin, later traced what happened across 24 categories between 1990 and 1996.

P&G lost share. Across those 24 categories it gave up about 18 per cent of its share on average, close to five share points in each. Deals and coupons, the researchers found, had been doing a specific job: bringing in buyers. Advertising did that job too, but less powerfully. Take the deals away and fewer people came in.

So was value pricing a mistake? That depends entirely on what you were counting. Measured on share, it cost P&G dearly. Measured on profit, the researchers thought it “plausible that P&G gave up share points in return for profits”, and their own estimate, built on assumptions rather than P&G’s accounts, put the gain at more than a billion dollars. Same policy, same years, two scoreboards and two verdicts.

That is the subject of this chapter. Every promotion in this book has been judged on something: sales for five of the six jobs, fame for the sixth. This chapter is about how that judgement is actually made, and about the one decision that settles many arguments about whether a promotion worked, which is agreeing before launch what will be counted.

What is marketing ROI for a promotion?

Marketing ROI, for a promotion, is a simple sum with one hard number in it. Take the extra gross profit the promotion produced, subtract everything the promotion cost, and divide by the cost. The costs are knowable, if someone does the work: the prizes or rewards actually paid out, not the forecast; the insurance premium; fulfilment and claims handling; the agency, the media, the point-of-sale, the retailer’s fee or funding; and for any discount, the margin given away on sales that would have happened anyway.

The hard number is the word extra. The promotion has to be credited only with the sales that would not have happened without it, and nobody can observe those directly. Every sale during a promotion looks the same at the till. Some were caused by the promotion; many were not. Entries and claims do not settle it, as this book has said since chapter 1: they are purchases the promotion can see, not uplift.

How big is the gap? Nielsen looked at promotion in Australian retail in 2018 and found that the share of products sold on promotion had risen over eight years from 30 per cent to 40 per cent, about $51 billion of sales. Its estimate was that about 48 per cent of those promoted sales would have happened anyway, and that around $11.3 billion went through promotions that generated little or no extra sales. A Nielsen analysis of the United States in 2014, covering 39 million promotional events, found that almost three-quarters of promotions did not break even. Those studies are about price promotion, the discount at the shelf, rather than the prizes and gifts this book mostly deals with. The question they raise is the same for every mechanic: how much of what sold would have sold anyway?

How to measure sales uplift

Uplift is measured against a baseline: an estimate of what would have sold in the same stores, in the same weeks, without the promotion. The whole craft of measurement is building a believable baseline. There are four ways to do it, and they are listed here from strongest to weakest.

Test and control. Run the promotion in some stores or regions and not in others that look like them, and compare the two over the same weeks. Differences in weather, competitor activity and the calendar fall on both groups, so what is left is mostly the promotion. It is the most convincing method and the hardest to agree to, because it means deliberately not running a promotion somewhere. Retailer groups often make a natural split, though chapter 14’s warning applies: their footprints differ, so compare rates and trends, not totals.

Before, during and after, in the same stores. Compare the promotion weeks with the weeks before and, just as important, with the weeks after. Chapter 10 explained why: about a third of a typical promotional bump is borrowed from other weeks, and it comes back as a trough. A read that stops on the last day of the promotion counts the bump and misses the bill. Set the same weeks last year alongside, to take out the calendar.

A modelled baseline. For brands with years of sales history, marketing mix models estimate the baseline statistically, separating price, distribution, advertising, competition and promotion. They are built by specialists and are best at portfolio questions, such as how much of the budget should go to promotion at all. They are less reliable for one small promotion inside a busy quarter.

Who bought, from panels and loyalty data. Household panels and retailer loyalty data show who bought, not only how much. That is the only way to answer a Breaker’s question, whether the buyers were new. Chapter 8’s research is the warning: on established brands, the extra buyers during a price promotion had almost all bought the brand before.

Whichever method is used, a few things fool the scoreboard every time. Sell-in is not sell-out: shipments into the retailer before a promotion measure the retailer’s confidence, not the shopper’s. A price change, a new listing or a competitor’s promotion in the same weeks will be credited to your promotion unless someone looks. And an out-of-stock in week two makes a good promotion look like a bad one.

A scoreboard for each job

Chapter 2 said that the job decides what is counted afterwards. Here is that scoreboard, job by job, with the data each one needs.

Breaker (trial). New buyers, and how many of them bought again at full price. Panel or loyalty data, or at least a “have you bought this before?” question at entry, which gives a useful, imperfect signal.

Builder (frequency). Purchases per buyer during the promotion and after it, against before. Coles reported comparable supermarket sales growth of 5.1 per cent in the quarter that included the first Little Shop, and 0.1 per cent a year later in the quarter that included the sequel: a company-level read, not a clean one, but a public reminder that the second run is measured against the first.

Loader (basket). Units and value per trip across the whole cycle, including the weeks after, so the trough is counted.

Harvester (data). Valid, unique contacts with consent, the speed of the first follow-up, and what those contacts buy afterwards compared with similar buyers who did not enter.

Keeper (loyalty). The share of the best customers still buying a year later, against a comparison group. Member counts and email opens are activity, not loyalty.

Headliner (fame). Coverage, conversation, share of search and the trade’s response, as chapter 13 set out, with sales read afterwards rather than inside the run.

Short-term sales and long-term brand

A promotion can win the quarter and cost the year. The evidence on this has been building for decades. Ehrenberg, Hammond and Goodhardt found in 1994 that price promotions on leading grocery brands had no lasting effect on later sales or loyalty: the bump came and went. Carl Mela, Sunil Gupta and Donald Lehmann followed shoppers for more than eight years and found, in 1997, that they became more sensitive to price and promotion over time, as brands shifted money from advertising into promotion. And Les Binet and Peter Field’s 2013 study for the IPA, The Long and the Short of It, put the balance that is now widely quoted: roughly 60 per cent of the budget on brand building and 40 per cent on sales activation, of which promotion is part.

None of that makes promotion the enemy. It makes it a tool with a known side effect, and the scoreboard should watch for the side effect. Add three long lines to any promotion that runs often: the average price shoppers actually paid, the share of the brand’s sales sold on promotion, and the brand’s share of search between promotions. If all three are drifting the wrong way, the promotions are teaching people to wait, which is chapter 1’s appliance floor. A promotion that is not a discount does not cut the shelf price, which is one reason this book spends most of its pages on prizes, gifts and experiences rather than on price.

How much should a promotion cost?

One of us once asked a client, a large hospitality business that ran a promotion every single month, what would happen if they ran it every other month instead and kept the difference. The promotion was expensive, and it had long since become part of the furniture. They thought he was mad. It was a serious question, and it is the one this industry has never properly answered: what is the least a promotion can cost for the most it can do, how should the money be split between the prize, the media and the retailer, and what would have happened if the promotion had not run at all?

Nobody in that room could answer it, and the reason was simple: nobody had ever seen the promotion’s absence. There is no universal answer either, and anyone who offers one is selling something. But the question can be answered for any one brand, with the method this chapter has been describing all along: compare the promotion with its absence.

The month you do not run it. A promotion that runs every month stops being a promotion. It becomes part of the price, and shoppers fold it into what they expect, which is chapter 4’s expectation problem. Nielsen’s 2018 Australian study found that 59 per cent of the most heavily promoted items did not react to their discounts. The only way to know what a regular promotion is worth is to stop it, on purpose, somewhere or for a while, and watch. Every other month is not madness. It is a test. Run the promotion in alternate months, or in half the stores or venues, read the difference in sales and margin, and the business learns in a quarter what it has been guessing at for years. If sales hold up in the months off, the promotion was paying for customers who were coming anyway.

Three buckets, one rule. Most promotion budgets go into three places. The reward: the prizes, cashbacks or gifts. The reach: the media that tells people the promotion exists. And the venue: what it takes to be met in store, from display and catalogue to retailer funding and incentives for store staff. Each has diminishing returns. The first prize makes a promotion possible; the tenth adds far less. The first weeks of media make it known; later weeks mostly reach the same people again. The first end displays put it in front of shoppers; the last few stores add the least. The right balance is the one where the last dollar in each bucket buys about the same amount of extra sales. Nobody knows where that point is in advance. It is found by moving money between the buckets, one at a time, and reading the result.

Where our experience points. Without a test, our experience leans one way. The reward is the bucket most often overfilled: chapter 3 showed that the prize pool is the easiest number in the room to argue about and rarely the one that decides the result, and chapter 7 showed that insurance can buy a big headline for a known premium rather than its full value. The venue is the bucket most often underfilled: the Harley in chapter 14 sold tea because the buyer gave it the floor. Media earns its place most clearly for Headliners and for Breakers, which need strangers; for a Loader run in store to people already shopping, the shelf is the media.

Against doing nothing. Every budget conversation should include the option of spending nothing, because it is the baseline every other option is measured against, and sometimes it wins. A promotion that only moves people who were coming anyway costs its full price and buys nothing, and that money is better kept, or put into the product or the price. Saying so is part of the craft. Almost every good idea has a version that works, and sometimes the version that works is the month off.

The Budget Test

Run it on and off: alternate months, or half the stores or venues, for at least two cycles. Change one bucket at a time, prizes or media or retailer support, never all three at once. Read the extra margin, not the entries, and include the weeks after. Keep an off period or a set of untouched stores in every plan, so the comparison is always there. Expect the answer to move as the market moves, and test again.

What entries are for

None of this means entries and claims are useless. They are the best diagnostic a promotion has. They show where the idea was met and acted on, which stores and which days, which pack sizes people bought to take part, what else was in the basket when receipts are read, and where the claim process lost people. That is how the next promotion gets better. What entries cannot do is stand in for the sales number, and a promotion report that leads with entries because the sales number was never asked for is a report written for the agency, not the client.

When entries are compared, the rules of chapter 1 apply: like with like, one-off promotions against one-off promotions, the same category, per week where the durations differ, and always-on programs kept to one side.

Agree the scoreboard before launch

Many arguments about whether a promotion worked are really arguments about what it was for, held too late. Settle it at the brief, on one page, and get it agreed by the people who will judge the result.

The Scoreboard On One Page

The job, and the one measure that will decide it. The comparison: control stores, an off period, last year, or all three. The data, who supplies it and when, written into the trading terms where a retailer holds it. The date of the read, including the weeks after. The full cost, line by line, with actual redemption replacing the forecast when it is known. The result that would count as success, and the result that would mean doing it differently next time. If nobody can fill in the last line, the promotion does not yet have a job.

Two habits make the page worth having. The first is writing the forecast as a ceiling, not a plan, as chapter 2 asked, so a lower redemption is read as the budget working rather than the promotion failing. The second is publishing the result to the people who approved the promotion even when it is dull. A team that only reports good promotions will never learn which of its ideas were the good ones.

Where this leaves the brief

Every chapter in this book has ended on a brief, and this is the page that makes the rest of them accountable. A brief without a scoreboard asks for a promotion that cannot fail, because nobody will be able to say whether it did. A brief with one asks for a promotion that can be learned from, which over a few years is worth far more than any single result.

The next chapter is about the other thing that decides whether a promotion can run at all: the rules. Permits, terms and conditions, consumer law and privacy are not the fine print of the craft. They are part of it.