Increasingly, brands are rethinking how they measure the efficiency of their ad campaigns. For example, Uber is cutting unnecessary ad spend, and the food delivery app Deliveroo evaluates the feasibility of investments in each new channel. To measure the effectiveness of marketing campaigns, these brands started measuring incrementality. Exactly this metric that answers the question: “Would users make their purchases in the same way if they did not see any ads?”
What is incrementality?
Incrementality determines which conversions were the result of marketing campaigns and which would have happened organically anyway. Many marketers calculate campaign and media channel revenue based on last-click attribution. It is easy to measure, and it is also available on many analytics platforms. But we cannot trust this data and say that the last click brought a sale – the user could have contacted the company via other channels.
What is incrementality testing?
The modern digital advertising market is gradually moving from measuring conversions to measuring incremental user growth. Incrementality testing is a guided experiment that calculates the number of new unique users coming from each media channel. In this case, incrementality is measured in relation to other channels, as well as in comparison with the complete refusal to launch paid advertising campaigns.
Incrementality testing allows you to determine the added value of each channel and understand whether it is attracting new users or the budget is being spent on buying potential organic traffic.
Why is incrementality important in app marketing?
If you know the incremental growth in traffic for each of the channels, you can effectively distribute the budget between media partners. According to Appsumer, in Q3 2020, advertisers with a monthly budget of over $1M used an average of 12 platforms to acquire users.
When evaluating the efficiency of mobile advertising, brands and app developers most often pay attention to correctly attributed installs and who made them – new or existing customers. For this, they use identifiers such as cookies and suppression lists to exclude active customers and leave only “clean” traffic. While not perfect, this approach meets the needs of most advertisers.
With the inevitable cancellation of the IDFA and the disappearance of cookie fingerprints, advertisers need to look for an alternative approach to evaluate the efficiency of advertising campaigns. Measuring the incrementality helps to optimize the strategy and increase the monetization of each channel by reducing costs and finding new ideas to influence the metrics.
How incrementality works
Measuring incrementality allows you to compare how advertising affects control groups of users in order to objectively measure the effectiveness of each channel without considering organic growth.
By correctly testing incrementality across all channels, advertisers can see what percentage of purchase-based ad reimbursement is satisfactory. Based on this data, they can adjust ad campaigns accordingly.
However, advertisers can easily get it wrong in determining the efficiency of their chosen strategy if they measure incrementality only for certain channels or within certain campaigns. The incrementality test will not work if the effectiveness of some channels is measured in isolation from others, as it is primarily a comparative test. Therefore, ideally, the incrementality test should run across all channels.
Not every channel delivers the same percentage of new users. In terms of incrementality, customers from different channels complement each other. It means that even if the channel attracts fewer customers, do not abandon it, as these users may simply not find you on other traffic sources.
Source: businessofapps.com
When is the best time to calculate incrementality?
A common mistake many advertisers make is trying to measure incrementality on low traffic channels. Before starting the incrementality test, you should perform a basic efficiency test for each channel. For example, does the channel meet the minimum criteria for measuring conversions and achieving the CPA (pay-per-action) goals?
The effectiveness of channel incrementality changes over time, so advertisers should regularly monitor campaigns and channels in their media mix. For example, as an app becomes more popular, a “network” effect can occur and some channels can become less effective. Other possible reasons for the change in incrementality are the addition of new channels to the media plan and a decrease in the efficiency of existing ones.
Incrementality calculation example
Let’s go through the calculation of incrementality by using the example of bulk email newsletters. We’ll try to answer the question: “Do emails trigger a purchase, or is it rather a reminder and the purchase would have taken place anyway?”
Let’s say we launched an email campaign that brought in an income of $100,000. You can calculate income in different ways: by the last click, looking at end-to-end analytics, or by the performance of an individual campaign. Let’s suppose that the cost of this email was $10,000, and the product margin was 50%.
Step 1: Calculate the net profit. To do this, subtract expenses from income and multiply them by the margin.
Profit = (100,000 – 10,000) * 0.5 = $45,000
By the numbers, we see that the mailing is successful, the incomes exceed the expenses. We can work with this further.
Step 2: Disable the mailing. What if you avoid the mailing and still look at the income from the same audience? Perhaps at this stage, it will turn out that the email campaign is not a purchase trigger.
Step 3: Calculate profit with the disabled mailing. Let’s say the income from the audience for the time until the email was sent out had been $50,000 with expenses of $0:
Profit with disabled mailing = (50,000 – 0) * 0.5 = $25,000
Step 4: Compare income from the same audience with the disabled and working mailing lists. To calculate the “net” incremental income from the mailing, subtract the income when the mailing was disabled from the income when the mailing was enabled. In our example, these are: 45,000 – 25,000 = $20,000. This will be the added value of our newsletter.
Conclusion: In our example, spending $10,000 will generate a “net” incremental income of $20,000 as a result of the email campaign.
In our case, the net profitability of email campaigns is $20,000, not $45,000. It turns out that expenses are half the incremental income, so we can say that the newsletter works and acts as a purchase trigger.
How to apply the method in practice
The above example is an abstraction with invented numbers. Here’s how to measure incrementality in practice.
First, determine the channel whose incrementality you are going to measure. For newsletters, you can consider the following channels:
- The entire email channel – to understand if mailings bring money at all. But this is too radical – you will have to disable all newsletters for a while.
- A specific strategy. For example, what is more profitable – to send 5 letters per week or 5 letters per month?
- Email threads. You can find out how effective and profitable each subsequent thread is.
- Specific mailing lists – to find out if the mass mailing really brings in additional profit.
Then choose an audience that will not receive letters. There are two options for creating such an audience:
- Segmentation. Select a control audience from the mass mailing to which you will not be sending anything. In the case of threads, we select a segment of users on which we will not start a new thread. This is the easiest and most affordable way, which is suitable if you need to measure the incrementality of a single letter, regular mass mailing, or a new email thread.
- A separate list with new subscribers. Send everything as usual, but save the control audience from some of the new subscribers to a separate list. You will not be sending them marketing emails, only transactional ones. Creating a separate list is good for calculating the incrementality of bulk email newsletters and welcome threads or choosing a strategy.
After that, choose the main metric for the email channel. Every business has its own set of basic email channel metrics. As a rule, e-commerce chooses revenue, and content projects – the retention rate. The main point is to choose the metric that is really important for your email channel.
Finally, determine the number of conversions before stopping the test. It is necessary to measure at what number of conversions in the control group this test will be statistically significant. When you reach this figure, you can stop the test and start calculating the results.
Over to You
Incrementality testing is a great way to measure the efficiency of ad campaigns, save on advertising costs, and have a chance to still get organic conversions without ads.
