In our latest webinar “Bigger Returns. Smarter Decision Making. Marketing Budgets that Work.”, hosted by the American Marketing Association, experts from Cardinal Path and Data2Decisions Canada shared their deep experience gained from executing optimization and media mix modeling projects for some of the world’s leading brands.
One of the most difficult challenges marketers face is the ability to understand the different media touch points that are involved throughout the entire consumer journey. As the landscape becomes increasingly complex, organizations are missing out on profits. So, how can marketers best tap into the potential that exists, without relying on guesswork? In this webinar, our presenters Irina Pessin, Managing Partner at Cardinal Path, New York, and Andrew Dodds, Managing Partner, Data2Decisions Canada, showed us how best to unlock this potential through modeling and optimization. In case you missed it, you can watch the on-demand webinar here.
Below, you will find some of the questions that our audience asked during the webinar, as well as those questions that we didn’t have time to answer in the live session.
Q: What data is needed for the modeling and optimization work?
A: Andrew: There is a long list of data required for modeling. You shouldn’t think of it as a list in the sense that you can check it off as you go, and then never think about it again. Your data should reflect the conversion you are trying to measure, for example, traffic. All of the other data is an independent variable (execution, environmental impacts, pricing etc). This should be all media data, such as impressions, CTR, time spent on page. Through modeling, we test those variables for their relationship in explaining our influence on sales. The data itself should not be limited to being of a particular region or market, it could be as granular as the marketing data available, and can be built from store level. We recommend having granular data and an abundance of it. For optimization work, its more limited because it’s about the effects of sales on the business.
Q: How do you evaluate the accuracy of your models?
A: Andrew: A variety of statistical techniques that we use (e.g. R2 or adjusted R2) will help us to understand the ‘goodness of fit’ meaning how much of sales will it actually explaining and what is the margin of error. That is not to say it’s the sole determinant of a good model. A lot of it is dependent on the amount, and the quality of the data used to build the model. Statistics should reflect the validity of a variable and whether it is a meaningful and explanatory variable. Typically we use a T-stat and P-value. We also use ‘hold out’ samples to test the accuracy of the model – if it is a good model, it should be an accurate predictor of the dependent variable when the actual independent variables for the holdout period are inserted into the model.
Q: How long does it typically take to deliver a project?
A: Irina: Typically, a project can be delivered over a 12-14 week period. This includes time to gather the data using an existing MMM, or using benchmarks, setting up the tool, and working with stakeholders to make sure the optimizations are as relevant as possible.
Q: How much does a project typically cost?
A: Irina: I will outline an optimization-only project for the purposes of this question. However, an optimization-only project’s cost will still rely heavily on the scope. If we’re talking about one brand, the starting cost will be around 15K, if we are talking about 3 brands the cost will typically go up to 30 or 40K. Of course, there are economies of scale which can be applied across brands.
Q: Does modeling only work for sales?
A: Andrew: No. Although sales tends to be the most common measurement that the model will be built for, it can also work against other forms of conversion. Depending on the environment and vertical we are operating in, it could come in some other form such as store traffic, footfall, click through rates, or other action.
Q: Which markets and verticals do your benchmarks cover?
A: Irina: In terms of marketing, our benchmarks mainly cover all of North America, most countries in Europe, some countries in Asia, and Australia. We don’t yet cover South America right now, but generally speaking all of the other main markets are covered. Some of the benchmark verticals include: home care, healthcare, food and beverage, retail, banking, insurance, and more.
Q: How detailed can an MMM get for digital media channels?
A: Andrew: Digital is very dependent on the complexity of digital execution – there may be few “channels” used, or there could be many. In the event there are many, there is a very good chance we will experience auto-correlation which is a fancy way of saying everything is happening at the same time, and therefore it is difficult or impossible to assign value to the specific pieces. Instead, we manage it as a whole. In this case, there are other options such as digital attribution which are better suited to get at the minutia of the digital execution while the MMM can be used for a more strategic and macro assessment of the digital environment.
