Mental gymnastics for online marketers - calculating payback (return-on-ad-spend)
Sep 14, 2015
Determining how to allocate an organization's online marketing budget can be a painful process. Understanding the metrics behind conversions and campaigns can help understand the return on investment and optimize spending. Drew Gillson, VP of Technology at Live Out There, speaks on how his organization analyzes customer cohorts to understand which programs are succeeding.
How should we spend our online marketing budget? You will find few early-stage SaaS or e-commerce businesses that don’t grapple repeatedly with this question. Despite the proliferation of tools available to understand website visitor behaviour in 2015, the ability of these tools to identify the relationship between online ad spend and growth is underwhelming. In our experience at LiveOutThere.com , we have found that we can’t rely on vendor data to make spending decisions. Sound familiar? Your retargeting network rep is probably telling you that their service returns 5x too! We also can’t rely on traditional analytics offerings like Google Analytics to make spending decisions, because of issues with data sampling and a general distaste for not having complete control over our data (although with newer features like the Management API, you can import cost data, and this may work just fine for your own use-case).
Despite the offerings available in the vast marketing technology ecosystem, we felt there had to be a better way to allocate our online marketing budget.
This post describes our attempt to solve the “where & when should we spend money” problem for ourselves. We use Looker, a modern web-based business intelligence platform, to generate activation curves for paid traffic cohorts. We’ve developed a method to “repatriate” cumulative ad spend to acquisition cohorts, which yields a running cost for each cohort over time. We then compare the repatriated cohort cost to the contribution margin dollars for the cohort, and calculate a payback figure. Finally, we set the maximum duration in which we expect a cohort to “pay back”, and we tune, tweak, and turn on and off paid traffic sources accordingly. Using this method, our marketing team allocates budget to campaigns that generate new buyers who add more dollars to our bottom line than the amount we’ve spent to acquire and remarket to them.
Example activation curves for cohorts of paid traffic. The X axis shows the age of a cohort in weeks. The Y axis shows the percentage of the cohort who have become buyers each week. The X axis is normalized so cohorts starting in different weeks can be more easily compared to one another.
Attribution modeling the right way
Through the process of developing our method, we concluded that fancy approaches to attribution modelling are still not particularly useful when looking for simple answers to that all-important question: “How should we spend our online marketing budget?” As online marketers have moved on from last-touch attribution to blended attribution models, we feel the problem has only become more complicated. Simply put, we believe that the CMOs of most* growing online business in 2015 are no better equipped to allocate their online marketing dollars with these complicated attribution models.
*There are certainly larger, omni-channel organizations that need the complexity— our method is most appropriate for SMBs like Live Out There.
Our opinion is that early-stage marketing organizations (companies in “Acquisition Mode”) need to start with first-touch attribution analysis. Typically, our acquisition cost for a new buyer exceeds the cumulative costs incurred (for remarketing & sending email) during our desired payback window for new buyers. For this reason we place the most emphasis on optimizing spend for channels that send first-time visitors to our online store who become new buyers, regardless of the impact future touches have on the likelihood these visitors become buyers.
In simpler terms, we accept that remarketing has an important place in the customer journey, but we don’t lose sleep over the exact mix of remarketing touches that create a buyer. Instead, we determine which first-touch acquisition cohorts pay back the fastest, we attempt to control for factors like seasonality and sale pricing, and then we invest more money in that channel, hoping to acquire more new visitors.
The problem is that recording a reliable “first-touch” for a visitor is a non-trivial problem. If it wasn’t, online marketers wouldn’t have focused on last-touch attribution for the better part of 15 years. Fortunately, software like Snowplow makes it easier than ever before. We use Snowplow to record page views, UTM parameters, and other event data from our visitors, and then we sink it into Looker for analysis.
Assigning costs to the visitor journey
Once we have this data in Looker, we can combine it with cost data from our advertising platforms and ask questions like, “What was the cost of attracting new* visitors from Google AdWords this week?” This number forms the basis of our payback analysis, which we call the cohort acquisition cost. We’ve chosen to bucket our costs into weeks for analysis. If you had the data pipeline, there’s no reason you couldn’t do this daily, but most people do tend to look at larger cohorts.
*Today, we define new visitors as visitors who have only had 1 session with us, but we hope to implement an identity stitching algorithm in the future that can consolidate visits from multiple devices into a single visitor profile.
Example chart showing a cohort cost over time (in blue) and the contribution dollars associated to the cohort (in green). This cohort is almost paid back in just over 3 months. The reason there is a dip in the contribution dollars in the 3rd week is that purchased goods were returned.
The cumulative cost of a cohort over time consists of the acquisition cost, plus any additional costs that can be repatriated to the cohort. So what are these “repatriated costs” for a cohort, and how do we measure them?
If we spent $250 to acquire the cohort of visitors shown above, that would only have been a portion of our spend in that channel on that particular week. To make the math easy, let’s assume we spent $1,000. That means that 25% of our weekly AdWords cost was used to acquire new visitors, and the acquisition cost for the cohort was $250.
Naturally, the remaining $750 we spent on AdWords that week was spent on repeat visitors. Because these repeat visitors themselves belong to older cohorts, the next step in the exercise is to determine the proportional representation of traffic from older cohorts and allocate the $750 accordingly.
Example matrix showing the costs incurred for several weekly first-touch channel cohorts. Costs following the acquisition event are incurred from multiple channels.
A deeper dive into cumulative cohort costs. The money spent the week of June 1st that we attribute to the cohort acquired the week of May 18th comprises costs from 3 separate paid channels.
We determine how to divvy up the weekly spend by allocating the same percentage of the cost to the acquisition cohort as the percentage of traffic the cohort accounted for, from that channel, in that week. So if Facebook drove 100 visits total the week of June 1st, and 10 of those visits belonged to visitors who first visited during the week of May 18th from Google AdWords, we would assign 10% of the Facebook advertising cost for the week of June 1st to the May 18th cohort. This is what we mean by repatriated cost.
We believe that this method of cost allocation is the best possible simple solution to a rather complicated problem. It isn’t perfect, but it gives our marketing team some great indicators they can use to understand the entire customer journey and its associated costs.
Unsolved problems and next steps
Now, we would like to understand the impact of each touch. This is what the more complex attribution models attempt to do. In other words, how much more valuable does a cohort become when you do juice the remarketing budget?
Crunching the underlying datasets is sloooow; the SQL queries involve many large tables with torturous joins. We want to analyze payback at the keyword level, but are running into database performance issues. How would an implementation of our method in Python perform?
If you’re interested in talking more about our approach to calculating return-on-ad-spend, please don’t hesitate to reach out to me at firstname.lastname@example.org, or just leave me a comment.
Thanks for reading!
View the original posting from Live Out There, VP of Technology, Drew Gillson.