Affinity analysis for recommendation engines
Mar 9, 2016
Lots of businesses make good use of recommendation engines based on affinity analysis to improve their offering. Let’s look at three examples before diving into what often can seem like black magic.
Playlists & song recommendations
Have you ever been listening to a playlist in a public place and thought to yourself, “this playlist is sooo good, whoever made this is a genius!” only to discover it was Pandora? That Pandora can deliver that experience is their competitive advantage and the basis of that is taking a song selection and making great recommendations based on it.
Amazon product recommendations
Amazon.com has gained notoriety for countless things from drones to their razor thin margins. One less talked about part of Amazon.com is their recommendation engine which suggests high-affinity products based on customers’ purchase and browsing history. Although it may not be a culture-shaking innovation, it’s a safe bet that the strategy pays significant dividends. And why shouldn’t it? If you know people who buy staplers tend to also buy paper (or people who buy “Office Space” on DVD have a high affinity for red Springline staplers) then it stands to reason that offering those products, in context, will result in more sales than a control scenario.
In brick and mortar retail, affinity analysis is a classic strategy to increase revenue. For example, if we know that toothbrushes and toothpaste are often bought together, putting them both on sale at the same time is unlikely to significantly increase revenue. But, if we put just one on sale then we can rely on the buyer’s affinity to boost the sales of the other. Sometimes retailers will discount one item below the point of profitability because this effect is so strong. If you have the discipline to buy your toothbrushes and toothpaste separately then there is a good chance you will come out ahead!
These three examples make it clear that affinity analysis can be profitable in lots of different contexts and that it can lead to big business optimizations.
Now I’m not an engineer and if I were I would probably take issue with the following explanation of affinity analysis (especially if I was in the product recommendation division of Amazon) which is fair. I want to explain the 20% of affinity analysis that produces 80% of the results in simple terms that everyone can understand and apply to their business.
Let’s look at the music recommendation example because who doesn’t love music? Imagine you had access to a list of popular playlists, maybe it was the 500,000 most followed playlists on Spotify. One thing you could do with this list is rank how often a particular song appears in the list. The song that appears in the most playlists is ranked #1, and so on down. Then, let’s do the same for songs from particular artists. Now we know which songs are most popular overall and which songs are the most popular for each artist.
Now let’s take this data and format it a little differently. Let’s make a new table where each row represents two different artists appearing in the same playlist. In the far right column we have the total number of co-appearances in playlists and with that information we can rank them from most co-appearances to least.
Cool. Now here is what we do when we are asked to create a playlist based on a song. Go to the table of co-appearances table and find the 5 artists who co-appear the most with the artists we are basing the playlist off of. Now go to the table where we ranked the songs by popularity and artist and find the 3 most popular for each artist. Go find the next most popular songs for the original artist and put them all in a list together. Boom, we’re done. No black magic, just simple logic applied to lots of good data.
Skeptical that this actually works? Fair enough, but before you start tearing it to pieces let your ears be the judge. Lloyd, our founder, built this in Looker in a matter of hours, which is what got me thinking about affinity analysis to begin with. It comes complete with playable song links and everything. Seriously. Check it out. It’s no Pandora but we think it’s a pretty good demonstration of the power of affinity analysis and it’s way better at making playlists than most DJs I know.
Want to learn more? Check out our Chief Analytics Officer, Colin Zima's post on recommendations.