The magic of Looker
Feb 11, 2014
Blog post originally published on Dan DeMeyere’s site here. Dan is the lead engineer for the web team at thredUP.com
Any company that makes data-driven decisions needs to have easy access to comprehensive views of their application's data. Average order value (AOV), customer lifetime value (CLV), cart abandonment rates, etc. are all crucial to understanding where to focus development efforts to maximize the conversion funnel (visits -> sign-ups -> adding to cart -> completing an order).
In thredUP's early days, we built a set of internal metric dashboards to help provide insight into this data. There were two flaws with these dashboards. One, all new dashboards or changes to the current dashboards required engineering resources. Two, dashboards are mostly hollow from a data perspective. Even with historical data that could be sliced by a number of dimensions, interesting data points would surface that would require investigation. The only way to investigate was by asking a developer for a custom metric pull. As the business grew, more and more of these metric requests started to flood the developer backlog. The inevitable moment had reached us as a company—we needed a business intelligence (BI) tool.
We spent weeks checking out the available BI services in the market, but each one we looked at either required significant setup time or had a hefty contract that would make any startup cringe. As we were weighing our options, our CEO arranged a meeting for Chris (our CTO) and me with Lloyd Tabb. Lloyd came to discuss the company he founded, Looker, which offers a query-based BI solution. I figured we would get a staged demo or a pitch deck with screenshots and features lists. Boy, was I wrong.
Lloyd came in, pulled up a real database that Chris and I were familiar with, and asked us for common metric requests developers typically have to pull. We started firing out metric requests, for which Lloyd easily whipped up the views. Because he made it look so effortless, we started thinking of the nastiest data pulls we could think of—some of which we only knew how to pull using SQL in conjunction with a scripting language. Not only was he able to do these data pulls with Looker (which executes pure SQL), I ended up using some of the queries that were being run (you can view every SQL query Looker executes) to help us speed up our application.
Every item thredUP sells is unique. We currently have 115,000 items for sale and we've sold multiples of that to date—so we have plenty of data to analyze. If we asked the following question “group items sold into a cohort based by the date they were first listed for sale and then give us the average days it took those items to sell,” not only would Lloyd know exactly how to whip up that data for us, he would go a step further and show us additional ways to dig into the data.
This is the magic of Looker. When you pull data, the resulting set is not static. Every data point is clickable and every data point you click results in a new set of data. If you're looking at order data and want to drill down on those orders, you just click the order data point and, boom, you're now on a view with all those orders. You can 'looker' like this for hours, and if at any individual look you want to change the query or bookmark your current query, they have that functionality. If you arrive at a great data set and want to share it with your co-worker, they have that too, and your co-worker can pick up right where you left off. There's no way I can do justice to this feature in words. If you were to use it for 30 seconds, you would immediately understand why I'm writing this post.
Three other reasons Looker is great:
- Setup. It took...wait for it...one day to set up looker. And by setup, I mean 100% ready to start querying our application's data.
- Customer service. If there's ever an issue or question, they have a chat window available so you can immediately have your question addressed. I should also mention that the person they have helping you is not a customer service representative. It's an engineer that’s able to dig into the question right away.
- RESTful API - JSON, XML, .txt, .csv — they have it all. We're a Rails shop at thredUP so I can't speak for their other SDKs, but their Ruby SDK makes it even easier to talk to their API.
The last thing I'll say is that I can't remember the last time I saw a custom metric pull request in our developer backlog, which can be attributed solely to Looker. Our product, marketing, operations, and engineering teams are more data-driven than ever thanks to Looker.