Hotwire hotel search results in 2011
Conversion rate optimization
Creating Hotwire's first UX A/B testing program and making millions with microcopy
In 2011, I was Hotwire's Lead Product Manager for hotel booking. I reserved 15% of the Scrum team's bandwidth for UX optimization experiments. I was the first PM at Hotwire to do this methodically on a live product.
I worked with the interaction designer to compile a backlog of ideas, starting with classic techniques like social proof.
My strategy was to run as many tests as possible. I did this by prioritizing simple microcopy changes. I believed that success wouldn't correlate with scope.
We used a homegrown A/B testing system to measure the impact on booking rates. I could translate that into contribution margin (CM), or revenue minus costs. CM was Hotwire's key metric.
Here are a few examples of these experiments:
Hotwire uses opaque pricing. Customers don't know the hotel name until they make a non-refundable booking. In exchange, they get a deep discount.
Qualitative research confirmed this was too risky for a lot of people. So we added a tooltip showing a list of hotels they might get.
It was enough reassurance for many users. The ones who saw the new microcopy booked at a higher rate, worth $1.4 million per year in additional CM.
What else helps people take an action when they're unsure? Knowing that other people have already done it. We added a daily count of bookings for each result. It was worth $1.5 million per year in CM.
On Hotwire, there's not a single full-price rate corresponding to the discounted one. We calculated a representative rate and showed it as a strikethrough price.
I worked with Legal to make sure our approach was okay and that the explanatory footnote was clear.
Now customers could see what a great bargain they'd be getting. The booking rate increased more than 5%. That was worth over $6 million incremental CM per year.
After 8+ years, my microcopy is still in the UI. During that time, it would have earned Hotwire at least $50 million (and counting).
In the first year of the program, we completed ten A/B tests. Six of them showed a statistically significant increase in bookings. They added up to $15 million CM per year, far beyond expectations.
I had demonstrated the value of simple optimizations. As a result, Hotwire spun up a new team to focus on them full time.