Hotwire hotel search results in 2011
Kristen Mirenda
UX Writing + Design Thinking + Product Know-How
Conversion rate optimization
Creating Hotwire's first UX A/B testing program and making millions with microcopy
Context
In 2011, I was Hotwire's Lead Product Manager for hotel booking. I reserved 15% of the Scrum team's bandwidth for UX optimization experiments. I was the first PM at Hotwire to do this methodically on a live product.
I worked with the interaction designer to compile a backlog of ideas, starting with classic techniques like social proof.
My strategy was to run as many tests as possible. I did this by prioritizing simple microcopy changes. I believed we'd learn that impact didn't correlate with scope.
We used a homegrown A/B testing system to measure the impact on booking rates. I could translate that into contribution margin (CM), or revenue minus costs. CM was Hotwire's key metric.
Here are a few examples of these experiments:
Confidence building
Hotwire uses opaque pricing. Customers aren't assigned a specific hotel until they make a non-refundable booking. In exchange, they get a deep discount.
Qualitative research confirmed this was too risky for a lot of people. So we added a tooltip showing a list of hotels they might get.
It was enough reassurance for many users. The ones who saw the new microcopy booked at a higher rate, worth $1.4 million per year in additional CM.
Social proof
What else helps people take an action when they're unsure? Knowing that other people have already done it. We added a daily count of bookings for each result. It was worth $1.5 million per year in CM.
Strikethrough pricing
TL;DR I earned Hotwire $50 million with this one bit of microcopy.
On Hotwire, the user books a discounted room that will end up being one of several possible hotels. Since those hotels have different full-price rates, there's not a single "real" rate to show as a comparison.
But we knew from user insights that it would really help them evaluate the offer. So we calculated a single representative rate that we could show as a strikethrough price.
I worked with Legal to make sure our approach was okay and that the explanatory footnote was clear.
Now customers could see what a great bargain they'd be getting. The booking rate saw a lift of more than 5%. That was worth over $6 million incremental CM per year.
8+ years later, it's is still in the UI. Based on that annual CM calculation, my microcopy has earned about $50 million (and counting).
Outcomes
In the first year of the program, we completed 10 A/B tests. 6 of them showed a statistically significant increase in bookings, adding up to $15 million CM per year.
I had demonstrated the value of simple copy-based optimizations. As a result, Hotwire spun up a new team to focus on them full time.