In 2011, I was Hotwire's Lead Product Manager for hotel booking. I reserved 15% of the Scrum team's bandwidth for UX optimization experiments. I was the first PM at Hotwire to do this methodically on a live product.
I worked with the interaction designer to compile a backlog of ideas, starting with classic techniques like social proof.
My strategy was to run as many tests as possible. I did this by prioritizing simple microcopy changes. I believed that success wouldn't correlate with scope.
We used a homegrown A/B testing system to measure the impact on booking rates. I could translate that into contribution margin (CM), or revenue minus costs. CM was Hotwire's key metric.
Here are a few examples of these experiments: