Re point one “testing micro conversions”:
can we really rely on the fact that the drop off rates in the next steps of the funnel starting from a micro conversion , will remain unchanged regardless of the testing variant(s)?

Please refer to: http://www.widerfunnel.com/should-i-optimize-for-micro-conversions/

Specifically:

” Since you will be sending double the traffic to each website version, you will have more data than you’d have had if you followed the traditional split testing way.”

I don’t get this.

How is running a test with 1 version getting 100% of the traffic for a week and then another version getting 100% of the traffic for the next week “more data” compared running an A/B test at 50-50 for two weeks?

It’s not more data. It’s just worse data.

@Lucas – That’s sequential testing. You will have to be very careful about monitoring external factors and your traffic sources to avoid any noise in the data.

@Alex – To declare a winner, number of conversions and traffic both play an important role. If your conversions are significantly high, you will get a statistically significant winning variation even with relatively small traffic.

By micro-conversions, I mean things like, ‘add to cart,’ ‘visits to the pricing page,’ ‘download brochure,’ etc. That is, something that shows the intent to buy. Since these conversions need less commitment from consumers and are easier to come by, it can help you find a winner quickly than tracking macro-conversions, like product purchase, paid signups, and so on.

As long as you know the conversion ratio of micro to macro conversions, you can calculate which version is working for you without having to wait too long.

I like article, but I’m not clear on point number 1:

“When you have traffic crunch, instead of tracking test data by macro-conversions, declare test results with micro-conversions that indirectly contribute to sales.”

Can you give some examples of how this would work?

It’s confusing because if I already know the ratio of micro to macro, then aren’t my micro-tests really macro anyway.

Maybe I’m missing something, and that’s why I’m hoping you have some examples of what you mean.

I think that the learning is not so important when it comes to low traffic, I usually focus on generating value by constantly testing and tweaking the tests as the time goes in order to increase CR, even if I don’t reach statistcial significance.

I also focus on the peak conversion days of the week to make sure my potential winner is outperforming the control on those days and then declare it as a winner.

This reminds me of a popular adage that says, “A ship in the port is safe, but that’s not what ships are built for”

Websites are made to generate revenue. So, you must keep testing to improve conversion rates. 🙂