article thumbnail

Conservation of Intent: The hidden reason why A/B tests aren’t as effective as they look

Andrew Chen

This is why you can’t add up your A/B test results. If you’re at a company that A/B tests everything and then announces the great results – that’s wonderful, of course, but just run the thought experiment of summing together all of those A/B tests. Rarely does it match.

article thumbnail

The red flags and magic numbers that investors look for in your startup’s metrics – 80 slide deck included!

Andrew Chen

Of course, as an investor you can’t run A/B tests or analyze results directly, but you can form hypotheses, ideate, and apply the same type of thinking. There’s a couple common frameworks to try to understand this, and one is the Growth Accounting Framework. How do you predict the future? Is it working and will it sustain?

Metrics 111
article thumbnail

a16z Podcast: Why paid marketing sucks, Network effects, Viral Growth, and more

Andrew Chen

You know, ecommerce, a lot of companies struggle with, “Okay, how do I get organic ecommerce traffic?” ” So most ecommerce companies rely heavily on paid user acquisition, you know, typically one of the interesting things is they degrade over time and they’re all competing for the same user.