article thumbnail

Conservation of Intent: The hidden reason why A/B tests aren’t as effective as they look

Andrew Chen

This is why you can’t add up your A/B test results. If you’re at a company that A/B tests everything and then announces the great results – that’s wonderful, of course, but just run the thought experiment of summing together all of those A/B tests. Rarely does it match.

article thumbnail

my boss has terrible ideas about social media, video games on your resume, and more

Alison Green

The other option would be for you and your fellow team members to push back on your PI’s preferred strategy, possibly suggesting some A/B testing so you get real data on what does and doesn’t work … but it doesn’t sound like you want to take that on, which is completely reasonable. Putting video game playing on your resume.

Media 94
Insiders

Sign Up for our Newsletter

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Trending Sources

article thumbnail

How to build a billion dollar digital marketplace – examples from Uber, eBay, Craigslist, and more

Andrew Chen

If you tend to get homesick after a few days or don’t sleep well unless you’re in your own bed, this is definitely not the position for you. Optimization : A/B testing growth levers – from email/SMS/push copy – to when/how to reach out.

article thumbnail

Organizational Development: Theories on Organizational Change

Walk Me

The definition of organizational development comes in three sections : Uses a critical/ science-based process – Scientific and intensive market research is the basis for organizational development processes. Although we can include individual and group interventions within this, they are all achieved from the organizational perspective.

article thumbnail

What to look for when you’re hiring a Head of Growth

Andrew Chen

This is where the classical definition of a “network effect” is wrong. There were projects to help with A/B testing (Morpheus!), But it’s not — to the contrary, this is when technology companies start to hire thousands of people, launch a series of ambitious new projects, and try to continue the product’s rapid trajectory.

Scaling 82
article thumbnail

28 ways to grow supply in a marketplace — by Lenny Rachitsky, ex-Airbnb

Andrew Chen

Converting travelers to hosts definitely moved the needle when we launched a new market. The biggest gains in optimization don’t come from brute-force A/B tests, but from trying to understand the real barriers to people using your product. This lever works, but can easily become addicting. Tactic #9: Convert demand to supply.

article thumbnail

The red flags and magic numbers that investors look for in your startup’s metrics – 80 slide deck included!

Andrew Chen

Of course, as an investor you can’t run A/B tests or analyze results directly, but you can form hypotheses, ideate, and apply the same type of thinking. I use MAUs in this example but you could also use active subscribers, or users who have bought something in the past 30 days, or some other definition.

Metrics 111