Email content

We did something most teams are too scared to try.

We let AI redesign one of our landing pages, minimal human input, and ran it against the original.

It won by 44%.

So we did it again on a different page.

It won by 34%.

At that point we stopped asking "does AI win?" and started asking "how do we make this repeatable?"

Here's the short version of what we learned:

Pick the right page. You need ~2,000 monthly visitors minimum. Target pages that are underperforming quietly, not broken ones. Broken pages are a different problem.

Don't over-direct the AI. If you want it to genuinely challenge your assumptions, give it facts (audience, goal, differentiators) and get out of the way. Every edit you make reintroduces the blind spots you were trying to test around.

Use 99% statistical significance, not 95%. At 95% you have 19 people saying yes and 1 saying no. At 99% it's 99-to-1. That gap matters more than it looks. It also lets you skip complex sample size calculations.

Chase big signals. This workflow isn't for detecting 2-3% lifts on button color changes. You're testing a fundamentally different page. If it's working, you'll know.

Kill tests after 30 days with no clear winner. A flat test running forever is just a clogged pipeline.

The full post walks through every step: tool selection, the brief-to-build workflow, the critique loop, and when to roll out vs. keep testing.

Worth a read if you're sitting on a landing page you know could be doing more.

— Hiten

P.S. The thing that surprised us most: the AI caught objections our own team had stopped seeing. That alone made it worth it.

P.P.S. The reddit thread is here for any questions/comments you have.

P.P.P.S. You can now target specific audience segments with A/B Tests in Crazy Egg. More info here.

Crazy Egg, 13337 South St, #269, Cerritos, CA 90703, United States

Unsubscribe

Manage preferences