first_action_rate
Share of users who took the first meaningful step.
Scroll depth without action may just be wandering.
first hall
Landing tests
Tests for home, halls, search hub, and first screens: users must quickly know where to go.
A landing test should reduce wandering, not add decorative fog.
hall_selection_rate
Shows whether hall doors help choose a method.
Do not confuse a pretty door with a clear door.
mobile_tap_errors
Taps near buttons and repeated taps.
If the finger misses, copy will not save the UX.
Write the hypothesis
Formula: if we change X for audience Y, metric Z will move because a specific friction disappears.
- There is one main changed element.
- There is one primary metric and one guardrail metric.
- There is a pre-written decision after the test.
Launch quietly
Start with a small traffic share or one segment, especially when testing payment, email, trust, or beginners.
- The test can be turned off without deploy chaos.
- Events are logged without private questions or scroll text.
- Support knows what changed.
Read the result
Do not declare a winner from one day. Check device split, traffic source, complaints, refunds, and behavior quality after payment.
- The result is compared with baseline.
- The guardrail did not worsen.
- The outcome is written into the experiment log.
experiment cards
- Centered question box versus hall doors first.
- Search bar in hero versus search as secondary door.
- Beginner path above oracle catalog versus below catalog.
rollback rules
- Rollback if mobile first_action_rate drops.
- Rollback if users use search more because the primary CTA became unclear.
output
- Landing decision with desktop/mobile split and screenshots.
- Updated navigation if the winning path changes user intent.
red flags
- Hero became pretty but no longer explains the action.
- First screen is overloaded with oracles, trust, pricing, and emails all at once.
related doors