beginner_path_start
Shows whether a beginner started the guided path.
Do not force everyone through onboarding.
first visit
Onboarding tests
How to explain OmenHall to a beginner without a lecture, overload, or closed-club feeling.
Onboarding should reduce anxiety and provide the first safe step.
free_first_success
Whether the user reached the first free result.
Free path must not be a trap with no exit.
trust_page_visit
Shows need for boundary explanation.
High trust visits can be a good sign, not a problem.
Write the hypothesis
Formula: if we change X for audience Y, metric Z will move because a specific friction disappears.
- There is one main changed element.
- There is one primary metric and one guardrail metric.
- There is a pre-written decision after the test.
Launch quietly
Start with a small traffic share or one segment, especially when testing payment, email, trust, or beginners.
- The test can be turned off without deploy chaos.
- Events are logged without private questions or scroll text.
- Support knows what changed.
Read the result
Do not declare a winner from one day. Check device split, traffic source, complaints, refunds, and behavior quality after payment.
- The result is compared with baseline.
- The guardrail did not worsen.
- The outcome is written into the experiment log.
experiment cards
- Onboarding card after first question versus before first question.
- One paragraph safety note versus three-card explainer.
- Start with My Day versus start with recommender.
rollback rules
- Rollback if onboarding lowers first_action_rate.
- Rollback if users perceive the service as too complex.
output
- Beginner path decision and onboarding copy patch.
- Updated FAQ if users ask the same beginner question.
red flags
- A beginner sees too many systems immediately.
- Onboarding sounds patronizing or mystically unclear.
related doors