method_fit_click
Click to a method after guidance.
Click does not prove the recommendation was right.
oracle choice
Recommender tests
How to test method selection: Tarot, Lenormand, Runes, I-Ching, Moon, Astrology, Symbols, Dreams.
The recommender should help choose a language of response, not claim one oracle is stronger than another.
switch_method_rate
How many users switch method after guidance.
High switching can mean curiosity or confusion.
question_completed
Whether the user reached question completion after recommendation.
Do not complicate entry with a long quiz.
Write the hypothesis
Formula: if we change X for audience Y, metric Z will move because a specific friction disappears.
- There is one main changed element.
- There is one primary metric and one guardrail metric.
- There is a pre-written decision after the test.
Launch quietly
Start with a small traffic share or one segment, especially when testing payment, email, trust, or beginners.
- The test can be turned off without deploy chaos.
- Events are logged without private questions or scroll text.
- Support knows what changed.
Read the result
Do not declare a winner from one day. Check device split, traffic source, complaints, refunds, and behavior quality after payment.
- The result is compared with baseline.
- The guardrail did not worsen.
- The outcome is written into the experiment log.
experiment cards
- Three-question recommender versus one-question recommender.
- Method cards by situation versus method cards by tradition.
- Beginner default path versus free choice first.
rollback rules
- Rollback if users abandon more often before the question.
- Rollback if copy sounds like 'only this method knows the truth'.
output
- Winning recommender rule and affected routes.
- Updated method descriptions in search and halls.
red flags
- The recommender collects overly personal data.
- The system hides the free path.
related doors