return_7d
Return within 7 days after first scroll.
Do not count anxious repeated readings as healthy retention.
return without hook
Retention tests
How to test My Day, journal, inbox, echoes, and favorites without anxious dependence on readings.
Good retention returns users to meaning, not panic.
journal_entry_created
Whether the user created a journal entry after an answer.
Do not store personal details without clear consent.
inbox_open_helpful
Email opened and led to a helpful action.
Open rate alone is often an empty firefly.
Write the hypothesis
Formula: if we change X for audience Y, metric Z will move because a specific friction disappears.
- There is one main changed element.
- There is one primary metric and one guardrail metric.
- There is a pre-written decision after the test.
Launch quietly
Start with a small traffic share or one segment, especially when testing payment, email, trust, or beginners.
- The test can be turned off without deploy chaos.
- Events are logged without private questions or scroll text.
- Support knows what changed.
Read the result
Do not declare a winner from one day. Check device split, traffic source, complaints, refunds, and behavior quality after payment.
- The result is compared with baseline.
- The guardrail did not worsen.
- The outcome is written into the experiment log.
experiment cards
- Echo reminder after 7 days versus after 3 days.
- My Day card with one action versus card with three prompts.
- Journal suggestion immediately after reading versus next visit.
rollback rules
- Rollback if users make many repeated readings about one fear.
- Rollback if unsubscribe or complaint rate rises.
output
- Retention decision with return quality, not only return count.
- Updated inbox cadence and journal prompt if test wins.
red flags
- Email implies the user will miss a sign without a new reading.
- Journal prompts become therapy or medical advice.
related doors