delivery_rate
Deliverability before reading copy results.
Poor delivery makes subject testing meaningless.
email without noise
Email tests
Tests for subject lines, timing, service notifications, echo reminders, and moon letters.
Email should help find a scroll or return to a ritual, not create worry.
useful_click_rate
Click to useful action: open scroll, journal, privacy settings.
Do not chase clickbait subjects.
unsubscribe_or_complaint
The main email-test guardrail.
Complaint growth matters more than open-rate growth.
Write the hypothesis
Formula: if we change X for audience Y, metric Z will move because a specific friction disappears.
- There is one main changed element.
- There is one primary metric and one guardrail metric.
- There is a pre-written decision after the test.
Launch quietly
Start with a small traffic share or one segment, especially when testing payment, email, trust, or beginners.
- The test can be turned off without deploy chaos.
- Events are logged without private questions or scroll text.
- Support knows what changed.
Read the result
Do not declare a winner from one day. Check device split, traffic source, complaints, refunds, and behavior quality after payment.
- The result is compared with baseline.
- The guardrail did not worsen.
- The outcome is written into the experiment log.
experiment cards
- Subject: 'Your scroll is ready' versus 'The answer is sealed'.
- Echo timing: 7 days versus 10 days.
- Plain service email versus branded service email.
rollback rules
- Rollback if complaint rate rises.
- Rollback if email sounds like an urgent omen.
output
- Email decision with delivery, useful click, unsubscribe, and complaint notes.
- Updated macro in support ops if users misunderstand email purpose.
red flags
- Subject hides that the email is commercial or recurring.
- Email includes a private question in the subject.
related doors