If we deploy the tone-adaptive content strategist to all users, then we will increase the confirmed publish rate from 13% to 30%.
In the last 30 days, 62 external users visited the content module. Of those, only 8 converted to a confirmed publish via Gia’s scheduler — a 13% rate. The gap between users who enter the content module and those who actually publish is the activation problem this experiment targets.
The content strategist reduces the friction between a user having an idea and producing something they’re willing to put their name on. With better tone alignment and a more guided drafting experience, more users will cross the threshold from draft to publish.
All external users (GA deployment). The tone-adaptive content strategist is live as of March 16, 2026. Run for 14 days with a one-week check-in on March 23.
scheduled_posts.PUBLISHEDThe normalized target based on the same 50% relative lift Tukan described (20% → 30%) would be ~20% given the real baseline of 13%. That was set aside because moving from 8 to 12 users is too small a population shift to constitute a meaningful signal. 30% was chosen to ensure the result is readable — 19 publishers vs 8 is a real move.
Supporting signals — tracked for post-experiment analysis, not used to judge the outcome
| Event | Baseline (last 30d) | What it tells us |
|---|---|---|
| createSchedulePost | 10 users | Committed to a publish time — intent signal |
| clickMarkAsPosted | 17 users | Published outside the scheduler — captures Gia-assisted publishes not tracked by primary |
| clickEditPost / clickCopyPost | 13 / 5 users | Engaging but not committing — helps locate where drop-off happens |
| Hit 30% | Extend content strategist improvements to the Done-for-You cohort. Analyse which supporting signals correlated most strongly with publish to understand the mechanism. |
|---|---|
| Miss 30% | Use the supporting signal breakdown to diagnose the drop-off: are users engaging but not scheduling (intent problem), or not engaging at all (discovery problem)? Each diagnosis points to a different next experiment. |
The experiment is deployed and running. One issue is noted for the record before the March 23 check-in.
The contentAgent collection in MongoDB shows 24 users (19 external, 5 internal) — not the full external user base of 220. The meeting stated “general availability to everyone” but the database does not confirm this. If the flag is still restricted to 24 users, the experiment population is not GA and both the baseline and the target denominator need to be recalculated.