CASE 1/3 The mistranslated streak: rebuilding the retention vocabulary in 16 languages
The core retention mechanic of a learning app is the streak. Ours, it turned out, didn't mean anything in most of our languages.
This surfaced through the research program I ran: structured interviews with churned and retained users in Turkey, Thailand and Indonesia, 90+ survey responses, plus a localization audit my interns ran. The same class of defect kept appearing: strings shipped untranslated, machine-translated, or plain wrong. One user found a microphone permission that said we needed the mic "to speak Spanish", inside the English course, a hardcoded leftover leaking into every course.
At the center sat the word streak. In Hindi the app used a literary word closer to silsila, which no learner connects to a daily run. We were building loss aversion around a word that produced no mental image of anything to lose.
The move
A terminology pass, not a translation pass. For each of 16 languages we chose the term a learner would actually recognize, and verified each against what the market leader uses there, since their term has effectively taught the concept to millions: seri in Turkish, runtunan in Indonesian, racha in Spanish, sequência in Portuguese. For Hindi we deliberately transliterated "streak" into Devanagari, because the concept word matters more than a dictionary equivalent. For Vietnamese we kept the English word, because that's what the market knows. Then the same discipline went outward: D0/D1 notification copy re-localized and A/B tested (reverting where the new copy lost, no ego), all 48 award names, and the learn tab.
Mechanics don't retain users, meanings do. If the word at the center of your retention loop means nothing in a user's language, you don't have a retention loop in that market. You have a counter.
CASE 2/3 The poisoned metric: when "US retention" wasn't measuring US users
A reported US retention drop had been escalated to the founder. The product was fine. The denominator was sick.
It started as a routine anomaly in the weekly metrics review: a US activation drop matching nothing in the product timeline. Digging into the cohort, the US bucket was contaminated: a meaningful share of "US" users were actually Russian users on VPNs, geolocating to the US because of app-store and payment restrictions at home. They behaved nothing like US learners, and their weak retention dragged the whole series down.
The move
Separate the populations, correct the metric, then re-read history: every experiment judged against the polluted US baseline deserved a second look. The mystery that had reached the founder closed as a measurement artifact.
Before you debug the product, debug the denominator. A retention problem appearing from nowhere is sometimes a population problem wearing a product costume.
CASE 3/3 The push that couldn't wake the phone: an investigation in eliminations
Android push delivery had been below iOS for months, and the organization's working theory pointed at the vendor. Eliminating a vendor takes the same rigor as blaming one.
I owned the investigation. We tested the vendor's own root-cause theory against our data (it died, twice), shipped their recommended SDK upgrade (null result), audited our own instrumentation (found and fixed a real deep-link regression mis-stamping notification attribution), and audited campaign configuration (found a conversion definition silently zeroing iOS numbers). Real fixes, none of them the answer.
Changing the question
Instead of asking what broke, I asked what was different in our best period, and brought engineering just the timeline and our release calendar, no theory. The falsification logic: if the vendor pipeline had broken, day-7 delivery would cliff on the same date as day-1. It declined gradually instead, the shape of something spreading through the installed base with update adoption. Conclusion accepted: our build, fixable, and engineering took the two candidate builds to diff.
The structural discovery
Mapping why Android is structurally harder surfaced the biggest delivery gap in the product: Android pushes arrive as data-only messages, so the phone must wake our app before any notification can be drawn. And half of our Android daily actives are on Vivo, Oppo, Realme and Xiaomi-family devices, the OEMs that kill background apps hardest to save battery.
That one comparison ended the "Android problem" debate: where the OS doesn't kill the app, Android delivers at iOS parity. It also explained a confusing "sent" gap: on killer OEMs a day-0 user's phone refuses the first wake attempts, gets flagged unreachable, and is written off the same day, not old expired tokens.
What shipped forward
- An in-app prompt after the first lesson, targeted only at killer-OEM brands, deep-linking to the right settings screen per manufacturer (the setting is literally "Autostart" on Oppo/Realme, "Auto-launch" on Vivo).
- A message-type change for that segment so the OS draws the notification without waking the app at all.
- Sized honestly: a structural fix for all notifications, explicitly not sold as a D1 silver bullet.
A percentage metric hid a population problem. Every average delivery number was really two numbers: Samsung at iOS parity, and half the base ten points below. No amount of vendor tuning fixes a phone that refuses to wake your app.
ALSO IN THE LOG Migrations and recoveries
- CRM migration for 5M users (CleverTap → MoEngage): redesigned event schema and SDK integration, and overrode a vendor path that would have force-logged-out every user, designing push-token regeneration on launch instead.
- Email deliverability recovery: rebuilt sender authentication, isolated campaigns to a subdomain, recovered root-domain reputation, campaign mail back in the inbox.
- Research rigor: mixed-methods program across 3 markets, 90+ survey responses, managing 3 market-based interns.