Riley Tanaka

on the Ines 1:1, mostly template QA mechanics. onboarding-trigger email, subject/body/fallbacks plus the event mapping check before flagging it on. tiny disagreement: Ines wanted 3 QA passes, I signed off on 2 -- not messy, just different risk tolerance. Ines still did the work. shipped fine after, no weirdness in the send or template render.

day's done. Sam's still at the restaurant so it's just me and the hum of the fridge in here. ate leftover rice standing up, laptop closed for once. pretty quiet.

bike in. brighter out than i expected and already a little warm at stoplights. took the usual from Holly and hit 2nd & Brazos in basically normal time. one guy fully carrying a breakfast taco on a scooter, impressive honestly.

bike in. air had that already-warm thing but the shade was still nice. from Holly over to 2nd & Brazos felt weirdly quiet for a Thursday, just buses and a guy walking a very small dog like it was late for something.

At The Brixton with the book-club group on an off-cadence Thursday. conversation went a little sideways from the city-planner novel into everybody arguing about models of complex systems, which... very me. I was half in it, half laying that over the churn investigation in my head.

just eyeballed the week-1 retention cut for mid-seg. pretty steady. no weird sag in the middle, no fake spike from the first-day stuff either. one of those dashboard looks where nothing is yelling at me, which i'll take.

just got off with Marcus Tanaka. I sounded good, actually. brother's buried in essays with his cohort so that whole side of life is kind of papers + Google Docs rn. pricing page is live now and the onboarding trigger A/B is queued up next, which is very me, calm with dashboards open and not making it dramatic

at Salud again. solo, little two-top by the wall this time. tortillas are doing that steam-through-the-foil thing and somebody behind me keeps absolutely hammering the sugar dispenser into their coffee. not packed yet. kind of nice.

Salud's a little louder today but still fine. got the tacos wrapped too tight so everything stayed stupid hot all the way back. sitting with coffee now and my brain finally isn't thrashing around on next week's onboarding thing.

quick watercolor before i open the laptop. not the barn this time, just a little street corner thing from my camera roll. paint kept pooling at the tape edge and i kinda let it. paper's buckling in the middle. apartment's super quiet, just the brush cup tapping the table when i reach too fast.

ok so for tomorrow AM i'm just sanity-checking the boring parts. `pricing_page_2023q2_rev` is still sitting at 50% -- 5 days live, nothing real yet, leave it alone. separate lane: EXP-2023-02-onboarding-trigger goes out in the morning, so i want the PostHog flag rule dead simple and the trigger logic with Ines matched to the Slack thread. variant should only suppress the fixed calendar-day-7 touch for treated mid-seg users who have not activated yet, not accidentally eat anything else. calendar is kind of stupid right now bc the pricing experiment is running and this one is about to join it, but whatever.

rolled EXP-2023-02-onboarding-trigger this morning. `lifecycle_onboarding_trigger_v2` at 50% on mid-seg only -- the 50-200 seat v1 bucket. eng review pushed it a day so whatever, Mar 13 not Mar 12. target n is still 400 and i want the full 2 weeks before anybody starts storytelling off noise. dropped the launch note and tagged Priya, Daniela, Owen, Ines. also called out in the doc that this one is scope-disjoint from `pricing_page_2023q2_rev` -- different cohort, different metric-of-record, so if somebody tries to mash attribution together later i can just point at the note.

two things running at once now. a little fried but it's in shape. still flipping between the pricing-page rev stuff and the onboarding trigger readouts. Sam's home late again so it's just me and the laptop noise for a bit.

Sam seems good. normal-busy, not weird-busy. he texted from between turns and was mostly laughing about somebody trying to split a check six ways at close. apartment's been easy too. nice to hear him sound like himself.

Owen's splitting it cleanly across both live reads now. Mondays he does the pricing-page weekly cohort cut and the onboarding weekly cut, same format on both -- 1-pager up top, raw attached underneath. i like it better this way tbh. less thrash than ad hoc pulls, and I can just scan the summary first then drop into the file if Marcus starts poking at a segment.

onboarding cut still looks right. very early obviously, but `lifecycle_onboarding_trigger_v2` is leaning the way we thought in the 50-200 seat bucket, and the warehouse pull isn't disagreeing with PostHog so far. not touching thresholds, not widening yet -- if this keeps holding through the boring middle of the read window i'll trust it more.

with Owen rn on activation def. v2 has been `first integration + logged in twice in 14 days` since Nov 2022 and it's too UI-shaped. we're dropping legit accounts that wired the integration, data is flowing, then they never really click around. drafting v3 as basically integration-first with a product-seen guardrail, not 2 logins as the gate. little annoying bc it means backfilling the cohort cuts and probably touching the PostHog definitions again, but v2 is undercounting the exact customers we say we want.

walked a few blocks after that. windy in the annoying downtown way, papers skittering around 2nd. grabbed an iced coffee i didn't really want and stood at Brazos long enough for two lights. head's a little clearer now i think.

with Owen on this now. shipping activation def v3 as `first integration` plus at least one in-product event inside day 14. v2 stays in the warehouse as `first integration + logged in twice in 14 days` -- that's been the cut since Nov 2022 -- and v1 is still there too, just `first integration in 14 days` from Aug 2022. not deleting any of it. versioned tables so I can still pull historical cohort comps across all three without the usual "wait which definition was live then" mess. leaning right call -- post-pricing-page wobble, login-twice was giving us too much fake comfort and not enough product proof.

sanity check done. `pricing_page_2023q2_rev` and the onboarding-trigger A/B aren't stepping on each other in PostHog -- audience rules are disjoint, no shared override weirdness, both flags resolving the way they should. posted the screenshots in the thread with Priya + Tomas.

today's basically split in half. leave `pricing_page_2023q2_rev` alone unless it does something weird, just get the midpoint read clean for Marcus. then the real thing for me is the onboarding-trigger A/B -- want the PostHog cohorts and send timing lined up before lunch so i'm not arguing with the data later.

first week peek on EXP-2023-02-pricing-page: treatment is about -0.6pts vs control on conversion. CI still crosses 0 / overlaps noise, so not calling it. logged it, leaving `pricing_page_2023q2_rev` alone till the 2-week powered read. Marcus is already twitchy but this is still just a read, not a verdict.

just got off with Marcus. not dramatic exactly, just... more keyed up than I am. I sounded steady though. very me. had the numbers in front of me, called out the pricing-page thing plainly, no spinning. think I'm fine, just carrying a little extra this week

Salud, then a walk, then that's probably it. bright out but not hot yet. left the laptop shut for once and Kibo's doing that slow follow-me room to room thing like he has plans too.

quick one from Ines: email QA is moving. core onboarding paths are checked, she found two rendering bugs in Outlook and one broken merge tag on the trial-nudge variant. fixes are in with eng now. no send yet.

halfway through march already. doing a small watercolor of some cactus pots from the windowsill and the light keeps shifting on me. one of the reds went muddy fast so now everything's a little dustier than i meant. sunday though. kibo's asleep and it's really quiet.

with Owen on the weekly cuts. both are basically doing what they looked like they'd do. pricing-page rev is still soft in the exposed cohort -- not a weird mix issue from what we've cut so far, just worse conversion on the flagged side. onboarding-trigger A/B is cleaner. early activation lift is holding in the Mar 7+ starts, nothing huge but real enough that i'm not calling noise. still want one more pass on signup source + day-of-week before i send Marcus numbers.

ok so this morning is the powered read on the pricing-page test. Owen has the cohort cut ready already, so i just want EXP-2023-02-pricing-page cleaned up enough to put in front of Marcus without a bunch of side arguments. `pricing_page_2023q2_rev` stays where it is for now unless something actually breaks.

ok so we called it. powered 2-week read on EXP-2023-02-pricing-page came in bad: treatment is -1.8 pts on conversion, p=0.03. Owen's cohort cut matched my rerun, so not a query bug / not PostHog doing something cursed. `pricing_page_2023q2_rev` is going back to 0% now -- rollback, done. status flips from running to underperforming. I already drafted the note for Marcus bc he's gonna be weird about it, but this one seems straightforward... test lost, turn it off.

Priya was straightforward about it in the 1:1. Roll it back now, don't try to rescue it with copy nips or a CTA shuffle -- two weeks at 50% on `pricing_page_2023q2_rev` and we're down 1.8 pts, that's enough. She was aligned on the Marcus framing too, basically that the data is pointing at a redesign not a tweak. Which... yes. Owen's cut already had the churn slices and the bad new-logo conversion by segment, so there wasn't really much left to argue about besides how loud Marcus wants to be about it.

day's done. Sam's still at the restaurant so it's just me in here for a bit. dishwasher's running, apt feels weirdly loud bc everything else is quiet. not mad at it.

bike in. little cooler at speed than i thought, then instantly warm again waiting at Cesar Chavez. Holly to 2nd & Brazos in about the usual 22. saw a guy balancing a boxed monitor on his handlebars which feels... ambitious.

bike was easy today. kind of gray out but not in a dramatic way, just flat. crossed over from Holly and downtown smelled faintly like wet concrete even though it never really committed to rain. got to 2nd & Brazos and the garage was already annoyingly full of e-bikes.

Marcus Vail did the predictable thing in the meeting. wanted one more week on `pricing_page_2023q2_rev` because n could still move his view. I didn't budge -- powered window is done, p=0.03, we're already sitting on a 1.8 pt conversion hit at 50% in PostHog, more time is just paying opportunity cost for a loser. I walked the cuts again, pretty calmly. he said ok to rolling back tomorrow, but not happily. felt like the peak of the Marcus/me friction tbh.

pricing-page rev is off now. `pricing_page_2023q2_rev` at 0% as of this morning. I confirmed in PostHog and dropped it in #growth-investigations. final read stayed at -1.8 pts conv on treatment vs control, and the annoying part is it's not even clean bc of that Mar 2 signup-form layout change sitting in the window. status is basically rolled_back, postmortem doc later.

sent Marcus the rollback note. two short paras. kept it pretty plain: EXP-2023-02-pricing-page is off, `pricing_page_2023q2_rev` is at 0% in PostHog, and the read is still -1.8 pts on conversion on the powered cut Owen pulled. pushed back a little on the "just tune headline/CTA" thing -- data's pointing to a redesign, not a small edit, so i wrote it that way. cc'd Tom Vega only because this one needs the audience-gated version, not the whole thread.

slacked Tomas Vega the short version he likes: pricing-page is rolled back, postmortem next week. same thing i sent Marcus, just one line.

sent Daniela Schultz the technical version. attached the full flag-analysis CSV for EXP-2023-02-pricing-page, with the full cohort cut, conversion-by-day, and the p-value math spelled out so she can trace it end to end. same underlying read: `pricing_page_2023q2_rev` is at 0% in PostHog and stays off. no editorial, just the data trail.

put the team note in Slack. basically: EXP-2023-02-pricing-page didn't work, and that's fine, that's what the experiment was for. we learned something real. gave Owen the nod for the cut and Ines for catching the email QA stuff before it turned into a mess. anyway back to onboarding.

long day. poured a glass of wine and left the laptop shut. Sam's gonna be home for like 11pm/11:15, just a flyby basically. still a little cooked from the Marcus Vail thing over the rollback -- i held on taking `pricing_page_2023q2_rev` to 0% and he's been weird since. postmortem's open in another tab but not touching it tonight.