Alex Valdez

at the 7th Avenue spot in Park Slope. same 7th-Ave block as the bagel place, opposite corner. cortado's fine, crossword open at the counter at the back, and yeah 4-letter for 'pulled' was DREW lol. pretty quiet.

Devika's home, we're doing the Friday leftovers. Gonna do a quick read-through of the migration sequencing doc after. Keeping the evening wrap to the usual short 1-3 sentence status note; that pattern's been stable for me since Q4 2021.

ok so first monday back. migration spine is basically metrics-router this week -- config-loader cleanup first, then cut legacy-aggregator out of the path in pieces. I own the whole metrics pipeline anyway, all three services: metrics-router, ingest-edge, shard-keeper. Hema can pull me in if the order gets weird but leaning router -> edge -> keeper unless prod does something dumb

Move the standing infra sync Wednesday to Thursday for the migration sprint. Hema's call on Mar 6 -- keep Tuesday afternoon clear.

thinking metrics-router cutover is gonna be 30% -> 60% -> 95% -> 100%, with a 48h soak at each step. put that in the cutover doc already -- early March decision so nobody "helpfully" compresses it later. 95 to 100 is the only part i don't totally trust yet bc legacy-aggregator has weird tail behavior when traffic mix shifts, but i still don't want a bigger jump.

Day done. Went through Nadia's ingest-edge config-loader diff. Looks fine from here, no weirdness in the loader path or the env fallback stuff. I'll send the approval in the morning.

just got off w Hema + Nadia. order is fixed now: ingest-edge first -- lowest blast radius, then metrics-router, shard-keeper last bc that's where the long-tail cardinality logic gets weird. Hema's basically tracking it at the manager level, not trying to run it day to day. Nadia had one real ask: keep label names parity on the dashboards during cutover so alert defs don't turn into a cleanup project at the same time. fine w that. shard-keeper itself hasn't started yet anyway, which honestly makes the ordering easier

lol it landed bc you weren't even in the Migration kickoff sync on Mar 7 and still somehow clocked shard-keeper immediately. it was just me + Hema + Nadia in there. hairy one is still the Python one, sorry

Devika just got back, normal shift for once. Heating up leftovers. Tuesday kickoff was clean -- no weirdness, people had the right docs open, and nobody tried to turn it into a bigger thing.

10:17 dip again. 0.4%, same dumb cardinality thing in legacy-aggregator from Q4. not even interesting now, just the usual little 10am wobble and then it settles.

Anya asked me figma vs sketch for the new pitch deck and I gave her a very confident "figma" despite not being a designer. pure vibes. she ended up using Figma, so apparently fine.

ok so morning is `metrics-router` config-loader review, then I want the staging shadow deploy config actually finished, not half-there. lunch w Cyrus in the middle. after that probably back on the router cutover doc -- still leaning 30 -> 60 -> 95 -> 100 w the 48h soak unless staging looks weird

Lunch with Cyrus today -- he pulled me into this weird data-platform throughput regression and I said yeah, I'll look next week. kind of my lane tbh.

Devika just got back from a normal shift, which still feels rare when you're with a 4th-year internal-medicine resident at a Manhattan hospital. Leftovers now. We've been doing this almost 3 years so I should be used to the night-shift week blocks vs my dumb work hours, but the apartment rhythm gets weird.

Hold the staging cutover for Monday morning. Friday afternoon is code review + prep, no prod deploys -- that's still how I work after that Q4 2021 outage at my prior job.

Anya texted him 'brunch sunday? the place near you with the eggs' and he was just yes lol. pretty good. she's settling in without making a production out of it, which seems more his speed too.

Ren's V5 boulder project is gonna be a whole three weeks at this rate. first go today, Mar 11, and he slipped off twice on the same move. no send. coffee after at the spot by the gym, both of us dusty and pretending we'll remember the sequence on Mar 18. maybe the Mar 24 send, lol.

around the park w Kibo. pretty empty except one guy doing sprints way too hard for a sunday. cold but not bad once you're moving. Devika's still asleep, which honestly tracks for a normal weekend. coffee in hand, dog stopping every 8 feet to inspect the same bush like it's new

ok so for the metrics-router config-loader cleanup i'm not trying to get fancy with it. mostly want one path for env/file overrides and kill the little helper pile that grew around the rewrite. if a thing stays in metrics-router it should look like the rest of the Go repo: gofmt strict, table-driven tests on the routing logic, and no log.Fatal in handlers bc i don't want request code deciding the process dies. that's been the standing convention there since last year's rewrite anyway. leaning small PRs over one huge sweep -- loader first, then test fixtures, then the handler bits that still have old assumptions from legacy-aggregator.

stayed late on the config-loader and found a dumb `metrics-router` merge bug. on Mar 12 of course. if two routing rules had overlapping label predicates, merge was picking the alphabetically-later rule instead of the more-specific one. fix was literally 4 lines. test was the real work -- ~80 lines table-driven just to pin the cases so we don't re-break it when someone touches ordering again. PR is up now. still a little annoyed this logic lived in three helpers but whatever

`ingest-edge` on staging cut over clean. let the 48h shadow traffic run and it came back zero dropped events. nailed it. back to the dumb `legacy-aggregator` edge cases now

Cyrus already claiming a beer off one "lol it works" is fair tbh. I gave him the "briefly, yeah" while ingest-edge staging shadow just sat there for 48h on Mar 11-9 under shadow traffic with zero dropped events.

48h shadow on staging finished. `ingest-edge` stayed at zero drops the whole run. wrote it down once in my notes and moved on. still have `legacy-aggregator` being weird about bucket flush timing lol

Day done. Let staging shadow sit the full 48 and it held clean the whole time on ingest-edge, zero dropped events. Tomorrow I'm gonna spec the prod cutover plan and write down the rollback bits so nobody gets cute mid-switch.

Devika's home, leftovers in those square glass containers again, tv on but neither of us really watching. pretty normal Tuesday. nice though. gonna miss this a little when nights start getting eaten by the migration.

ok so today is mostly the metrics-router cutover doc. want the spine down before lunch and keep the whole metrics-router cutover doc at ~4 pages if i can. don't want to overcomplicate it unless the rollout math gets weird.

Saturday sounds pretty chill then. Anya brunch + maybe drinks after is kind of ideal tbh. probably nothing big, even if i've got three services sitting in my head at all times.

graph did the Quarterly cardinality-drop bug thing again. legacy-aggregator service still has so much personality. it's literally marked scheduled_for_removal for 2023 and i'm over here watching its little farewell tour on a Wednesday. replacement path is still OTel collector + Prometheus, which... yeah, cannot come soon enough. whoever wrote this and left us with it, lol.

Devika got in late and we did the standing-in-the-kitchen version of dinner. she was happy bc one of her residents from earlier this year matched at Mount Sinai. mostly traded tiny stories from our days while Kibo kept trying to patrol for dropped rice.

Ping Nadia on the label-name parity doc. She said EOW and it's Thursday; doc's still empty -- need a nudge today.

ok so before i write the migration touch points doc i'm doing a full label_map inventory in shard-keeper today, Mar 16. not just the obvious routing path -- every read site, incl the weird admin/debug bits and the backfill-ish code that still sniffs label_map before assigning shards. i do not want to propose a clean cutover and then find some side path still depending on legacy shape. leaning grep first, then trace from the loader outward and mark each read as routing-critical vs incidental. shard-keeper is small enough that this is faster than pretending i remember it all, and the old label handling is exactly where surprise breakage likes to hide.

brunch w Anya at the Brooklyn diner she's apparently decided is her place now. kind of sweet. younger sister moved to NYC like 6 weeks ago for the design job at a creative agency and already has a diner. she asked him about Figma auto-layout over coffee and he gave a very engineer answer, which felt on brand lol

brunch was good. Anya seemed a lot more settled than she did in december, which was nice to see. she's really adopted that diner by me as her spot lol. Devika's home now so pretty low-key tonight.

same cafe situation. cortado, crossword, him getting fully stuck on 6-letter for 'lustrous' for like ten min. SHEENY, apparently. awful word. Anya owes him a coffee from last Sunday so he's feeling pretty good about that lol.

ok so shard-keeper inventory came back uglier than i wanted. counted 19 label_map read sites as of Mar 18. that's the surface area for the cutover, so i can't treat this like just the hot path plus one cleanup pass. a bunch are obvious shard assignment / routing reads, but there are also side bits where old label_map shape leaks into admin-ish code and some validation paths. leaning write the touch matrix first, then kill them in groups instead of file-by-file grep heroics -- too easy to miss one and ship a fake-clean cutover.

just checked staging dashboards. all green. metrics-router / ingest-edge / shard-keeper look boring in the best way. gonna head to bed early before i convince myself to poke at legacy-aggregator again

ok so today is final staging pass on all three -- `metrics-router`, `ingest-edge`, `shard-keeper` -- then i want the rollback dry run actually executed end to end, not just read off the doc. after that filing the cutover doc to the team. still assuming tomorrow unless staging does something stupid

Confirm with Hema: Tuesday 10:30am for cutover, 90-min hold. No Friday -- keep that off the calendar.

ingest-edge prod cutover is done. Tuesday Mar 21, mid-morning like we put in the doc, not Friday and not late in the day. Clean. Sat on the 90-min hold, watched the dashboards, nothing weird.

team-channel cutover events note, 2-3 sentence note max: ingest-edge cut clean this morning. monitored drops/latency/error rate plus the usual dashboards through the hold, nothing weird. gonna leave it under watch for 24h before calling next state done. Cyrus replied "noice".