02 / alex
Alex Valdez
Infrastructure engineer / Sphere (initial profile)
Infrastructure migrations, incident response, team coordination, and life outside work.
000201May 31, 202313:00 UTC-04:00Yes. Take the 30 with Iris + Theo next week. Any slot that doesn't blow up my Tuesday.
Yes. Take the 30 with Iris + Theo next week. Any slot that doesn't blow up my Tuesday.
000202May 31, 202316:00 UTC-04:00May 31 note to self: my pre-Lantern observability sketch is the interesting version, not another status layer. basically a way to surface "what eng is doing" without making people write "what i did". if this gets real, I'd want it pulled from the metrics-pipeline layer directly -- deploys, ownership churn, incident load, maybe query volume shifts, code movement if it's clean enough, but not a creepy employee dashboard thing. more like durable surfaces Theo/Iris can look at before the formal Theo/Iris meeting next week and immediately see where attention is going. leaning event stream + service graph + a couple opinionated views, not a feed nobody opens.
May 31 note to self: my pre-Lantern observability sketch is the interesting version, not another status layer. basically a way to surface "what eng is doing" without making people write "what i did". if this gets real, I'd want it pulled from the metrics-pipeline layer directly -- deploys, ownership churn, incident load, maybe query volume shifts, code movement if it's clean enough, but not a creepy employee dashboard thing. more like durable surfaces Theo/Iris can look at before the formal Theo/Iris meeting next week and immediately see where attention is going. leaning event stream + service graph + a couple opinionated views, not a feed nobody opens.
000203Jun 1, 202308:30 UTC-04:00today is mostly getting the restart date locked. need Hema to confirm `shard-keeper` is actually June 6, not floating. also want the pre-push parity sync on calendar before EOD friday with Nadia + whoever owns `rollup-service` now -- don't want that booked at the last second and turn into another weird config mismatch.
today is mostly getting the restart date locked. need Hema to confirm `shard-keeper` is actually June 6, not floating. also want the pre-push parity sync on calendar before EOD friday with Nadia + whoever owns `rollup-service` now -- don't want that booked at the last second and turn into another weird config mismatch.
000204Jun 1, 202314:00 UTC-04:00Restart date is locked now. Hema and I agreed this afternoon to unpause `shard-keeper` and pick it back up the first week of June, under the new runbook/rollback rules from the audit. June 6 is the date on the board now, not floating.
Restart date is locked now. Hema and I agreed this afternoon to unpause `shard-keeper` and pick it back up the first week of June, under the new runbook/rollback rules from the audit. June 6 is the date on the board now, not floating.
000205Jun 1, 202316:30 UTC-04:00Cyrus DM'd right after I sent the date over. just "so we're back" and I hit him with "we're back." perfect honestly. that's about the exact amount of ceremony this one needed.
Cyrus DM'd right after I sent the date over. just "so we're back" and I hit him with "we're back." perfect honestly. that's about the exact amount of ceremony this one needed.
000206Jun 2, 202314:00 UTC-04:00Saturday -- Anya's coming for dinner. Devika wants to cook. Grab a nice bottle or two before then.
Saturday -- Anya's coming for dinner. Devika wants to cook. Grab a nice bottle or two before then.
000207Jun 3, 202320:00 UTC-04:00Anya came by for dinner. Devika made shakshuka. Pretty nice, actually. She didn't touch the layoff thing once -- just ate, talked about random Brooklyn nonsense, helped clear plates. Feels like she's settling back into normal with him. good night
Anya came by for dinner. Devika made shakshuka. Pretty nice, actually. She didn't touch the layoff thing once -- just ate, talked about random Brooklyn nonsense, helped clear plates. Feels like she's settling back into normal with him. good night
000208Jun 3, 202322:30 UTC-04:00Devika clocked it right away after Anya left. we were on the couch and she was like she's holding it together, and I just went yeah, tougher than she looks. I think I was good with her tonight. no overtalking it, just normal.
Devika clocked it right away after Anya left. we were on the couch and she was like she's holding it together, and I just went yeah, tougher than she looks. I think I was good with her tonight. no overtalking it, just normal.
000209Jun 4, 202316:00 UTC-04:00date's finally real so tuesday can be tuesday. weirdly quieter in here now. devika just got home and we're gonna go walk before it gets dark -- first one together since april. kibo already doing circles by the door lol
date's finally real so tuesday can be tuesday. weirdly quieter in here now. devika just got home and we're gonna go walk before it gets dark -- first one together since april. kibo already doing circles by the door lol
000210Jun 5, 202308:30 UTC-04:00today is the boring setup for tomorrow. pre-push sync w Nadia and the rollup-service owner, make sure label-name parity is actually locked and nobody has a cute interpretation of the cutover steps. if that meeting gets hand-wavey, stop it -- shard-keeper restart needs to be dull.
today is the boring setup for tomorrow. pre-push sync w Nadia and the rollup-service owner, make sure label-name parity is actually locked and nobody has a cute interpretation of the cutover steps. if that meeting gets hand-wavey, stop it -- shard-keeper restart needs to be dull.
000211Jun 5, 202317:00 UTC-04:00Pre-restart sync done for Jun 5. Three in the room: Nadia, Roman from Cyrus's data platform team -- he's the rollup-service owner now -- and me. Label-name parity checked out. Restart's on for tomorrow.
Pre-restart sync done for Jun 5. Three in the room: Nadia, Roman from Cyrus's data platform team -- he's the rollup-service owner now -- and me. Label-name parity checked out. Restart's on for tomorrow.
000212Jun 5, 202319:00 UTC-04:00Day's done. Jun 5 pre-push sync was clean -- Nadia (alerting), Roman (rollup-service), me in the room, Hema attended remotely. got label-name parity in writing, which is the whole thing. Devika's still at the hospital so I'm eating random noodles w Kibo staring at me.
Day's done. Jun 5 pre-push sync was clean -- Nadia (alerting), Roman (rollup-service), me in the room, Hema attended remotely. got label-name parity in writing, which is the whole thing. Devika's still at the hospital so I'm eating random noodles w Kibo staring at me.
000213Jun 6, 202308:30 UTC-04:00restart push is 11, not 9. tuesday helps and it gives Roman + the rollup-service side a minute to exist before we touch anything. this morning is just windows, parity check w Nadia one more time, make sure shard-keeper restart path is the same one we already said out loud in the room yesterday.
restart push is 11, not 9. tuesday helps and it gives Roman + the rollup-service side a minute to exist before we touch anything. this morning is just windows, parity check w Nadia one more time, make sure shard-keeper restart path is the same one we already said out loud in the room yesterday.
000214Jun 6, 202311:00 UTC-04:00Jun 6 restart is underway. `shard-keeper cutover` started Tuesday morning, on purpose not Monday. I did the pre-push sync with Nadia and the rollup-service owner before touching it; label-name parity checked out. First day was clean.
Jun 6 restart is underway. `shard-keeper cutover` started Tuesday morning, on purpose not Monday. I did the pre-push sync with Nadia and the rollup-service owner before touching it; label-name parity checked out. First day was clean.
000215Jun 6, 202318:30 UTC-04:00day one of the restart was clean. `shard-keeper` moved this morning, nobody paged me on the way through, labels stayed lined up with Nadia/Roman's side. first time in like two months i can say i'm going home and not half-expecting my phone to light up. Devika already texted about noodles, so yeah.
day one of the restart was clean. `shard-keeper` moved this morning, nobody paged me on the way through, labels stayed lined up with Nadia/Roman's side. first time in like two months i can say i'm going home and not half-expecting my phone to light up. Devika already texted about noodles, so yeah.
000216Jun 7, 202315:00 UTC-04:00`shard-keeper` is at 30% now and holding clean. No weirdness from labels, rollup-service side stayed quiet, nothing spilled over to the router/edge stuff. Letting it sit there and bake until Friday before pushing further.
`shard-keeper` is at 30% now and holding clean. No weirdness from labels, rollup-service side stayed quiet, nothing spilled over to the router/edge stuff. Letting it sit there and bake until Friday before pushing further.
000217Jun 8, 202308:30 UTC-04:00first pass on Nadia's comments this morning. gonna just answer the code/doc stuff and leave the extra stuff alone. cross-review at 2. want `shard-keeper` label-name parity squared away before that so this doesn't turn into another weird side thread.
first pass on Nadia's comments this morning. gonna just answer the code/doc stuff and leave the extra stuff alone. cross-review at 2. want `shard-keeper` label-name parity squared away before that so this doesn't turn into another weird side thread.
000218Jun 8, 202317:00 UTC-04:00ok yeah. Jun 8 makes four cross-reviews with Nadia since Apr 28. first one on May 10 was icy, third on May 31 felt basically back to normal, and now it's "boring, that's the goal." honestly progress. Cyrus clocked it too.
ok yeah. Jun 8 makes four cross-reviews with Nadia since Apr 28. first one on May 10 was icy, third on May 31 felt basically back to normal, and now it's "boring, that's the goal." honestly progress. Cyrus clocked it too.
000219Jun 9, 202308:30 UTC-04:00back to the cortado apparently. barista just made it without asking and slid it over like we'd all agreed on this already. little clatter of cups, AC too cold in here, that's kinda it.
back to the cortado apparently. barista just made it without asking and slid it over like we'd all agreed on this already. little clatter of cups, AC too cold in here, that's kinda it.
000220Jun 9, 202308:30 UTC-04:00first cortado before work in a while. tasted fine. no weirdness, no internal committee meeting about it, just coffee. kind of nice realizing apparently i'm allowed to enjoy things again.
first cortado before work in a while. tasted fine. no weirdness, no internal committee meeting about it, just coffee. kind of nice realizing apparently i'm allowed to enjoy things again.
000221Jun 9, 202318:00 UTC-04:00day's done. think i'm actually gonna sleep like a normal person tonight. that cortado this morning helped somehow -- been weeks since coffee just felt... normal. Devika's got leftovers out, kibo's parked by the fridge like he's involved.
day's done. think i'm actually gonna sleep like a normal person tonight. that cortado this morning helped somehow -- been weeks since coffee just felt... normal. Devika's got leftovers out, kibo's parked by the fridge like he's involved.
000222Jun 10, 202311:00 UTC-04:00Jun 10 soccer this morning. Diego looked at me at one point and went "you look less wrecked" lol. I laughed bc fair. weird little check-in marker honestly -- Diego doesn't normally comment on affect.
Jun 10 soccer this morning. Diego looked at me at one point and went "you look less wrecked" lol. I laughed bc fair. weird little check-in marker honestly -- Diego doesn't normally comment on affect.
000223Jun 10, 202313:00 UTC-04:00post-soccer Diego did the dad check lol. just hits me with "you eating?" and I go "a lot, actually". Diego: "good, we missed you." that's basically his version of a hug, I think.
post-soccer Diego did the dad check lol. just hits me with "you eating?" and I go "a lot, actually". Diego: "good, we missed you." that's basically his version of a hug, I think.
000224Jun 11, 202309:30 UTC-04:00back table again. cortado's good. 9-letter for 'understated' was LOW-KEY which, ok, fair. took me 20 min so either the puzzle was weird today or i'm just slow. little kid at the next table keeps stirring an empty cup like it's very serious business.
back table again. cortado's good. 9-letter for 'understated' was LOW-KEY which, ok, fair. took me 20 min so either the puzzle was weird today or i'm just slow. little kid at the next table keeps stirring an empty cup like it's very serious business.
000225Jun 12, 202308:30 UTC-04:00coffee first, then shard-keeper. aiming to get it to 50% today if it stays this boring. Wes is the backup pair on it so i'm not doing the weird half-looking-over-my-shoulder thing all day.
coffee first, then shard-keeper. aiming to get it to 50% today if it stays this boring. Wes is the backup pair on it so i'm not doing the weird half-looking-over-my-shoulder thing all day.
000226Jun 12, 202315:30 UTC-04:00Jun 12 2023 14:20 ET had a sev3 canary alert on metrics-router in the middle of the shard-keeper cutover. Wes was on backup, took the first pass, got the diagnosis, then pulled me in inside the 45-min rule. resolver ended up Wes. I dropped a quick "nice job" in the rotation channel. also cutover's past 50 now.
Jun 12 2023 14:20 ET had a sev3 canary alert on metrics-router in the middle of the shard-keeper cutover. Wes was on backup, took the first pass, got the diagnosis, then pulled me in inside the 45-min rule. resolver ended up Wes. I dropped a quick "nice job" in the rotation channel. also cutover's past 50 now.
000227Jun 12, 202319:30 UTC-04:00Jun 12, after Wes's canary assist, I DM'd Cyrus the actual version: the kid's better than I was at 9 months. Cyrus came back with "that's a real bar" lol. channel gets "nice job," Cyrus gets the actual version.
Jun 12, after Wes's canary assist, I DM'd Cyrus the actual version: the kid's better than I was at 9 months. Cyrus came back with "that's a real bar" lol. channel gets "nice job," Cyrus gets the actual version.
000228Jun 13, 202308:30 UTC-04:00coffee, then `shard-keeper` again. it's already past 50%, so today is just keep it moving if it stays this quiet. Wes is backup on it so standing windows can stay put -- not touching those unless the graph does something dumb.
coffee, then `shard-keeper` again. it's already past 50%, so today is just keep it moving if it stays this quiet. Wes is backup on it so standing windows can stay put -- not touching those unless the graph does something dumb.
000229Jun 13, 202322:00 UTC-04:00Jun 13 so we're back on the weird clock. Devika started the third night-shift block of the spring/summer tonight -- 10 days. more ready than the Mar 31 block, at least on paper. kitchen's too bright, Kibo already did one lap to the door and gave up.
Jun 13 so we're back on the weird clock. Devika started the third night-shift block of the spring/summer tonight -- 10 days. more ready than the Mar 31 block, at least on paper. kitchen's too bright, Kibo already did one lap to the door and gave up.
000230Jun 13, 202322:00 UTC-04:00quick hug at the door and she was gone again. scrubs, badge, hair half up, already in hospital mode. third night block this spring... not ideal. he put the laundry on right after, like ok, this is the house now for the next 10 days.
quick hug at the door and she was gone again. scrubs, badge, hair half up, already in hospital mode. third night block this spring... not ideal. he put the laundry on right after, like ok, this is the house now for the next 10 days.
000231Jun 14, 202314:00 UTC-04:00`shard-keeper` has been sitting at 50% and it's boring in the good way. no anomalies, labels stayed clean, nothing odd from rollup-service. plan is 75% on Thursday if it keeps acting like this.
`shard-keeper` has been sitting at 50% and it's boring in the good way. no anomalies, labels stayed clean, nothing odd from rollup-service. plan is 75% on Thursday if it keeps acting like this.
000232Jun 14, 202316:00 UTC-04:00Reviewed Nadia's cleanup PR. Two-line approval on Jun 14, basically done. Kinda wild compared to May 10 being cold and even May 31 still pretty distant. Jun 8 was the one where it finally felt like "boring, that's the goal". No drama now.
Reviewed Nadia's cleanup PR. Two-line approval on Jun 14, basically done. Kinda wild compared to May 10 being cold and even May 31 still pretty distant. Jun 8 was the one where it finally felt like "boring, that's the goal". No drama now.
000233Jun 15, 202308:30 UTC-04:00pre-Lantern call with Iris this morning. she has the v0 UI sketch in Figma, I have the v0 architecture sketch from the whiteboard, and I mostly want us aligned on what 'show your work' actually means before tomorrow's kickoff. if that gets crisp, the rest is probably fine.
pre-Lantern call with Iris this morning. she has the v0 UI sketch in Figma, I have the v0 architecture sketch from the whiteboard, and I mostly want us aligned on what 'show your work' actually means before tomorrow's kickoff. if that gets crisp, the rest is probably fine.
000234Jun 15, 202317:00 UTC-04:00Just before the Jun 15 Lantern call -- Iris Kemper seems good. product eng on the Sphere side, pretty UI-heavy, and she was at Figma before this. She has a v0 sketch already and it's basically lining up with his architecture pass on the 'show your work' thing. feels easy in a useful way.
Just before the Jun 15 Lantern call -- Iris Kemper seems good. product eng on the Sphere side, pretty UI-heavy, and she was at Figma before this. She has a v0 sketch already and it's basically lining up with his architecture pass on the 'show your work' thing. feels easy in a useful way.
000235Jun 16, 202308:30 UTC-04:00Lantern at 2. Iris and I are basically lined up on the v0 shape, so this morning is mostly making sure Theo doesn't book this as an 8-week thing before we've finished shard-keeper. that goes first -- then we can talk about how fat v1 gets.
Lantern at 2. Iris and I are basically lined up on the v0 shape, so this morning is mostly making sure Theo doesn't book this as an 8-week thing before we've finished shard-keeper. that goes first -- then we can talk about how fat v1 gets.
000236Jun 16, 202314:00 UTC-04:00Project Lantern started today, Jun 16. Theo (CTO) is pushing it pretty hard for Q3 and trying not to let it turn into a giant platform rewrite in the first meeting. I had a better read on the observability-side shape than i expected -- pulled from the actual work systems, not another place people have to type status. Iris Kemper is on the UI side and she's sane about v0, which helps. Hema was asking the right annoying questions re permissions / who sees what / whether PMs get raw incident noise. 8-week target to v0, which is tight but not fake-tight. leaning we keep the first cut to deploy movement, ownership changes, incident load, maybe query-volume deltas if the data is clean enough. anything with code movement gets risky fast unless Theo wants to pay the Git tax.
Project Lantern started today, Jun 16. Theo (CTO) is pushing it pretty hard for Q3 and trying not to let it turn into a giant platform rewrite in the first meeting. I had a better read on the observability-side shape than i expected -- pulled from the actual work systems, not another place people have to type status. Iris Kemper is on the UI side and she's sane about v0, which helps. Hema was asking the right annoying questions re permissions / who sees what / whether PMs get raw incident noise. 8-week target to v0, which is tight but not fake-tight. leaning we keep the first cut to deploy movement, ownership changes, incident load, maybe query-volume deltas if the data is clean enough. anything with code movement gets risky fast unless Theo wants to pay the Git tax.
000237Jun 16, 202316:30 UTC-04:00Jun 16 kickoff for Project Lantern is done. Theo (CTO) was in, reports run up through Hema, and they put an 8-week target on v0. I'm co-leading the infra/architecture side with Iris on UI.
Jun 16 kickoff for Project Lantern is done. Theo (CTO) was in, reports run up through Hema, and they put an 8-week target on v0. I'm co-leading the infra/architecture side with Iris on UI.
000238Jun 17, 202314:00 UTC-04:00apartment's doing the saturday afternoon thing where every sound feels louder bc she's asleep. tried to tweak the third-block setup and it's taking stupidly longer than the first two. I went out, came back with that post-walk/read quiet, kibo's mostly just staring at the bedroom door.
apartment's doing the saturday afternoon thing where every sound feels louder bc she's asleep. tried to tweak the third-block setup and it's taking stupidly longer than the first two. I went out, came back with that post-walk/read quiet, kibo's mostly just staring at the bedroom door.
000239Jun 17, 202319:00 UTC-04:00day's done. she's asleep again so whole apartment is on mute. made eggs, read for a bit, kibo did one slow lap and settled by the bedroom door. third block, saturday version.
day's done. she's asleep again so whole apartment is on mute. made eggs, read for a bit, kibo did one slow lap and settled by the bedroom door. third block, saturday version.
000240Jun 18, 202311:00 UTC-04:00Jun 18 Sunday, skipped soccer. Laundry day anyway and Devika needed to sleep through the morning of the third night-block, so whole apartment stayed on mute. I texted Diego and got a simple 'next week' back. Kibo keeps parking himself by the hamper like he's helping.
Jun 18 Sunday, skipped soccer. Laundry day anyway and Devika needed to sleep through the morning of the third night-block, so whole apartment stayed on mute. I texted Diego and got a simple 'next week' back. Kibo keeps parking himself by the hamper like he's helping.