SRE Weekly - SRE Weekly Issue #301
Articles
This one perhaps belongs in a security newsletter, but the failure mode is just so fascinating. A CDN bug led to the loss of millions of dollars worth of Bitcoin.
Badger
Google posted a report for the Google Calendar outage last week.
Jeli, authors of the Howie post-incident guide has their own “howie”. It’s a great example of a thorough incident report.
Vanessa Huerta Granda — Jeli
Hopefully not too late, here are some tips as we head into the thick of it.
JJ Tang — Rootly
This article is published by my sponsor, Rootly, but their sponsorship did not influence its inclusion in this issue.
Using their own incident retrospective template, Blameless shows us how to write an incident retrospective.
Emily Arnott — Blameless
Meta has their own in-house-built tool for tracking and reporting on SLIs.
A Posten, Dávid Bartók, Filip Klepo, and Vatika Harlalka — Meta
These folks put everyone on-call by default, and also pay them extra automatically for each shift and even covering for coworkers.
Chris Evans — incident.io
Code that was deployed under a feature flag inadvertently affected all traffic, even with the flag disabled.
Steve Lewis — Honeycomb
By creating SLOs for microservices at various levels of the request tree, they ended up with a morass of arbitrary targets that didn’t relate clearly to the user experience.
Ben Sigelman — Lightstep
Outages
- AWS us-west-1 and us-west-2
-
Hot on the heels of last week’s us-east-1 outage, AWS had a shorter outage in us-west-1 and us-west-2.
-
- PagerDuty
-
PagerDuty alert notifications were affected by the AWS us-west-2 outage, and the impact lasted about twice as long as AWS’s.
-
- Slack
- Cloudflare
- Solana
|
Older messages
SRE Weekly Issue #300
Monday, December 13, 2021
View on sreweekly.com 300 issues. 6 years. Wow! I couldn't have done it without all of you wonderful people, writing articles and reading issues. Thanks, you make curating this newsletter fun! A
SRE Weekly Issue #299
Monday, December 6, 2021
View on sreweekly.com A message from our sponsor, Rootly: Manage incidents directly from Slack with Rootly 🚒. Automate manual admin tasks like creating incident channel, Jira and Zoom, paging the right
SRE Weekly Issue #298
Monday, November 29, 2021
View on sreweekly.com Email subscribers, my apologies for the double-send last week. I upgraded WordPress and subsequently further cemented my distrust of all version upgrades ever. I carefully tested
SRE Weekly Issue #297
Monday, November 22, 2021
View on sreweekly.com A message from our sponsor, Rootly: Manage incidents directly from Slack with Rootly 🚒. Automate manual admin tasks like creating incident channel, Jira and Zoom, paging the right
SRE Weekly Issue #297
Monday, November 22, 2021
A message from our sponsor, Rootly: Manage incidents directly from Slack with Rootly 🚒. Automate manual admin tasks like creating incident channel, Jira and Zoom, paging the right team, postmortem
You Might Also Like
Ranked | The World's Top Media Franchises by All-Time Revenue 📊
Saturday, May 4, 2024
From Pokémon to Star Wars, some media franchises are globally recognizable. How do media franchises compare in terms of all-time revenue? View Online | Subscribe Presented by Voronoi: The App Where
Noonification: Read Code Like a Hacker With the SAST
Saturday, May 4, 2024
Top Tech Content sent at Noon! Get Algolia: AI Search that understands How are you, @newsletterest1? 🪐 What's happening in tech today, May 4, 2024? The HackerNoon Newsletter brings the HackerNoon
Weekend Reading — May the fourth
Saturday, May 4, 2024
This week we setup our new Minecraft server, play Spacewar, avoid burnout, wonder about Facebook AI spam, lose our passkeys, and claim stairs on the way back home. 😎 Labnotes (by Assaf Arkin) Weekend
Google lays off workers
Saturday, May 4, 2024
Plus: Tesla cans its Supercharger team and UnitedHealthcare reveals security lapses View this email online in your browser By Kyle Wiggers Saturday, May 4, 2024 Image Credits: Tomohiro Ohsumi / Getty
When It Rains, It Pours ☔
Saturday, May 4, 2024
Why the umbrella's design can't be beat. Here's a version for your browser. Hunting for the end of the long tail • May 04, 2024 Hey there, Ernie here with a refreshed piece about umbrellas
🐍 New Python tutorials on Real Python
Saturday, May 4, 2024
Hey there, There's always something going on over at realpython.com as far as Python tutorials go. Here's what you may have missed this past week: Python's unittest: Writing Unit Tests for
Microsoft Outlook Flaw Exploited by Russia's APT28 to Hack Czech, German Entities
Saturday, May 4, 2024
THN Daily Updates Newsletter cover Webinar -- Data Security is Different at the Petabyte Scale Discover the secrets to securing fast-moving, massive data sets with insights from industry titans
Stop spam texts today!
Saturday, May 4, 2024
Have you been receiving a ton of spam texts lately? In April alone, spammers sent out over 19.2 billion texts. Ever wonder where they get your info to begin with? Data brokers. They sell your address,
DeveloPassion's Newsletter #165 - She Said Yes ❤️
Saturday, May 4, 2024
Edition 165 of my newsletter, discussing Knowledge Management, Knowledge Work, Zen Productivity, Personal Organization, and more! Sébastien Dubois DeveloPassion's Newsletter DeveloPassion's
📧 Implementing API Gateway Authentication With YARP
Saturday, May 4, 2024
Implementing API Gateway Authentication With YARP Read on: my website / Read time: 5 minutes BROUGHT TO YOU BY Supercharging Development With AI and APIs Announcing Postman v11: Streamline API