SRE Weekly - SRE Weekly Issue #290
Articles
Despite carefully testing how they would handle this week’s expiration of the root CA that cross-signed Let’s Encrypt’s CA certificate, they had an outage. The reason? Poor behavior in OpenSSL. See the next article for a deeper explanation of what went wrong with OpenSSL.
Oren Eini — RavenDB
This article explains why some versions of OpenSSL are unable to validate certificates issued by Let’s Encrypt now, even though the certificates should be considered valid.
Ryan Sleevi
This says it all:
It turns out that the path to safety isn’t increased complexity.
Matt Asay — TechRepublic
The thrust of this article is that reliability applies to and should matter to the entire company, not just engineering. I really like the term “pitchfork alerting”.
Robert Ross — FireHydrant
Lesson learned: always make your application server’s timeout longer than your reverse proxy’s.
Ivan Velichko
Who deploys the deploy tool? The deploy tool, obviously — unless it’s down.
Lorin Hochstein
Their approach: group tables into “schema domains”, make sure that queries don’t span schema domains, and then move a schema domain to its own separate database cluster.
Thomas Maurer — GitHub
Groot is about helping figure out what’s wrong during an incident, not about analyzing an incident after the fact. I totally get why they need this tool, since they have over 5000 microservices!
Hanzhang Wang — eBay
SRE is a broad, overarching responsibility that needs a multitude of role considerations to pull off properly.
Ash P — Cruform
Outages
- Heroku
- (also this one)Heroku had a major outage that coincided with an Amazon EBS failure in a single availability zone in us-east1. Customers of Heroku such as Dead Man’s Snitch were impacted.
- Slack
- Slack had a big disruption related to DNSSEC. Here’s an interesting analysis of what may have gone wrong (link).
- Let’s Encrypt
- Let’s Encrypt saw heavy traffic as everyone clamored to renew their certificates, causing certificate issuance to slow down.
- Microsoft 365
- Apple’s “Find My” service
- Signal
- Xero
- This one coincided with the same Amazon EBS outage mentioned above. Xero also had another outage on October 1.
|
Older messages
SRE Weekly Issue #289
Monday, September 27, 2021
View on sreweekly.com A message from our sponsor, StackHawk: Semgrep and StackHawk are showing you what's new with automated security testing on September 30. Grab your spot: https://sthwk.com/
SRE Weekly Issue #288
Monday, September 20, 2021
View on sreweekly.com A message from our sponsor, StackHawk: Want to see what's new with automated security tooling? Tune in on September 30 to see how StackHawk and Semgrep are making it possible
SRE Weekly Issue #287
Monday, September 13, 2021
View on sreweekly.com A message from our sponsor, StackHawk: Trying to figure out how to keep your APIs secure? You're not the only one. See how DataRobot is automating API security testing with
SRE Weekly Issue #286
Monday, September 6, 2021
View on sreweekly.com A message from our sponsor, StackHawk: Trying to scale AppSec across engingeering is no joke. Check out the 3 main reasons developers struggle with AppSec and how to make it
SRE Weekly Issue #285
Monday, August 30, 2021
View on sreweekly.com A message from our sponsor, StackHawk: Check out the latest from StackHawk's Chief Security Officer, Scott Gerlach, on why security should be part of building software, and
You Might Also Like
JSter #218 - Libraries and more
Wednesday, May 1, 2024
All JavaScript is good JavaScript. I'm close to done with my SurviveJS rework. The new site will have more content while being much lighter and faster to compile so that's all good. Libraries
BetterDev #258 - Build an 8-bit computer from scratch and Home automation with ESP8266
Wednesday, May 1, 2024
Better Dev #258 Apr 30, 2024 Hi all, We come back with a new issue this week. If you like BetterDev, please help spead word out by refer to your friends. Buy me a coffee would be great too. Build an 8-
Interface Interference 👎
Wednesday, May 1, 2024
Amid the AI device dunking, should everything “just be an app”? Here's a version for your browser. Hunting for the end of the long tail • April 30, 2024 Interface Interference The problem
Some Tesla Supercharger jobs get a jolt
Tuesday, April 30, 2024
Plus: Amazon CodeWhisperer changes its name and Arc gets a Windows version View this email online in your browser By Christine Hall Tuesday, April 30, 2024 Welcome to TechCrunch PM, bringing you the
Relief From Tinnitus: Free Discovery Call!
Tuesday, April 30, 2024
Do you suffer from tinnitus or a ringing in your ears? 1 in 3 adults over the age of 65 will suffer from this condition and often don't know there are things you can do to help. Our friends at
WebAIM April 2024 Newsletter
Tuesday, April 30, 2024
WebAIM April 2024 Newsletter Read this newsletter online at https://webaim.org/newsletter/2024/april Feature Web Accessibility in the 2024 Presidential Campaigns WebAIM's John Northup ran the US
👀 Being More Productive on a Smaller Screen — How to Hide Games on Steam Family Sharing
Tuesday, April 30, 2024
Also: What to Expect From Apple's "Let Loose" Event, and More! How-To Geek Logo April 30, 2024 Did You Know The letter J is the only letter that makes no appearance on the Periodic Table.
PEP 686, Lazy Evaluation, Serverless Python, and More
Tuesday, April 30, 2024
PEP 686: Make UTF-8 Mode Default #627 – APRIL 30, 2024 VIEW IN BROWSER The PyCoder's Weekly Logo PEP 686: Make UTF-8 Mode Default This Python Enhancement Proposal outlines making UTF-8 the default
Daily Coding Problem: Problem #1427 [Easy]
Tuesday, April 30, 2024
Daily Coding Problem Good morning! Here's your coding interview problem for today. This problem was asked by Amazon. Given an array and a number k that's smaller than the length of the array,
🎙 My advice for film + TV creatives on the AI wave
Tuesday, April 30, 2024
Learning AI fast + Karate Kid references