SRE Weekly - SRE Weekly Issue #337

Every one of these 10 items is enough reason to read this article! This makes me want to go investigate some incidents right now.

  Fischer Jemison — Jeli

Slack shares with us in great detail why they use circuit breakers and how they rolled them out.

  Frank Chen — Slack

My favorite part of this one is the section on expectations. We need to socialize this to help reduce the pressure on folks going on call for the first time.

  Prakya Vasudevan — Squadcast

Status pages are marketing material. Prove me wrong.

  Ellen Steinke — Metrist

incidents have unusually high information density compared with day-to-day work, and they enable you to piggy-back on the experience of others

  Lisa Karlin Curtis — incident.io

These folks realized that they had two different use cases for the same data, real-time transactions and batch processing. Rather than try to find one DB that could support both, they fork two copies of the data.

  Xi Chen and Siliang Cao — Grab

It’s all about gathering enough information that you can ask new questions when something goes wrong, rather than being stuck with only answers to the questions you thought to ask in advance.

  Charity Majors

They needed the speed of local ephemeral SSDs but the reliability of network-based persistent disks. The solution: a linux MD option to mirror but prefer to read from the local disks. Neat!

  Glen Oakley — Discord

OS upgrades can be risky. LinkedIn developed a system to unify OS upgrade procedures and make them much less risky.

  Hengyang Hu, Dinesh Dhakal, and Kalyanasundaram Somasundaram — LinkedIn