Every socket connection in Unix/Linux systems is represented by a file. Files opened by a process are represented by file descriptors - integer numbers that are used in I/O syscalls like read/write. it’s easy to deploy a bug that leaks connections, causes the system to run out of file descriptors, and prevent new connections. It’s difficult to solve this problem in a truly general sense due to the various code paths that may have connection leaks. However, if we’re looking to solve only for internal service communication, we control internal RPC frameworks and deployments, and we can make it simple to debug these failures, catch them before full production rollout, and possibly eliminate them.

Wikipedia defines Root Cause Analysis (RCA) as “a method of problem-solving used for identifying the root causes of faults or problems.” Essentially, root cause analysis means to dive deeper into an issue to find what caused a non-conformance. What’s important to understand here is that Root Cause Analysis does not mean just looking at superficial causes of a problem. Rather, it means finding the highest-level cause- the thing that started a chain of cause-effect reactions and ultimately led to the issue at hand.

Slack serves 2.3 million QPS at peak. 2M of those queries are reads and 300K are writes. Median query latency is 2 ms, and p99 query latency is 11 ms. This shareds detail how they evaluate Vitess and migrate to it at their scale. They are 99% on track of migrating to Vitess at this point

GitHub’s architecture is deeply rooted in Ruby on Rails,” said Ma, adding that “a monolithic architecture got us pretty far,” including multiple code deploys every day and high scale, “serving over a billion API calls daily.” Now they are migrating to microservices, why? Because they acquires so many companies with different tech stacks from Python, Node, C#, and merge sister team from Microsoft. In other words, it isn’t a scalability issue but more about collaboration across teams

A large scale incident of Kinesis in US-East-1 last week. They hit a limit of OS thread, they lack of alerting on OS thread limit to know the root cause earlier. Check out DataDog incident on Infrastructure connectivity issue impacting multiple systems too

In a 2016 TED interview (14:10) Linus Torvalds speaks about what he considers good taste in coding. As an example, he presents two implementations of item removal in singly linked lists (reproduced below). In order to remove the first item from a list, one of the implementations requires a special case, the other one does not. Linus, obviously, prefers the latter

Suppose you double-click on a file on your computer. You’re doing this so you can open the file and work with it. But does your operating system know what that means? How does it know what to open the file in? Let’s look at some solutions that have been proposed over the years to solving this issue.

A pipe is a first-in-first-out interprocess communication channel. This is frequently observed in UNIX shell constructs that utilize the | operator. In this artcile, we look at the pipes from C API perspective.

An experiment on using Postgres and PostGIS as the main workhorse for data over space and time. This post outlines examples of how to use these new PostGIS trajectory tricks with OpenStreetMap data for path findinng

There are currently ~2.5 billion Android devices — consisting of ~1,300 discrete brands and ~24,000 unique device models. I’m exploring tapping into this latent resource pool to make automated testing affordable & unlock a more comprehensive configuration coverage.