Better Dev #245 Mar 20, 2023

Scaling Kubernetes to 7,500 nodes

At this point no one doesn’t know OpenAI and their ChatGPT. They run a massive K8S cluster with 7,500 nodes. Needless to say that come with many challenges they share with us here.

Things DBs Don't Do - But Should

Writing or talking about things databases don’t do may sound a bit silly. There are obviously many things databases don’t do. Example, automatic backup, version control of row…

Tinkering with Unix domain sockets

A most normal way for client <-> server communication is to open a port, and listen to traffic to that port. To secure sometine we use localhost, but there is also other way call Unix socket. The most famous one use it is docker with its /var/run/docker.sock which sometime we can mount into a container and interact with docker on host. In this post, we explore this socket concept.

Generate diagrams programmatically

Drawing Diagram by hand can be very tedious. To connect relationship we have to constantly move thing around to make room and space. D2 is a modern text-to-diagram language. It is built with extensibility in mind. This post show an example of how we can use SQL to generate the diagram of table relationship.

Inside logical replication in PostgreSQL: How it work

Logical replication uses a publish and subscribe model with one or more subscribers subscribing to one or more publications on a publisher node. Subscribers pull data from the publications they subscribe to and may subsequently re-publish data to allow cascading replication or more complex configurations.

The Wonders of Postgres Logical Decoding Messages

Did you know there’s a function in Postgres that lets you write data which you can’t query? A function that lets you persist data in all kinds and shapes but which will never show up in any table? Let me tell you about pg_logical_emit_message()! It’s a Postgres function that allows you to write messages to the write-ahead log (WAL) of the database.

Introduction to Postgres Backups

There’s a variety of ways to back up your data and this post aims to explain the basic tools involved in backups and what options you have, from just getting started to more sophisticated production systems.

The tar archive format, its extensions, and why GNU tar extracts in quadratic time

Can you imagine being annoy by a very very slow untar and think you can beat it by writing your own untar? The OP did just that, lead us through the wolrd if tar format, why it is slow and a few tip we can do to speed up.

Beautiful and mind-bending effects with WebGL Render Targets

How We Keep Our Government Apps Running With High Reliability: A Peek at Our Incident Management Strategy

Get to know Nadinastiti, our Technical Program Manager, and Estu Fardani, our Cloud Platform Engineer at GovTech Edu, as they shed light on the best practices for incident management in GovTech Edu’s engineering teams

awesome-crdt

A collection of awesome CRDT resources. Useful when you want to learn about collaborative editing

Code to read

basaran

an open-source alternative to the OpenAI text completion API. It provides a compatible streaming API for your Hugging Face Transformers-based text generation models.

Python

tragic-method

A collection of script depicting the strange quirks of programming languages.

Gopythonjavascriptctypescriptphp

noseyparker

a command-line program that finds secrets and sensitive information in textual data and Git history.

Rust

Minisearch

Tiny and powerful JavaScript full-text search engine for browser and Node

JavaScript

transformers.js

Run 🤗 Transformers in your browser

JavaScript

go-ansi-parser

Have you ever see code like \u001b[1;31;40mHello World\033[0m in your log because color code get send to text log? How can we parse them?

Go

revive

~6x faster, stricter, configurable, extensible, and beautiful drop-in replacement for golint

Go

coltrane

🎹🎸A music theory library with a command-line interface. Super cool as a guitar beginner to look up chord I must say

Ruby

Tools

chainloop

an open source software supply chain control plane, a single source of truth for artifacts plus a declarative attestation process. SecOps teams can declaratively state the attestation and artifacts expectations for their organization’s CI/CD workflows, while also resting assured that latest standards and best practices are put in place.

libgsqlite

A SQLite extension which loads a Google Sheet as a virtual table.

steampipe

Use SQL to instantly query your cloud services (AWS, Azure, GCP and more). Open source CLI. No DB required

Tremor Components

Ready, Beautiful UI Components for building analytical interfaces. Feed data and they show nice chart.

jsoncrack

Have you ever want to dig through a huge JSON object? filter out a certain array. Nested it etc. jq is great if you know the shape of object ahead of time. But if you’re looking for a needle in a huge JSON, a visual tool might be better

modern font stacks

Use system font avoid many gotcha such as layout shift, flashing. istant render because the font is pre-installed. But which font stack offer the best on all the OS, how does they looks like. Use this site to choose

readyset

a lightweight SQL caching engine written in Rust that helps developers enhance the performance and scalability of existing applications.

pg_timetable

Schedule job in Postgrs by using Postgres table itself. Using a cron you always have to deal with lock to ensure there are no 2 instances run at same time. Putting this in the db itself make operating simpler. You can schedule SQL query, system command, or built-in function. Refresh materialize view is a perfect match.

pgcat

PostgreSQL pooler with sharding, load balancing and failover support. There is a post by instacart on adopting it

greptimedb

An open-source, cloud-native, distributed time-series database with PromQL/SQL/Python supported.

That's it for this round, have a great day! If you like this newsletter, please tell the world or tweet about this!
You can view this issue in web browser.

If you have any suggestion/feedback, do tell me by replying to this email. I read them all.

No longer want to receive these emails? Unsubscribe

want to support my work? buy me a coffee

OpenAI text completion API Postgres Logical Decoding Messages compatible streaming API sophisticated production systems WebGL Render Targets Technical Program Manager Cloud Platform Engineer collaborative editing Code Hugging Face Transformer Unix domain sockets Incident Management Strategy awesome CRDT resources tar archive format

BetterDev #245 - Scaling Kubernetes to 7,500 nodes

Better Dev #245 Mar 20, 2023

Code to read

Tools

Key phrases

Older messages

BetterDev #244 - How discord stores trillions of messages and How Cloudflare run Prometheus at scale

BetterDev #243 - Scaling Etsy Payments with Vitess and Husky DataDog Exactly Once Ingestion and Multi Tenancy at Scale

BetterDev #242 - Neural Networks: Zero to Hero and Bit Hacking with Go code

BetterDev #241 - How a single line of code brought down a half-billion euro rocket launch

BetterDev #240 - Hello, PNG

You Might Also Like

WP Weekly 191 - Essentials - Duplicate in Core, White Label Kadence, Studio for Mac

SRE Weekly Issue #422

Quick question

Kotlin Weekly #404 (NOT FOUND)

📲 Why Is It Called Bluetooth? — Check Out This AI Text to Song Generator

Daily Coding Problem: Problem #1425 [Easy]

PD#571 Software Design Principles I Learned the Hard Way

When Procrastination is Productive & Ghost integrating with ActivityPub

C#503 Building pipelines with System.Threading.Channels

RD#453 Get your codebase ready for React 19