BetterDev #273 - Operating System in 1,000 Lines
Better Dev #273 Jan 12, 2025
Hi all,
Happy new year. Welcome to the first issue of 2025. I’m trying to become more regular this year. Looking forward to a new year and hope everyone continue to build cool thing. For me, I was unable to launch my another SaaS last year. I did overhault my cloudcost https://ec2.shop and much more happy with its state now.
This week our sponsor is WarpStream, they have build an amazing Kafka alternative while 100% compatible with Kafka and way more cheaper. If you are running Kafka in Kubernetes, or looking to replace Redis with Kafka, like Honeybadger team did, then definetely give WarpStream a try.
If you enjoy BetterDev, please spread the word by sharing it with your friends. And if you’d like to support my work, buying me a coffee would be much appreciated.
The video game company Pixel Federation uses Kafka as the message bus to power its real-time data streaming infrastructure. However, they were leveraging Amazon MSK and their bill was growing faster than their data volumes and they had to use complex networking like VPC peering. By switching to WarpStream, they saved 83% and ditched VPC peering and manual partition rebalancing
build a small operating system from scratch, step by step. You might get intimidated when you hear OS or kernel development, the basic functions of an OS (especially the kernel) are surprisingly simple. Even Linux, which is often cited as a huge open-source software, was only 8,413 lines in version 0.01. Today’s Linux kernel is overwhelmingly large, but it started with a tiny codebase, just like your hobby project. We’ll implement basic context switching, paging, user mode, a command-line shell, a disk device driver, and file read/write operations in C. Sounds like a lot, however, it’s only 1,000 lines of code!
So without further ado, I present to you … Regex Chess: sequence of 84,688 regular expressions that, when executed in order, will play a (valid; not entirely terrible) move given a chess board as input.
Working in large established codebases is one of the hardest things to learn as a software engineer. You can’t practice it beforehand (no, open source does not give you the same experience). Personal projects can never teach you how to do it, because they’re necessarily small and from-scratch.
People, myself included, like to say that POSIX time, also known as Unix time, is the number of seconds since the Unix epoch, which was 1970-01-01 at 00:00:00. This is not true. Or rather, it isn’t true in the sense most people think. For example, it is presently 2024-12-25 at 18:51:26 UTC. The POSIX time is 1735152686. It has been 1735152713 seconds since the POSIX epoch. The POSIX time number is twenty-seven seconds lower.
How 2 devices under a NAT can connect directly to each other without a server beind in the middle to relay traffic?
Routing table define what what happens when an IP packet is sent from or through a Linux box. But the simplify is which destination a package should send to? how do your computer know that a LAN ip need to send to othe laptop, and the rest go out to internet?
When you need memory for your program, you can malloc. Similarly to malloc, a pool allocator allows the user to allocate memory at run time. The pool allocator, however, is much faster than malloc1, at the cost of having a fixed pool size. It allows the user to allocate and free memory blocks (referred to as chunks, from now on) in O(1) constant time. This implementation also uses very little memory: when creating the pool, a very small Pool structure is allocated, along with the pool itself. Free chunks are used to store information, so the memory impact is minimal.
Stream Live accepts audio/video content from broadcasters and makes that content available to viewers around the world in real time through the Cloudflare network. This post look into how they build broadcast ingestion with RTMPS, SRT, WHIP and Spectrum. The data is converted and store in their Durable Object, then serve to viewer through HLS or DASH protocol.
B-Trees were essentially “better” Binary Search Trees, with some hand-waving done that they had improved performance when used in database applications. But it’s more than just that, today we will look into disk constraint, slotted pages to see how B-Tree node fit naturally into a page - a chunk of data that the harddrive read all in the same time.
Arrays are in every web developer’s toolbox, and there are a dozen ways to iterate over them. Choose wrong, though, and all of that processing time will happen synchronously in one long, blocking task. The thing is, the most natural ways are the wrong ways.
Code to read
a machine learning library in Ruby. Rumale provides machine learning algorithms with interfaces similar to Scikit-Learn in Python.
RubyTrack personal Bluetooth devices via Apple’s “Find My” network using OpenHaystack and Macless-Haystack with tools written in Go/TinyGo. No Apple hardware required!
GoFile Parser optimised for LLM Ingestion with no loss 🧠 Parse PDFs, Docx, PPTx in a format that is ideal for LLMs.
PythonVideo
Tools
a simple terminal emulator that can be used to create a terminal-like interface on your website
a source code-based static analysis tool that automatically identifies the list of missing security patches in the target system. By default, Vanir pulls up-to-date CVEs from Open Source Vulnerabilities (OSV) together with their corresponding signatures so that users can transparently scan missing patches for an up-to-date list of CVEs.
OpenGFW is a flexible, easy-to-use, open source implementation of GFW (Great Firewall of China) on Linux
Ever wished installing or upgrading PostgreSQL extensions didn’t feel like digging through outdated readmes, cryptic configure scripts, or random GitHub forks & patches? The painful truth is that Postgres’s richness of extension often comes at the cost of complicated setups—especially if you’re juggling multiple distros or CPU architectures. Enter Pig, a Go-based package manager built to tame Postgres and its ecosystem of 340+ extensions in one fell swoop.
Self Hosted
an easy way to send form data that is submitted on your website to your email, Telegram, Webhooks, Etc. It is a “Form Backend” which means it allows you to have functional forms that send places without having to write any server side code yourself
Storyteller is a self-hosted platform for creating and reading ebooks with synced narration. It’s made of three components: the API server, the web interface, and the mobile apps. Together, these components allow you to take audiobooks and ebooks that you already own and automatically synchronize them, as well as read or listen to (or both!) the resulting synced books.
an open-source authentication and authorization solution for your applications. Bring your database and have complete control over the user information. You can self-host authorizer instances and connect to any database (Currently supports 11+ databases including Postgres, MySQL, SQLite, SQLServer, YugaByte, MariaDB, PlanetScale, CassandraDB, ScyllaDB, MongoDB, ArangoDB).
You can view this issue in web browser.
If you have any suggestion/feedback, do tell me by replying to this email. I read them all.
No longer want to receive these emails? Unsubscribe
Older messages
BetterDev #272 - Compromising OpenWrt Supply Chain via Truncated SHA-256 Collision and Command Injection
Tuesday, December 10, 2024
Better Dev #272 Dec 09, 2024 Hi all, It's chrismast time. Time for led lighting project. This year I came across this WLED project, a fast and feature-rich implementation of an ESP8266/ESP32
BetterDev #271 - Memory: The Forgotten History and Why did Windows 95 setup use three operating systems?
Wednesday, November 27, 2024
Better Dev #271 Nov 26, 2024 Hi all, Welcome to thanksgiving issue of BetterDev. Hope everyone had a safe and warm thanksgiving. It's getting so cold these days. If you are in warzone such as
BetterDev #270 - Should We Chat, Too? Security Analysis of WeChat’s MMTLS Encryption Protocol
Tuesday, October 22, 2024
Better Dev #270 Oct 21, 2024 Hi all, Welcome to another issue of BetterDev! This week I come across Colmi, a smart ring where you can write your own software to interact with it. It's also have a
BetterDev #269 - LLM from scratch with Pytorch
Sunday, October 20, 2024
Better Dev #269 Oct 14, 2024 Hi all, Welcome to another issue of BetterDev! I've been exploring LLMs more and, while they're not perfect or likely to replace programming jobs, they're great
BetterDev #268 - Compiling to Assembly from Scratch and A Reintroduction to Programming
Tuesday, October 1, 2024
Better Dev #268 Sep 30, 2024 Hi all, Welcome to another issue of BetterDev. This week PostgreSQL 17 is released. It has a lot of amazing feature. Time to update and look over the release note. If you
You Might Also Like
How to avoid spam texts
Tuesday, January 14, 2025
Let me ask you something: How many times have you shared your phone number online this month? Every time you do—whether for a delivery, online shopping, or signing up for a new service—you're
Daily Coding Problem: Problem #1667 [Hard]
Monday, January 13, 2025
Daily Coding Problem Good morning! Here's your coding interview problem for today. This problem was asked by Airbnb. We're given a hashmap associating each courseId key with a list of courseIds
🧠 Are Supercomputers Dead? — This 90s Tech Is Perfect for Smart TVs
Monday, January 13, 2025
Also: How to Make Sense of Linux Ping Stats, and More! How-To Geek Logo January 13, 2025 Did You Know The original name of the iconic SR-71 Blackbird was actually the RS-71 Blackbird, but Lyndon
Consistency means nothing & Bluesky is reportedly valued at $700
Monday, January 13, 2025
Sill Beta Update #3, Miro AI starts storing AI interactions from free users, Mastodon transfers to a new non-profit organization, and a lot more in this week's issue of Creativerly. Creativerly
Ranked | The AI Models With the Lowest Hallucination Rates 🤖
Monday, January 13, 2025
Hallucination rate is the frequency that an LLM generates false or unsupported information in its outputs. Which models have the lowest rates? View Online | Subscribe | Download Our App FEATURED STORY
GCP Newsletter #433
Monday, January 13, 2025
Welcome to issue #433 January 13th, 2025 News Official Blog Vertex AI Introducing Vertex AI RAG Engine: Scale your Vertex AI RAG pipeline with confidence - Vertex AI RAG Engine is a fully managed
Spyglass Dispatch: It's Political & Personal
Monday, January 13, 2025
On Meta's Moderation Changes • Inside DOGE • Zuck Slams Apple (Again) • Apple's Muted 2025 • CES 2025 Recap The Spyglass Dispatch is a newsletter sent on weekdays featuring links and commentary
$200 to invest today... (USA Only)
Monday, January 13, 2025
Join me in investing in blue chip art on Masterworks, and you will receive $200 to invest on the platform. Not kidding. Founder interview coming soon! ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏
The Sequence Knowledge #468: A New Series About RAG
Monday, January 13, 2025
Exploring key concepts of one of the most popular methods in generative AI solutions. ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏