BetterDev #245 - Scaling Kubernetes to 7,500 nodes
Better Dev #245 Mar 20, 2023
At this point no one doesn’t know OpenAI and their ChatGPT. They run a massive K8S cluster with 7,500 nodes. Needless to say that come with many challenges they share with us here.
Writing or talking about things databases don’t do may sound a bit silly. There are obviously many things databases don’t do. Example, automatic backup, version control of row…
A most normal way for client <-> server communication is to open a port, and listen to traffic to that port. To secure sometine we use localhost, but there is also other way call Unix socket. The most famous one use it is docker with its /var/run/docker.sock
which sometime we can mount into a container and interact with docker on host. In this post, we explore this socket concept.
Drawing Diagram by hand can be very tedious. To connect relationship we have to constantly move thing around to make room and space. D2 is a modern text-to-diagram language. It is built with extensibility in mind. This post show an example of how we can use SQL to generate the diagram of table relationship.
Logical replication uses a publish and subscribe model with one or more subscribers subscribing to one or more publications on a publisher node. Subscribers pull data from the publications they subscribe to and may subsequently re-publish data to allow cascading replication or more complex configurations.
Did you know there’s a function in Postgres that lets you write data which you can’t query? A function that lets you persist data in all kinds and shapes but which will never show up in any table? Let me tell you about pg_logical_emit_message()! It’s a Postgres function that allows you to write messages to the write-ahead log (WAL) of the database.
There’s a variety of ways to back up your data and this post aims to explain the basic tools involved in backups and what options you have, from just getting started to more sophisticated production systems.
Can you imagine being annoy by a very very slow untar and think you can beat it by writing your own untar? The OP did just that, lead us through the wolrd if tar format, why it is slow and a few tip we can do to speed up.
Get to know Nadinastiti, our Technical Program Manager, and Estu Fardani, our Cloud Platform Engineer at GovTech Edu, as they shed light on the best practices for incident management in GovTech Edu’s engineering teams
A collection of awesome CRDT resources. Useful when you want to learn about collaborative editing
Code to read
an open-source alternative to the OpenAI text completion API. It provides a compatible streaming API for your Hugging Face Transformers-based text generation models.
PythonA collection of script depicting the strange quirks of programming languages.
Gopythonjavascriptctypescriptphpa command-line program that finds secrets and sensitive information in textual data and Git history.
RustHave you ever see code like \u001b[1;31;40mHello World\033[0m
in your log because color code get send to text log? How can we parse them?
~6x faster, stricter, configurable, extensible, and beautiful drop-in replacement for golint
Go🎹🎸A music theory library with a command-line interface. Super cool as a guitar beginner to look up chord I must say
RubyTools
an open source software supply chain control plane, a single source of truth for artifacts plus a declarative attestation process. SecOps teams can declaratively state the attestation and artifacts expectations for their organization’s CI/CD workflows, while also resting assured that latest standards and best practices are put in place.
Use SQL to instantly query your cloud services (AWS, Azure, GCP and more). Open source CLI. No DB required
Ready, Beautiful UI Components for building analytical interfaces. Feed data and they show nice chart.
Have you ever want to dig through a huge JSON object? filter out a certain array. Nested it etc. jq
is great if you know the shape of object ahead of time. But if you’re looking for a needle in a huge JSON, a visual tool might be better
Use system font avoid many gotcha such as layout shift, flashing. istant render because the font is pre-installed. But which font stack offer the best on all the OS, how does they looks like. Use this site to choose
a lightweight SQL caching engine written in Rust that helps developers enhance the performance and scalability of existing applications.
Schedule job in Postgrs by using Postgres table itself. Using a cron you always have to deal with lock to ensure there are no 2 instances run at same time. Putting this in the db itself make operating simpler. You can schedule SQL query, system command, or built-in function. Refresh materialize view is a perfect match.
PostgreSQL pooler with sharding, load balancing and failover support. There is a post by instacart on adopting it
An open-source, cloud-native, distributed time-series database with PromQL/SQL/Python supported.
You can view this issue in web browser.
If you have any suggestion/feedback, do tell me by replying to this email. I read them all.
No longer want to receive these emails? Unsubscribe
Older messages
BetterDev #244 - How discord stores trillions of messages and How Cloudflare run Prometheus at scale
Monday, March 13, 2023
Better Dev #244 Mar 13, 2023 How discord stores trillions of messages Rewrite in Rust, migrate to ScylaDB from cassandra. But the most important thing to me is the request coalescing. If two user read
BetterDev #243 - Scaling Etsy Payments with Vitess and Husky DataDog Exactly Once Ingestion and Multi Tenancy at Scale
Monday, February 27, 2023
Better Dev #243 Feb 26, 2023 Scaling Etsy Payments with Vitess: Part 1 – The Data Model Etsy Payment system reached the point where vertical scale won't work anymore. They used biggest instance on
BetterDev #242 - Neural Networks: Zero to Hero and Bit Hacking with Go code
Monday, February 13, 2023
Better Dev #242 Feb 13, 2023 Neural Networks: Zero to Hero A course by Andrej Karpathy on building neural networks, from scratch, in code. We start with the basics of backpropagation and build up to
BetterDev #241 - How a single line of code brought down a half-billion euro rocket launch
Monday, February 6, 2023
Better Dev #241 Feb 06, 2023 How a single line of code brought down a half-billion euro rocket launch How a rocket launch fail? Re-use of code written for 32-bit system on a 64-bit data range. Novo
BetterDev #240 - Hello, PNG
Tuesday, January 24, 2023
Better Dev #240 Jan 23, 2023 Start speaking a new language in just three weeks with Babbel (SPONSOR) Learning to speak a new language goes beyond just vocabulary: it's about being able to hold a
You Might Also Like
The Sequence Chat: Why are Foundation Models so Hard to Explain and What are we Doing About it?
Wednesday, November 27, 2024
Addressing some of the interpretability challenges of foundation models and the emerging fields of mechanistic interpretability and behavioral probing. ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏
Overcoming Perfectionism: How to Break Free from the Enemy of Progress
Wednesday, November 27, 2024
Discover how perfectionism hinders progress and learn practical strategies to overcome the fear of imperfection, boost productivity, and achieve your goals without getting stuck in the pursuit of
🖤 Laravel Black Friday Deals!
Wednesday, November 27, 2024
The biggest deals of the year Laravel Black Friday Deals View in browser Laravel News Editor Note: We are sending this outside the regular Sunday newsletter schedule because some of these specials end
BetterDev #271 - Memory: The Forgotten History and Why did Windows 95 setup use three operating systems?
Wednesday, November 27, 2024
Better Dev #271 Nov 26, 2024 Hi all, Welcome to thanksgiving issue of BetterDev. Hope everyone had a safe and warm thanksgiving. It's getting so cold these days. If you are in warzone such as
Mapped | Unemployment Rate By U.S. State in 2024 💼
Tuesday, November 26, 2024
As of October 2024, DC and Nevada tied for the highest unemployment rate in the US at 5.7%. Which states saw the lowest rates? View Online | Subscribe | Download Our App FINAL CHANCE - ENDS TONIGHT!
🔊 7 DIY Tips for Soundproofing a Room — Why I Switched to Xfce for Linux Mint
Tuesday, November 26, 2024
Also: Home Theater Sound Terms Explained, and More! How-To Geek Logo November 26, 2024 Did You Know The shiny layer of a CD doesn't contain the data; the plastic polycarbonate layer does. The shiny
JSK Daily for Nov 26, 2024
Tuesday, November 26, 2024
JSK Daily for Nov 26, 2024 View this email in your browser A community curated daily e-mail of JavaScript news JavaScript Certification Black Friday Offer – Up to 54% Off! Certificates.dev, the trusted
NumPy, Loop Targets, Vector Animation, and More
Tuesday, November 26, 2024
NumPy Practical Examples: Useful Techniques #657 – NOVEMBER 26, 2024 VIEW IN BROWSER The PyCoder's Weekly Logo NumPy Practical Examples: Useful Techniques In this tutorial, you'll learn how to
Your Games Quarterly newsletter has arrived
Tuesday, November 26, 2024
What's new for games in Google Play and Android Email not displaying correctly? View it online November 2024 The First Developer Preview of Android 16 The First Developer Preview of Android 16
Daily Coding Problem: Problem #1620 [Hard]
Tuesday, November 26, 2024
Daily Coding Problem Good morning! Here's your coding interview problem for today. This problem was asked by Dropbox. Sudoku is a puzzle where you're given a partially-filled 9 by 9 grid with