BetterDev #218 - Scaling Large Production Clusters with Partitioned Synchronization
Better Dev #218 Oct 12, 2021
Happy tuesday everyone. I hope you enjoy this issue, as much as I do.
This issue make this newsletter feel like a database newsletter. It just happens we have so many database news this week and database is always the hardest thing to learn and operate at scale.
A team at Microsoft build a system for Halo 4 game. They talked about all scaling aspect, stress and load test and Vladimir was able to learn a lot from those public available resources. The system that they build is really impressive. They had to rework their legacy backend code, so they can achieve that their system can sustain an initial load of over 11 million clients in a span of a few days period. Besides that, they have to make sure their system remains resilient and available with the load of several 100,000 requests per second.
They took Notion down for five minutes of scheduled maintenance. In that lone 5 minutes, they managed to cut over some Postgres upgrade which shard their database to help deal with the massive scale and leave room for future growth.
This week’s paper review won a best paper award at Usenix ATC, and discusses Alibaba’s approach to scaling their production environment. In particular, the paper focuses on the evolution of the scheduling architecture used in Alibaba datacenters in response to growth in workloads and resourcesAn increase in resources or workloads impacted the load on the existing scheduler architecture. The former translates into more options for the scheduler to choose from when scheduling, and the latter means more computation that needs to be performed by the scheduler.. Beyond discussing Alibaba’s specific challenges and solutions, the paper also touches on the landscape of existing scheduler architectures
On the surface, load shedding is simple. When a server approaches overload, it should start rejecting excess requests so that it can focus on the requests it decides to let in. The goal of load shedding is to keep latency low for the requests that the server decides to accept so that the service replies before the client times out. With this approach, the server maintains high availability for the requests it accepts, and only the excess traffic’s availability is affected.
To our knowledge, this is the first time a major cloud provider has announced GA for a new Postgres major version on their platform one day after the official release. We’ll then describe the work involved in making Postgres extensions compatible with new major Postgres versions, including our distributed database Citus as well as other extensions such as HyperLogLog (HLL), pg_cron, and TopN. Finally, you’ll learn how packaging, testing, and deployments work on Hyperscale (Citus). This last part ties everything together and enables us to release new versions on Azure, with speed.
supply chain attacks are more popular nowadays. What if NPM, Docker, Chef or any upstream sdk/provider are hacked and inject malicious code? Even a project as big as PHP had their Git server compromised. What can we learn and how can we prevent these.
Modern JavaScript has many low-level building block which looks like not useful at all. In this article we look into WeakMap and see why they are useful for our daily life of a front-end developer.
Debugging in CSS means figuring out what might be the problem when you have unexpected layout results. We’ll look at a few categories bugs often fit into, see how we can evaluate the situation, and explore techniques that help prevent these bugs.
Code to read
It’s a massive project to implement a terminal in browser. I don’t think I can ever read all of these but a few thing are useful such as see how they handle ascii color code.
TypeScriptmultitouch gesture recognizer. This gem makes your linux able to recognize swipes or pinchs and assign commands to them. define your touchpad actions in yaml file and trigger command. Example, 3 left swipe, 4 up swipe etc.
RubyVideo
See how we can reverse engineer the solarwinds malware, tooling, methodology. 1 hour long but very much worth yoru time.
Tools
Disposable webmail server (similar to Mailinator) with built in SMTP, POP3, RESTful servers; no DB required.
You can view this issue in web browser.
If you have any suggestion/feedback, do tell me by replying to this email. I read them all.
No longer want to receive these emails? Unsubscribe
Older messages
BetterDev #217 - Self-parking car in 500 lines of code
Tuesday, October 5, 2021
Better Dev #217 Oct 05, 2021 We skipped one issue last week due to my personal workload on my side project, email forwarding service. I also get into a few blockchain projects recently and it was a lot
BetterDev #216 - Why Authorization is Hard and The pitfalls of using ssh-agent, or how to use an agent safely
Monday, September 20, 2021
Better Dev #216 Sep 20, 2021 A very practical issue. Dealing with authorization, SSH agent, design API, optimize big JS bundle, text vs varchar in database design. I hope you like these as much as I do
BetterDev #215 - Can Podcasts Predict the Stock Market?
Monday, September 13, 2021
Better Dev #215 Sep 13, 2021 Hi everyone, full of security related articles this week. I want to shift gear a bit to give everyone gain more knowledge and exposure to cyber security. Can Podcasts
BetterDev #214 - Picturing Git: Conceptions and Misconception
Tuesday, September 7, 2021
Better Dev #214 Sep 07, 2021 This issue is arrived one day later than our usual schedule due to US holiday. We're back now and hope everyone had a great week despite of the holiday or not Picturing
BetterDev #213 - An amazing error message if you put more than 2^24 items in a JS Map object
Monday, August 30, 2021
Better Dev #213 Aug 30, 2021 An amazing error message if you put more than 2^24 items in a JS Map object Can you guess that? a map with 2^24 items? Probaly some limit exceed error? Indeed, JS will
You Might Also Like
Import AI 399: 1,000 samples to make a reasoning model; DeepSeek proliferation; Apple's self-driving car simulator
Friday, February 14, 2025
What came before the golem? ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏
Defining Your Paranoia Level: Navigating Change Without the Overkill
Friday, February 14, 2025
We've all been there: trying to learn something new, only to find our old habits holding us back. We discussed today how our gut feelings about solving problems can sometimes be our own worst enemy
5 ways AI can help with taxes 🪄
Friday, February 14, 2025
Remotely control an iPhone; 💸 50+ early Presidents' Day deals -- ZDNET ZDNET Tech Today - US February 10, 2025 5 ways AI can help you with your taxes (and what not to use it for) 5 ways AI can help
Recurring Automations + Secret Updates
Friday, February 14, 2025
Smarter automations, better templates, and hidden updates to explore 👀 ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏
The First Provable AI-Proof Game: Introducing Butterfly Wings 4
Friday, February 14, 2025
Top Tech Content sent at Noon! Boost Your Article on HackerNoon for $159.99! Read this email in your browser How are you, @newsletterest1? undefined The Market Today #01 Instagram (Meta) 714.52 -0.32%
GCP Newsletter #437
Friday, February 14, 2025
Welcome to issue #437 February 10th, 2025 News BigQuery Cloud Marketplace Official Blog Partners BigQuery datasets now available on Google Cloud Marketplace - Google Cloud Marketplace now offers
Charted | The 1%'s Share of U.S. Wealth Over Time (1989-2024) 💰
Friday, February 14, 2025
Discover how the share of US wealth held by the top 1% has evolved from 1989 to 2024 in this infographic. View Online | Subscribe | Download Our App Download our app to see thousands of new charts from
The Great Social Media Diaspora & Tapestry is here
Friday, February 14, 2025
Apple introduces new app called 'Apple Invites', The Iconfactory launches Tapestry, beyond the traditional portfolio, and more in this week's issue of Creativerly. Creativerly The Great
Daily Coding Problem: Problem #1689 [Medium]
Friday, February 14, 2025
Daily Coding Problem Good morning! Here's your coding interview problem for today. This problem was asked by Google. Given a linked list, sort it in O(n log n) time and constant space. For example,
📧 Stop Conflating CQRS and MediatR
Friday, February 14, 2025
Stop Conflating CQRS and MediatR Read on: my website / Read time: 4 minutes The .NET Weekly is brought to you by: Step right up to the Generative AI Use Cases Repository! See how MongoDB powers your