BetterDev #218 - Scaling Large Production Clusters with Partitioned Synchronization
Better Dev #218 Oct 12, 2021
Happy tuesday everyone. I hope you enjoy this issue, as much as I do.
This issue make this newsletter feel like a database newsletter. It just happens we have so many database news this week and database is always the hardest thing to learn and operate at scale.
A team at Microsoft build a system for Halo 4 game. They talked about all scaling aspect, stress and load test and Vladimir was able to learn a lot from those public available resources. The system that they build is really impressive. They had to rework their legacy backend code, so they can achieve that their system can sustain an initial load of over 11 million clients in a span of a few days period. Besides that, they have to make sure their system remains resilient and available with the load of several 100,000 requests per second.
They took Notion down for five minutes of scheduled maintenance. In that lone 5 minutes, they managed to cut over some Postgres upgrade which shard their database to help deal with the massive scale and leave room for future growth.
This week’s paper review won a best paper award at Usenix ATC, and discusses Alibaba’s approach to scaling their production environment. In particular, the paper focuses on the evolution of the scheduling architecture used in Alibaba datacenters in response to growth in workloads and resourcesAn increase in resources or workloads impacted the load on the existing scheduler architecture. The former translates into more options for the scheduler to choose from when scheduling, and the latter means more computation that needs to be performed by the scheduler.. Beyond discussing Alibaba’s specific challenges and solutions, the paper also touches on the landscape of existing scheduler architectures
On the surface, load shedding is simple. When a server approaches overload, it should start rejecting excess requests so that it can focus on the requests it decides to let in. The goal of load shedding is to keep latency low for the requests that the server decides to accept so that the service replies before the client times out. With this approach, the server maintains high availability for the requests it accepts, and only the excess traffic’s availability is affected.
To our knowledge, this is the first time a major cloud provider has announced GA for a new Postgres major version on their platform one day after the official release. We’ll then describe the work involved in making Postgres extensions compatible with new major Postgres versions, including our distributed database Citus as well as other extensions such as HyperLogLog (HLL), pg_cron, and TopN. Finally, you’ll learn how packaging, testing, and deployments work on Hyperscale (Citus). This last part ties everything together and enables us to release new versions on Azure, with speed.
supply chain attacks are more popular nowadays. What if NPM, Docker, Chef or any upstream sdk/provider are hacked and inject malicious code? Even a project as big as PHP had their Git server compromised. What can we learn and how can we prevent these.
Modern JavaScript has many low-level building block which looks like not useful at all. In this article we look into WeakMap and see why they are useful for our daily life of a front-end developer.
Debugging in CSS means figuring out what might be the problem when you have unexpected layout results. We’ll look at a few categories bugs often fit into, see how we can evaluate the situation, and explore techniques that help prevent these bugs.
Code to read
It’s a massive project to implement a terminal in browser. I don’t think I can ever read all of these but a few thing are useful such as see how they handle ascii color code.
TypeScriptmultitouch gesture recognizer. This gem makes your linux able to recognize swipes or pinchs and assign commands to them. define your touchpad actions in yaml file and trigger command. Example, 3 left swipe, 4 up swipe etc.
RubyVideo
See how we can reverse engineer the solarwinds malware, tooling, methodology. 1 hour long but very much worth yoru time.
Tools
Disposable webmail server (similar to Mailinator) with built in SMTP, POP3, RESTful servers; no DB required.
You can view this issue in web browser.
If you have any suggestion/feedback, do tell me by replying to this email. I read them all.
No longer want to receive these emails? Unsubscribe
Older messages
BetterDev #217 - Self-parking car in 500 lines of code
Tuesday, October 5, 2021
Better Dev #217 Oct 05, 2021 We skipped one issue last week due to my personal workload on my side project, email forwarding service. I also get into a few blockchain projects recently and it was a lot
BetterDev #216 - Why Authorization is Hard and The pitfalls of using ssh-agent, or how to use an agent safely
Monday, September 20, 2021
Better Dev #216 Sep 20, 2021 A very practical issue. Dealing with authorization, SSH agent, design API, optimize big JS bundle, text vs varchar in database design. I hope you like these as much as I do
BetterDev #215 - Can Podcasts Predict the Stock Market?
Monday, September 13, 2021
Better Dev #215 Sep 13, 2021 Hi everyone, full of security related articles this week. I want to shift gear a bit to give everyone gain more knowledge and exposure to cyber security. Can Podcasts
BetterDev #214 - Picturing Git: Conceptions and Misconception
Tuesday, September 7, 2021
Better Dev #214 Sep 07, 2021 This issue is arrived one day later than our usual schedule due to US holiday. We're back now and hope everyone had a great week despite of the holiday or not Picturing
BetterDev #213 - An amazing error message if you put more than 2^24 items in a JS Map object
Monday, August 30, 2021
Better Dev #213 Aug 30, 2021 An amazing error message if you put more than 2^24 items in a JS Map object Can you guess that? a map with 2^24 items? Probaly some limit exceed error? Indeed, JS will
You Might Also Like
The 165+ best Black Friday deals
Wednesday, November 27, 2024
Windows Super God Mode; Bluesky starter packs; Tech gifts under $100 -- ZDNET ZDNET Tech Today - US November 27, 2024 Black Friday 2024 live blog Best Black Friday deals 2024: 165+ sales live now
⚙️ Neuralink's new trial
Wednesday, November 27, 2024
Plus: Zoom is becoming an AI-first company
Matrix Botnet Exploits IoT Devices in Widespread DDoS Botnet Campaign
Wednesday, November 27, 2024
THN Daily Updates Newsletter cover The AI Value Playbook ($35.99) FREE for a Limited Time Business leaders are challenged by the speed of AI innovation and how to navigate disruption and uncertainty.
The Sequence Chat: Why are Foundation Models so Hard to Explain and What are we Doing About it?
Wednesday, November 27, 2024
Addressing some of the interpretability challenges of foundation models and the emerging fields of mechanistic interpretability and behavioral probing. ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏
Overcoming Perfectionism: How to Break Free from the Enemy of Progress
Wednesday, November 27, 2024
Discover how perfectionism hinders progress and learn practical strategies to overcome the fear of imperfection, boost productivity, and achieve your goals without getting stuck in the pursuit of
🖤 Laravel Black Friday Deals!
Wednesday, November 27, 2024
The biggest deals of the year Laravel Black Friday Deals View in browser Laravel News Editor Note: We are sending this outside the regular Sunday newsletter schedule because some of these specials end
BetterDev #271 - Memory: The Forgotten History and Why did Windows 95 setup use three operating systems?
Wednesday, November 27, 2024
Better Dev #271 Nov 26, 2024 Hi all, Welcome to thanksgiving issue of BetterDev. Hope everyone had a safe and warm thanksgiving. It's getting so cold these days. If you are in warzone such as
Mapped | Unemployment Rate By U.S. State in 2024 💼
Tuesday, November 26, 2024
As of October 2024, DC and Nevada tied for the highest unemployment rate in the US at 5.7%. Which states saw the lowest rates? View Online | Subscribe | Download Our App FINAL CHANCE - ENDS TONIGHT!
🔊 7 DIY Tips for Soundproofing a Room — Why I Switched to Xfce for Linux Mint
Tuesday, November 26, 2024
Also: Home Theater Sound Terms Explained, and More! How-To Geek Logo November 26, 2024 Did You Know The shiny layer of a CD doesn't contain the data; the plastic polycarbonate layer does. The shiny
JSK Daily for Nov 26, 2024
Tuesday, November 26, 2024
JSK Daily for Nov 26, 2024 View this email in your browser A community curated daily e-mail of JavaScript news JavaScript Certification Black Friday Offer – Up to 54% Off! Certificates.dev, the trusted