Architecture Weekly #190 - Queuing, Backpressure, Single Writer and other useful patterns for managing concurrency
Welcome to the new week! In the last edition, we explored why connection pooling is crucial for using databases efficiently. Now, let's take a step further and perform a thought experiment: How would you actually implement a connection pool? We'll use Node.js and TypeScript as an illustration, as we need to choose a specific stack to make the concepts concrete. If you think, “Well, I won’t ever do it myself”, don’t worry—the intention here isn’t to drown you in the imaginary code. Instead, I will show you how these micro-scale coding considerations can impact macro-scale design analysis. As we go, we’ll learn a few valuable concepts, like Queuing, Backpressure and single Writer, and discuss why queuing can be useful in managing concurrency and beyond! Connection PoolLet’s start where we ended, so with our database access code: As we discussed in detail in the previous article, a connection pool manages reusable database connections shared among multiple requests. Instead of opening and closing a connection for each request, we borrow a connection from the pool and return it after database operation. Thanks to that, we’re cutting the latency required to establish a connection and get a more resilient and scalable solution (at least if we’re not into serverless). Done? Now, think about the potential limitations of your thinking and the potential solutions to them. Ready? Let’s compare that with my thoughts! Basic definitionLet’s start by defining the basic API. It could look like this: There is not much to explain besides that we want to manage the pool to the database represented by the connection string and limit the number of open connections. Why? Imagine a service like a payment gateway that handles transactions for a popular online retailer. During peak shopping times, such as Black Friday, the system experiences a huge increase in payment requests. Each transaction requires a secure connection to the database to verify payment details and update the order status. Yet, the database has its limits, so we might need to try somehow to squeeze that into the limit of 100 concurrence connections. When all 100 connections are occupied processing transactions, new incoming requests must be managed effectively. We could try to: - Fail Immediately: The system could reject new requests outright, leading to a poor user experience and lost sales. - Retry Aggressively: The requests might enter a loop, repeatedly attempting to acquire a connection, which can further strain the system. The naive implementation of a fail-fast strategy could look as such: Now, While doing retries in the loop or even immediate failure can be a temporary solution, for obvious reasons, it won’t scale and will result in a subpar experience. Consider a busy restaurant with limited seating. When all tables are occupied, you can be asked to wait a bit near the entrance. As tables become available, the waiter will invite you to be seated. This ensures that the restaurant operates smoothly without overcrowding or turning away customers. We could do the same with our connection pool implementation. Instead of rejecting new requests or letting them endlessly retry (which could crash your system), they're placed in a queue and handled sequentially. The next part of the article is for paid users. If you’re not such yet, till the end of August, you can use a free month's trial: https://www.architecture-weekly.com/b3b7d64d. You can check it out and decide if you like it and want to stay. I hope that you will!... Unlock this post for free, courtesy of Oskar Dudycz.A subscription gets you:
|
Older messages
Architecture Weekly #189 - Mastering Database Connection Pooling
Monday, August 5, 2024
Boom! This is the first Architecture Weekly in the new format. It's a deep dive to database connection pooling. Check to learn how to set it up starting from simple solution and learning how the
Architecture Weekly #188 - 29th July 2024
Monday, July 29, 2024
This time in an unusual edition, we focused on the underestimated aspect of Software Design: resting. We discussed the typical mistakes we do and why repeating them will not help you in making right
Architecture Weekly #187 - 8th July 2024
Monday, July 8, 2024
This week, we discussed the challenges of finding the right architecture strategy and pivoting it when needed. We brought and discussed a few insightful articles on how/when/why to move to serverless,
Architecture Weekly #186 - 1st July 2024
Monday, July 1, 2024
This week, we covered the Polyfill JS supply chain attack that potentially touched 4% of the Internet. We also checked the DataDog report on the state of Cloud Spendings. We discussed whether it's
Architecture Weekly #185 - 24th June 2024
Monday, June 24, 2024
This edition is 50 shaded of coupling! Yeah, I know how that sounds, but we discussed why coupling is not an end goal but a metric to evaluate tradeoffs. We also discussed how to actually make
You Might Also Like
New Blogs on ThomasMaurer.ch for 11/05/2024
Tuesday, November 5, 2024
View this email in your browser Thomas Maurer Cloud & Datacenter Update This is the update for blog posts on ThomasMaurer.ch. Honored to Receive the YouTube Silver Creator Award By Thomas Maurer on
📱 I Tried Running Ubuntu on My Phone — Samsung's One UI Is How Android Should Be
Monday, November 4, 2024
Also: The Most Realistic Game Simulations, and More! How-To Geek Logo November 4, 2024 Did You Know Peter Weller, best known for his role as Robocop, is an accomplished academic and actor. He has a
Ranked | America’s Most Popular Drugs by Dollars Spent 💰
Monday, November 4, 2024
Tired of hearing about Ozempic? This chart isn't for you. It's one of America's most popular drugs in 2023. Here are some numbers. View Online | Subscribe | Download Our App Presented by:
Ranked | America’s Most Popular Drugs by Dollars Spent 💰
Monday, November 4, 2024
Tired of hearing about Ozempic? This chart isn't for you. It's one of America's most popular drugs in 2023. Here are some numbers. View Online | Subscribe | Download Our App Presented by:
Spyglass Dispatch 1: AI for Startups • RIP Quincy Jones • Days of Thunder 2 • Microsoft's Copilot Complaints • Apple's Shifting Vision Pro Strategy • A Game of Thrones Film • On 43
Monday, November 4, 2024
AI for Startups • RIP Quincy Jones • Days of Thunder 2 • Microsoft's Copilot Complaints • Apple's Shifting Vision Pro Strategy • A Game of Thrones Film • On 43 The Spyglass Dispatch is a free
Q3 Movers and Shakers
Monday, November 4, 2024
Top Tech Content sent at Noon! NODES 2024, a Dev Conference on AI, Knowledge Graphs & Apps Read this email in your browser How are you, @newsletterest1? 🪐 What's happening in tech today,
Learn more the future of access management with an IDC analyst
Monday, November 4, 2024
Join us on November 13th ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏
GCP Newsletter #423
Monday, November 4, 2024
Welcome to issue #423 November 4th, 2024 News Compute Engine Official Blog C4A VMs now GA: Our first custom Arm-based Axion CPU - Google has announced the general availability of C4A virtual machines,
How this election will determine tech's future
Monday, November 4, 2024
Netscape lives on; Gen AI experiments; Best early phone deals -- ZDNET ZDNET Tech Today - US November 4, 2024 gettyimages-1995802253 How the 2024 US presidential election will determine tech's
⚙️ Disney AI
Monday, November 4, 2024
Plus: Deepfake fraud & the US election