Architecture Weekly #190 - Queuing, Backpressure, Single Writer and other useful patterns for managing concurrency
Welcome to the new week! In the last edition, we explored why connection pooling is crucial for using databases efficiently. Now, let's take a step further and perform a thought experiment: How would you actually implement a connection pool? We'll use Node.js and TypeScript as an illustration, as we need to choose a specific stack to make the concepts concrete. If you think, “Well, I won’t ever do it myself”, don’t worry—the intention here isn’t to drown you in the imaginary code. Instead, I will show you how these micro-scale coding considerations can impact macro-scale design analysis. As we go, we’ll learn a few valuable concepts, like Queuing, Backpressure and single Writer, and discuss why queuing can be useful in managing concurrency and beyond! Connection PoolLet’s start where we ended, so with our database access code: As we discussed in detail in the previous article, a connection pool manages reusable database connections shared among multiple requests. Instead of opening and closing a connection for each request, we borrow a connection from the pool and return it after database operation. Thanks to that, we’re cutting the latency required to establish a connection and get a more resilient and scalable solution (at least if we’re not into serverless). Done? Now, think about the potential limitations of your thinking and the potential solutions to them. Ready? Let’s compare that with my thoughts! Basic definitionLet’s start by defining the basic API. It could look like this: There is not much to explain besides that we want to manage the pool to the database represented by the connection string and limit the number of open connections. Why? Imagine a service like a payment gateway that handles transactions for a popular online retailer. During peak shopping times, such as Black Friday, the system experiences a huge increase in payment requests. Each transaction requires a secure connection to the database to verify payment details and update the order status. Yet, the database has its limits, so we might need to try somehow to squeeze that into the limit of 100 concurrence connections. When all 100 connections are occupied processing transactions, new incoming requests must be managed effectively. We could try to: - Fail Immediately: The system could reject new requests outright, leading to a poor user experience and lost sales. - Retry Aggressively: The requests might enter a loop, repeatedly attempting to acquire a connection, which can further strain the system. The naive implementation of a fail-fast strategy could look as such: Now, While doing retries in the loop or even immediate failure can be a temporary solution, for obvious reasons, it won’t scale and will result in a subpar experience. Consider a busy restaurant with limited seating. When all tables are occupied, you can be asked to wait a bit near the entrance. As tables become available, the waiter will invite you to be seated. This ensures that the restaurant operates smoothly without overcrowding or turning away customers. We could do the same with our connection pool implementation. Instead of rejecting new requests or letting them endlessly retry (which could crash your system), they're placed in a queue and handled sequentially. The next part of the article is for paid users. If you’re not such yet, till the end of August, you can use a free month's trial: https://www.architecture-weekly.com/b3b7d64d. You can check it out and decide if you like it and want to stay. I hope that you will!... Unlock this post for free, courtesy of Oskar Dudycz.A subscription gets you:
|
Older messages
Architecture Weekly #189 - Mastering Database Connection Pooling
Monday, August 5, 2024
Boom! This is the first Architecture Weekly in the new format. It's a deep dive to database connection pooling. Check to learn how to set it up starting from simple solution and learning how the
Architecture Weekly #188 - 29th July 2024
Monday, July 29, 2024
This time in an unusual edition, we focused on the underestimated aspect of Software Design: resting. We discussed the typical mistakes we do and why repeating them will not help you in making right
Architecture Weekly #187 - 8th July 2024
Monday, July 8, 2024
This week, we discussed the challenges of finding the right architecture strategy and pivoting it when needed. We brought and discussed a few insightful articles on how/when/why to move to serverless,
Architecture Weekly #186 - 1st July 2024
Monday, July 1, 2024
This week, we covered the Polyfill JS supply chain attack that potentially touched 4% of the Internet. We also checked the DataDog report on the state of Cloud Spendings. We discussed whether it's
Architecture Weekly #185 - 24th June 2024
Monday, June 24, 2024
This edition is 50 shaded of coupling! Yeah, I know how that sounds, but we discussed why coupling is not an end goal but a metric to evaluate tradeoffs. We also discussed how to actually make
You Might Also Like
💻 Issue 428 - C# different way to do a proof of concept
Thursday, September 19, 2024
This week's Awesome .NET Weekly Read this email on the Web The Awesome .NET Weekly Issue » 428 Release Date Sep 19, 2024 Your weekly report of the most popular .NET news, articles and projects
💎 Issue 435 - Ruby-SAML pwned by XML signature wrapping attacks
Thursday, September 19, 2024
This week's Awesome Ruby Newsletter Read this email on the Web The Awesome Ruby Newsletter Issue » 435 Release Date Sep 19, 2024 Your weekly report of the most popular Ruby news, articles and
💻 Issue 435 - Oracle, it's time to free JavaScript
Thursday, September 19, 2024
This week's Awesome JavaScript Weekly Read this email on the Web The Awesome JavaScript Weekly Issue » 435 Release Date Sep 19, 2024 Your weekly report of the most popular JavaScript news, articles
📱 Issue 429 - iOS 18 breaks IMAPS self-signed certs
Thursday, September 19, 2024
This week's Awesome iOS Weekly Read this email on the Web The Awesome iOS Weekly Issue » 429 Release Date Sep 19, 2024 Your weekly report of the most popular iOS news, articles and projects Popular
💻 Issue 353 - Why React Won the Front-End Race
Thursday, September 19, 2024
This week's Awesome React Weekly Read this email on the Web The Awesome React Weekly Issue » 353 Release Date Sep 19, 2024 Your weekly report of the most popular React news, articles and projects
💻 Issue 435 - DevSecOps Project: "Secure Full-Stack Node.js Web Application Deployment with Jenkins, Docker, Kubernetes, and HashiCorp Vault"
Thursday, September 19, 2024
This week's Awesome Node.js Weekly Read this email on the Web The Awesome Node.js Weekly Issue » 435 Release Date Sep 19, 2024 Your weekly report of the most popular Node.js news, articles and
📱 Issue 432 - Swift 6
Thursday, September 19, 2024
This week's Awesome Swift Weekly Read this email on the Web The Awesome Swift Weekly Issue » 432 Release Date Sep 19, 2024 Your weekly report of the most popular Swift news, articles and projects
💻 Issue 430 - Days since last Minecraft server written in Rust was released
Thursday, September 19, 2024
This week's Awesome Rust Weekly Read this email on the Web The Awesome Rust Weekly Issue » 430 Release Date Sep 19, 2024 Your weekly report of the most popular Rust news, articles and projects
Ranked | The Largest Producers of Wind Power, by Country ⚡
Thursday, September 19, 2024
Global wind power capacity hit fresh records in 2023 thanks to strategic government investment and lower technology costs. View Online | Subscribe | Download Our App Presented by: NEW REPORT: Brought
🧠 ChatGPT Passed the Turing Test — 5 Tips to Make Your Laptop Last Longer
Thursday, September 19, 2024
Also: How to Sideload Apps on Android TV, and More! How-To Geek Logo September 19, 2024 Did You Know Babies seem to have such large eyes because humans are born with eyes approximately 75 percent of