Architecture Weekly #190 - Queuing, Backpressure, Single Writer and other useful patterns for managing concurrency
Welcome to the new week! In the last edition, we explored why connection pooling is crucial for using databases efficiently. Now, let's take a step further and perform a thought experiment: How would you actually implement a connection pool? We'll use Node.js and TypeScript as an illustration, as we need to choose a specific stack to make the concepts concrete. If you think, “Well, I won’t ever do it myself”, don’t worry—the intention here isn’t to drown you in the imaginary code. Instead, I will show you how these micro-scale coding considerations can impact macro-scale design analysis. As we go, we’ll learn a few valuable concepts, like Queuing, Backpressure and single Writer, and discuss why queuing can be useful in managing concurrency and beyond! Connection PoolLet’s start where we ended, so with our database access code: As we discussed in detail in the previous article, a connection pool manages reusable database connections shared among multiple requests. Instead of opening and closing a connection for each request, we borrow a connection from the pool and return it after database operation. Thanks to that, we’re cutting the latency required to establish a connection and get a more resilient and scalable solution (at least if we’re not into serverless). Done? Now, think about the potential limitations of your thinking and the potential solutions to them. Ready? Let’s compare that with my thoughts! Basic definitionLet’s start by defining the basic API. It could look like this: There is not much to explain besides that we want to manage the pool to the database represented by the connection string and limit the number of open connections. Why? Imagine a service like a payment gateway that handles transactions for a popular online retailer. During peak shopping times, such as Black Friday, the system experiences a huge increase in payment requests. Each transaction requires a secure connection to the database to verify payment details and update the order status. Yet, the database has its limits, so we might need to try somehow to squeeze that into the limit of 100 concurrence connections. When all 100 connections are occupied processing transactions, new incoming requests must be managed effectively. We could try to: - Fail Immediately: The system could reject new requests outright, leading to a poor user experience and lost sales. - Retry Aggressively: The requests might enter a loop, repeatedly attempting to acquire a connection, which can further strain the system. The naive implementation of a fail-fast strategy could look as such: Now, While doing retries in the loop or even immediate failure can be a temporary solution, for obvious reasons, it won’t scale and will result in a subpar experience. Consider a busy restaurant with limited seating. When all tables are occupied, you can be asked to wait a bit near the entrance. As tables become available, the waiter will invite you to be seated. This ensures that the restaurant operates smoothly without overcrowding or turning away customers. We could do the same with our connection pool implementation. Instead of rejecting new requests or letting them endlessly retry (which could crash your system), they're placed in a queue and handled sequentially. The next part of the article is for paid users. If you’re not such yet, till the end of August, you can use a free month's trial: https://www.architecture-weekly.com/b3b7d64d. You can check it out and decide if you like it and want to stay. I hope that you will!... Unlock this post for free, courtesy of Oskar Dudycz.A subscription gets you:
|
Older messages
Architecture Weekly #189 - Mastering Database Connection Pooling
Monday, August 5, 2024
Boom! This is the first Architecture Weekly in the new format. It's a deep dive to database connection pooling. Check to learn how to set it up starting from simple solution and learning how the
Architecture Weekly #188 - 29th July 2024
Monday, July 29, 2024
This time in an unusual edition, we focused on the underestimated aspect of Software Design: resting. We discussed the typical mistakes we do and why repeating them will not help you in making right
Architecture Weekly #187 - 8th July 2024
Monday, July 8, 2024
This week, we discussed the challenges of finding the right architecture strategy and pivoting it when needed. We brought and discussed a few insightful articles on how/when/why to move to serverless,
Architecture Weekly #186 - 1st July 2024
Monday, July 1, 2024
This week, we covered the Polyfill JS supply chain attack that potentially touched 4% of the Internet. We also checked the DataDog report on the state of Cloud Spendings. We discussed whether it's
Architecture Weekly #185 - 24th June 2024
Monday, June 24, 2024
This edition is 50 shaded of coupling! Yeah, I know how that sounds, but we discussed why coupling is not an end goal but a metric to evaluate tradeoffs. We also discussed how to actually make
You Might Also Like
Daily Coding Problem: Problem #1647 [Medium]
Tuesday, December 24, 2024
Daily Coding Problem Good morning! Here's your coding interview problem for today. This problem was asked by Square. In front of you is a row of N coins, with values v 1 , v 1 , ..., v n . You are
Sentiment Analysis, Topological Sort, Web Security, and More
Tuesday, December 24, 2024
Exploring Modern Sentiment Analysis Approaches in Python #661 – DECEMBER 24, 2024 VIEW IN BROWSER The PyCoder's Weekly Logo Exploring Modern Sentiment Analysis Approaches in Python What are the
🤫 Do Not Disturb Mode Is My Secret to Sanity — 8 Gadgets I Want To See Nintendo Make
Tuesday, December 24, 2024
Also: The Best Christmas Movies to Watch on Netflix, and More! How-To Geek Logo December 24, 2024 Did You Know Their association with the Christmas season might make you think poinsettias hail from a
😱 AzureEdge.net DNS Retiring Jan. 2025, 🚀 Microsoft Phi-4 AI Outperforms, 🔒 Microsoft Secure Future Initiative
Tuesday, December 24, 2024
Blog | Advertise | View Online Your trusted source for Cloud, AI and DevOps guidance with industry expert Chris Pietschmann! Phi-4: Microsoft's New Small Language Model Outperforms Giants in AI
Mapped | The Top Health Insurance Companies by State 🏥
Tuesday, December 24, 2024
In 13 US states, a single company dominates the health insurance market, holding at least half of the total market share. View Online | Subscribe | Download Our App Presented by: Global X ETFs Power
The Stanford Grad Who Forgot How To Think
Tuesday, December 24, 2024
Top Tech Content sent at Noon! Boost Your Article on HackerNoon for $159.99! Read this email in your browser How are you, @newsletterest1? 🪐 What's happening in tech today, December 24, 2024? The
The next big HDMI leap is coming
Tuesday, December 24, 2024
Sora side hustles; Casio's tiny watch comes to the US -- ZDNET ZDNET Tech Today - US December 24, 2024 Ecovacs Deebot T30S Combo robot vacuum and mop The next big HDMI leap is coming next month -
⚙️ Robo-suits
Tuesday, December 24, 2024
Plus: The data center energy surge
Apache Tomcat Vulnerability CVE-2024-56337 Exposes Servers to RCE Attacks
Tuesday, December 24, 2024
THN Daily Updates Newsletter cover The Data Science Handbook, 2nd Edition ($60.00 Value) FREE for a Limited Time Practical, accessible guide to becoming a data scientist, updated to include the latest
Edge 459: Quantization Plus Distillation
Tuesday, December 24, 2024
Some insights into quantized distillation ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏