#99: Scaling to 1000 K8s Pods

Hold on to your keyboards, the latest DevOps weekly digest is here and it's packed with content 😅

Embark on a developer's voyage to AWS Lambda mastery, and fine-tune your pull request game with writing tips that'll make your team productive. We'll also delve into setting up event-driven, trace-based testing in Kubernetes and present the ultimate guide to load balancers.

Ever wondered how HTTPie lost 54k GitHub stars? I've got the scoop! Discover why one team ditched RabbitMQ for a Postgres queue, and learn valuable lessons from scaling a Kubernetes cluster to a whopping 1000 pods.

As always, I've handpicked some outstanding open-source projects for you to explore. KubeStalk unveils Kubernetes and related infrastructure attack surfaces from a black-box perspective. Unleash powerful search, tagging, filtering, and sorting capabilities for SQLite databases with a simple text query language. And don't miss the open-source database package registry for Postgres, taking your DB game to the next level.

Plus, I'm hosting a free workshop this Wednesday on Discord about building a Jenkins cluster on AWS from scratch with Terraform and Packer - Join the event here.

This week's DevOps digest is bursting with insights, so don't let it slip away!

📖 POSTS OF THE WEEK

Developers' journey to AWS Lambda
AWS Lambda has a surprise learning curve. You create a new function, write your code, and it executes. Easy, right? Then you discover just how deep the rabbit hole goes!
Read more »

"Writing tips for improving your pull requests" - Learn how to write concise and effective pull request descriptions for your code changes - Read more »

"Event-driven Kubernetes testing with Testkube and Tracetest" - Learn how to set up event-driven, trace-based testing in Kubernetes - Read more »

"Introducing AWS Lambda response streaming" - You can use response streaming to send responses larger than Lambda’s 6 MB response payload limit up to a soft limit of 20 MB - Read more »

"Deploy Komiser to AWS with Terraform" - A step-by-step tutorial to deploy Komiser on AWS using Terraform and how to uncover cost-saving opportunities, security threats, and compliance issues - Read more »

"Debating the benefits of AWS spending restrictions" - Everybody should be using AWS Budget regardless if you’re a hobbyist or a multi-billion dollar company - Read more »

"Lesson learned while scaling Kubernetes cluster to 1000 pods" - Things don’t behave as expected, especially when we start to push outside of the normal scale - Read more »

"Load Balancing" - Focus on the ways that a single load balancer might distribute HTTP requests to a set of servers - Read more »

🛠 PROJECTS OF THE WEEK

1️⃣ KubeStalk discovers Kubernetes and related infrastructure based attack surface from a black-box perspective - Learn more »

2️⃣ CasaOS is a simple, easy-to-use, elegant open-source Personal Cloud system - Learn more »

3️⃣ Generate d2 AWS architure diagrams from CDK manifest and deployed CloudFormation - Learn more »

4️⃣ An open-source database package registry for Postgres - Learn more »

5️⃣ Powerful search, tagging, filtering and sorting via simple text query language, for SQLite databases - Learn more »

6️⃣ A mind-blowing code to accompany "A Method for Animating Children's Drawings of the Human Figure" - Learn more »

📰 NEWSWORTHY STORIES

"How we lost 54k GitHub stars" - Read more »

"Intelligent, automatic restarts for unhealthy Kafka consumers" - Read more »

"Why we ditched RabbitMQ and replaced it with a Postgres queue" - Read more »

"You can’t solve AI security problems with more AI" - Read more »

💡 THREAD OF THE WEEK

😂 MEMES OF THE WEEK

Sign up for AWS free credits .... and it's gone!

📕 BOOK OF THE WEEK

The tech resume inside-out is a book written by Gergely Orosz, author of the largest tech newsletter in Substack and ex engineer manager at Uber.
It contains actionable advice on how to write a good tech resume or CV for developer or engineering manager positions. It was co-written by tech folks at Airbnb, Uber, Shopify and Spotify.

💼 OPEN JOBS OF THE WEEK

Hand-picked DevOps opportunities to work from home, remotely, freelance, full-time, part-time, contract and internships - See more »

DevOps Engineer @Appsmith
Terraform, Kubernetes, AWS
🌎 Remote, India
Read more »

Site Reliability Engineer @Reddit
Kubernetes, AWS, GCP
🌎 Remote, USA
Read more »

Infrastructure Engineer @Opal
Postgres, Kubernetes, Go
🌎 Remote, USA
Read more »

I hope you enjoy this week’s newsletter! Share it with a friend or colleague if you find it helpful, drop me an email or send me a DM on Twitter about topics you’d like to hear about in future editions.

Remember to share if you enjoyed this issue!

Share

Forward

DevOps Bulletin - Digest #99: Scaling to 1000 K8s Pods 😳

#99: Scaling to 1000 K8s Pods

Older messages

Digest #98: Multiple Environments with Terraform 😎

Digest #97: Hacking AWS Account 😰

Digest #96: Kubernetes Broke Reddit 😳

Digest #95: ChatGPT for DevOps 😎

Digest #94: Postgres Explained 😎

You Might Also Like

Import AI 399: 1,000 samples to make a reasoning model; DeepSeek proliferation; Apple's self-driving car simulator

Defining Your Paranoia Level: Navigating Change Without the Overkill

5 ways AI can help with taxes 🪄

Recurring Automations + Secret Updates

The First Provable AI-Proof Game: Introducing Butterfly Wings 4

GCP Newsletter #437

Charted | The 1%'s Share of U.S. Wealth Over Time (1989-2024) 💰

The Great Social Media Diaspora & Tapestry is here

Daily Coding Problem: Problem #1689 [Medium]

📧 Stop Conflating CQRS and MediatR