#107: End-to-End Monitoring Platform

This week, I've once again handpicked a diverse range of fresh insights, tutorials, and tools in the rapidly-evolving DevOps landscape.

Leading off, we delve into the world of automated IaC tests with a walkthrough on how to use Terratest, coupled with GitHub Actions, to automate the testing process. Next, we tackle application monitoring through open-source tools, where you'll learn how to build a monitoring platform using Loki, Grafana, Tempo, and Mimir.

Optimizing Kubernetes clusters can be a challenge, so we'll discuss some key considerations, encompassing various cluster, node, and tenancy configurations. Speaking of configuration, I've included a useful guide on converting entire Terraform projects to Pulumi.

For AWS Lambda enthusiasts, I found a brilliant piece on building a Lambda extension to provision a SOCKS5 proxy inside the runtime environment. In terms of application monitoring, I've highlighted how to monitor a Next.js application with Komiser. For those aspiring to break into the SRE role, I've curated some resources to set you on the right path.

This week's open-source projects feature some fascinating tools, including a library for locally debugging JS/TS AWS Lambda functions, a CLI tool for performing cost analysis on your Azure subscription, and an API Traffic Analyzer providing real-time, protocol-level visibility into Kubernetes. These are just a few among the many gems we have this week.

As always, there is a lot more content in the full newsletter, so maintain your curiosity, persist with your learning, and I hope you'll enjoy this week's digest!

📖 TUTORIALS OF THE WEEK

"How infrastructure testing can help boost your deployment" - How to use Terratest to write and run automated infrastructure tests, and GitHub Actions to automate the testing process - Read more »

"Setting up an End-to-End monitoring platform with Grafana stack" - How to build an application monitoring platform with open-source tools like Loki, Grafan, Tempo and Mimir - Read more »

"Optimized Kubernetes cluster architecture" - Various key considerations for optimizing a Kubernetes cluster, including different cluster, node, and tenancy configurations - Read more »

"Converting full Terraform programs to Pulumi" - How to convert full Terraform projects to Pulumi with tf2pulumi CLI - Read more »

"Using Tailscale on Lambda for a live development proxy" - How to build an AWS Lambda extension to provide a SOCKS5 proxy inside of the Lambda runtime environment - Read more »

"Monitoring a Next.js application with Komiser" - How even a simple Next.js app, when coupled with mismanaged cloud infrastructure, can quickly accumulate costs that could have a significant impact on your budget - Read more »

"Power up your serverless application with AWS Lambda extensions" - How to use extensions in order to create your own monitoring tool for lambdas without compromising their performances - Read more »

🛠 PROJECTS OF THE WEEK

1️⃣ Library to help debug JS/TS AWS Lambda functions locally - Learn more »

2️⃣ CLI tool to perform cost analysis on your Azure subscription - Learn more »

3️⃣ An example workload to stream a game from Kubernetes using Sunshine - Learn more »

4️⃣ Trolley is a multi cloud Kubernetes management system - Learn more »

5️⃣ API Traffic Analyzer for Kubernetes providing real-time, protocol-level visibility into Kubernetes - Learn more »

6️⃣ An easy, pleasant, speedy, and exciting way to use your favorite OS - Learn more »

📰 NEWSWORTHY STORIES

"Incident vs. bug" - Read more »

"How you use DynamoDB makes a lot of difference!" - Read more »

"The ElasticSearch rant" - Read more »

"How to get an SRE role" - Read more »

"CS:GO: from zero to 0-day" - Read more »

💡 THREAD OF THE WEEK

😂 MEMES OF THE WEEK

Hand-picked DevOps opportunities to work from home, remotely, freelance, full-time, part-time, contract and internships - See more »

I hope you enjoy this week’s newsletter! Share it with a friend or colleague if you find it helpful, drop me an email or send me a DM on Twitter about topics you’d like to hear about in future editions.

Remember to share if you enjoyed this issue!

Share

Forward

Digest #107: End-to-End Monitoring Platform 📊

#107: End-to-End Monitoring Platform

Older messages

Digest #106: I love git-blame-someone-else! 😳

Digest #105: Kubernetes CronJobs at Scale 🚀

Digest #104: Testing CloudFormation Stacks 🧪

Digest #103: Run Any Linux Distribution Inside Your Terminal 😍

Digest #102: Are Terraform’s days numbered? ⏰

You Might Also Like

Import AI 399: 1,000 samples to make a reasoning model; DeepSeek proliferation; Apple's self-driving car simulator

Defining Your Paranoia Level: Navigating Change Without the Overkill

5 ways AI can help with taxes 🪄

Recurring Automations + Secret Updates

The First Provable AI-Proof Game: Introducing Butterfly Wings 4

GCP Newsletter #437

Charted | The 1%'s Share of U.S. Wealth Over Time (1989-2024) 💰

The Great Social Media Diaspora & Tapestry is here

Daily Coding Problem: Problem #1689 [Medium]

📧 Stop Conflating CQRS and MediatR