BetterDev #232 - Reducing BigQuery Costs: How We Fixed A $1 Million Query
Better Dev #232 Nov 07, 2022
Hi everyone, welcome to first issue of November. This week, where I live, red leaves start to emerge and my family went outside, enjoy seeing the beautiful color of autumn. Social media had been a tough week with Twitter and Elon, same as finance system and crypto drama. The world looked so depressing, but no matter what, being a programmer, knowing how to code is really a privilege, we have a way to make a living while sitting at a deck doing thing we enjoy. No matter what happen, there will always a demand to make website, application or service and a continous stream of jobs require to run these things. We are thankful for the jobs and opportunities we had, be optimist and always leaning. on ward. have a good week everyone.
This week I shared a small tip to debug CI build failure easiser. If you ever debug some tricky CI failure and say to your self: “If I can SSH into this server or container that running the build, I will be able to figure out immediately”, then this could be very useful. It can also apply anytime we need to SSH back into some private infrastructure.
During the infrastructural exploration of a pipeline my team was building, we discovered a query that could have cost us nearly $1 million USD a month in BigQuery. Below, we’ll detail how we reduced this and share our tips for lowering costs in BigQuery.
Recently, the Chrome developers announced their decision to remove the behind-a-flag support for JPEG XL. Cloudinary argue(to which I agree) JPEG XL can bring these benefits without introducing a regression in other areas, at least in terms of technical strengths. We can only hope that the Chrome developers reverse their decision and help JPEG XL catch up with the old formats in terms of software support, so we can all enjoy the benefits it brings.
Almost web app needs to make call to third party API such as taking a payment, send out an email. By wrapping an adapter around each API, presenting only the features of that API the app needs, we can more easily manage and test the app.
when we run diff
or git diff
, we see a visualization of what were add, insert or delete? what algorithm power that?
If you’ve ever wanted to build a web app that you can control with hand gestures as if by magic, this article is for you. With a couple of APIs and some JavaScript, you can build apps that behave like sorcery.
BTree is the most common index type we used every day. For exact lookup, BTree probably best choice most of time, but if you’re looking for a range query, Brin maybe an option
We’re going to dig into some of the flexibility on architecting your database and terminology within Postgres and the trade-offs for different approaches.
When you tune performance for a system, very quickly you run into stuff like nofile, transaparent huge page which usually either has some high level command to set it, or sometime require you to change a
certain parameters by writing to /proc/sys/dev
. So today we will have a look around it.
This is a step by step guide on how to become a DevOps engineer, with links to relevant learning resources. If you plan to become a DevOps engineer, could be a great learning resource.
Code to read
AWS Lambda deployment requires a bundle of entire of your app and all dependencies, package in a certain structure. Zip it and ship it do exactly that, create a zip bundle for a lambda app in node go or rust and deploy to aws.
JavaScripta library for building concurrent applications in Ruby. Polyphony harnesses the power of Ruby fibers to provide a cooperative, sequential coroutine-based concurrency model. Under the hood, Polyphony uses io_uring or libev to maximize I/O performance.
RubyA Pure-go redis driver use RESP3 protocol. If you curios about writing driver, this is a great example
Goa cross-platform embeddable sandboxing library allowing restrictions to Filesystem and Network operations using native operating system APIs. It should be ued for education purpose only at this point
RustVideo
Tools
The Open Source Shopify Alternative. Medusa is a composable engine that combines an amazing developer experience with endless customizations for merchants to scale
a powerful, flexible, Markdown-based authoring framework to create blog or document site in markdown.
It answers thousands of questions about your infrastructure, and integrates with over 300 resources across cloud accounts, Kubernetes, containers, services, VMs, APIs, and more.
finds vulnerabilities and misconfigurations on all systems in your infrastructure including: public and private cloud environments, Kubernetes clusters, containers, container registries, servers and endpoints, SaaS products, infrastructure as code, APIs, and more.
Lambda function for executing pg_dump and upload the output to s3. Can be useful to take a daily/hourly pg_dump of database
You can view this issue in web browser.
If you have any suggestion/feedback, do tell me by replying to this email. I read them all.
No longer want to receive these emails? Unsubscribe
Key phrases
Older messages
BetterDev #231 - Time is an illusion, Unix time doubly so
Monday, October 31, 2022
Better Dev #231 Oct 31, 2022 Hi, everyone, happy halloween. This issue is a haloween treat :-), it has a lot of short and small articles which you can apply to your daily work and small tips and tricks
BetterDev #230 - Secure your server by limit outgoing connection
Monday, October 24, 2022
Better Dev #230 Oct 24, 2022 Hi, everyone, welcome to our second issue of the come back. I had received many emails with encouragement about my resumt. I really really appreciate that. Thank you so
BetterDev #229 - BetterDev is back
Monday, October 17, 2022
Better Dev #229 Oct 17, 2022 Hi, everyone. It has been almost 6 months with out any betterdev issue :-). It was due to my busy schedule. I finally started to get thing back on track. I will come back
BetterDev #228 - Memory leaks: the forgotten side of web performance
Monday, February 7, 2022
Better Dev #228 Feb 07, 2022 Memory leaks: the forgotten side of web performance I've researched and learned enough about client-side memory leaks to know that most web developers aren't
BetterDev #227 - How to avoid layout shifts caused by web fonts and Scaling Kubernetes to Over 4k Nodes and 200k Pods
Tuesday, February 1, 2022
Better Dev #227 Feb 01, 2022 How to avoid layout shifts caused by web fonts Fonts cause layout shifts when the size of the containing element (eg a or paragraph) changes when the web font is downloaded
You Might Also Like
Upgrade Your Git Game, Visual Studio Getting 'Command Palette,' Python/Java in VS Code, .NET 9 Preview, More
Thursday, April 25, 2024
Home | News | How To | Webcasts | Whitepapers | Advertise .NET Insight April 25, 2024 THIS ISSUE SPONSORED BY: ■ dtSearch® - INSTANTLY SEARCH TERABYTES Upgrade Your Git Game in Visual Studio 2022
🔒 The Vault Newsletter: April issue 🔑
Thursday, April 25, 2024
Get the latest business security news, updates, and advice from 1Password. ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏
Top Tech 🏆 Lenovo ThinkPad X1 Carbon Gen 12 Laptop Review — Testing an AI Voice Recorder
Thursday, April 25, 2024
Also: The Roborock S8 MaxV Ultra Vacuum is Excellent, and More! How-To Geek Logo April 25, 2024 Take a look at our latest reviews, featuring fun tech like the Lenovo ThinkPad X1 Carbon laptop,
⚙️ r1
Thursday, April 25, 2024
Plus: UK investigating OpenAI
Charted | Economic Growth Forecasts for G7 and BRICS Countries in 2024 📊
Thursday, April 25, 2024
The IMF has released its economic growth forecasts for 2024. How do the G7 and BRICS countries compare in expected real GDP growth? View Online | Subscribe Presented by: Access European benchmarks with
Build5Nines Newsletter - April 25, 2024
Thursday, April 25, 2024
View this email in your browser Build5Nines Build5Nines Newsletter Thank you for subscribing! I look forward to sharing with you the latest cloud news, technical help, and other thoughts around DevOps
Discover the World's Easiest Parallel File System
Thursday, April 25, 2024
Join us in exploring the future of data management with Bjorn Kolbeck, a Google engineer turned CEO and Co-founder of Quobyte, the creators of the world's easiest parallel file system. ͏ ͏ ͏ ͏ ͏ ͏
Issue 314 - New Model 3 Performance is here
Thursday, April 25, 2024
View this email in your browser If you are just now finding out about Tesletter, you can subscribe here! If you already know Tesletter and want to support us, check out our Patreon page Issue 314 - New
Programmer Weekly - Issue 202
Thursday, April 25, 2024
View this email in your browser Programmer Weekly Welcome to issue 202 of Programmer Weekly. Let's get straight to the links this week. Quote of the Week "Computer science inverts the normal.
Python Weekly - Issue 647
Thursday, April 25, 2024
View this email in your browser Python Weekly Welcome to issue 647 of Python Weekly. Let's get straight to the links this week. From Our Sponsor Get Your Weekly Dose of Programming A weekly