Programming Digest #477: Lessons I've Learned While Scaling Up a Data Warehouse
#477 — June 20, 2022 | View in browser |
Programming Digest
Spread the word, build the community, share the knowledge with your friends.
sponsor
Open source product analytics you can self-host
PostHog's product analytics suite has everything product-led teams need in one place. Heatmaps, Session Recording, Funnels, Feature Flags, Experimentation and more — all seamlessly integrated. And you can self-host, so user data never leaves your infrastructure.
this week's favorite
Lessons I've Learned While Scaling Up a Data Warehouse
In this post, we will cover the main scale obstacles you might face when using a data warehouse. We’ll also cover what you can do to overcome these challenges in terms of technological tools and whether it pays to build these tools in-house or to use a managed service. Addressing these challenges could be very important for a young startup, whose data is just starting to pile up and questions from different stakeholders are popping up, or for an existing data warehouse that has reached its infrastructure limit.
The Configuration Complexity Clock
When I was a young coder, just starting out in the big scary world of enterprise software, an older, far more experienced chap gave me a stern warning about hard coding values in my software. “They will have to change at some point, and you don’t want to recompile and redeploy your application just to change the VAT tax rate.” I took this advice to heart and soon every value that my application needed was loaded from a huge .ini file. I still think it’s good advice, but be warned, like most things in software, it’s good advice up to a point. Beyond that point lies pain.
It was the second game of a double-header, and the Washington Nationals had a problem. Not on the field, of course: The soon-to-be World Series champions were performing beautifully. But as they waited out a rain delay, something went awry behind the scenes. A task scheduler deep within the team’s analytics infrastructure stopped running.
Uber’s Unified Signup and Login Stack
Every year we have millions of users going through signup and login on our various apps. Over the years we’ve built independent signup and login experiences for each of our lines of business which allowed us to innovate and move a lot quicker. However, as we scaled and added additional lines of business, our experiences began to diverge leading to some of these inconsistencies being amplified.
Mobile-First CSS: Is It Time for a Rethink?
The mobile-first design methodology is great—it focuses on what really matters to the user, it’s well-practiced, and it’s been a common design pattern for years. So developing your CSS mobile-first should also be great, too…right?
how did you like this issue?
1 = didn't like it at all and 5 = loved it and shared it with everyone I know
newsletters
Older messages
Programming Digest #476: Fallacies of Distributed Systems
Sunday, June 12, 2022
And more news, tutorials and articles about programming and technology in this week's issue. #476 — June 13, 2022 View in browser Programming Digest Spread the word, build the community, share the
Programming Digest #475: Programming in the Apocalypse
Sunday, June 5, 2022
And more news, tutorials and articles about programming and technology in this week's issue. #475 — June 06, 2022 View in browser Programming Digest Spread the word, build the community, share the
Programming Digest #474: Learnings from 5 years of tech startup code audits
Sunday, May 29, 2022
And more news, tutorials and articles about programming and technology in this week's issue. #474 — May 30, 2022 View in browser Programming Digest Spread the word, build the community, share the
Programming Digest #473: The overengineered Solution to my Pigeon Problem
Sunday, May 22, 2022
And more news, tutorials and articles about programming and technology in this week's issue. #473 — May 23, 2022 View in browser Programming Digest Spread the word, build the community, share the
Programming Digest #472: The Other Kind of Staff Software Engineer
Sunday, May 15, 2022
And more news, tutorials and articles about programming and technology in this week's issue. #472 — May 16, 2022 View in browser Programming Digest Spread the word, build the community, share the
You Might Also Like
🎮 5 Cheap Apple AirPlay Receiver Alternatives — Your Game Controllers Need Firmware Updates Too
Tuesday, January 7, 2025
Also: The Best Free Offline Music Player Apps For Android How-To Geek Logo January 7, 2025 Did You Know It's a common practice in Japan to package toys with a single cheap piece of candy in order
Daily Coding Problem: Problem #1661 [Medium]
Tuesday, January 7, 2025
Daily Coding Problem Good morning! Here's your coding interview problem for today. This problem was asked by Triplebyte. Implement a data structure which carries out the following operations
DRF, Temp Files, Dataclasses, and More
Tuesday, January 7, 2025
Building HTTP APIs With Django REST Framework #663 – JANUARY 7, 2025 VIEW IN BROWSER The PyCoder's Weekly Logo Building HTTP APIs With Django REST Framework This course will get you ready to build
Charted | The Pyramid of S&P 500 Returns (1874-2024) 💰
Tuesday, January 7, 2025
In 2024, the S&P 500 surged 23%, setting a series of record highs. We show these returns in a historical context spanning 150 years. View Online | Subscribe | Download Our App Presented by: Global
LW 164 - How to create new arrivals collection in Shopify using Shopify Flow
Tuesday, January 7, 2025
How to create new arrivals collection in Shopify using Shopify Flow Shopify Development news and
Tic-Tac-D’Oh 💻
Tuesday, January 7, 2025
The latest from the dull side of the internet. Here's a version for your browser. Hunting for the end of the long tail • January 07, 2025 Tic-Tac-D'Oh Dell decides to rebrand its machines along
Spyglass Dispatch: CaptAIn AmerIca...
Tuesday, January 7, 2025
Hulu, Fubo, Venu • NVIDIA's Cosmos • NVIDIA's DIGITS • Meta's Board Addition • Meta's Fact-Checking Subtraction • Dude, You're Getting a Dell Pro Max Premium The Spyglass Dispatch
DeveloPassion's Newsletter #183 - Knowledge Management for All
Tuesday, January 7, 2025
A newsletter discussing Knowledge Management, Knowledge Work, Zen Productivity, Personal Organization, and more! Sébastien Dubois DeveloPassion's Newsletter DeveloPassion's Newsletter #183 -
CES 2025 ICYMI: 8 top reveals so far
Tuesday, January 7, 2025
Bluesky's most-needed feature; A mulching robot mower; Linux man pages -- ZDNET ZDNET Tech Today - US January 7, 2025 ces55gettyimages-2191705850 CES 2025: ZDNET's 8 most impressive products we
Post from Syncfusion Blogs on 01/07/2025
Tuesday, January 7, 2025
New blogs from Syncfusion Introducing the New Blazor Chat UI Component By Silambarasan Ilango Enhance real-time communication with the Blazor Chat UI. Discover its features and use cases for creating