[Python Dependency Pitfalls] How to set the world on fire

Hey there,

#1 on my list of dependency management pitfalls is there for a good reason:

It lead to a single developer causing mayhem and breaking thousands of open-source projects around the world in one fell swoop.

Here's how it all went down:

A few years back, Azer Koçulu wrote a tiny library he published on Npm, the package repository for NodeJS. (The Python equivalents would be PyPI + pip.)

That "library" contained only 11 lines of code in total, but it was downloaded MILLIONS of times every month of as a dependency in other JavaScript projects.

What Azer's magical "micro library" did, you ask?

It added a tiny piece of functionality that was frequently needed but wasn't a part of the NodeJS standard library:

The ability to pad out the lefthand-side of strings with zeroes or spaces. For example, to format numbers for display.

In Python you'd probably do something like this:

>>> n = '4'
>>> n.zfill(3)
'004'

Anyway, this tiny "left-pad" library was used across many projects, including important applications like Node itself.

One fine day, Azer decided to close his NodeJS package repository account…

Which removed all of the packages associated with it.

And suddenly, "left-pad" was no longer available for download…

Can you guess what happened to the dependency install setup steps on projects using "left-pad?"

Well, they came to a SCREECHING HALT:

App deployments became stuck dead in their tracks. Automated tests stopped working.

And thousands of developers couldn't even RUN their apps locally…

All because "left-pad" had disappeared—and some important build tools required it to work.

It was quite crazy. Even some newspapers reported about the "left-pad incident."

You can imagine that there was a lot of "bruhaha" about the NodeJS packaging system—

But to tell you the truth the *exact same thing* could happen at any time with Python's packaging repository, PyPI.

It's easy to think that pulling in functionality from 3rd party dependencies is always a net benefit.

But every time you're adding an external dependency to your own project you're walking a fine line…

BAD external dependencies can make your stomach churn as a developer or project maintainer.

The people who got burned by the NodeJS "left-pad" incident know what I'm talking about...

On the other hand, a great quality third-party package can save you hours or even days of work.

The challenge is deciding whether a dependency adds value or is just a liability:

>> See step-by-step how to research and make Python dependency decisions (and how to explain them to your team/manager)

— Dan Bader

P.S. There's an important skill that Python dev managers look for in a candidate, but they rarely find it. More on that tomorrow.

Older messages

[Sublime + Python Setup] Sublime Text is just a blank canvas…

Friday, July 16, 2021

Hey there, When I became serious about optimizing Sublime Text with plugins, it was hard for me to separate the wheat from the chaff. Without a real guideline or roadmap I resorted to installing *any*

[PythonistaCafe] What's in PythonistaCafe for you?

Friday, July 16, 2021

Hey there, A couple of years ago I'd become quite interested in martial arts. Hours upon hours of watching "The Karate Kid" growing up must've taken their toll on me... And so, I

[Sublime + Python Setup] How to become a happier & more productive Python dev

Friday, July 16, 2021

Hey there, I really struggled with setting up an effective development environment as a new Python developer. It was difficult to build the right habits and to find a set of tools I enjoyed to use.

[Sublime + Python Setup] The Ctrl+s "Heisenbug"

Friday, July 16, 2021

"What the **** is going on?!" I heard Keith yell. Returning from my lunch break and in a helpful mood I grabbed my coffee mug and shuffled over to my coworker's desk. "What's

[PythonistaCafe] Q&A

Friday, July 16, 2021

Hey there, At this point you should have a pretty good idea of what PythonistaCafe is about and what makes it special. In this email I want to answer some common questions that I get asked about the

Alpine.js Weekly #69

Saturday, July 24, 2021

Alpine plugins are dropping, keep an eye on Caleb Porzio's announcements. 🥳 New Alpine Plugin: Persist 🍾 Easily persist data across page loads by using a nifty little decorator function: $persist()

Olympics: Scammers are trying to cash in on the Olympic Games

Saturday, July 24, 2021

Smart Cities Robotic Challenge; KDE is to Linux what 7 was to Windows Subscription | Read Online | Twitter Facebook LinkedIn Top Story of the Day July 23, 2021 Top Story of the Day Scammers offer

Weekly Xamarin - Issue 313

Saturday, July 24, 2021

Moar MAUI! Weekly Xamarin View on the Web Archives ISSUE 313 24th July 2021 KYM PHILLPOTTS G'day Everyone, We are now on the downhill slope towards MUAI releases and you can really see it in the

[Python Dependency Pitfalls] "Re-inventing the wheel" disease

Saturday, July 24, 2021

Hey there, PyPI, the Python packaging repository, now contains more than 100000 third-party packages in total. That's an *overwhelming* number of packages to choose from... And this feeling of

The Plastic World of RealSelf

Saturday, July 24, 2021

Cyberbits #10: “The TripAdvisor of Boob Jobs” ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌

The Framework Laptop is now shipping — BirdNet – Identify Birds by Sound — and AWS's Egregious Egress

Friday, July 23, 2021

Issue #473 — Top 20 stories of July 24, 2021 Issue #473 — July 24, 2021 You receive this email because you are subscribed to Hacker News Digest. You can open it in the browser if you prefer. 1 The

Daily Crunch - Bitcoin 'is a big part of our future,' says Twitter CEO Jack Dorsey

Friday, July 23, 2021

TechCrunch Newsletter TechCrunch logo The Daily Crunch logo Friday, July 23, 2021 • By Alex Wilhelm Hello and welcome to Daily Crunch for July 23, 2021. It's been an interesting week for the crypto

Software Testing Weekly - Issue 81

Friday, July 23, 2021

Do you ask the right questions? View on the Web Archives ISSUE 81 July 23rd 2021 COMMENT Welcome to the 81st issue! Here's one thing I'd like to highlight this week. It's a meaningful post

The Dial-Up Volunteer Army 💾

Friday, July 23, 2021

How AOL was built on a an army of unpaid volunteers. Here's a version for your browser. Hunting for the end of the long tail • July 23, 2021 Today in Tedium: In some ways, social media started with

JSK Daily for Jul 23, 2021

Friday, July 23, 2021

JSK Daily for Jul 23, 2021 View this email in your browser A community curated daily e-mail of JavaScript news Snapshot Testing for Frontends There are many types of software testing. Among these, one