[Python Dependency Pitfalls] How to set the world on fire

Hey there,

#1 on my list of dependency management pitfalls is there for a good reason:

It lead to a single developer causing mayhem and breaking thousands of open-source projects around the world in one fell swoop.

Here's how it all went down:

A few years back, Azer Koçulu wrote a tiny library he published on Npm, the package repository for NodeJS. (The Python equivalents would be PyPI + pip.)

That "library" contained only 11 lines of code in total, but it was downloaded MILLIONS of times every month of as a dependency in other JavaScript projects.

What Azer's magical "micro library" did, you ask?

It added a tiny piece of functionality that was frequently needed but wasn't a part of the NodeJS standard library:

The ability to pad out the lefthand-side of strings with zeroes or spaces. For example, to format numbers for display.

In Python you'd probably do something like this:

>>> n = '4'
>>> n.zfill(3)
'004'

Anyway, this tiny "left-pad" library was used across many projects, including important applications like Node itself.

One fine day, Azer decided to close his NodeJS package repository account…

Which removed all of the packages associated with it.

And suddenly, "left-pad" was no longer available for download…

Can you guess what happened to the dependency install setup steps on projects using "left-pad?"

Well, they came to a SCREECHING HALT:

App deployments became stuck dead in their tracks. Automated tests stopped working.

And thousands of developers couldn't even RUN their apps locally…

All because "left-pad" had disappeared—and some important build tools required it to work.

It was quite crazy. Even some newspapers reported about the "left-pad incident."

You can imagine that there was a lot of "bruhaha" about the NodeJS packaging system—

But to tell you the truth the *exact same thing* could happen at any time with Python's packaging repository, PyPI.

It's easy to think that pulling in functionality from 3rd party dependencies is always a net benefit.

But every time you're adding an external dependency to your own project you're walking a fine line…

BAD external dependencies can make your stomach churn as a developer or project maintainer.

The people who got burned by the NodeJS "left-pad" incident know what I'm talking about...

On the other hand, a great quality third-party package can save you hours or even days of work.

The challenge is deciding whether a dependency adds value or is just a liability:

>> See step-by-step how to research and make Python dependency decisions (and how to explain them to your team/manager)

— Dan Bader

P.S. There's an important skill that Python dev managers look for in a candidate, but they rarely find it. More on that tomorrow.

Older messages

[PythonistaCafe] What's in PythonistaCafe for you?

Wednesday, July 21, 2021

Hey there, A couple of years ago I'd become quite interested in martial arts. Hours upon hours of watching "The Karate Kid" growing up must've taken their toll on me... And so, I

[PythonistaCafe] Why PythonistaCafe exists

Tuesday, July 20, 2021

Hey there, In one of my last emails I talked about how some online communities in the tech space devolve over time and turn into cesspools of negativity. This relates directly to how and why I started

[Sublime + Python Setup] Sublime Text is just a blank canvas…

Tuesday, July 20, 2021

Hey there, When I became serious about optimizing Sublime Text with plugins, it was hard for me to separate the wheat from the chaff. Without a real guideline or roadmap I resorted to installing *any*

[Python Dependency Pitfalls] A total mess?

Monday, July 19, 2021

Hey there, Recently I watched a Pythonista ask for advice on setting up a Python project on his work machine. This new developer had some prior experience with NodeJS and had just started to get his

[Sublime + Python Setup] Grumpy old greybeard with a whitespace problem

Monday, July 19, 2021

One fateful day, the Agile Gods that be decided to “add some firepower” to my little team… And so, developer Paul joined (name changed to protect the guilty). Before I dive into this story, let me ask

[Python Dependency Pitfalls] "Re-inventing the wheel" disease

Saturday, July 24, 2021

Hey there, PyPI, the Python packaging repository, now contains more than 100000 third-party packages in total. That's an *overwhelming* number of packages to choose from... And this feeling of

The Plastic World of RealSelf

Saturday, July 24, 2021

Cyberbits #10: “The TripAdvisor of Boob Jobs” ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌

The Framework Laptop is now shipping — BirdNet – Identify Birds by Sound — and AWS's Egregious Egress

Friday, July 23, 2021

Issue #473 — Top 20 stories of July 24, 2021 Issue #473 — July 24, 2021 You receive this email because you are subscribed to Hacker News Digest. You can open it in the browser if you prefer. 1 The

Daily Crunch - Bitcoin 'is a big part of our future,' says Twitter CEO Jack Dorsey

Friday, July 23, 2021

TechCrunch Newsletter TechCrunch logo The Daily Crunch logo Friday, July 23, 2021 • By Alex Wilhelm Hello and welcome to Daily Crunch for July 23, 2021. It's been an interesting week for the crypto

Software Testing Weekly - Issue 81

Friday, July 23, 2021

Do you ask the right questions? View on the Web Archives ISSUE 81 July 23rd 2021 COMMENT Welcome to the 81st issue! Here's one thing I'd like to highlight this week. It's a meaningful post

The Dial-Up Volunteer Army 💾

Friday, July 23, 2021

How AOL was built on a an army of unpaid volunteers. Here's a version for your browser. Hunting for the end of the long tail • July 23, 2021 Today in Tedium: In some ways, social media started with

JSK Daily for Jul 23, 2021

Friday, July 23, 2021

JSK Daily for Jul 23, 2021 View this email in your browser A community curated daily e-mail of JavaScript news Snapshot Testing for Frontends There are many types of software testing. Among these, one

iOS Dev Weekly - Issue 517

Friday, July 23, 2021

Focusing on positivity and balance. ⚖️ View on the Web Archives ISSUE 517 July 23rd 2021 Comment I'm determined to write something more positive today! I've been far too negative recently, so

Can Ethereum Be Ultra-Sound Money? - DeFriday #10

Friday, July 23, 2021

Or is it all ultra-hopium? ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌ ‌

You probably don't need Redux: Use React Context + useReducer hook

Friday, July 23, 2021

Keep up-to-date with the latest programming news Codementor Your Weekly Digest TOP POSTS FROM THIS WEEK Nikhil Kumaran S You probably don't need Redux: Use React Context + useReducer hook I would