I’ve worked for some Hot Silicon Valley™ companies in the past. Oftentimes, these companies would roll their own projects instead of buying off-the-shelf SaaS solutions. This was great for a junior engineer like myself as I got to see how you would build and architect companies like Segment and Datadog. Now, I’m at a company where the culture is the exact opposite. If we need a solution, we can buy it. If you ask the team for permission to add a library to the React app for a new WhizBang feature, you’re normally met with shrugs.

Generally, I tend to be pretty indifferent when the script is flipped. I may not totally understand why we need to bring in a particular package or change the paradigm for how we’re handling, say, buttons but it’s normally not a big enough deal to spend political capital on.

I do want to share a fun story of why I am going to change my tune going forward. A couple of weeks ago, we were unable to deploy our latest commits on master to prod. This project leverages continuous deployment and subsequent merges to master pre-empts any build in flight. Thus, it wasn’t until after several people had merged that we realized the build was failing (of course, I was on-call at the time).

An investigation followed and what the engineer found was that an innocous commit to bump a library had caused the failure. This library wasn’t even used in production! It was simply a feature to help developers keep their environments clean. It turns out that we were including all these packages, including ones not needed for production, in our Docker container. This version bump had increased the size of the Docker container by ~50MB, a substantial enough jump that our container failed to build within the 60s timeout. So increase the timeout to 120s and problem solved!

Of course, we shouldn’t have been including packages like this in the prod Docker container. Of course, we should have set alerts on the build time as we would have probably hit this problem eventually. But was it really worth all this effort to pull in a package that, realistically, less than 10% of the dev team uses and provides little lift?

Just an interesting example of how something seemingly innocous caused great pain.