How a Rogue Developer Ruined Millions of Software (happened this weekend)

Publish Date: Jan 10 '22

69 26

TLDR: A software developer who made some highly used open source software, decided to go rogue and inject a bug into his software, making it usable. This affected every other dependency (and developer) using his software.

Bug Breaks my Software Deployment

Over the weekend, I was deploying some software (to Firebase) with CI/CD pipelines. But for some reason, the pipelines were failing. The failure occurred at this stage of my GitHub Actions workflow:

    - uses: FirebaseExtended/action-hosting-deploy@v0
    with:
        repoToken: '${{ secrets.GITHUB_TOKEN }}'
        firebaseServiceAccount: '${{ secrets.FIREBASE_SERVICE_ACCOUNT_ANTHONYDELLAVECCHIA }}'
        channelId: live
        projectId: anthonydellavecchia
        entryPoint: "./anthonyjdella"

This is the visual representation of my failed pipeline:

I then went over to the Firebase Extended Github repo to see if anyone else was having similar issues. And yep, many others were experiencing the same issue:

(link)

Rogue Developer, Marak

Well, it turns out, Action-hosting-deploy was using a dependency called colors, created by Marak (the rogue developer), which is a tool that colors and styles your node.js console. This npm package gets over 20 million downloads per week, so its very popular! The dependency tree for this GitHub action looks something like this:

Action-hosting-deploy
- Firebase-tools
  - cli-table
    - colors (subdependancy which is causing the issue)
- winston
  - logform
    - colors (subdependancy which is causing the issue)

So what? Well, Marak, the creator of colors (mentioned above) added some code into his project to purposely break it. He added an infinite loop to purposely break his code!

(link to Marak's evil commit)

This is very much intentional and not an accidental bug. It was malicious.

Why is Breaking his own Software Bad?

You may be wondering why breaking his own software is bad? Well, Marak knows that his software is being used by other software. So if his breaks, so will theirs. Think of it as a chain reaction. If his breaks, other software that uses it will break too. Because of "dependency hell", this affects millions of developers.

Why Did Marak Do This?

Marak was upset that corporations were using his open-source software and not paying for it. It's basically that simple. He posted an article on his blog.

How Do You Fix It?

If your software was using colors, you would have to revert to the previous (non-broken) version. But because of this developer's poor work, you should definitely use another package instead. Chalk is another alternative that is recommended.

It's really important to have a dependency management system in place for your projects. Tools like Snyk, or SonarQube will help you detect dependency issues so you can quickly resolve them.

For more information:

Thanks for reading! 🙌

For more of my articles, check out (anthonydellavecchia.com)[https://anthonydellavecchia.com]!

Comments 26 total

George JohnsonJan 10, 2022
I'm not going to start a language "turf war" but the depdency system on NPM, now it's very mature, is coming up more and more in discussions. When you're discussing a package dependency tree that counts packages in the hundreds, that's mental.
Vasilii PollockJan 10, 2022
Is there a safer way to change your piece of coding from being open-source and free of charge to a paid one rather than injecting a bug into it?
- Vincent A. CicirelloJan 10, 2022
  They could have just stopped maintaining it. Versions that were already released would still be open. Nothing you can feasibly do about that. But if they no longer wanted future versions to be open source, they could have just announced that, and offered consulting services to add features, fix bugs, etc. Albeit, there'd be no way of stopping someone else with a fork of the last MIT licensed version from continuing open source development.
  - AnthonyJan 10, 2022
    Completely agree here. Not much else you can do if older versions of your software are OSS, free of charge.
- michelemauroJan 11, 2022
  It is well within the authors' rights. It's not on him to provide you with bug-free code, it's on you to check if the code you use with an open licence is up to what you need.
  - sigzeroJan 12, 2022
    Uh...except he deliberately did it.
    - JamesJan 15, 2022
      It's still within his right's. It's MIT, it's very forward about being warranty free.
      - sigzeroJan 16, 2022
        I am not saying it isn't. I am saying he deliberately broke it knowing it would break all the projects who relied on it and had nothing to do with the kerfuffle.
        
        michelemauroJan 24, 2022
        They HAD something to do with it: they depended on it.
        Was it an aggressive move, that caused problems? yes.
        Was it within what the licence and copyright laws permit him to do? yes, too.
Vincent A. CicirelloJan 10, 2022
If they had a problem with closed source use of their software, they should have been using a copyleft license such as the GPLv3 rather than the permissive MIT license that they were using for colors. By licensing under the MIT license, the developer of colors explicitly declared that anyone can use their software for any purpose within any other software whether open or closed, etc.

Now this also highlights some important practices to employ in relation to project dependencies. First, don't use "latest" tags to pull in dependencies. Use an explicit version number instead. Second, always run all tests when upgrading dependencies. This malicious change shouldn't have gotten into as many packages/apps as easily as it did, at least not into production.
- AnthonyJan 10, 2022
  Excellent comment, well put!
  
  Many devs "by default", use the MIT license because it's so popular. But using a license like you mention is the way to go IMO. I'm kinda glad that this happened because it brings more awareness to other types of licenses.
  
  And another excellent point on my using "latest" is a bad practice! Having a dependency scanner in your pipelines would have helped alot!
- lepinekongJan 12, 2022
  I think many including myself don't want to bother with copyright but this story shows one should. Reading his story I understand his rage against big companies or startups but he should have alerted his fellows developers instead of acting badly because he's the one who is going to the pay again. I'm sad for him.
АнонимJan 10, 2022
[deleted]
- AnthonyJan 10, 2022
  Thanks for commenting. Yea you're right but any type of issue like this will bring on great discussion and people can learn from it. For example, maybe we should be using a different software license, instead of MIT as a standard.
  - Melvyn SopacuaJan 12, 2022
    It's not about the license. It's about evaluating your goals in life periodically. As with most open source software, you start out noble: to solve a problem you've solved multiple times already and helping others at the same time.
    If circumstances in your life change so that you no longer can afford to be that noble, then stop maintaining.
    
    Secondly, it's dellusional to think you can become wealthy with a tiny package that can only do one thing (say, 80% of npm's packages) - no matter how complex internally. But you are going to have millions of users, because the more abstract and smaller in scope, the easier it is to integrate into other things. If you cannot handle that "disconnect", then by all means, don't do open source (but, then also don't expect to get millions of users if you charge for it). The projects that can fund themselves with open source, through (corperate) donations are frameworks and complex tools and even then the total yearly revenue of the companies that are using your framework is many orders of magnitude larger than your rake in donations (for example, Django ~200k donations over 2021 versus Instagram / Chaturbate / Dropbox / Pinterest as some of its coroporate users).
    
    So no license is going to help you there.
    
    Then you'd have to go old school: hire programers, make software that needs to be installed and invest in anti-piracy. Then patent some of your programming principles and get noticed enough to be bought up.
    
    Or, like Marak did: create a SAAS with a pricing model, but can't afford to patent the idea, before it gets stolen.
    
    So I'm really surprised why we're blaming the license and not the fact that you can just blatantly use someone elses SAAS solution while copying it and saying it's your new invention. Not saying we need laws for that. Patent laws are bad enough as is, but why is there not more outrage and backlash towards those copy cats from Retool?
Fahmi Noor FiqriJan 10, 2022
Marak also removes faker.js from his GitHub repo
Ben CalderJan 11, 2022
I think you have misrepresented this. If you read the blog post it's clear that it's not just about corporations using his library for no money. He had just built an application on top of the library to monetise his work. A corporation came along and literally copied that application. They used their superior resources to market the copied product, effectively blowing any chance he has of profiting from his work.
You can argue that his actions are irresponsible, but there's no doubt in my mind who the bad guys are here. Whilst it's likely to be inconvenient for many people I think he has sent an important message to tech corporations: work with the OS community instead of seeing it solely as a resource to be exploited for profit.
- Melvyn SopacuaJan 12, 2022
  Well put. And this is the only way he has to shout louder than Retool's 70mil in marketing budget (obviously, none of the budget went into inspiration, innovation or originality).
PNS11Jan 11, 2022
If you don't test library upgrades before they are applied somewhere serious the malicious party is you.

If you don't have a contract with the developer maintaining a library you can't expect them to keep it maintained, backwards compatible &c. So if you just grab a MIT licensed lib, fork it and maintain your own repo, merging in whatever the upstream adds that you've reviewed and tested.

This is common sense, and not a fault on Marak's part.
- PieterjanMay 2, 2022
  Problem is that package dependencies are specified as boundaries. You write a library, and in this library you specify
  
  "peerDependencies": { "colors": "^1.3.0" }
  
  Then, when using this new library in one of your applications, NPM finds the most optimal version of this "colors" package (which for example is "1.4.0") which in this case contains the corrupted code. So to be perfect, each package should publish another version excluding this version of colors:
  
  "peerDependencies": { "colors": "^1.3.0 <1.4.0" }
૮༼⚆︿⚆༽つJan 13, 2022
Hopefully more npm packages broken in the future.
Especially a package that are trivial but still being used as dependencies on many other packages and projects.
stevesotoJan 14, 2022
I agree with the term rogue. I know this is a coder's forum.

What I would also like to point out is mental health is every bit as important as intelligence and skill. I am sure the "rogue" developer was also a frustrated and stressed developer. Maybe a depressed human.

Many projects get abandoned for life reasons. There was a good case for this coder to walk away rather than cause damage. But without support and understanding from his peers or loved ones he got damaged first and lashed out.
- PieterjanMay 2, 2022
  I think he just needed money to buy his narcotics
okyanusozJan 14, 2022
Marak is breaking every single library he has (Faker, colors, ...) because he doesn't get money from OSS...

I don't think this is the good way of shutting down a project. Why not simply archive it?
- PieterjanMay 2, 2022
  The tragic part is that apparently he did get money from some companies. So he punished them too...
Imaduddin HaetamiFeb 19, 2022
Not sure why it is breaking anyone software. Don't people use automated or manual integration test before merging his commit?

Add comment

Anthony @anthonyjdella