Eight Considerations for Thwarting Malicious Packages
We’re currently seeing a concerted effort from malicious actors to attack the supply chain through intentionally malicious packages. Our recent research shows a 315 percent rise in the publication of malicious packages to open source registries such as npm and RubyGems between 2021 and the end of Q3, 2022; about 85 percent of those packages stole credentials. This trend requires an urgent shift from detection to prevention. With this in mind, here are eight considerations to arm you against the threat of malicious packages and harden your security posture.
1. Malicious packages: a different kind of threat
Malicious packages are a different kind of threat because they target you directly. Vulnerabilities may or may not be exploited. They may be accidental. Malicious packages are intentional. They’re specifically designed to disrupt or disable targets, and you need to proactively detect and block them to avoid them contaminating your code base.
You should be extra cautious about the threats they pose and what you can fall prey to, such as malicious hacks such as event-stream or Circle CI. This holds particularly true in npm or the JavaScript ecosystems — the most popular programming language for open source projects and therefore most prone to vulnerabilities and malicious packages.
Realistically, developers can’t spend tons of time reviewing every single open source component. The smart way to spend AppSec dollars is to focus on implementing a secure SDLC, keep abreast of alerts, and respond quickly to threats that arise, in line with the application security procedures that you already have in place.
2. Watch your adversaries
Attackers rely on their targets not paying full attention or keeping track of what they’re using. When you inadvertently open or download a malicious package, they gain access to your code base and data.
So watch your adversaries and understand their methods. Then you can better anticipate them and take measures to thwart them.
For example, Google’s Project Zero team releases information about the chatter they hear on the dark web concerning which vulnerabilities and packages to look for, and they release it as an intelligence feed. They aim to give users the intelligence to help take preventative measures against such threats. It’s valuable information because if you know what attackers are doing, you know what you need to fix or prevent before you fall victim to a cyberattack.
That information also helps you comprehensively check and document your software supply chain, starting with a software bill of materials (SBOM). It’s important to check frequently and regularly. You can have an exploitable vulnerability in your system for years and never be affected, but the moment that you download that malicious dependency, you’re impacted. Don’t let this happen. Be consistently vigilant throughout the software development lifecycle (SDLC).
3. Pay attention to package provenance
The community relies a lot on ecosystem providers to provide protection when issues arise. Our advice? Don’t wait for their intervention. Be proactive about identifying the provenance of packages.
For example, we recently issued a wave of malicious package notifications to our customers. In one case, a customer notified us of a false positive because there appeared to be nothing malicious in this package in the particular release 4.12 on GitHub. However, the attacker had intentionally put one lot of code on GitHub and different code in the actual package that was published.
It’s not necessary to check everything, provided you can at least verify that the code that you are consuming comes from the source it claims to come from. Then you obviate the need to analyze binaries or artifacts.
4. The importance of industry collaboration
The issue of package provenance raises the question of whether you can trust certain open source packages and their maintainers. That requires collaboration between the security industry, vendors, and researchers, to better understand what vulnerabilities are out there and what malicious packages should be removed from the repository.
Repository managers are good at removing these, but working with them to react quickly is only effective when everyone pays full attention. In the past, when researchers found malicious packages, the repositories were notified, but packages could sit there for days and get downloaded thousands of times before action was taken. We must work faster, together.
5. Keep up with best practices for the repository
Public forums like the Open Software Security Foundation (Open SSF) or the Linux Foundation have central repository groups that introduce best practices for the repository because that’s the distribution point to the global development community and an important part of keeping the whole supply chain secure.
Repositories handle a huge volume of downloads annually — billions, even trillions in some cases. Maintainers of those repositories are working towards improving their responsiveness. Maven Central is probably the model that everyone would like to emulate. And those are specifically Java open source components. They have the most rigor behind them. Once npm was acquired by Microsoft and GitHub, things improved in that repository. There is still much work to do, and that’s what the Open SSF working group is focused on.
Companies need to play their part too. For example, in Canada, Shopify has a bug bounty. Its AppSec team writes in Ruby and uses Ruby on Rails. They constantly fix bugs within RubyGems and push the fixes into the gems for anyone to use. They consider it their responsibility. However, many companies use open source but don’t give anything back.
6. The value of dependency management
Some developers might argue that there’s little point in updating dependencies to a new version of a package if their code is working acceptably. However, updating your dependencies is like going to the dentist. It’s less drastic if you do it regularly. If you do it every four years, you have to rectify four years of neglect and it’s going to hurt. In security terms, that’s technical debt. Mature software teams understand they need to stay updated.
And that’s when you get a choice about updates. When dependencies are transitive, there isn’t a choice about which are pulled in. Solving problems arising from them relies on the package maintainer to bump them. You can’t do it yourself.
In these cases, some customers feel like they lack control because a malicious dependency could hit them despite following best practices. Or, a vulnerability is disclosed and a new version is released, but they can’t get an upgrade to that version because the direct dependency hasn’t been upgraded yet.
7. Automate to handle volume
Although open source can be exposed to vulnerabilities and malicious packages, the benefits still outweigh the risks with the right security strategy and tools. Ultimately, keeping it up to date is the best practice, and that’s where automation comes into play.
Take, for example, Mend Renovate. It’s a dependency update automation tool that scans your software, discovers dependencies, automatically checks to see if an updated version exists, and helps you by submitting automated pull requests.
Importantly, a tool like Renovate can handle volume. Millions of developers downloading trillions of packages per year require tooling and automation. Automation is key, especially as attacks will get more sophisticated as we get better at detecting them.
8. Handling CI/CD pipelines to remain secure
The good news is that we now have tools that can be injected into your CI/CD pipeline to thwart malicious packages. And they’ll start to become a standard part of security tooling.
One issue to address is centralization. Companies assume centralizing open source downloading makes them more secure. But really, they’ve just centralized the path through which malicious packages can be downloaded. You still need to actively stop the malicious packages. However, developers don’t download from the same code repository. They use a range of CI/CD pipelines and tech stacks. That’s challenging to police. Ideally, everyone would use the same CI/CD but that’s unlikely, so the compromise is to use a tool that works with different CI/CD pipelines.
Evolving solutions as threats evolve
There’s no magic recipe yet to completely cure security vulnerabilities and eradicate malicious packages. You need to implement a strategy that combines an approach and tools that work best for you.
What’s undeniable is that we all face the challenges of a growing attack surface and fast-changing attack methods. This means we must all remain nimble in our responses to evolving threats. Start by using the tools and technology we currently have — but keep in mind that further improvement will come as technology evolves.
