On May 6, 2022, a critical CVE was published for RubyGems, the primary packages source for the Ruby ecosystem.
This vulnerability created a window of opportunity for malicious actors to take over gems that met the following criteria:
Because RubyGems provides data dumps that include a lot of information, it is unfortunately relatively simple to create an automated mining process for these criteria.
Moreover, we cannot assume that this vulnerability has gone previously unnoticed by malicious actors. While it was reported by a security researcher, the investigators proceeded with the assumption of existing compromises using this vulnerability. With that in mind, we revisited all available packages on RubyGems when looking for suspicious activities.
A bug in RubyGems that allowed unauthorized actors to yank (remove) a package version without being its owner. The request would be dispatched for the controlling package actor but due to how the package version fetch was performed, the yank would be performed on a different package. The relevant code looked as follows:
Because the slug was not filtered/checked correctly, it allowed strings into the query that could reach versions of other packages.
For example, with googleapis case, the following package version could be built: slug: googleapis-common-protos-types-1.3.1 when sending common-protos-types-1.3.1 as a slug.
This effectively would remove this version of the targeted package.
Things escalate quickly with issues of such a nature. By removing all the versions, under certain circumstances the name could be available for a reuse. This means that we’ve got ourselves a great new package name available for use.
RubyGems package versions are immutable and irreplaceable by design. This means that unless there is a security incident where someone compromises RubyGems and tampers with package content, users cannot simply replace or update a given version. Versions can be removed and a new one can be uploaded, but the new one needs to have a different number.
Or does it?
What is often missed here is that a single RubyGems version is unique only within the scope of the platform on which it was released. Platforms are based on the CPU architecture, operating system type, and sometimes the operating system version. Examples include “x86-mingw32” or “java”. The platform indicates that the gem only works with a ruby built for that platform. RubyGems will automatically download the correct version for your platform.
The default platform is known as -ruby, which should work on any platform. And while you cannot replace its content, nothing prevents you from removing it and releasing a new platform-specific version with the same number. For example, you could remove karafka-testing-1.4.3 and upload karafka-testing-1.4.3-i686-linux. The version itself would be unchanged, but it would specify the platform.
Nothing prevents a malicious actor from listing all the platforms available and releasing a version per platform to make sure everyone is affected.
Actually, no. There are cases where Bundler can re-resolve dependencies despite having a lockfile. It is an expected behavior, but nonetheless it may pose both legal and security risks. More than half a year ago I wrote an article on using the –frozen flag and why it should be a standard for production, testing, staging, and actually any other non-dev environment.
With a –frozen as the default and a compromised package, you would get a message similar to this one:
“Your bundle only supports platforms [“x86_64-linux”] but your local platform is x86_64-darwin. Add the current platform to the lockfile with `bundle lock –add-platform x64-mingw32` and try again.”
While it is not descriptive in this context or state any security risk, at least it might draw attention to the fact that something has changed.
No. Not in a case as stated above Bundler might make a decision to re-resolve dependencies. New dependency means an update lockfile. Updated lockfile means a new checksum for the same version but targeting a specific platform.
As part of our ongoing initiative to help open source software communities and package registries protect all users, Mend provides intelligence derived from our Supply Chain Defender platform.
The moment we were notified about this incident, we ran an assessment using Supply Chain Defender to make sure that:
When analyzing such a case, we start with the impact assessment. Because we collect information about RubyGems in real time, we were able to check, that for the last year there were:
Because Supply Chain Defender tracks ownership transitions, we are aware of any ownership changes that occur for packages.
Because this attack requires a new owner, we could then reduce the scope to 1,101 packages.
When a regular ownership transfer happens, there should be a phase in which there are two owners:
In the case of a package takeover of that nature, there should be no transition phase. One owner disappears, and a new one appears.
Expected ownership transition flow:
Unexpected ownership transition flow:
Note: This pattern can have legitimate cases, but for us it acts as a part of the funnel.
When applying this logic to our 1,101 packages, we end up with 174 packages. Now after filtering for packages with a hyphen, we end up with 60 left. Out of those 60, only three had per platform specific releases:
Note: There were more proof of concept packages of this issue, but those did not have ownership changes and were thus irrelevant. Those packages were all of a research nature.
Just to be sure, we double-checked all 60 and did not find any malicious signatures.
Can we really assume no one noticed this earlier?
No. That is why we also checked all the packages that would have the ruby platform one yanked while having other platforms for the same version present:
SELECT versions.package_id from versions inner join ( SELECT "versions"."package_id", "versions"."number" FROM "versions" WHERE yanked_at is not null ) yanked on versions.package_id = yanked.package_id and versions.number = yanked.number where versions.yanked_at is null
This gave us 85 packages, out of which 29 had a hyphen. For those 29 we have performed a manual review and again, did not find any signs of malicious compromise.
This was also checked. We have identified 25 packages that were removed in the last two weeks, but none of them was popular enough to raise our awareness.
While this is out of scope of this investigation, we also monitor various registries for potential signs of package tampering. So far, we have not found any issues in RubyGems indicating any problems. On top of that, we have also not found any malicious packages that would anyhow correlate to this incident.
While this issue was indeed critical, as it may have caused havoc in the Ruby community, based on our data and the following investigation, to the extent of our knowledge, we have concluded that no gems were compromised and the issue was mitigated.
Mend’s automated malware detection platform, Supply Chain Defender, checks to make sure you’re only using verified package sources and prevents you from importing any malicious package into your organization or personal machine. Mend Supply Chain Defender is free to use. Sign up here >>