XZ Utils Unmasked: Exposing Social Engineering Tactics and the Infiltration of a Sophisticated Backdoor
Backstory
XZ Utils, along with its underlying library liblzma, are pivotal open-source projects that facilitate lzma compression and decompression, widely integrated into numerous Linux distributions by default. These utilities hold substantial esteem within the developer community and serve as integral components throughout the Linux ecosystem.
Nearly two years ago, a developer by the name of Jia Tan began actively participating in the project, initiating pull requests aimed at rectifying bugs and enhancing functionalities, a customary practice in the realm of open-source collaboration. Over time, Jia Tan garnered trust and recognition, ultimately securing elevated privileges within the repository, starting with commit permissions and progressing to the role of release manager.
Evidently, Jia Tan employed a nuanced form of social engineering to acquire these privileges. This tactic involved utilizing fabricated accounts to inundate the original maintainer with a barrage of feature requests and bug reports, coercing the addition of another maintainer to alleviate the mounting workload.
After contributing to the codebase for approximately two years, in 2023, Jia Tan introduced several modifications to XZ, integrated into release version 5.6.0. Regrettably, concealed within these alterations was a sophisticated backdoor, compromising the integrity and security of the software.
The backdoor
The intricacies of the backdoor within XZ Utilities are notably complex. Notably, the backdoor eludes detection within the XZ GitHub repository, which is presently disabled. Employing a strategy to evade detection, the malevolent maintainer refrains from incorporating segments of the backdoor into the public git repository. Instead, these elements are exclusively included within source code tarball releases. This tactic conceals portions of the backdoor while enabling its utilization during the build process of dependent projects.
The backdoor comprises various components introduced across multiple commits:
- Utilization of IFUNCs during the build process to hijack symbol resolve functions by the malware.
- Integration of an obfuscated shared object concealed within test files.
- Execution of a script during the library’s build process to extract the shared object, solely included in releases and added to .gitignore.
- Disablement of landlocking, a security feature designed to restrict process privileges.
The execution chain encompasses several stages:
- Execution of the malicious script, build-to-host.m4, during the library’s build process, decoding the “test” file bad-3-corrupt_lzma2.xz into a bash script.
- Subsequent decoding process within the bash script for another “test” file, good-large_compressed.lzma, yielding another script.
- Extraction of a shared object, liblzma_la-crc64-fast.o, incorporated into the compilation process of liblzma.
This convoluted process may be challenging to decipher. For clarity, we recommend referring to Thomas Roccia’s infographic for visual guidance and comprehensive analysis.
The shared object itself is compiled into liblzma, supplanting the conventional function name resolution process. During process loading, function names are resolved into pointers within the process memory, referencing binary code. The malevolent library disrupts this function resolving process, enabling the replacement of the function pointer for the OpenSSH function RSA_public_decrypt. This redirection directs the function to a malicious counterpart, as detailed in research by Filippo Valsorda. This nefarious action intercepts commands from the authenticating client’s certificate, subsequently passing them to the system() function for execution, thereby achieving Remote Code Execution (RCE) prior to authentication.
Potential impact
The discovery of the backdoor reveals a critical vulnerability within the SSH daemon on affected machines, providing a gateway for remote attackers to execute arbitrary code. This poses a significant threat to any system hosting the vulnerable package and exposing SSH services to the internet, potentially compromising the security of a wide range of machines.
The severity of this backdoor nearly rivaled that of the SolarWinds incident, posing a substantial risk of widespread intrusion. If exploited, attackers would have gained immediate access to Linux machines utilizing infected distributions, including but not limited to Fedora, Ubuntu, and Debian. The scale of potential compromise was immense.
Fortunately, the catastrophic implications were thwarted by Andres Freund. During investigation into a latency issue following a software update, Freund traced the anomaly to the xz package, ultimately uncovering the backdoor’s existence.
This revelation prompts significant apprehension. The detection of the backdoor relied on fortuitous discovery by a vigilant engineer. Had it gone unnoticed, the duration of its activity remains a troubling unknown.
Moreover, the possibility of previous occurrences raises profound concerns regarding the efficacy of security measures and the extent of prior compromises.
Detection and Mitigation:
In response to the identified backdoor within XZ Utils, the Cybersecurity and Infrastructure Security Agency (CISA) advocates for immediate action to mitigate the risk. Their recommended approach entails downgrading to a verified, uncompromised version, specifically version 5.4.6.
Version Control:
To determine the version of XZ Utils or liblzma present on your systems and ensure prompt action, the following query can be executed within Akamai Guardicore Segmentation Insight. This query is designed to identify loaded instances of the liblzma library, facilitating swift identification of potentially compromised versions
SELECT DISTINCT path AS liblzma_path
FROM process_memory_map
WHERE LOWER(path) LIKE “%liblzma%”
Alternatively, a supplementary query can be executed to identify the package manager associated with the installed version. This information is invaluable for swift and efficient remediation efforts.
SELECT name AS vulnerable_item, 'DEB' AS type, version
FROM deb_packages
WHERE (LOWER(name) LIKE '%xz-utils%' OR LOWER(name) LIKE '%liblzma%')
UNION
SELECT name AS vulnerable_item, 'RPM' AS type, version
FROM rpm_packages
WHERE (LOWER(name) LIKE '%xz-utils%' OR LOWER(name) LIKE '%liblzma%')
you can also filter to show only vulnerable assets.
SELECT path AS vulnerable_item, "Loaded Library" AS type, '5.6%' AS version
FROM process_memory_map
WHERE LOWER(path) LIKE "%liblzma%5.6%"SELECT name AS vulnerable_item, 'DEB' AS type, version
FROM deb_packages
WHERE (LOWER(name) LIKE '%xz-utils%' OR LOWER(name) LIKE '%liblzma%')
AND version LIKE '5.6.%'
UNION
SELECT name AS vulnerable_item, 'RPM' AS type, version
FROM rpm_packages
WHERE (LOWER(name) LIKE '%xz-utils%' OR LOWER(name) LIKE '%liblzma%')
AND version LIKE '5.6.%'Given that the backdoor facilitates the execution of system commands rather than solely enabling authentication, it presents an opportunity for detection through process tracking mechanisms.
Traditionally, during the login process, a new shell is spawned for the logging user, initiating the default shell process such as bash. However, in the case of this backdoor, the malicious command is executed by the SSH daemon process, sshd, potentially triggering an anomaly.
Kill Switch:
Notably, certain analyses of the backdoor suggest the presence of an environment variable-based kill switch. It is purported that appending the key “yolAbejyiejuvnup=Evjtgvsh5okmkAvj” to the system’s environment variables may deactivate the backdoor. This action could serve as an immediate mitigation measure pending further investigation and remediation efforts.
