Trust Domain Extensions Guest Fuzzing Framework for Security Vulnerability Detection

Eran Dahan, Itzhak Aviv () and Michael Kiperberg
Additional contact information
Eran Dahan: Research Institute for Crypto, 859423 Vienna, Austria
Itzhak Aviv: Research Institute for Crypto, 859423 Vienna, Austria
Michael Kiperberg: Modelyo, Tel Aviv Yafo 5211200, Israel

Mathematics, 2025, vol. 13, issue 11, 1-25

Abstract: The Intel ® Trust Domain Extensions (TDX) encrypt guest memory and minimize host interactions to provide hardware-enforced isolation for sensitive virtual machines (VMs). Software vulnerabilities in the guest OS continue to pose a serious risk even as the TDX improves security against a malicious hypervisor. We suggest a comprehensive TDX Guest Fuzzing Framework that systematically explores the guest’s code paths handling untrusted inputs. Our method uses a customized coverage-guided fuzzer to target those pathways with random input mutations following integrating static analysis to identify possible attack surfaces, where the guest reads data from the host. To achieve high throughput, we also use snapshot-based virtual machine execution, which returns the guest to its pre-interaction state at the end of each fuzz iteration. We show how our framework reveals undiscovered vulnerabilities in device initialization procedures, hypercall error-handling, and random number seeding logic using a QEMU/KVM-based TDX emulator and a TDX-enabled Linux kernel. We demonstrate that a large number of vulnerabilities occur when developers implicitly rely on values supplied by a hypervisor rather than thoroughly verifying them. This study highlights the urgent need for ongoing, automated testing in private computing environments by connecting theoretical completeness arguments for coverage-guided fuzzing with real-world results on TDX-specific code. We discovered several memory corruption and concurrency weaknesses in the TDX guest OS through our coverage-guided fuzzing campaigns. These flaws ranged from nested #VE handler deadlocks to buffer overflows in paravirtual device initialization to faulty randomness-seeding logic. By exploiting these vulnerabilities, the TDX’s hardware-based memory isolation may be compromised or denial-of-service attacks may be made possible. Thus, our results demonstrate that, although the TDX offers a robust hardware barrier, comprehensive input validation and equally stringent software defenses are essential to preserving overall security.

Keywords: Intel TDX; guest fuzzing; formal verification; security invariants; coverage theorems (search for similar items in EconPapers)
JEL-codes: C (search for similar items in EconPapers)
Date: 2025
References: View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www.mdpi.com/2227-7390/13/11/1879/pdf (application/pdf)
https://www.mdpi.com/2227-7390/13/11/1879/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jmathe:v:13:y:2025:i:11:p:1879-:d:1671674

Access Statistics for this article

Mathematics is currently edited by Ms. Emma He

More articles in Mathematics from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().