The technical architecture of kernel anti-cheat software is one of the more interesting corners of Windows systems programming. The engineering is genuinely clever, the mechanisms are well-documented in Microsoft’s own WDK, and the arms race between cheat developers and anti-cheat vendors has pushed both sides into territory that illuminates real constraints in how trust works on a general-purpose PC.
This deep-dive into kernel anti-cheat mechanics covers the foundational architecture clearly. What I want to do here is trace where the escalation leads, specifically the DMA problem that no software running inside the OS can fully solve, and the security irony of using kernel drivers as your protection mechanism.
Privilege Rings and Why They Matter
The x86 processor architecture defines four privilege rings (0 through 3), though Windows uses only two in practice. Ring 3 is user mode: where your game, your browser, and every ordinary application run. Ring 0 is kernel mode: where device drivers and the Windows kernel itself operate. Code in ring 3 cannot directly inspect or manipulate kernel structures, but ring 0 code can access any user-mode memory freely.
Early cheats were user-mode programs. They called OpenProcess to get a handle to the game process, then used ReadProcessMemory and WriteProcessMemory to extract enemy positions or modify values. The fix for this was simple in principle: protect the game process, strip dangerous access rights from any handles another process tried to open.
The escalation happened when cheat authors moved into ring 0 themselves. A kernel driver does not need OpenProcess at all. It can call MmCopyVirtualMemory directly, or walk page table entries manually, reading any process’s memory without going through the handle system. Anti-cheat vendors responded by moving their own code into ring 0, which is how we arrived at the current situation where installing a game often means installing a kernel driver that starts at boot.
What the Callbacks Actually Do
The Windows kernel exposes a documented set of notification callback APIs that security software can register with. These are the primary tools anti-cheats use.
PsSetCreateProcessNotifyRoutine lets a driver receive a callback each time a process starts or exits. This is how anti-cheats know to begin monitoring when the game launches, and can terminate suspicious processes before they fully initialize.
PsSetLoadImageNotifyRoutine fires when any PE image is mapped into any process, including injected DLLs. An anti-cheat can inspect the image path, check its signature, and take action before execution reaches the injected code.
ObRegisterCallbacks is the mechanism with real teeth. It allows a driver to intercept object manager operations, specifically handle creation and duplication for process and thread objects. When cheat software calls OpenProcess(PROCESS_VM_READ, FALSE, game_pid), the anti-cheat callback fires first and can strip the requested access before the call returns. The caller gets a handle, but it lacks the rights needed to do anything useful.
A simplified illustration of this pattern:
// Register a pre-operation callback on process objects
OB_OPERATION_REGISTRATION opReg = {0};
opReg.ObjectType = PsProcessType;
opReg.Operations = OB_OPERATION_HANDLE_CREATE
| OB_OPERATION_HANDLE_DUPLICATE;
opReg.PreOperation = ProtectionPreCallback;
OB_CALLBACK_REGISTRATION cbReg = {0};
cbReg.Version = OB_FLT_REGISTRATION_VERSION;
cbReg.OperationRegistrationCount = 1;
cbReg.OperationRegistration = &opReg;
ObRegisterCallbacks(&cbReg, &RegistrationHandle);
// Inside the callback:
VOID ProtectionPreCallback(
PVOID RegistrationContext,
POB_PRE_OPERATION_INFORMATION Info)
{
if (Info->Object == ProtectedProcess) {
Info->Parameters->CreateHandleInformation.DesiredAccess &=
~(PROCESS_VM_READ | PROCESS_VM_WRITE | PROCESS_VM_OPERATION);
}
}Registry callbacks via CmRegisterCallback round out the toolkit, letting drivers watch for key reads that cheat software uses to detect whether an anti-cheat is present before deciding to load.
None of this is anti-cheat-specific. These APIs exist in the WDK for security products generally, and antivirus software has used the same callbacks for years. What anti-cheats add is the specific policy: protect the game process’s memory from being read, and monitor for indicators of tampering.
The End of SSDT Hooking
Older anti-cheats and older cheats both relied on SSDT hooking: modifying the System Service Descriptor Table to intercept syscalls at the kernel level. Microsoft closed this on 64-bit Windows with PatchGuard (Kernel Patch Protection), introduced in Windows Vista x64. PatchGuard runs periodically, checksums the SSDT along with the IDT, GDT, and other critical structures, and triggers a CRITICAL_STRUCTURE_CORRUPTION bugcheck (0x109) if it detects unauthorized modification. Both sides had to abandon the technique on any supported platform.
This was a meaningful stabilization. It forced both cheat developers and anti-cheat vendors to work through documented interfaces rather than patching kernel internals, which made the entire space more legible even if it did not end the arms race.
The DMA Ceiling
Here is where the callback-based model runs into a structural limit it cannot engineer around.
PCIe devices on a system can, by design, access physical memory directly through DMA without involving the CPU or operating system. This is how network cards, storage controllers, and GPUs move data efficiently. A PCIe card with the right firmware can issue DMA reads to any physical memory region, extracting whatever is there, and the operating system has no visibility into these transfers by default.
DMA-based cheating hardware exploits this directly. Using FPGA boards or commercial PCIe DMA cards, cheat authors build devices that masquerade as innocuous hardware and read the game process’s physical memory pages. The cheat software itself runs on a separate machine, communicating with the DMA hardware over USB or a network link. From the game’s perspective, from the anti-cheat driver’s perspective, nothing is happening. No process is running, no handle is open, no syscall is issued.
The open-source project pcileech-fpga by Ulf Frisk is the reference implementation of the underlying technique. It was published as a forensics and research tool, and it demonstrates exactly how DMA access to physical memory works at the hardware level. Cheat hardware borrows from the same principles.
The correct defense against DMA attacks is IOMMU enforcement. Intel’s VT-d and AMD’s AMD-Vi are IOMMU implementations that let the OS restrict which physical memory regions each PCIe device can access, effectively sandboxing DMA traffic per device. Game consoles use IOMMU enforcement as a matter of course because they control the entire hardware stack. Consumer PCs support IOMMU in the silicon and firmware, but enforcing it in a way that actually catches malicious PCIe hardware requires the OS to configure it aggressively, which general-purpose Windows does not do for gaming workloads.
Riot’s Vanguard requires TPM 2.0 and Secure Boot, which establish a hardware root of trust and prevent boot-level compromise. These requirements generated significant controversy when they were announced, since they imposed hardware requirements on players and, in the case of the always-on kernel driver requirement, sparked legitimate questions about what ring-0 access implies for privacy. But neither TPM nor Secure Boot addresses DMA attacks from a PCIe device that spoofs its PCI configuration space to look like a known-good network adapter. An IOMMU that is not enforcing per-device memory isolation provides no protection against it.
The Vulnerability Surface Problem
The security argument that deserves more attention in these conversations is what happens when the protection mechanism itself has bugs.
A kernel driver with an arbitrary read/write vulnerability is far more dangerous than a vulnerable user-mode application. Exploiting a kernel bug typically produces immediate ring-0 code execution with no privilege escalation step needed. This is the basis for Bring Your Own Vulnerable Driver (BYOVD) attacks, where malware bundles a legitimately signed but vulnerable driver, loads it, exploits it to gain kernel execution, and then unloads it.
Anti-cheat drivers make attractive BYOVD targets for a specific reason: they are distributed to very large numbers of machines, they often run at boot before the user does anything, and they carry Microsoft’s WHQL signature or an EV code signing certificate that makes them load without complaint even on machines with Driver Signature Enforcement enabled. A vulnerability in an anti-cheat driver is, in practice, a vulnerability in the security posture of every machine that has the associated game installed.
Multiple anti-cheat systems have carried kernel read/write vulnerabilities that were exploited in exactly this way. The irony is complete: software installed to harden a game’s execution environment has been used by malware to gain the kernel access the anti-cheat was designed to prevent.
HVCI (Hypervisor-Protected Code Integrity, part of Windows Virtualization Based Security) addresses part of this. Under HVCI, code integrity enforcement runs inside a hypervisor context that the kernel itself cannot tamper with. New kernel-mode code cannot execute unless it was signed and the hypervisor approves it before page protections are locked. This defeats the most common kernel exploit payloads that involve writing shellcode into kernel memory and executing it. Vanguard now requires HVCI on supported hardware configurations. The tradeoff is compatibility: HVCI requires that every loaded driver meet its requirements, and some older hardware drivers do not.
Where This Leaves Things
Kernel anti-cheat has raised the cost of cheating substantially for casual users, and that is worth something. User-mode cheats are effectively eliminated on well-protected titles. Kernel-mode cheats require signing workarounds and careful engineering. But the ceiling is DMA hardware, which remains outside what software-based enforcement can reliably detect without IOMMU cooperation, and the security cost of deploying millions of ring-0 drivers with imperfect security review is real and ongoing.
The path forward probably involves hardware attestation becoming meaningful at the platform level. Microsoft Pluton, which moves TPM functionality into the processor die and allows remote attestation of the execution environment, could eventually enable a model where the game server verifies that the client is running an approved, unmodified environment before accepting connections. That shifts the enforcement from the local OS to the network boundary, which is harder to spoof. It also requires hardware that is only beginning to appear in consumer systems, and depends on the attestation chain remaining trustworthy, which is its own security problem.
The anti-cheat arms race is fundamentally a question of whether a general-purpose, user-controlled PC can be made to reliably enforce constraints on its own execution. Game consoles answer this by giving the manufacturer control of the hardware. PCs answer it by giving the user control, which is the property that makes them valuable and makes anti-cheat a permanent engineering problem rather than a solved one.