Detecting GPU Model Weight Tampering from ROME-Class Memory Attacks

Security vendor Stealthium published research detailing detection techniques for Rank-One Model Editing (ROME) attacks, where malicious code modifies neural network weights directly within GPU memory during inference. Because these attacks operate entirely inside standard Python and CUDA execution contexts without altering files on disk, spawning child processes, or triggering unusual network connections, traditional endpoint and application detection tools fail to register anomalous activity. In a laboratory demonstration using PyTorch, an imported malicious package applied a targeted weight edit to an in-memory transformer upon entering evaluation mode, altering model answers while leaving checkpoints intact. To address the visibility gap, Stealthium demonstrated runtime telemetry that links CUDA kernel launches to originating Python call stacks, flagging unusual callers and anomalous math kernels. The vendor also presented ongoing research monitoring framework-level signals to capture in-place tensor modifications alongside the execution paths responsible for them.

Key takeaways

• ROME-class attacks modify targeted model weights directly in GPU memory, altering inference outputs without changing model files on disk or generating detectable OS-level process anomalies.
• Traditional host EDR and ADR systems fail to detect in-memory tampering because the malicious operations consist of ordinary CUDA kernel launches executed by trusted Python processes.
• Stealthium's sensor records CUDA kernel launches and library loads with associated Python call stacks, allowing responders to identify unusual callers and abnormal computation kernels.
• Ongoing research tracks framework-level in-place tensor write events, capturing the exact call stack, process, and tensor identity when model weights are altered at runtime.