Using eBPF for Real-Time AI Workload Threat Detection in Cloud-Native Environments

eBPF (Extended Berkeley Packet Filter) gives security engineers kernel-level observability into AI workloads without modifying application code. This post explains how to use eBPF to detect the attack classes that matter most for ML systems: data poisoning, model exfiltration, inference manipulation, and supply chain compromise — at up to ~600k events per second of ingest throughput (rules disabled, internal audit).

Why Traditional Security Fails AI Workloads

AI and ML systems running on Kubernetes expose a different attack surface than conventional web services. The threats that matter most — training data poisoning, model weight exfiltration, inference manipulation, and MLOps pipeline compromise — are largely invisible to network-perimeter tools, container image scanners, and SIEM log analysis.

Consider what a compromised ML training job looks like:

• No malicious network signature: The attacker exfiltrates model weights by encoding them in DNS queries — traffic that looks like normal DNS
• No process anomaly at image level: The container image is clean; the poisoning happens via a mounted volume with compromised training data
• No log entry: The attack modifies training data at the filesystem level before any ML framework log is written

Traditional detection misses all three. eBPF sees all three — at the kernel system call layer, before any application abstraction can hide the activity.

eBPF Architecture for AI Security Monitoring

eBPF programs attach to kernel hooks and execute in a verified, sandboxed environment. For AI workload monitoring, the relevant attachment points are:

Kernel Attachment Points for AI Security:
│
├── kprobes/kretprobes (kernel function entry/exit)
│   ├── sys_read / sys_write — I/O to model files, training data
│   ├── sys_connect / sys_sendto — Outbound network from inference pods
│   └── sys_execve — New process spawning inside ML containers
│
├── tracepoints (stable kernel trace events)
│   ├── syscalls:sys_enter_openat — File access patterns (model weight reads)
│   ├── net:netif_receive_skb — Incoming network to training pods
│   └── sched:sched_process_exec — Process execution lineage
│
├── XDP (eXpress Data Path)
│   ├── Pre-stack packet inspection at NIC driver level
│   └── Detect covert channels (DNS tunneling, ICMP payload exfiltration)
│
└── LSM hooks (Linux Security Module via eBPF)
    ├── file_open / file_permission — Fine-grained model file access control
    ├── socket_connect — Allowlist outbound connections from ML pods
    └── bpf_map_update_elem — Detect eBPF map manipulation by attacker programs

CO-RE (Compile Once — Run Everywhere): Modern eBPF programs using BTF (BPF Type Format) and CO-RE can be compiled once and deployed across kernel versions 5.8+, covering all major Kubernetes distributions (EKS, GKE, AKS, self-managed). This eliminates the kernel-version fragmentation problem that plagued earlier eBPF security tools.

The 7 AI-Specific Attack Classes eBPF Detects

Class 1: Training Data Poisoning via Filesystem

Attack pattern: Attacker compromises a mounted volume or data pipeline and modifies training data files before the ML framework reads them. The modification could inject backdoor triggers, degrade model accuracy, or bias outputs toward attacker-controlled predictions.

eBPF detection approach:

// Detect unexpected writes to training data directories
SEC("tracepoint/syscalls/sys_exit_write")
int detect_training_data_write(struct trace_event_raw_sys_exit *ctx) {
    // Get current process namespace and cgroup (container identity)
    struct task_struct *task = (struct task_struct *)bpf_get_current_task();
    
    // Check if write target is in training data path
    // Flag writes from unexpected process identities (not the expected data loader PID)
    if (is_training_data_path(file_path) && !is_authorized_writer(task)) {
        emit_alert(ALERT_TRAINING_DATA_WRITE, task, file_path);
    }
    return 0;
}

What this catches: A supply chain attack where a compromised data preprocessing library writes modified samples to the training directory. The write syscall is intercepted regardless of what the library claims to be doing at the application layer.

Class 2: Model Weight Exfiltration

Attack pattern: A compromised ML container reads model weight files and transmits them out of the cluster via DNS queries, HTTP to a lookalike domain, or ICMP payload encoding.

eBPF detection approach: Correlate sys_read on model files (.pt, .bin, .safetensors, .onnx) with subsequent sys_sendto/sys_connect calls from the same process within a 500ms window. Flag connections to external IPs not in the approved egress list.

Nora Vision implements this as a weight-read-to-egress correlation rule that fires within one ring buffer flush cycle (~50ms), well before the exfiltration completes over a TCP connection.

Detection signature:

ALERT: Model Exfiltration Attempt
  Process: python3 (PID 4821, container: ml-inference-7d9f)
  Sequence: open(/models/gpt-finetuned.bin) → read(2.1GB) → connect(185.220.101.x:443)
  Time delta: 340ms
  Model file hash: sha256:e3b0c44298fc...
  Destination: not in egress allowlist
  MITRE: T1041 (Exfiltration Over C2 Channel)

Class 3: Inference Manipulation (Adversarial Input at Runtime)

Attack pattern: Attacker submits crafted adversarial inputs to a live inference API — not to extract training data but to force specific model outputs. In a fraud detection context, this means inputs designed to make the model classify fraudulent transactions as legitimate.

eBPF detection: Monitor the inference request handler at the socket layer. Build a statistical baseline of input vector distributions per model version. Use eBPF ring buffers to stream input feature summaries to a userspace anomaly detector running as a sidecar.

Key insight: Standard application-layer logging only captures what the application chooses to log. eBPF captures the raw bytes received by the inference service socket — the adversarial input is detectable before the inference function executes, giving an opportunity to reject it before it reaches the model.

Class 4: MLOps Pipeline Compromise

Attack pattern: An attacker with access to the CI/CD pipeline (via a compromised GitHub token, Argo Workflows misconfiguration, or Kubeflow RBAC escalation) modifies the training job definition to poison the model during a scheduled re-training run.

eBPF detection:

• Attach kprobes to sys_execve inside the training namespace
• Baseline the expected process tree for a normal training job
• Alert on any sys_execve spawning a shell (/bin/sh, /bin/bash) from inside the training container during a scheduled run — legitimate ML frameworks don't spawn shells
• Detect unexpected sys_clone calls indicating fork-exec patterns inconsistent with the ML framework's process model

MITRE ATT&CK mapping: T1609 (Container Administration Command), T1610 (Deploy Container)

Class 5: Container Escape Attempts from ML Pods

ML training jobs often run with elevated privileges (GPU device access, shared memory for distributed training). This makes them high-value targets for container escape.

eBPF detects escape attempts via:

• sys_unshare with CLONE_NEWUSER flag — namespace manipulation for privilege escalation
• Write to /proc/sysrq-trigger, /sys/kernel/debug/ — kernel interface abuse
• sys_open on /host/etc/shadow, /host/proc/1/maps — host filesystem access via volume mounts
• sys_ptrace targeting PID 1 — breakout via host process attachment

eBPF LSM hooks (bpf_lsm_file_open) can block these operations in real time rather than merely alerting after the fact — converting detection into prevention.

Class 6: Covert Channel Exfiltration (DNS Tunneling)

Attack pattern: The attacker encodes model weights or training data in DNS query payloads, bypassing outbound firewall rules that permit DNS traffic.

eBPF XDP detection:

SEC("xdp")
int detect_dns_tunnel(struct xdp_md *ctx) {
    // Parse DNS query at XDP layer (pre-stack, before any socket buffer)
    struct dns_hdr *dns = parse_dns(ctx);
    
    // Flag: query name > 63 chars, or QNAME contains base64-looking subdomains
    if (dns->qdcount > 1 || get_qname_len(dns) > 63) {
        // Compute entropy of QNAME — high entropy = likely encoded data
        u32 entropy = compute_qname_entropy(dns);
        if (entropy > DNS_TUNNEL_ENTROPY_THRESHOLD) {
            emit_xdp_alert(ALERT_DNS_TUNNEL, ctx);
            return XDP_DROP; // Block before kernel networking stack
        }
    }
    return XDP_PASS;
}

XDP operates before the kernel networking stack, providing sub-100µs detection and blocking at line rate — unlike userspace packet capture which has kernel-to-userspace copy overhead.

Class 7: Supply Chain Attack on ML Libraries

Attack pattern: A compromised version of a popular ML library (torch, transformers, safetensors) contains a backdoor that activates when certain model weight patterns are loaded.

eBPF detection: Monitor dlopen and library loading syscalls. Maintain a hash allowlist of approved library versions. Alert immediately when a container loads a library version not on the allowlist — before any backdoor code can execute.

ALERT: Unapproved Library Version Loaded
  Container: training-job-a7f2
  Library: /opt/conda/lib/python3.11/site-packages/torch/lib/libtorch.so
  Hash: sha256:b8d4c91... (NOT in approved library allowlist)
  Expected: sha256:7f3a2bb... (torch==2.3.1)
  Action: Container quarantine triggered
  MITRE: T1195.002 (Compromise Software Supply Chain)

Performance: eBPF vs. Alternative Monitoring Approaches

A common concern about runtime security is performance overhead. eBPF's kernel-native execution model addresses this directly:

| Monitoring Approach | Throughput Overhead | Latency Addition | Deployment | |--------------------|--------------------|--------------------|------------| | eBPF (Nora Vision) | low CPU overhead | <10µs per syscall | Daemonset, no app changes | | Sysdig Falco (eBPF mode) | 2–5% CPU | 20–50µs | Daemonset | | Traditional HIDS (auditd) | 8–15% CPU | 100–500µs | Per-node agent | | Network TAP + DPI | 3–8% CPU + NIC cost | Not applicable | Infrastructure | | Userspace ptrace | 50–200% CPU | ms-range | Incompatible with production |

Nora Vision benchmark (AWS EC2 c6i.4xlarge, 16 vCPU, 32GB RAM):

• Syscall throughput: 250,000 events/second monitored without drops
• CPU overhead: 0.7% average under production ML training load (PyTorch DDP, 8 GPU workers)
• Memory: 48MB per node (ring buffers + eBPF maps + userspace daemon)
• Latency: P99 syscall intercept latency: 8.4µs

The low overhead means Nora Vision can be deployed on GPU training nodes without measurable impact on training throughput or GPU utilisation.

MITRE ATT&CK Coverage for AI Workloads

Nora Vision's eBPF detection rules map to 7 MITRE ATT&CK for Cloud tactics relevant to AI workloads:

| Tactic | Technique | Detection Method | |--------|-----------|-----------------| | Initial Access | T1190 (Exploit Public-Facing Application) | XDP packet inspection on inference API ports | | Execution | T1059.004 (Unix Shell) | kprobe on sys_execve inside ML containers | | Persistence | T1053.007 (Cron in Container) | tracepoint on cron/anacron spawns | | Privilege Escalation | T1611 (Escape to Host) | LSM hook on namespace operations | | Defense Evasion | T1562.006 (Disable Security Tools) | Detect writes to eBPF program maps | | Exfiltration | T1041 (Exfiltration Over C2) | Read-to-egress correlation rules | | Impact | T1565.001 (Stored Data Manipulation) | Filesystem write monitoring on model paths |

The complete MITRE ATT&CK coverage matrix for Nora Vision is published in the Nora Vision product documentation.

Deployment: Nora Vision as a Kubernetes DaemonSet

Nora Vision deploys as a privileged DaemonSet — the minimal privilege model needed for eBPF program attachment:

apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: nora-vision
  namespace: innora-security
spec:
  selector:
    matchLabels:
      app: nora-vision
  template:
    spec:
      hostPID: true        # Required for process tree visibility
      hostNetwork: true    # Required for XDP attachment
      containers:
      - name: nora-vision
        image: innora/nora-vision:latest
        securityContext:
          privileged: true   # Required for eBPF program loading
          capabilities:
            add:
            - SYS_ADMIN     # eBPF program loading
            - NET_ADMIN     # XDP attachment
            - SYS_PTRACE    # Process inspection
        resources:
          limits:
            cpu: "500m"
            memory: "256Mi"
          requests:
            cpu: "100m"
            memory: "64Mi"
        env:
        - name: NORA_ALERT_WEBHOOK
          value: "https://your-siem-endpoint/webhook"
        - name: NORA_KERNEL_MIN
          value: "5.8"
        volumeMounts:
        - name: debugfs
          mountPath: /sys/kernel/debug
        - name: modules
          mountPath: /lib/modules
          readOnly: true
      volumes:
      - name: debugfs
        hostPath:
          path: /sys/kernel/debug
      - name: modules
        hostPath:
          path: /lib/modules

Alert output goes to webhook, syslog, or direct SIEM integration (Splunk, Elastic, Chronicle). Detection rules are hot-reloadable without DaemonSet restart.

Practical: Protecting a Production LLM Inference Fleet

Concrete example: an LLM inference fleet running 40 replicas of a fine-tuned language model on Kubernetes.

Nora Vision policy configuration for LLM inference:

policies:
  - name: model-weight-protection
    description: Prevent model weight exfiltration from inference pods
    selector:
      labels:
        workload: llm-inference
    rules:
      - type: file_open
        paths: ["/models/**/*.bin", "/models/**/*.safetensors"]
        allowed_processes: ["python3", "uvicorn"]  # Only inference server can open weights
        
      - type: egress_correlation
        trigger: file_read
        path_pattern: "/models/**"
        window_ms: 1000
        alert_on: external_connection  # Flag if model read followed by external connect
        
  - name: no-shell-in-inference
    description: Inference containers should never spawn a shell
    selector:
      labels:
        workload: llm-inference
    rules:
      - type: exec_blacklist
        processes: ["/bin/sh", "/bin/bash", "/usr/bin/python3 -c"]
        action: block  # Prevent execution, not just alert
        
  - name: dns-tunnel-prevention
    description: Detect DNS-based exfiltration
    rules:
      - type: xdp_dns
        max_qname_length: 63
        entropy_threshold: 4.2
        action: drop_and_alert

With this policy active, any attempt to exfiltrate model weights — whether via direct TCP connection, DNS tunneling, or spawning a shell to use curl — is blocked within one kernel round trip, before any data leaves the pod.

Getting Started with eBPF-Based AI Security

Option 1: Deploy Nora Vision — Our managed eBPF security solution for Kubernetes. Pre-built detection rules for AI workloads, MITRE ATT&CK mapping, and alert integrations. Request access.

Option 2: Open-source foundation — Start with Tetragon (Cilium project) for basic eBPF observability. Nora Vision adds the AI-workload-specific detection logic on top of a production-hardened eBPF core.

Option 3: Security audit — If you're unsure about your AI workload's current security posture, our AI security audit service includes an eBPF-based runtime observation phase that maps your actual syscall behaviour against expected baselines.

Summary

eBPF provides the only practical approach to securing AI workloads at kernel level with production-acceptable overhead. The seven attack classes that matter most for ML systems — training data poisoning, model exfiltration, inference manipulation, MLOps pipeline compromise, container escape, DNS tunneling, and supply chain attacks — are all detectable via eBPF syscall monitoring, network-layer inspection, and LSM enforcement. At 250,000+ events/second with sub-1% CPU overhead, eBPF security monitoring can be deployed on GPU training nodes and inference fleets without impacting model performance.

Related reading:

• Nora Vision: eBPF Cloud-Native IDS
• Singapore AI Security Regulations 2026
• AI Red Teaming: LLM Security Testing Methodology
• Innora Security Research