🔐

Kubernetes RBAC Violations That kubectl Never Shows: Building Permission Audit Detection Through Linux System Calls

· Server Scout

Most Kubernetes security teams rely on kubectl audit logs to track permission violations, believing they have comprehensive visibility into RBAC enforcement. They don't. The Kubernetes API server only logs requests that reach it, but containers can escalate privileges through direct system calls that completely bypass kubectl and the audit system.

The real security gap lies below the orchestration layer. When processes inside containers call setuid(), execve(), or access the container runtime socket directly, standard Kubernetes logging sees nothing. These operations modify the underlying Linux process security context without triggering any API server events.

The RBAC Visibility Gap in Standard Kubernetes Logging

Kubernetes RBAC operates at the API level, controlling which users can perform specific operations on cluster resources. However, several attack vectors remain invisible to standard audit logging:

  • Direct container runtime API calls through Docker or containerd sockets
  • Privilege escalation through Linux capabilities that containers shouldn't possess
  • Process spawning with elevated UIDs that bypass pod security contexts
  • File system permission changes that grant access to host resources

These operations modify the security posture of running containers without generating any kubectl-visible events. The Kubernetes control plane remains completely unaware that privilege boundaries have been violated.

Setting Up Linux Auditd Rules for Container Runtime Monitoring

First, configure auditd to monitor system calls that indicate privilege changes. Add these rules to /etc/audit/rules.d/kubernetes.rules:

# Monitor privilege escalation system calls
-a always,exit -F arch=b64 -S setuid,setgid,setreuid,setregid,setresuid,setresgid -k k8s_privilege_escalation

# Track container runtime socket access
-w /var/run/docker.sock -p rwxa -k k8s_runtime_socket
-w /run/containerd/containerd.sock -p rwxa -k k8s_runtime_socket

# Monitor process status changes
-w /proc/ -p wa -k k8s_proc_monitor

Restart auditd with service auditd restart. These rules capture every attempt to change process privileges or access container runtime APIs, regardless of whether kubectl is involved.

Configuring System Call Filters for Kubernetes Operations

Add specific monitoring for execve() calls that might spawn privileged processes:

auditctl -a exit,always -F arch=b64 -S execve -F success=1 -k k8sexecmonitor

This tracks successful process execution, allowing you to identify when containers spawn processes with unexpected privilege levels. The success=1 filter reduces noise by ignoring failed execution attempts.

Analysing /proc Filesystem Changes During Permission Escalation

The /proc filesystem reveals privilege escalation attempts that occur after initial container startup. Monitor /proc/PID/status files for changes to Uid, Gid, and Groups fields that indicate privilege modifications.

Create a monitoring script that checks process credentials against their expected container security context. Parse /proc/PID/loginuid to track the original user context - this field persists even after privilege escalation, revealing the audit trail that attackers cannot erase.

Identifying Suspicious Process Tree Modifications

Examine /proc/PID/children and /proc/PID/task to identify process spawning patterns that violate pod security policies. Legitimate Kubernetes processes follow predictable parent-child relationships, whilst privilege escalation attacks often create unusual process hierarchies.

Monitor /proc/PID/fd for file descriptors that provide access to host resources outside the container's permitted scope. Containers that suddenly gain access to host filesystem paths or network namespaces indicate successful privilege escalation.

Building Automated Detection Scripts

Create a bash script that correlates auditd events with /proc filesystem analysis. Parse audit logs for privilege-related system calls, then cross-reference with container metadata to identify violations:

#!/bin/bash
# Parse auditd logs for Kubernetes privilege violations
ausearch -k k8s_privilege_escalation --format csv | while IFS=',' read timestamp pid uid syscall; do
  # Check if PID belongs to container process
  if [[ -f "/proc/$pid/cgroup" ]]; then
    container_id=$(grep -o 'docker-[a-f0-9]*' "/proc/$pid/cgroup" | head -1)
    echo "Privilege escalation detected: PID $pid, Container $container_id"
  fi
done

This correlates audit events with running containers, identifying which specific pods experienced privilege violations.

Parsing Audit Logs for RBAC Bypass Patterns

Search for sequences of system calls that indicate coordinated attacks. A typical bypass pattern involves:

  1. Container runtime socket access (docker.sock or containerd.sock)
  2. Process creation with elevated privileges (execve with setuid)
  3. File system permission changes (chmod, chown)
  4. Network namespace modifications (unshare, setns)

Identify these patterns through temporal correlation of audit events occurring within short time windows from the same container.

Integration with Existing Kubernetes Security Workflows

Integrate detection results with your existing alert infrastructure. Server Scout's monitoring capabilities can track these custom security metrics alongside standard infrastructure health checks, providing unified visibility across both system-level and application-level security events.

Forward audit findings to your incident response system using webhook notifications. Include container metadata (pod name, namespace, image) alongside the privilege violation details to enable rapid threat containment.

This approach complements our comprehensive security monitoring strategies by extending detection capabilities into the container orchestration layer. The lightweight nature of bash-based detection scripts aligns with Server Scout's zero-dependency philosophy whilst providing enterprise-grade security visibility.

For teams managing complex infrastructures, this system-level approach provides the infrastructure monitoring depth needed to secure modern Kubernetes deployments without relying on heavyweight security platforms.

Linux kernel-level monitoring through auditd and /proc analysis provides the comprehensive security visibility that standard Kubernetes RBAC logging cannot deliver. By implementing these detection mechanisms, security teams gain complete oversight of privilege escalation attempts, regardless of whether they use legitimate kubectl commands or attempt to bypass the API server entirely. The resulting security posture protects against the sophisticated attacks that traditional Kubernetes security tools simply cannot see.

FAQ

Will auditd monitoring impact container performance significantly?

Minimal impact when properly configured. The key is using specific syscall filters rather than monitoring all system activity. Focus on privilege-related calls (setuid, execve) and container runtime access patterns.

How do I distinguish between legitimate privilege changes and malicious escalation?

Cross-reference audit events with pod security contexts and expected application behaviour. Legitimate privilege changes typically occur during container startup, whilst attacks happen during runtime after initial deployment.

Can this approach detect privilege escalation in managed Kubernetes services like EKS or GKE?

Yes, but with limitations. You can monitor worker nodes where you have access, but managed control plane components remain outside your audit scope. Focus on monitoring your application pods and any privileged system containers you control.

Ready to Try Server Scout?

Start monitoring your servers and infrastructure in under 60 seconds. Free for 3 months.

Start Free Trial