Building Linux Intrusion Detection with auditd and Bash: A Step-by-Step Security Monitor

Most companies delay implementing intrusion detection because commercial solutions demand massive infrastructure investments and monthly fees that eclipse their entire monitoring budget. Meanwhile, attackers exploit this gap, knowing that basic log monitoring misses the system call patterns that reveal their presence.

Linux's audit framework offers a different path. With auditd and targeted bash scripts, you can build production-grade intrusion detection that catches compromise patterns heavyweight solutions miss, using tools already installed on your servers.

Understanding System Call Patterns in Normal vs Compromised Systems

Compromised systems exhibit distinct system call signatures. Normal applications follow predictable patterns: web servers open network sockets, databases read from specific paths, cron jobs execute scheduled tasks. Attackers disrupt these patterns through privilege escalation attempts, unexpected network connections, and file system modifications that bypass application logic.

The key insight is frequency analysis rather than individual event detection. A single execve call launching /bin/sh might be legitimate. Fifty such calls in rapid succession, especially from processes that normally don't spawn shells, indicates compromise.

Step 1: Configure auditd for Security-Focused System Call Logging

Install and enable the audit daemon:

sudo apt-get install auditd audispd-plugins
sudo systemctl enable auditd
sudo systemctl start auditd

Edit /etc/audit/auditd.conf with security-optimised settings:

log_file = /var/log/audit/audit.log
log_format = RAW
log_group = adm
max_log_file = 100
max_log_file_action = ROTATE
space_left = 100
space_left_action = SYSLOG
admin_space_left = 50
admin_space_left_action = SUSPEND
disk_full_action = SUSPEND
disk_error_action = SUSPEND

Step 2: Deploy Essential Audit Rules for Intrusion Detection

Create /etc/audit/rules.d/intrusion-detection.rules:

# Monitor privilege escalation attempts
-a always,exit -F arch=b64 -S execve -F euid=0 -F auid!=0 -k privilege_escalation
-a always,exit -F arch=b32 -S execve -F euid=0 -F auid!=0 -k privilege_escalation

# Track network connections from unexpected processes
-a always,exit -F arch=b64 -S connect -k network_connect
-a always,exit -F arch=b32 -S connect -k network_connect

# Monitor sensitive file access
-w /etc/passwd -p wa -k passwd_changes
-w /etc/shadow -p wa -k shadow_changes
-w /etc/ssh/sshd_config -p wa -k ssh_config
-w /root/.ssh -p wa -k root_ssh

# Detect shell spawning from web processes
-a always,exit -F arch=b64 -S execve -F exe=/bin/bash -k shell_spawn
-a always,exit -F arch=b64 -S execve -F exe=/bin/sh -k shell_spawn

Load the rules and verify:

sudo augenrules --load
sudo auditctl -l

Step 3: Build the Baseline Generation Script

Create /opt/security-monitor/generate-baseline.sh:

#!/bin/bash

BASELINE_DIR="/var/lib/security-monitor"
LOG_RETENTION="7" # days

mkdir -p "$BASELINE_DIR"

# Generate system call frequency baselines
generate_baseline() {
    local period="$1"
    local output="$BASELINE_DIR/baseline-${period}.dat"
    
    ausearch -ts "${period} days ago" -te now 2>/dev/null | \
    grep -E "type=SYSCALL" | \
    awk -F' ' '{
        for(i=1;i<=NF;i++) {
            if($i ~ /^syscall=/) {
                split($i, a, "=")
                syscalls[a[2]]++
            }
            if($i ~ /^exe=/) {
                split($i, a, "=")
                executables[a[2]]++
            }
        }
    }
    END {
        for(s in syscalls) print "syscall", s, syscalls[s]
        for(e in executables) print "executable", e, executables[e]
    }' > "$output"
}

generate_baseline "1"
generate_baseline "7"

# Clean old logs
find "$BASELINE_DIR" -name "*.dat" -mtime +"$LOG_RETENTION" -delete

Step 4: Create Real-time Anomaly Detection

Build /opt/security-monitor/detect-anomalies.sh:

#!/bin/bash

BASELINE_DIR="/var/lib/security-monitor"
ALERT_THRESHOLD="300" # percent above baseline
ALERT_EMAIL="admin@yourcompany.com"

detect_anomalies() {
    local current_data=$(ausearch -ts recent 2>/dev/null | \
        grep -E "type=SYSCALL" | \
        awk 'END {print NR}')
    
    if [[ ! -f "$BASELINE_DIR/baseline-1.dat" ]]; then
        echo "No baseline found, generating..."
        /opt/security-monitor/generate-baseline.sh
        return 0
    fi
    
    local baseline_avg=$(awk '{sum+=$3; count++} END {print int(sum/count)}' \
        "$BASELINE_DIR/baseline-1.dat")
    
    if [[ $current_data -gt $(($baseline_avg * $ALERT_THRESHOLD / 100)) ]]; then
        send_alert "System call anomaly detected: ${current_data} events vs ${baseline_avg} baseline"
    fi
}

send_alert() {
    local message="$1"
    local hostname=$(hostname)
    
    echo "SECURITY ALERT: $hostname - $message" | \
    mail -s "Intrusion Detection Alert: $hostname" "$ALERT_EMAIL"
    
    logger -t security-monitor "ALERT: $message"
}

detect_anomalies

Step 5: Optimise Performance for Production Environments

Limit audit log size and processing overhead:

# Create log rotation specifically for audit
sudo cat > /etc/logrotate.d/audit-security << 'EOF'
/var/log/audit/audit.log {
    daily
    rotate 14
    compress
    delaycompress
    missingok
    notifempty
    postrotate
        /sbin/service auditd restart > /dev/null 2>&1 || true
    endscript
}
EOF

# Set CPU limits for monitoring processes
sudo systemctl edit --force security-monitor.service << 'EOF'
[Unit]
Description=Security Anomaly Monitor
After=auditd.service

[Service]
Type=oneshot
ExecStart=/opt/security-monitor/detect-anomalies.sh
CPUQuota=20%
IOSchedulingClass=3

[Install]
WantedBy=multi-user.target
EOF

The audit framework typically adds 2-5% CPU overhead on busy systems. In production environments handling thousands of requests per second, this overhead becomes noticeable but remains acceptable given the security benefits.

Step 6: Test Your Detection System

Simulate common attack patterns to verify detection:

# Test privilege escalation detection
sudo -u www-data sudo whoami

# Test shell spawning (from web process simulation)
sudo -u www-data /bin/bash -c "whoami; id; uname -a"

# Test rapid syscall generation
for i in {1..100}; do /bin/ls /tmp > /dev/null; done

Monitor /var/log/audit/audit.log and verify your detection script identifies these patterns.

Step 7: Production Deployment and Monitoring Integration

Schedule regular monitoring:

# Add to crontab
echo "*/5 * * * * /opt/security-monitor/detect-anomalies.sh" | sudo crontab -
echo "0 2 * * * /opt/security-monitor/generate-baseline.sh" | sudo crontab -

For comprehensive infrastructure monitoring, complement this intrusion detection with standard system metrics. Hidden packet drops in network interfaces often indicate lateral movement attempts that system call monitoring alone might miss.

Compared to enterprise monitoring solutions that can cost thousands monthly, this bash-based approach provides essential security monitoring with minimal resource overhead and no recurring fees.

This lightweight security monitor integrates naturally with existing infrastructure. Server Scout's bash-based monitoring approach follows similar principles, providing comprehensive system visibility with minimal overhead - the perfect complement to your intrusion detection system.

You now have production-ready intrusion detection that monitors system call anomalies, detects privilege escalation attempts, and alerts on suspicious network activity. The entire system runs with minimal resource overhead and requires no external dependencies beyond Linux's built-in audit framework.

FAQ