Socket Timing Analysis Reveals Coordinated SSH Attacks 40 Minutes Before Credential Compromise

The fail2ban logs looked completely normal. Individual IP addresses attempting 3-4 SSH connections every few minutes, well below any sensible rate limit threshold. Each attempt appeared to be standard brute force noise from different geographic regions. Nothing triggered alerts.

Forty minutes later, the attackers had successfully compromised credentials and were deploying lateral movement tools across the internal network.

The Attack Timeline - 40 Minutes of Invisible Coordination

This wasn't random brute force traffic. Analysis of /proc/net/tcp revealed a sophisticated coordination pattern across 47 source IP addresses spanning 12 countries. Each IP made precisely 3 attempts every 2-3 minutes, creating a sustained attack rate of approximately 150 attempts per hour while keeping individual IPs below traditional detection thresholds.

Initial Fail2ban Logs - Everything Looked Normal

Fail2ban's default SSH jail configuration triggers bans after 5 failed attempts within 10 minutes. The attackers stayed well below this threshold:

2026-02-15 14:23:12 sshd[12847]: Failed password for admin from 203.0.113.47 port 54321
2026-02-15 14:25:48 sshd[12891]: Failed password for root from 203.0.113.47 port 54322
2026-02-15 14:28:31 sshd[12934]: Failed password for test from 203.0.113.47 port 54323
2026-02-15 14:31:05 sshd[12978]: Connection closed by 203.0.113.47 port 54324 [preauth]

Each IP maintained this pattern - never exceeding 4 attempts per 10-minute window, never triggering rate limiting.

The Geographic Distribution Pattern

The attack originated from infrastructure across Eastern Europe, Southeast Asia, and compromised residential networks. IP addresses cycled through attempts in a pattern that suggested central coordination:

• Wave 1: IPs from Poland, Czech Republic, Romania (minutes 0-15)
• Wave 2: Malaysian and Indonesian endpoints (minutes 15-30)
• Wave 3: Compromised Irish residential broadband (minutes 30-45)

This geographic rotation provided additional cover, making the attack appear like normal internet background noise rather than coordinated activity.

Socket State Analysis Through /proc/net/tcp

While fail2ban examined individual connection attempts, /proc/net/tcp revealed the underlying socket state transitions that exposed the coordination. SSH connections create predictable socket state patterns:

awk '{if($4=="01") print $2,$3,$4}' /proc/net/tcp | \
while read local remote state; do
  echo "$(date): SSH socket $local -> $remote state $state"
done

Connection Timing Correlation Methodology

The breakthrough came from analysing socket creation timestamps. Normal brute force attacks show random timing distribution. This attack showed precise 2-3 minute intervals across all source IPs, indicating central command coordination.

Parsing /proc/net/tcp every 30 seconds revealed socket state transitions that followed an algorithmic pattern. New connections appeared exactly 167 seconds (+/- 12 seconds) after previous attempts from the same geographic cluster completed.

Identifying Coordinated Attack Signatures

Three key indicators emerged from socket analysis:

1. Temporal clustering: Connection attempts grouped into 15-minute geographic waves
2. Socket lifetime consistency: All connections maintained identical 45-second lifespans regardless of authentication outcome
3. State transition synchronisation: Multiple IPs showed identical TCP state progression timing

These patterns are invisible to application-layer monitoring but clearly visible through network socket analysis.

Building Detection Scripts for Multi-Vector Attacks

Traditional SSH monitoring focuses on successful vs failed authentication. Socket-level monitoring reveals attack coordination before credential testing begins. A monitoring system that tracks these patterns would have detected this coordination early through systematic /proc analysis.

Real-time Socket Monitoring Implementation

Effective detection requires correlating socket states across time windows longer than traditional intrusion detection:

• Track SSH socket creation rates across 60-minute sliding windows
• Monitor geographic clustering of source IPs within attack timeframes
• Alert on consistent socket lifetime patterns across multiple sources
• Detect timing synchronisation between connection attempts from different regions

Geographic IP Clustering Analysis

Socket timing analysis combined with IP geolocation data reveals coordination that individual connection logs miss. Attackers using infrastructure from hosting providers often exhibit latency patterns that correlate with their geographic distribution.

Analysing connection timing against expected network latency from each region exposed the artificial delays inserted to avoid detection.

Prevention Strategies Beyond Rate Limiting

This attack succeeded because it operated below the threshold of traditional detection methods. Prevention requires monitoring patterns that span longer timeframes and correlate seemingly unrelated events.

Adaptive Threshold Configuration

Static rate limits fail against sophisticated attackers. Dynamic thresholds based on historical connection patterns would have detected this attack within the first 15 minutes. Server Scout's alert system can be configured to track these multi-dimensional attack patterns.

Cross-Timeline Attack Detection

Effective SSH protection requires correlation across multiple timeframes:

• Real-time connection monitoring (1-5 minute windows)
• Geographic pattern analysis (15-60 minute windows)
• Historical baseline comparison (daily/weekly patterns)

This attack would have been visible in the 15-60 minute analysis window, well before credential compromise occurred.

The sophistication of this attack demonstrates why modern infrastructure needs monitoring that goes beyond traditional rate limiting. Socket-level analysis through /proc/net/tcp provides the visibility needed to detect coordinated attacks that bypass conventional security measures.

Geographic attack coordination will only become more sophisticated. Infrastructure teams need monitoring tools that can correlate network-level patterns across extended timeframes to stay ahead of these threats.

FAQ