The 20-Minute Window
A skilled botnet doesn't hammer one server until fail2ban triggers. Instead, it spreads authentication attempts across your entire infrastructure, staying just below each system's individual rate limits while collectively probing for weak credentials across dozens of hosts.
The pattern becomes visible only when you correlate logs from multiple servers. A single IP might hit server A twice, server B once, then server C three times - perfectly innocent when viewed individually, but clearly coordinated when seen together. This geographic clustering creates a critical 20-minute window between campaign initiation and successful compromise.
How Geographic Attack Clustering Works
Botnet operators understand that most SSH monitoring operates per-server. They'll configure their networks to rotate through your infrastructure systematically:
- IP 192.168.1.100 hits Dublin server at 14:23:15
- Same IP hits Cork server at 14:26:42
- Different IP from same subnet hits Amsterdam server at 14:28:11
Each individual server sees normal failed login patterns. Your centralised view reveals the coordination.
The 20-Minute Window Before Compromise
Successful SSH campaigns follow a predictable timeline. Initial reconnaissance spreads across 3-5 servers over the first 10 minutes, testing common usernames. If they find responsive systems, attackers focus their credential attacks on the most promising targets during minutes 15-20. After 20 minutes, they typically achieve their first successful login.
This timing gives you a narrow but reliable detection window - if your correlation can process logs fast enough.
Setting Up Cross-Server Log Correlation
First, configure log forwarding from all SSH-enabled systems to a central collection point. Most teams use rsyslog for this:
# Add to /etc/rsyslog.conf on each server
*.* @@central-log-server:514On your central system, create a simple correlation script that tracks authentication patterns across the fleet. The script should group failed attempts by source IP and correlate timing patterns across multiple destination servers.
Real-Time Pattern Recognition Scripts
Your correlation logic needs to identify three key patterns:
Distributed timing: Multiple servers receiving attempts from the same IP within a 5-minute window Geographic clustering: IPs from the same subnet hitting different infrastructure locations Username consistency: The same username tested across multiple systems
When all three patterns align, you're likely seeing a coordinated campaign rather than isolated attack attempts.
Practical Detection Implementation
Log Aggregation Setup
Configure your central log server to separate SSH authentication events from general system logs. Create a dedicated processing pipeline that can handle the volume from your entire server fleet without creating bottlenecks.
For teams managing 20+ servers, consider implementing a simple alert threshold: any IP appearing in authentication logs from 3 or more systems within 10 minutes triggers investigation.
Alert Threshold Configuration
Set your geographic correlation alerts to fire faster than traditional fail2ban rules. While individual servers might use a 10-minute ban window, your cross-server detection should alert within 5 minutes of pattern recognition.
This creates an early warning system that gives you time to implement coordinated blocks across your infrastructure before the attackers achieve credential success.
Case Study: Identifying a Live Botnet Campaign
Last month, a hosting company's correlation system detected an interesting pattern. Between 14:20 and 14:35, eight different IPs from a /24 subnet attempted SSH connections across 12 servers in their Dublin and Cork datacentres.
Individually, each server saw 2-3 failed attempts - well below their fail2ban thresholds. The correlation revealed the larger picture: a coordinated campaign testing common service accounts across their entire infrastructure.
By minute 18, they'd implemented subnet-level blocks across all systems. The attackers never achieved successful authentication because the correlation caught the pattern before individual systems triggered their local defenses.
Building Monitoring System Redundancy: A Complete Multi-Region Alert Infrastructure Guide covers the log forwarding architecture needed for this type of correlation.
For teams implementing security monitoring workflows, Complete Monitoring Implementation Guide: From Zero Infrastructure Visibility to Production-Ready Team Workflows provides the team coordination frameworks that make centralized threat detection practical.
Server Scout's unified dashboard includes SSH connection pattern tracking that correlates authentication attempts across your entire fleet, providing the geographic clustering detection that fail2ban's per-server approach cannot achieve. The system automatically identifies coordinated campaigns and provides 20-minute early warning before credential compromise typically occurs.
Building SSH Attack Pattern Recognition: Step-by-Step Geographic Clustering Detection That Provides 20-Minute Warning Before Credential Compromise offers detailed implementation steps for teams building their own correlation systems.
Traditional security tools focus on individual system protection. Modern threats require infrastructure-wide visibility. The auditd framework provides the low-level logging needed to track authentication patterns that span multiple systems.
FAQ
Can geographic correlation cause false positives from legitimate distributed teams?
Yes, but legitimate access typically shows consistent usernames and successful authentications mixed with failures. Botnet campaigns show random username attempts with no successful logins across multiple systems.
How much log storage does cross-server correlation require?
SSH authentication logs are relatively small - typically 200-500 bytes per attempt. A 50-server infrastructure generates roughly 10-20MB of SSH logs daily, manageable for most centralised systems.
Does this detection method work with key-based authentication?
Geographic correlation works best with password authentication failures. Key-based systems generate fewer correlation points, but you can still detect reconnaissance patterns through connection timing analysis.