SSH Hardening: 12 Steps to Secure Your Linux Server

SSH is the gateway to your Linux server. Every server exposed to the internet receives thousands of brute-force login attempts daily — automated bots scanning for weak passwords, default credentials, and misconfigured services. A single compromised SSH session gives an attacker full control. This guide walks you through 12 concrete steps to harden SSH, each with the exact configuration changes you need to make.

Before you begin: Always keep a separate terminal session open while making SSH configuration changes. If you misconfigure something, you can use the existing session to fix it. Locking yourself out of a remote server is a real risk, and it is entirely preventable.

Step 1: Disable Root Login

The root user exists on every Linux system, making it the primary target for brute-force attacks. Disable direct root login and use a regular user with sudo instead.

Edit /etc/ssh/sshd_config:

PermitRootLogin no

If you need root access for automated processes (like backups), use the more restrictive option:

PermitRootLogin prohibit-password

This allows root login only with SSH keys, never with a password.

Step 2: Use Key-Based Authentication

SSH keys are fundamentally more secure than passwords. A 256-bit Ed25519 key is virtually impossible to brute-force, whereas even a strong password can be guessed given enough time.

Generate a key pair on your local machine:

ssh-keygen -t ed25519 -C "yourname@workstation"

Copy the public key to the server:

ssh-copy-id -i ~/.ssh/id_ed25519.pub user@your-server-ip

Verify that key-based login works before proceeding to step 3:

ssh -i ~/.ssh/id_ed25519 user@your-server-ip

Tip: Use a passphrase when generating your key. Without a passphrase, anyone who gains access to your private key file can log in to every server that trusts it. Use ssh-agent to avoid retyping the passphrase on every connection.

Step 3: Disable Password Authentication

Once key-based authentication is confirmed working, disable passwords entirely.

Edit /etc/ssh/sshd_config:

PasswordAuthentication no
ChallengeResponseAuthentication no
UsePAM no

Warning: If you disable password authentication before setting up key-based login, you will be locked out of your server. Always verify key-based login works first using a separate terminal session.

Step 4: Change the Default SSH Port

Moving SSH off port 22 eliminates the vast majority of automated scanning bots. This is not security through obscurity alone — it is noise reduction that makes your logs cleaner and reduces attack surface.

Edit /etc/ssh/sshd_config:

Port 2222

Choose a port above 1024 and below 65535. Common choices include 2222, 2345, or a random high port.

Update your firewall to allow the new port before restarting SSH:

sudo ufw allow 2222/tcp
sudo ufw deny 22/tcp

Connect using the new port:

ssh -p 2222 user@your-server-ip

Step 5: Use SSH Protocol 2 Only

SSH Protocol 1 has known cryptographic weaknesses. While modern OpenSSH versions default to Protocol 2, explicitly enforcing it adds defense in depth.

Edit /etc/ssh/sshd_config:

Protocol 2

On recent versions of OpenSSH (7.0+), Protocol 1 support has been removed entirely, but adding this line does not hurt and protects against downgrade scenarios on older systems.

Step 6: Limit User Access with AllowUsers and AllowGroups

Restrict which users can log in via SSH. This is especially important on multi-user servers:

# Allow specific users
AllowUsers deployer admin

# Or allow a group
AllowGroups sshusers

To use the group-based approach, create the group and add your users:

sudo groupadd sshusers
sudo usermod -aG sshusers deployer
sudo usermod -aG sshusers admin

Only members of sshusers will be able to log in via SSH. All other system users (including service accounts) are automatically denied.

Step 7: Set Idle Timeout

Idle SSH sessions are a security risk — an unattended terminal can be exploited by someone with physical access to the workstation.

Edit /etc/ssh/sshd_config:

ClientAliveInterval 300
ClientAliveCountMax 2

This sends a keep-alive message every 300 seconds (5 minutes). If the client fails to respond after 2 attempts, the session is disconnected. Total idle timeout: 10 minutes.

Step 8: Use Strong Key Algorithms

Restrict SSH to modern, secure algorithms and disable weak ones:

# Host key algorithms
HostKey /etc/ssh/ssh_host_ed25519_key
HostKey /etc/ssh/ssh_host_rsa_key

# Key exchange algorithms
KexAlgorithms curve25519-sha256,[email protected],diffie-hellman-group16-sha512,diffie-hellman-group18-sha512

# Ciphers
Ciphers [email protected],[email protected],[email protected]

# MACs (Message Authentication Codes)
MACs [email protected],[email protected]

Remove small Diffie-Hellman moduli (less than 3072 bits):

sudo awk '$5 >= 3071' /etc/ssh/moduli > /etc/ssh/moduli.safe
sudo mv /etc/ssh/moduli.safe /etc/ssh/moduli

This prevents downgrade attacks that exploit weak key exchange parameters.

Step 9: Install and Configure Fail2ban

Fail2ban monitors log files and bans IP addresses that show malicious patterns (like repeated failed login attempts).

sudo apt update
sudo apt install fail2ban -y

Create a local configuration file (never edit the main jail.conf directly):

sudo nano /etc/fail2ban/jail.local

[DEFAULT]
bantime = 3600
findtime = 600
maxretry = 3
banaction = ufw

[sshd]
enabled = true
port = 2222
filter = sshd
logpath = /var/log/auth.log
maxretry = 3
bantime = 86400

This configuration bans an IP for 24 hours after 3 failed SSH attempts within 10 minutes. Adjust the port value to match your custom SSH port from Step 4.

Start and enable fail2ban:

sudo systemctl enable fail2ban
sudo systemctl start fail2ban

Check banned IPs:

sudo fail2ban-client status sshd

Step 10: Configure Firewall Rules with UFW

Use Ubuntu’s Uncomplicated Firewall (UFW) to restrict network access:

# Set default policies
sudo ufw default deny incoming
sudo ufw default allow outgoing

# Allow SSH on custom port
sudo ufw allow 2222/tcp comment 'SSH'

# Allow other services as needed
sudo ufw allow 80/tcp comment 'HTTP'
sudo ufw allow 443/tcp comment 'HTTPS'

# Enable the firewall
sudo ufw enable

# Verify rules
sudo ufw status verbose

For additional security, limit SSH connections to specific IP addresses:

# Allow SSH only from your office IP
sudo ufw allow from 203.0.113.50 to any port 2222 proto tcp comment 'SSH from office'

# Allow SSH from your VPN subnet
sudo ufw allow from 10.8.0.0/24 to any port 2222 proto tcp comment 'SSH from VPN'

# Delete the general SSH rule
sudo ufw delete allow 2222/tcp

Tip: If you use a VPN like WireGuard to access your servers, you can restrict SSH to only the VPN interface. See our WireGuard VPN setup guide for details.

Step 11: Enable Two-Factor Authentication (2FA)

Add a second layer of authentication using Google Authenticator:

sudo apt install libpam-google-authenticator -y

Run the setup as your regular user (not root):

google-authenticator

Answer the prompts:

• Time-based tokens: Yes
• Update .google_authenticator file: Yes
• Disallow multiple uses: Yes
• Increase time window: No (unless your clock is unreliable)
• Enable rate limiting: Yes

Edit /etc/pam.d/sshd and add at the end:

auth required pam_google_authenticator.so

Edit /etc/ssh/sshd_config:

ChallengeResponseAuthentication yes
AuthenticationMethods publickey,keyboard-interactive
UsePAM yes

This requires both an SSH key and a TOTP code for login. The AuthenticationMethods directive chains the two methods — the user must pass both to gain access.

Important: Make sure you save the emergency scratch codes from the google-authenticator setup. Store them securely offline. If you lose your authenticator device, these codes are your only way back in.

Restart SSH to apply:

sudo systemctl restart sshd

Test the login from a new terminal (keep your existing session open):

ssh -p 2222 user@your-server-ip

You should be prompted for your key passphrase and then a verification code.

Step 12: Use SSH Config for Convenience

With all these hardening steps, connecting requires remembering ports, key files, and usernames. Simplify this with an SSH config file on your local machine.

Edit ~/.ssh/config:

Host myserver
    HostName 203.0.113.50
    Port 2222
    User deployer
    IdentityFile ~/.ssh/id_ed25519
    IdentitiesOnly yes
    ServerAliveInterval 60
    ServerAliveCountMax 3

Host staging
    HostName 203.0.113.51
    Port 2222
    User deployer
    IdentityFile ~/.ssh/id_ed25519
    IdentitiesOnly yes

Host *.internal.knowledgexchange.xyz
    ProxyJump bastion
    User admin
    Port 2222

Now you can connect with a simple command:

ssh myserver

The ProxyJump directive lets you access servers behind a bastion host without manual port forwarding. The IdentitiesOnly yes setting prevents the SSH client from trying every key in your agent, which avoids accidental lockouts from too many authentication failures.

Complete sshd_config Reference

Here is a consolidated /etc/ssh/sshd_config incorporating all 12 steps:

# Network
Port 2222
Protocol 2
AddressFamily inet
ListenAddress 0.0.0.0

# Host keys
HostKey /etc/ssh/ssh_host_ed25519_key
HostKey /etc/ssh/ssh_host_rsa_key

# Authentication
PermitRootLogin no
PasswordAuthentication no
ChallengeResponseAuthentication yes
UsePAM yes
AuthenticationMethods publickey,keyboard-interactive
PubkeyAuthentication yes
MaxAuthTries 3

# Access control
AllowGroups sshusers

# Timeouts
ClientAliveInterval 300
ClientAliveCountMax 2
LoginGraceTime 30

# Security
X11Forwarding no
AllowTcpForwarding no
AllowAgentForwarding no
PermitTunnel no
Banner /etc/ssh/banner

# Algorithms
KexAlgorithms curve25519-sha256,[email protected],diffie-hellman-group16-sha512,diffie-hellman-group18-sha512
Ciphers [email protected],[email protected],[email protected]
MACs [email protected],[email protected]

# Logging
LogLevel VERBOSE

Testing Without Locking Yourself Out

Follow this protocol every time you change SSH configuration:

1. Open two terminal sessions to the server
2. Edit sshd_config in one session
3. Test the configuration syntax before reloading:

sudo sshd -t

4. Restart SSH if the test passes:

sudo systemctl restart sshd

5. Test login from a new terminal (keep the other two open)
6. Only close your original sessions after confirming the new configuration works

If something goes wrong, use your existing session to revert the changes.

Monitoring SSH Access

Use journalctl to monitor SSH login attempts:

# View recent SSH logs
sudo journalctl -u sshd --since "1 hour ago"

# Watch logs in real time
sudo journalctl -u sshd -f

# Find failed login attempts
sudo journalctl -u sshd | grep "Failed password"

# Count unique attacking IPs
sudo journalctl -u sshd | grep "Failed password" | awk '{print $(NF-3)}' | sort | uniq -c | sort -rn | head -20

For a more structured view, check the authentication log directly:

# View the last 50 authentication events
sudo tail -50 /var/log/auth.log

# Count successful logins today
sudo grep "Accepted" /var/log/auth.log | grep "$(date +%b\ %d)" | wc -l

Additional Recommendations

Beyond the 12 steps above, consider these additional measures:

• Set up a login banner to warn unauthorized users:

echo "Authorized access only. All activity is monitored and logged." | sudo tee /etc/ssh/banner

• Disable unused authentication methods like GSSAPI:

GSSAPIAuthentication no
KerberosAuthentication no

• Use SSH certificates instead of individual authorized_keys files for environments with many servers and users

• Rotate SSH keys regularly — generate new key pairs annually and remove old ones from authorized_keys

Summary

SSH hardening is not optional for any internet-facing server. Implement these 12 steps in order, testing each change before moving on. The most impactful steps are key-based authentication (Step 2), disabling password login (Step 3), and installing fail2ban (Step 9). Combined, they eliminate over 99% of common SSH attacks.

For related security topics, explore our Ubuntu server guides. If you want to add an extra layer of protection by routing SSH through a VPN, check out our articles on secure server networking.