Secure Shell, better known as SSH, is the standard way to log in to a Linux machine remotely and run commands as if you were sitting in front of it. Knowing how to allow SSH in Linux is one of the first skills any administrator, developer, or homelab enthusiast needs, because almost every server task — from installing packages to editing configuration files — happens over an SSH session. The good news is that enabling SSH is a short, predictable process: install the server package, start the service, open the firewall, and confirm the connection works. This guide walks through each step on the most common Linux distributions, then shows how to tighten security so your new remote door is not left wide open.
This is where a lot of people lose the thread Not complicated — just consistent..
What SSH Actually Does
SSH creates an encrypted tunnel between your computer (the client) and the Linux machine (the server). Everything sent through that tunnel — passwords, commands, file transfers — is scrambled, which is why SSH replaced insecure tools like Telnet and rlogin Which is the point..
Two pieces must be in place for a connection to succeed:
- An SSH server running on the Linux machine, usually the
openssh-serverpackage listening on port 22. - An SSH client on your local computer, such as the built-in
sshcommand on Linux and macOS, or PuTTY and Windows Terminal on Windows.
If either side is missing, or a firewall blocks the port, the connection fails. Most "SSH is not working" problems come down to one of those three things The details matter here..
Step 1: Check Whether SSH Is Already Installed
Many server distributions ship with OpenSSH installed but disabled. Before installing anything, check the current state:
# Is the server package present?
dpkg -l | grep openssh-server # Debian / Ubuntu
rpm -qa | grep openssh-server # RHEL / Fedora / Rocky
# Is the service running?
sudo systemctl status ssh # Debian / Ubuntu
sudo systemctl status sshd # RHEL family
If you see active (running), the server is already up and you can skip ahead to the firewall step. On top of that, if the command reports "Unit ssh. service could not be found," the package is not installed yet.
Step 2: Install the OpenSSH Server
Installation takes seconds and the command depends on your distribution's package manager.
Debian and Ubuntu:
sudo apt update
sudo apt install openssh-server
Fedora, RHEL, Rocky Linux, AlmaLinux:
sudo dnf install openssh-server
Arch Linux:
sudo pacman -S openssh
On Debian-based systems, the installer usually starts the service and generates host keys automatically. On RHEL-based systems you may need to start it manually, which is the next step.
Step 3: Start and Enable the SSH Service
Starting the service makes SSH available right now; enabling it makes SSH come back after a reboot. You want both.
sudo systemctl start ssh # or sshd on RHEL-based systems
sudo systemctl enable ssh
Verify that the daemon is listening on port 22:
sudo ss -tlnp | grep :22
You should see a line showing sshd bound to 0.0:22 (IPv4) and often [::]:22 (IPv6). Also, 0. So 0. If nothing appears, the service failed to start — check the logs with sudo journalctl -u ssh -n 50 to see why Practical, not theoretical..
Step 4: Allow SSH Through the Firewall
This is the step people most often forget. A perfectly healthy SSH server will still refuse connections if the firewall drops traffic on port 22. The right command depends on which firewall your system uses.
UFW (Ubuntu and Debian)
sudo ufw allow ssh
sudo ufw status
ufw allow ssh uses a predefined application profile that opens port 22 for both TCP and IPv4/IPv6. If you have changed the SSH port, use sudo ufw allow 2222/tcp instead.
firewalld (RHEL, Fedora, Rocky)
sudo firewall-cmd --permanent --add-service=ssh
sudo firewall-cmd --reload
sudo firewall-cmd --list-services
nftables and iptables
On systems managed with raw nftables:
sudo nft add rule inet filter input tcp dport 22 accept
With legacy iptables:
sudo iptables -A INPUT -p tcp --dport 22 -j ACCEPT
Remember that raw iptables rules vanish on reboot unless you save them with iptables-save or a persistence package.
Cloud provider firewalls
If your Linux machine is a cloud instance, the provider's own firewall sits in front of the operating system. AWS Security Groups, Google Cloud firewall rules, Azure Network Security Groups, and Oracle Cloud security lists all need an inbound rule allowing TCP port 22 from your IP address. No amount of ufw configuration will help if the cloud layer is blocking the traffic Still holds up..
Step 5: Test the Connection
From your local machine, connect using the server's IP address and your Linux username:
ssh username@192.168.1.50
The first connection shows a fingerprint prompt — type yes to trust the host key, then enter your password. If you get a shell prompt, SSH is fully working Nothing fancy..
To find the server's IP address, run ip a or hostname -I on the Linux machine itself. For a machine behind a home router, you will also need to forward port 22 on the router to reach it from the internet.
Step 6: Harden the Configuration Before You Walk Away
A working SSH server is not automatically a safe one. Automated bots scan the internet for open port 22 constantly, so spend a few minutes on the following settings in /etc/ssh/sshd_config Less friction, more output..
Disable direct root login
PermitRootLogin no
Log in as a normal user and escalate with sudo. This removes the single most attractive target for brute-force attacks.
Switch to key-based authentication
Generate a key pair on your local machine with ssh-keygen -t ed25519, then copy the public key to the server:
ssh-copy-id username@192.168.1.50
Once keys work reliably, disable password logins entirely:
PasswordAuthentication no
PubkeyAuthentication yes
Keys are effectively impossible to brute-force, which eliminates the main attack vector against SSH.
Consider changing the port
Moving SSH to a non-standard port such as 2222 reduces log noise from automated scanners. It is not real security — a determined attacker will find the port — but it dramatically cuts background
noise from bots hammering port 22. If you do move it, remember that three layers need updating: the firewall rule (--add-port=2222/tcp or the equivalent), the cloud security group, and SELinux on RHEL-family systems, which will refuse to let sshd bind to an unlabeled port until you run sudo semanage port -a -t ssh_port_t -p tcp 2222 The details matter here..
Limit who can log in and how often
A few more directives tighten the door without much effort:
MaxAuthTries 3
LoginGraceTime 20
AllowUsers alice bob
X11Forwarding no
PermitEmptyPasswords no
MaxAuthTries and LoginGraceTime shrink the window a brute-force tool has to work with, while AllowUsers means only the accounts you name can authenticate at all. If you run a larger fleet, AllowGroups ssh-users scales better than listing individuals That's the part that actually makes a difference..
Add fail2ban for automated blocking
Even with keys only, log spam is annoying and repeated attempts waste resources. fail2ban watches the auth log and temporarily bans IPs that fail too many times:
sudo apt install fail2ban # Debian/Ubuntu
sudo dnf install fail2ban # Fedora/RHEL
sudo systemctl enable --now fail2ban
The default jail covers SSH; a small file in /etc/fail2ban/jail.local lets you tune ban time and the number of retries Most people skip this — try not to..
Step 7: Apply and Verify the Changes
Never restart sshd blind. Check the configuration first:
sudo sshd -t
A silent return means the syntax is valid. Then reload the service — on most distributions sudo systemctl restart sshd, though Debian and Ubuntu name the unit ssh:
sudo systemctl restart ssh
Keep your current session open while you test a fresh login in a second terminal. If the new connection fails, you still have the working session to fix the mistake. Once you have confirmed that key-based login succeeds and password login is refused, you can close the old session with confidence.
Troubleshooting Common Failures
- Connection refused — sshd is not running or is bound to a different port. Check
sudo systemctl status sshandsudo ss -tlnp | grep ssh. - Connection timed out — something upstream is dropping packets: the host firewall, the cloud security group, or a NAT/router in between. Work outward from the server.
- Permission denied (publickey) — the key is not in
~/.ssh/authorized_keys, or the permissions are too open. The server rejects keys when the home directory,.ssh, orauthorized_keysare group- or world-writable. Fix withchmod 700 ~/.sshandchmod 600 ~/.ssh/authorized_keys. - Too many authentication failures — your client is offering every key in the agent. Point it at the right one with
ssh -i ~/.ssh/id_ed25519 user@hostor add anIdentityFileentry to~/.ssh/config. - Changes had no effect — you edited the wrong file or forgot to reload. Verify with
sudo sshd -T | grep -i passwordauth, which prints the effective configuration rather than what you think you wrote.
Logs are your fastest diagnostic tool: sudo journalctl -u ssh -n 50 on systemd distributions, or sudo tail -f /var/log/auth.log on Debian-family systems No workaround needed..
Conclusion
Setting up SSH on Linux comes down to four moving parts: install the server, make sure it starts at boot, open the port at every layer between you and the machine, and then tighten the configuration before exposing it to anything untrusted. The default install gets you a working connection in minutes, but the hardening steps — disabling root login, moving to key-based authentication
, and adding fail2ban, turns a stock install into a service that quietly absorbs the constant background noise of automated scans. None of it is exotic, and none of it takes long — but each step closes off a whole class of attack that the defaults leave wide open.
A short checklist before you call the job done:
sshd -treturns nothing, and the effective configuration (sshd -T) reports the values you intended for root login, password authentication, and the listening port.- A brand-new session can authenticate with your key and is refused when it tries a password.
- The port is reachable only where you expect it to be: on the host, in the local firewall, and in the cloud provider's security group.
- fail2ban is enabled, and its log shows actual bans after a burst of failed attempts.
- Your private key is backed up somewhere safe, and you still have a second way in — a console, out-of-band management, or a second admin account — in case the daemon misbehaves after a future edit.
SSH is one of those pieces of infrastructure that rewards a boring, deliberate setup. Do the work once, verify it while you still have a live session to fall back on, and the server stays both reachable and quiet for years That's the part that actually makes a difference..