Connect over SSH: the OpenSSH client guide

The ssh client is the entry door to every Linux server, and the ~/.ssh/config file turns long, repetitive commands into simple aliases. The basic ssh user@server connection is enough to start, but as soon as you manage several servers, several keys or a bastion, configuring the client becomes essential. This guide covers it all: from the first connection to everyday techniques (ProxyJump, ControlMaster, diagnostics), with real outputs captured on a two-machine lab. Intended audience: administrators and DevOps comfortable with the terminal.

This guide covers a direct LFCS objective (Networking domain, "Configure the OpenSSH server and client") and RHCSA objective (Understand and use essential tools domain, "Access remote systems using SSH").

What you will learn

• Establish a basic SSH connection with user, port and key.
• Create a ~/.ssh/config file with connection aliases.
• Bounce to an internal server through a bastion with ProxyJump.
• Speed up reconnections with ControlMaster.
• Diagnose a failing connection with -v.
• Manage fingerprints with known_hosts and avoid MitM false positives.

Prerequisites

This guide assumes you already have an SSH key. If not, start with create and manage SSH keys: key authentication is the basis of everything that follows, and it advantageously replaces the password.

Which technique for which context

Each technique of this guide answers a precise need. This table points you to the right section for your situation.

Situation	Technique
Connect to ten servers with different ports and keys	~/.ssh/config with aliases
Reach an internal server through a public bastion	ProxyJump
Run Ansible or rsync scripts without typing a password	SSH key + ssh-agent
Chain repeated connections during a deployment	ControlMaster (multiplexing)
Connection refused after rebuilding a VM	clean known_hosts
Debug an authentication refusal	ssh -vvv user@server

Swipe to see more

Basic connection

The minimal form

The simplest command gives the user and the host:

ssh bob@192.168.10.77

SSH connects on port 22 as the user bob. If it finds a matching key, it uses it; otherwise it asks for the password. Once connected, check that you are where you think you are:

hostname   # the name of the remote machine
whoami     # the effective user

Common options on the fly

Three options cover most special cases, without a configuration file:

# Non-standard port
ssh -p 2222 bob@192.168.10.77

# Specific private key
ssh -i ~/.ssh/id_ed25519_backup bob@backup.example.com

# Verbose mode, to understand what happens
ssh -v bob@192.168.10.77

Accept the fingerprint on first contact

On first contact, SSH shows the server's fingerprint and asks you to validate it:

The authenticity of host '192.168.10.77' can't be established.
ED25519 key fingerprint is SHA256:NqZlPkE/wI00lVSvf+jI6/ENp5tIeqiwM4P5nhuuBtE.
Are you sure you want to continue connecting (yes/no/[fingerprint])?

Answer yes to remember the fingerprint in ~/.ssh/known_hosts. From then on, SSH refuses the connection if the fingerprint changes: it is intentional, it is the protection against man-in-the-middle attacks (see below).

Configure ~/.ssh/config

The ~/.ssh/config file defines connection aliases with all their parameters. It is the clean way to manage several servers, and the first professional reflex to acquire.

The basic structure

Each Host block declares an alias and its options:

Host webserver
    HostName 192.168.10.20
    User deploy
    Port 2222
    IdentityFile ~/.ssh/id_ed25519_deploy

Host db
    HostName 10.0.1.5
    User dbadmin
    IdentityFile ~/.ssh/id_ed25519_db

With this configuration, a simple ssh webserver is equivalent to ssh -p 2222 -i ~/.ssh/id_ed25519_deploy deploy@192.168.10.20. The gain is immediate: on the lab behind this guide, the bastion alias answers directly.

ssh bastion 'echo "connected to $(hostname), user $(whoami)"'

connected to lab-bastion, user labuser

Required permissions

SSH refuses a config file that other users can modify: measured on Debian 13, a config in 666 stops the connection with Bad owner or permissions on ~/.ssh/config, whereas 644 passes. Still aim for the strict value:

chmod 700 ~/.ssh
chmod 600 ~/.ssh/config

Global configuration with Host *

The Host * block applies options to all hosts, unless overridden by a more specific block:

Host *
    ServerAliveInterval 60
    ServerAliveCountMax 3
    Compression yes

ServerAliveInterval 60 sends a keepalive every 60 seconds and avoids disconnections on idle sessions. It is the setting that spares you annoying "Broken pipe" errors on a session left open.

The main directives

These directives cover almost every administration need:

Directive	Role
Host	Connection alias
HostName	Real address or fully qualified name (FQDN)
User	Remote user
Port	SSH port (default: 22)
IdentityFile	Path to the private key
ServerAliveInterval	Keepalive (seconds)
ProxyJump	Bounce through a bastion
ControlMaster	Connection multiplexing
StrictHostKeyChecking	Fingerprint checking policy

Swipe to see more

ProxyJump: bounce through a bastion

In a healthy architecture, internal servers are not reachable directly from the Internet. You go through a bastion (also called a jump host), the only exposed and hardened entry point.

[workstation] --> [bastion:22] --> [internal server:22]

The configuration in ~/.ssh/config

The ProxyJump directive automates the bounce: you declare the bastion, then reference it in the internal server's block.

Host bastion
    HostName 192.168.10.77
    User labuser
    IdentityFile ~/.ssh/id_ed25519

Host internal
    HostName 192.168.10.210
    User labuser
    IdentityFile ~/.ssh/id_ed25519
    ProxyJump bastion

A simple ssh internal then establishes the connection to the bastion, and bounces automatically to the internal server:

ssh internal 'echo "connected to $(hostname), user $(whoami)"'

connected to lab-interne, user labuser

The strength of ProxyJump lies in a security detail: your private key never leaves your workstation. The verbose mode proves it, with its (via proxy) line:

ssh -v internal true

debug1: Connecting to 192.168.10.77 [192.168.10.77] port 22.
Authenticated to 192.168.10.77 ([192.168.10.77]:22) using "publickey".
Authenticated to 192.168.10.210 (via proxy) using "publickey".

You authenticate first on the bastion, then on the internal server through it, each time with your key. The bastion only relays the encrypted traffic; it never sees your key.

In one line, without a configuration file

For a one-off bounce, the -J option does the same without configuring anything:

ssh -J labuser@192.168.10.77 labuser@192.168.10.210

And for a chain of several hops, chain the bastions separated by commas: ProxyJump bastion1,bastion2.

ControlMaster: multiplex connections

During a deployment or a series of commands, SSH establishes a new TCP connection and a new authentication at each call. ControlMaster reuses an existing connection: the first one opens a socket, the following ones borrow it instead of starting over.

Host *.example.com
    ControlMaster auto
    ControlPath ~/.ssh/cm-%r@%h:%p
    ControlPersist 2m

• ControlMaster auto: reuses a connection if possible, creates one otherwise.
• ControlPath: path of the Unix socket (variables %r user, %h host, %p port).
• ControlPersist 2m: the socket stays open 2 minutes after the last session.

The gain is measurable. On the lab, a connection that reuses the master versus a new connection:

== connection through reused master ==
real    0m0,034s
== new connection ==
real    0m0,166s

On a local network, the gap looks modest (34 ms versus 166 ms), but it explodes as soon as authentication is costly (bastion, smart-card key, WAN latency). It is what makes rsync, scp or ansible nearly instant on the same host after the first connection.

The multiplexing socket is sensitive

The ControlPath file gives SSH access without a new authentication. Check its permissions (ls -la ~/.ssh/cm-*): it must belong only to your user. Anyone who can write to this socket can connect to it and borrow your session.

Manage known_hosts

The ~/.ssh/known_hosts file remembers the fingerprint of each visited server. It is what lets SSH detect that a server is no longer the one it claims to be.

The MitM warning after a rebuild

When a VM is rebuilt or an IP reassigned, the host key changes. SSH detects it and blocks the connection with a spectacular warning, captured here on the lab:

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@    WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!     @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
Someone could be eavesdropping on you right now (man-in-the-middle attack)!
...
Offending ED25519 key in /home/bob/.ssh/known_hosts:1
Host key verification failed.

Do not turn off this check by reflex: it is exactly the protection that would warn you of a real attack. If you know that the machine legitimately changed, remove the old entry. SSH even gives you the exact command:

ssh-keygen -f ~/.ssh/known_hosts -R 192.168.10.77

# Host 192.168.10.77 found: line 1
known_hosts updated.
Original contents retained as known_hosts.old

The next connection will ask again to validate the new fingerprint.

Pre-fill known_hosts without connecting

In an automation script, you do not want an interactive question on first contact. The ssh-keyscan command fetches the fingerprint in advance:

ssh-keyscan -H 192.168.10.77 >> ~/.ssh/known_hosts

Turn off checking (ephemeral environments only)

For disposable VMs recreated in a loop, checking becomes a hindrance. You then turn it off by IP range, never globally:

Host 192.168.122.*
    StrictHostKeyChecking no
    UserKnownHostsFile /dev/null

For labs only

StrictHostKeyChecking no turns off the protection against MitM attacks. Never use it in production nor on servers reachable from the Internet. Keep it for a strictly bounded network of disposable VMs.

Diagnose a connection

The verbosity levels

When a connection fails, the first command to run is ssh -v. Three levels exist, from the most useful to the most talkative:

ssh -v bob@192.168.10.77     # basic diagnosis, enough 9 times out of 10
ssh -vv bob@192.168.10.77    # handshake details
ssh -vvv bob@192.168.10.77   # everything, for extreme cases

On a connection that succeeds with a public key, the key lines look like this:

debug1: Reading configuration data ~/.ssh/config
debug1: Connecting to 192.168.10.77 [192.168.10.77] port 22.
debug1: Authenticating to 192.168.10.77:22 as 'labuser'
debug1: Offering public key: ~/.ssh/id_ed25519 ED25519 SHA256:NqZl... explicit
debug1: Server accepts key: ~/.ssh/id_ed25519 ED25519 SHA256:NqZl... explicit
debug1: Entering interactive session.

If Server accepts key does not appear, the problem is on the server side: key missing from authorized_keys, or wrong permissions.

Client-side permissions

On the client side, SSH warns you: a private key readable by others is ignored with a warning impossible to miss, WARNING: UNPROTECTED PRIVATE KEY FILE! followed by Permissions 0644 ... are too open, and the connection fails with code 255 (measured on Debian 13). The silent refusal comes from the server when ~/.ssh or authorized_keys are too open there: it shows up in its log. The values to set on the client:

chmod 700 ~/.ssh
chmod 600 ~/.ssh/id_ed25519
chmod 644 ~/.ssh/id_ed25519.pub
chmod 600 ~/.ssh/config

Read the logs on the server side

When the client is not enough, the server logs give the exact reason for the refusal:

# On Debian/Ubuntu, the service is called "ssh"
sudo journalctl -u ssh -n 50

# On RHEL/Fedora/AlmaLinux, it is called "sshd"
sudo journalctl -u sshd -n 50

Troubleshooting table

The table starts from the message displayed by the client, the only clue you have at the moment of failure. Read the left column, spot the likely cause, and check it with the suggested command before touching the configuration: most refusals come from the server, not from your workstation.

Symptom	Likely cause	Action
Connection refused	Port 22 closed or sshd stopped	sudo systemctl status ssh (Debian) or sshd (RHEL) on the server
WARNING: UNPROTECTED PRIVATE KEY FILE!	Private key readable by others	chmod 600 ~/.ssh/id_ed25519
Permission denied (publickey)	Wrong key or authorized_keys missing	Check with -v, fix the permissions
REMOTE HOST IDENTIFICATION HAS CHANGED	VM rebuilt, IP reassigned	ssh-keygen -R <ip>
Host key verification failed	Unknown fingerprint in strict mode	Check before accepting, or ssh-keyscan
Timeout without a message	Firewall blocks port 22	nc -zv <host> 22
Disconnection after inactivity	No keepalive	Add ServerAliveInterval 60

Swipe to see more

Security and points of attention

The SSH client is also an exposure surface: a few reflexes harden it effortlessly.

• One key per use, not a single key everywhere: a compromised key must not open the whole fleet.
• ProxyJump rather than a forwarded agent: avoid ForwardAgent yes to an uncontrolled bastion, where your agent could be hijacked.
• Encrypt your private keys with a passphrase, and delegate typing it to ssh-agent.
• ControlPath in a private directory: the multiplexing socket is an open door as long as it lives.

Guide validation

You master this guide if you can, without documentation: write a Host block with HostName, User and IdentityFile; reach an internal server with ProxyJump; read an ssh -v output to spot where authentication fails; and fix a known_hosts after rebuilding a machine.

Key points

• ~/.ssh/config centralises the connection parameters: an alias replaces a long, fragile command.
• ProxyJump bounces through a bastion without ever exposing your key: the (via proxy) line in -v proves it.
• ControlMaster makes the repeated connections of ansible, rsync or scp instant.
• ssh -v is the first command to run when a connection fails.
• Permissions are not optional: the client rejects a too-open private key with a warning, the server refuses a too-open authorized_keys without telling the client.
• The REMOTE HOST IDENTIFICATION HAS CHANGED warning is a protection, not a bug: understand it before purging known_hosts.

Knowledge check

Check that the essentials of the guide stuck. The questions cover only what has just been explained here.

Knowledge check

Check what you have learned with this interactive quiz

6 questions

6 min

70% to pass

How it works

• The timer starts when you click Start
• Multiple choice, true/false and short answer questions
• You can move between questions
• Detailed results are shown at the end

Start the quiz

Starts the quiz and the timer

×

See in the glossary →