Permanently Add a Private SSH Key So It Persists After Reboot

After adding an SSH key with ssh-add you might notice that after a system reboot you need to add it again. On most Linux distributions the SSH agent does not persist keys across sessions by default. The fix is to configure your SSH client to load the key automatically by adding an entry to the ~/.ssh/config file.

Problem: After adding a private SSH key with ssh-add, it works in the current terminal session but disappears after a reboot — the key must be added manually every time the machine restarts.

Solution: Configure the SSH agent to reload the key automatically on login: add an IdentityFile entry pointing to your private key in ~/.ssh/config, and set AddKeysToAgent yes to store the passphrase in the macOS Keychain or the system keyring on Linux.

Step 1. Find your SSH key location. If you are not sure, run:

ssh-add -l

If no identities are listed, your key is not currently loaded. Keys are typically stored in ~/.ssh/ (e.g. ~/.ssh/id_rsa or ~/.ssh/my-server-key.pem).

Step 2. Move your key file into the ~/.ssh/ directory if it is not already there, then restrict its permissions:

mv ~/Downloads/my-server-key.pem ~/.ssh/
chmod 600 ~/.ssh/my-server-key.pem

Step 3. Open (or create) ~/.ssh/config and add a host entry:

nano ~/.ssh/config

Host your-server-ip-or-hostname
    HostName        your-server-ip-or-hostname
    User            ubuntu
    IdentityFile    ~/.ssh/my-server-key.pem
    AddKeysToAgent  yes

AddKeysToAgent yes tells the SSH client to automatically load the key into the agent on first use, so you will not need to run ssh-add manually after a reboot.

On macOS, add one more line to persist the passphrase in the system Keychain so you are never prompted again:

Host your-server-ip-or-hostname
    HostName        your-server-ip-or-hostname
    User            ubuntu
    IdentityFile    ~/.ssh/my-server-key.pem
    AddKeysToAgent  yes
    UseKeychain     yes

Save the file and test the connection: ssh your-server-ip-or-hostname. The key should load automatically without requiring ssh-add.

NOTE: ~/.ssh/config permissions must be 600 (readable only by the owner), otherwise SSH ignores it. Run chmod 600 ~/.ssh/config if you created the file manually.