- Docker logs tail: Troubleshoot Docker containers with real-time logging - Wed, Sep 13 2023
- dsregcmd: Troubleshoot and manage Azure Active Directory (Microsoft Entra ID) joined devices - Thu, Aug 31 2023
- Ten sed command examples - Wed, Aug 23 2023
Working with SSH Keys
Since the SCP command relies on the SSH protocol for secure data transfer, we can use an SSH public key for authentication. This provides an additional layer of security compared to password-based authentication. For more robust security when using SCP, you should always opt to use SSH keys. It is much more difficult for an attacker to gain access to your private key in a key pair than to simply guess passwords for a user. You can also protect a private key with a password to add an additional layer of security.
By default, the SCP command uses password authentication. However, with an SCP parameter, you can use a private key instead of password authentication.
To specify a private key, use the -i option:
scp -i /path/to/private_key remote_user@remote_host:/path/to/remote_file /path/to/local_directory
Generating your SSH keys and copying to the remote server
To use public key authentication, make sure you have the public key of the key pair copied to the remote host.
You can generate an SSH key pair on the local system using the following command:
ssh-keygen
Once you have generated your SSH keys, you can copy the public key portion to your remote host using the following command:
ssh-copy-id <user>@<remote host>
I assume here that password authentication is enabled on the remote host. When you issue the ssh-copy-id command and give it the username and remote host, it will prompt you for the password of the user you want to use to authenticate with the remote host. After you correctly enter the password for the user, it will copy over the default SSH public key to ~/.ssh/authorized_keys.
Using ssh copy id to copy the public key to a remote host
SCP command syntax
Note the following syntax of the SCP command:
scp [options] [source] [destination]
You can see the options if you type scp from a shell prompt in Linux or WSL:
Viewing the SCP options in WSL
The source can be a file or directory on the local or remote system, while the destination can be a local or remote location. The source and destination can include a username and IP address, specifying the user account and remote host involved in the transfer.
SCP from remote to local
You can use the following command to copy a file from a remote system to your local machine as a local file:
scp remote_user@remote_host:/path/to/remote_file /path/to/local_directory
This will copy the remote file to the specified local directory. If you don't specify a local directory, the file will be stored in the current directory of your local system.
Below, we are remoting into a remote system at IP address 10.1.149.25 with the username linuxadmin. We are copying the file located at /home/linuxadmin/traefik/docker-compose.yml to the local home directory by specifying the "~" at the end.
If you have not connected to the host before, you will see the prompt to confirm that you want to accept the fingerprint and continue connecting to the remote host. Type yes if you want to continue.
Accept the fingerprint of the remote host
You will then be prompted for the password for the remote Linux user. After a successful login, you should see the file transfer begin and succeed.
Authentication and file copy are successful
Copy multiple files and directories
The SCP command supports copying multiple files and even entire directories from a remote host to your local system. To copy multiple files, you can use the following syntax:
scp remote_user@remote_host:/path/to/{remote file 1,remote file 2}
Below, we are copying two different files from the remote host.
Copying two different files from the remote host
To recursively copy entire directories, use the -r option:
scp -r remote_user@remote_host:/path/to/remote_directory /path/to/local_directory
This command will copy the entire directory, including its subdirectories and files, from the remote system to your local machine.
Below, we are copying the contents of a remote directory to the local home directory. The scp -r command tells the system to copy all the files recursively.
Recursively copying the contents of a remote directory
Advanced SCP options
Additional advanced options may be useful in certain situations. The SCP command offers several advanced options to enhance your file transfer experience. For example, the -p option preserves the access times and file permissions of the source files:
scp -p remote_user@remote_host:/path/to/remote_file /path/to/local_directory
For large files, the -C option compresses the data during transfer, speeding up the process:
scp -C remote_user@remote_host:/path/to/remote_file /path/to/local_directory
Displaying the progress meter is beneficial if you have a large file copy. To display a progress meter, use the -l option:
scp -l remote_user@remote_host:/path/to/remote_file /path/to/local_directory
Transferring files between two remote hosts
The SCP command is not limited to transferring files between a remote host and your local machine. It also enables users to transfer files between two remote systems. To achieve this, use the following syntax:
scp remote_user1@remote_host1:/path/to/remote_file remote_user2@remote_host2:/path/to/destination_directory
This command copies the file from the first remote host to the destination directory on the second remote host. The file transfer will occur directly between the two remote systems, without needing to pass through your local machine.
Securing file transfers with SCP
The SCP command ensures that your files are securely transferred by encrypting the data during transmission. As mentioned above, the secure copy protocol leverages the security features of the SSH protocol, providing authentication, confidentiality, and integrity.
In addition to using key files for authentication, note the following best practices:
- Use strong passwords for your user accounts on remote systems
- Keep your SSH keys secure and use passphrase protection
- Regularly update your systems and SSH software
- Limit the number of users who have access to remote systems
- Monitor and audit remote system access and file transfers
Common SCP errors and troubleshooting
You may encounter errors during file transfers using SCP. Note the following problems and solutions:
Subscribe to 4sysops newsletter!
- Permission denied: If you don't have the proper write permissions on the target system, you will encounter this error. To resolve the permission denied error, log in via SSH and check whether you have write permissions on the destination folder with the ls -l command.
- Connection timed out: This may be due to network issues or a remote host being down or inaccessible. If you see a connection timed out issue, check your network connection and make sure the remote host is online and no firewall is interfering with your connection.
- No such file or directory: This error occurs when the specified source file or directory does not exist. Verify that the file path is correct and that the file or directory is on the remote system.
- Host key verification failed: If the remote host's key has changed or is not trusted, you may see this error. To resolve the issue, update the known host's file on your local system or verify the remote host's key fingerprint.
Wrapping up
The SCP command is a great tool for copying files between remote and local hosts from the command line. As shown, many command line options are included with SCP, allowing you to perform advanced copy functions and leverage secure authentication using key files. It enables the efficient transfer of files across remote hosts and your local machine using manual commands or including them in bash scripts.
Great SCP/SSH post.
Please remember that working as root is not the best recommended practice.