Overview:
SCP (Secure Copy Protocol) allows you to securely transfer files between your computer and a remote server. SCP uses SSH (man/wiki) to copy files, and is typically available on any computer running ssh. SCP can operate in either a push (local --> remote) or pull (remote --> local) mode, specified by the ordering of options. It is assumed that most visitors of this article are here to learn how to copy homework to/from teaching and learning systems, usually small programs and text files. It is also assumed that you are probably running windows on your personal or lab computer. SCP is available on most modern windows/mac/linux computers, and will work very similarly on each. As with most GNU/Linux utilities, there are extensive man pages that you can and are encouraged to read if you have a desired usecase beyond the basic examples below.
Authentication:
Authentication will be different for each system. Some general-access systems like cps-sshd-student are AD-based (globalID and password), and access is controlled through automation that checks student groups in whitelisted course IDs for the current term. Other external or more specialized services might use local accounts that are not polled from active directory. Defer to your instructor or research sponsor for which authentication method your service uses.
Network Requirements:
CMU on-prem, ssh-based services typically require the following:
- Any domain-bound, ethernet connected computer (with an ssh client installed)
OR
- A virtual machine in the Citrix virtual lab (with an ssh client installed)
OR
- A personal computer connected to the CMU vpn (with an ssh client installed)
Basic Command Structure:
The basic command structure of an scp push command is as follows:
scp -P port /path/to/source_file user@host:/path/to/destination/
The basic command structure of an scp pull command is as follows:
scp -P port user@host:/path/to/source_file /path/to/destination/
Note that in both cases, the destination follows the source. The major difference between the two is where the "user@host" component goes. In the case of a push, the source is a local file. In the case of a pull, the source is a remote file. The [-P (port)] component is optional if ssh on your remote service is listening on port 22, mandatory if your remote service is listening on any other port. In the case of CMU services, it is common for ssh to use nonstandard ports. Once you enter an scp command you will be prompted for your password; upon successfully entering your password the file transfer will begin.
Example #1 (push):
Suppose you have the following situation:
- You would like to push a file from your personal windows computer to the remote service cps-sshd-student.se.cmich.edu
- This service runs an ssh daemon on port 30001
- your local file is in "C:\Users\local_username\Documents\" and is called "cps-999-homework1.py"
- you would like to push this file to your home directory (~/) on the remote server.
- you are currently connected to the vpn
- your CMU Global ID is "globa1id"
On your windows laptop, you would open powershell or the windows terminal, and probably see something like the following:
PS C:\Users\local_username>
you might type the following to change directory into your Documents directory:
cd ./Documents
In the directory you want to push from, you can now type the following scp command:
scp -P 30001 ./cps-999-homework1.py globa1id@cps-sshd-student.se.cmich.edu:~/
After entering your Global ID's password at the system prompt, the file transfer will begin.
Example #2 (pull):
Suppose you have the following situation:
- You would like to pull a file from the remote service cps-sshd-student.se.cmich.edu to your U drive via a lab computer
- This service runs an ssh daemon on port 30001
- your remote file is in ~/cps-999/ and is called "cps-999-homework1.py"
- you would like to pull this file to your U drive in a directory U:/cps-999/
- you are currently connected to the vpn
- your CMU Global ID is "globa1id"
On your windows laptop, you would open powershell or the windows terminal, and probably see something like the following:
PS C:\Users\local_username>
you might type the following to change directory to the desired subdirectory within your personal U drive:
cd U:/cps-999/
In the directory you want to pull to, you can now type the following scp command:
scp -P 30001 globa1id@cps-sshd-student.se.cmich.edu:~/cps-999/cps-999-homework1.py ./
After entering your Global ID's password at the system prompt, the file transfer will begin.
Important Considerations and Alternatives:
Encoding:
By default, many windows text editors will save files by default in an encoding schema of "UTF-8 w/BOM" and/or with "CR LF" line endings. Many Linux utilities, most notably bash, are incompatible with Byte Order Marks and CR LF line endings. It is recommended that you encode your scripts as plan UTF-8, using LF line endings. Once you save a file with these encoding standards, subsequent saves should respect and retain this formatting.
Pathing:
in scp commands, file paths on both the source and destination can be relative or absolute. Relative pathing is faster to type, but more prone to mental errors. Absolute pathing is longer to type but more prone to typing errors.
An absolute path is a path defined from the "root" of a drive e.g
- C:\Users\local_user\Documents\cps-999\file.txt
- /home/central/globa1id/cps-999/file.txt
A relative path is a path defined from the current working directory. If your active terminal is currently in your user's home directory on windows or linux, a file C:\Users\local_user\Documents\cps-999\file.txt or /home/central/globa1id/cps-999/file.txt could be referenced like
/ vs \:
On Linux, Mac, and Unix systems, path barriers are typically naively referenced using a forward slash /
- Most systems will only accept a forward slash to reference directory boundaries, as backward slashes are reserved for escape characters in strings.
On Windows, path barriers are naively referenced using a backward slash \
- Older windows systems will only accept \, modern windows systems will typically accept either \ or / interchangeably.
This is the reason most the examples above uniformly use /, and why we recomend using "windows terminal" or "powershell" instead of "cmd" to run these commands.
CLI-Based Editors:
There are other ways to make edits to text files on remote systems. You do not necessarily have to upload/download via scp.
While logged in via ssh: vim and nano are common cli-based text editors that many business-world workflows will expect you to know. They are simple, time-tested editors and are much more powerful than they appear on the surface.
nano:
- is the simplest, but least powerful by far.
- It's major advantage is that it dedicates a bottom section to permanently display hotkeys and their associated functions so that you don't have to memorize them.
vim:
- is more complex, but much more powerful.
- "vimtutor" is available on most CMU managed Linux-based systems, and is how many faculty and staff got started with vim themselves.
emacs:
- is a halfway-decent operating system, perpetually in search of a passable text editor
- fight me
Creative Solutions:
Some IDEs like vscode/vscodium have plugins for remote development directly to an ssh-backed system
Some ftp clients like FileZilla support sftp (very similar to, and compatible with ssh hosts the same way scp is)
While configuration for these utilities are out of scope for this article (and not officially supported) IT always encourages students, especially CPS/ITC students, to take ownership of their data and develop personal workflows that work for them. The philosophy behind open source, and open protocols, is that all programs supporting the same protocols should work intercompatibily with each other.