SSH

External command-line access of DoC machines via Secure Shell (SSH)

Secure shell ('ssh') is a network protocol allowing secure, remote machine access to Linux machines. 'ssh' also refers to applications that allow you to connect using this network protocol. All Linux machines in DoC are running a secure shell daemon/server.

The local machine (the client or your home machine) must be running a secure shell client; ssh client implementations are freely available for all major platforms (please see below).  If your home machine runs Windows 10/11, or MacOS X, or any version of Linux, ssh should come with it, or be trivially installable.  The rest of this guide concentrates on using your built-in ssh client.

Setting up SSH (new)

Ok, when you start using ssh in DoC, one of the first things you will need to do is to set up an SSH key with a strong passphrase.  This section explains, step by step, how to do that:  These instructions assume that you have no existing SSH setup either at DoC or on your laptop that you want to keep.  If that's not true, you might want to backup any existing .ssh directory either in your DoC home directory or on your laptop before starting.

1. Bring your laptop into College.  Sit down and login to any Linux lab machine on Level 2 Huxley.  I'm going to assume that you've logged into edge17.  Start a terminal.
2. Now, to set up your SSH key, rather than describe each step separately, we've written an experimental Perl script that will hopefully do it for you.  If you're absolutely sure you don't mind overwriting most of ~/.ssh in your DoC home directory:  run this command on your edge17 (DoC lab machine) terminal:  

   ~dcw/bin/setup-ssh --changereal.

3. This will ask you for the new SSH key's passphrase.  Make up and memorise a good strong passphrase, and enter it twice.  One way of generating a good passphrase might involve picking 3 or 4 unrelated words and stringing them together, either with spaces or with some punctuation to make it even more secure.  For example: herring fluffy Poster! would be an excellent passphrase (but don't use this one, obviously!).  Note that you are technically allowed to leave the passphrase blank: please do not do this, passphrases add an important extra level of security.
4. setup-ssh creates your new SSH key in ~/.ssh/id_ed25519 (the private key) and ~/.ssh/id_ed25519.pub  (the public key).  It also arranges for all DoC linux machines to trust your new SSH key, by appending the new public key to your ~/.ssh/authorized_keys file (creating it if necessary).  All these files are stored in your DoC home directory, which is common to every DoC Linux lab machine (and shell server) that you login to.
5. setup-ssh then writes you a possible ssh configuration file for your laptop/home machine, placing this temporarily in ~/.ssh/HomeConfig in your DoC home directory.  This picks a couple of shell servers at random and 3 lab machines at random, and adds suitable ssh Host declarations, one per shell, one per lab machine, and one for each combination of a shell server and a lab machine (this is what other parts of this page call "2-hop ssh").
6. setup-ssh then tells you precise commands that you must run on your laptop - connected via Imperial WPA wifi - that will copy your new private key and HomeConfig from your DoC home directory onto your laptop.  Run these commands on your laptop, being extremely careful to get them character-perfect.
7. Finally, setup-ssh tells you some test ssh commands to try from your laptop - try them now, verify that they work - and then suggests that you run a companion script ~dcw/bin/setup-sshagent to help you configure ssh-agent on your laptop.  You don't have to do this right now - but you might want to come back to it when you get bored of entering your SSH passphrase repeatedly.  Note that if your laptop runs Windows 10 or 11, you should run ~dcw/bin/setup-sshagent --windows, whereas if your laptop runs (any) Linux or MacOS X, just run ~dcw/bin/setup-sshagent.
8. These scripts are experimental, feel free to read them to see what commands they run.  If you get into difficulties, ask your lab organiser for help if you're in an organised session, or email doc-help@imperial.ac.uk otherwise (copy/pasting the output of the setup-ssh run would be helpful) and explaining where things went wrong.

Using SSH once you've setup your SSH key

To ssh into DoC Linux machines from outside the College network, individual lab machines are firewalled off - so you cannot directly ssh from your home machine to a lab machine.  You can directly ssh from outside the College network to  ONLY the following five DoC Linux servers:

• shell1.doc.ic.ac.uk

• shell2.doc.ic.ac.uk

• shell3.doc.ic.ac.uk

• shell4.doc.ic.ac.uk

• shell5.doc.ic.ac.uk 

(and use sftp to sftp.doc.ic.ac.uk).  Please do not run any CPU-intensive or memory-intensive code on the shell servers: they exist only as gateway machines.  Read the two hop section below!  Long running jobs (that is, over a day, and with very large RAM requirements) should run on batch1.doc.ic.ac.uk or batch2.doc.ic.ac.uk. 

In December 2024, password-based SSH authentication was disabled on the shell servers at ICT's request to increase system security.  To ssh to a shell server, you will need to setup an ssh key (shared between your laptop and DoC systems) and setup your .ssh/config on your laptop. This is covered in the new "Setting Up SSH" section below.

You may ssh to any other DoC machine either:

• in two hops (first ssh into one of the above shell servers, then ssh from that shell server to whichever DoC linux machine you wanted to connect to). See the later section on Two Hop SSH Access for details.

• or (when offsite) by first connecting to College Unified Access and then ssh'ing directly to your chosen DoC machine, using your College password - or your SSH passphrase.
• or (when onsite with a laptop), connecting to Imperial WPA wifi - and then ssh'ing directly to your chosen DoC machine, using your College password - or your SSH passphrase.

Please note that whenever you enter a hostname remember that you need to use the fully qualified hostnames as above, as in shell1.doc.ic.ac.uk not just shell1.

From any ssh window you can run text-mode programs (eg. editors such as Vim, Pico or Emacs) but you won't be able to run X clients (graphical programs). We'll cover graphical remote applications later.

Under Linux, Mac OS X, or Windows 10/11, you can run the command in your shell/command line interpreter (eg. Terminal, cmd or Powershell on Windows):

ssh YOURDOCLOGIN@shell3.doc.ic.ac.uk

You can replace shell3 with shell1, shell2, shell4 or shell5 in the above example to use one of the other shell servers.

For now, we recommend that you do not attempt to run graphical programs on lab machines remotely for now.  Learn to use the power of the command line.  If you want to edit files and you can't learn a Linux editor, use scp to grab a copy of the file, edit it locally, and scp it back again.

Simplifying ssh use

It's a pain to have to type:

ssh DOCUSERNAME@shell3.doc.ic.ac.uk

every time. That’s a mouthful, wouldn’t it be nicer to have to type only:

ssh shell5

and have it not ask who you are, or require you to fully qualify the hostname?  If you setup your SSH key by using the new setup-ssh script then you can already do this, because the script created .ssh/HomeConfig which you copied onto your laptop as .ssh/config, and added shell servers and a few random lab machines to it.

But if you haven't done that, here are the old instructions for how to do it manually: On your laptop/home PC, edit .ssh/config, creating the .ssh folder if it does not exist, using any locally available editor that you like. Note that your .ssh/config file may or may not already exist. Add the following to it (if it's not already there):

Host shell1
        User DOCUSERNAME
        HostName shell1.doc.ic.ac.uk

Host shell2
        User DOCUSERNAME
        HostName shell2.doc.ic.ac.uk

Host shell3
        User DOCUSERNAME
        HostName shell3.doc.ic.ac.uk

Host shell4
        User DOCUSERNAME
        HostName shell4.doc.ic.ac.uk

Host shell5
        User DOCUSERNAME
        HostName shell5.doc.ic.ac.uk

(replacing, as ever, DOCUSERNAME with your DoC/College username). You may wish to add additional entries, for particular DoC linux machines you regularly ssh to, such as batch1 and batch2, or specific lab machines that you like to use.

You can now ssh shell3 and have it know that you meant ssh USERNAME@shell3.doc.ic.ac.uk.

But it will still ask you for a password or passphrase, each time you ssh in.  That's still quite a pain.  We'll come back to that later.

Two Hop SSH Access via a Shell server

Suppose you want to ssh to edge17 from offsite.  Of course you can't just ssh USERNAME@edge17.doc.ic.ac.uk from offsite because it's firewalled off.  The easiest option is to go via a shell server.  You could do it in two explicit steps:

ssh DOCUSERNAME@shell3.doc.ic.ac.uk

(giving your SSH key passphrase) and then

ssh edge17

(given either your College password or your SSH key passphrase).  This should work, but there's a neater way:

ssh -J DOCUSERNAME@shell3.doc.ic.ac.uk DOCUSERNAME@edge17.doc.ic.ac.uk

should "jump" or "proxy" through shell3 to edge17.   It might ask for your passphrase twice, of course.  Read on, we'll show you ways of improving this below..

Note that you can hide away all this "-J shell server" stuff inside your .ssh/config, by adding an explicit "pseudo-host two hop" entry into your .ssh/config.  Suppose you often login to edge17 via shell3.  Edit your .ssh/config and add:

Host edge17
        User USERNAME
        HostName edge17.doc.ic.ac.uk

Host edge17viashell3
        User USERNAME
        HostName edge17
        ProxyJump shell3

Now, the equivalent of the enormous ssh -J command is simply:

ssh edge17viashell3

which is a lot simpler.  Note: the pseudo-hostname "edge17viashell3" can be anything you want, for instance "doclab1".

Using ssh-agent at home, to avoid ssh asking for a passphrase/password

Wouldn't it be nice if you could enter your ssh key passphrase once, and then do multiple ssh and scp commands, without having to re-enter your passphrase each time.

You can do this on your laptop/home PC via ssh-agent and your trusted SSH key.  I'm going to assume that your key is an ED25519 file whose private key is stored in .ssh/doc_ed25519 on your laptop.  If it's something different, eg an RSA key whose private key is stored in .ssh/id_rsa, say, substitute id_rsa for doc_ed25519 in the instructions below.

To use this key automatically on your Linux or Mac laptop/home PC, each time you reboot your laptop, start an ssh-agent:

eval `ssh-agent`  [Note: Recent Windows machines seem to not only come with ssh-agent, but have it automatically running.  Skip this step on such machines.]

Then add your DoC ssh key into the agent via:

ssh-add ~/.ssh/doc_id25519

It will prompt you for your passphrase - enter it. Check that it worked via:

ssh-add -l

Now that you’ve done all this, as long as the ssh-agent is running and has your key in it, you should be able to:

ssh USERNAME@shell3.doc.ic.ac.uk echo hi

and have it say hi without prompting you for a password or passphrase.

Of course, if you've done all the simplifications mentioned above you should even be able to say:

ssh shell3 echo hi

Note that this ssh-agent, loaded with your ssh key, lasts forever, and lets anyone with physical access to your home machine to ssh, as you, to all DoC linux machines.  This is convenient - but dangerous.  For security, at the end of your working day (or when leaving a secure environment with this setup on your laptop), we recommend that you destroy your keys via:

ssh-add -D

This leaves your ssh-agent running, but loaded with no keys.  The next day, simply redo

ssh-add ~/.ssh/doc_ed25519

and enter your passphrase again.