Program specific how-tos

Here we will share some guides on getting started with some specific applications. In many cases there are multiple ways to accomplish things so you can treat this as a starting point. If you found a good guide elsewhere which you successfully followed or if you would like your own guide added please feel free to let us know.

Jupyter notebooks

The Jupyter notebook is a web-based notebook environment for interactive computing. Running a Jupyter notebook server remotely on Viking and connecting to it from your local web browser is certainly possible. There are a few steps to ensure that your notebook is running on a compute node (not a login node) and you can remotely connect to it.

The connection would look something like this:

Your computer ➡️ Viking login node ➡️ Viking compute node ➡️ Jupyter notebook

One way to do this would be to start an interactive session on a compute node, from that session run the Jupyter notebook server:

you can adjust the ‘cpus-per-task’ and ‘time’ to suit your needs
$ srun --nodes=1 --ntasks=1 --cpus-per-task=5 --time=08:00:00 --pty /bin/bash
srun: job 76415 queued and waiting for resources
srun: job 76415 has been allocated resources
Creating user dir for 'Local Scratch'
flight start: Flight Direct environment is already active.
[abc123@node112[viking2] ~]$

Here we have an interactive bash session on the node112 compute node, with five CPU cores, for eight hours. Next we load the Jupyter module and run the server:

[abc123@node112[viking2] ~]$ module load JupyterLab/3.1.6-GCCcore-11.2.0
[abc123@node112[viking2] ~]$ jupyter notebook --no-browser
[I 15:01:51.756 NotebookApp] Writing notebook server cookie secret to /users/abc123/.local/share/jupyter/runtime/notebook_cookie_secret
[I 2023-11-03 15:01:52.190 LabApp] JupyterLab extension loaded from /opt/apps/eb/software/JupyterLab/3.1.6-GCCcore-11.2.0/lib/python3.9/site-packages/jupyterlab
[I 2023-11-03 15:01:52.190 LabApp] JupyterLab application directory is /opt/apps/eb/software/JupyterLab/3.1.6-GCCcore-11.2.0/share/jupyter/lab
[I 15:01:52.194 NotebookApp] Serving notebooks from local directory: /users/abc123
[I 15:01:52.194 NotebookApp] Jupyter Notebook 6.4.0 is running at:
[I 15:01:52.194 NotebookApp] http://localhost:8888/?token=9b0f6d6918f238c0e8543257a842b65cd4671ee1b55a4e3c
[I 15:01:52.194 NotebookApp]  or http://127.0.0.1:8888/?token=9b0f6d6918f238c0e8543257a842b65cd4671ee1b55a4e3c
[I 15:01:52.194 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[C 15:01:52.198 NotebookApp]

    To access the notebook, open this file in a browser:
        file:///users/abc123/.local/share/jupyter/runtime/nbserver-2295028-open.html
    Or copy and paste one of these URLs:
        http://localhost:8888/?token=9b0f6d6918f238c0e8543257a842b65cd4671ee1b55a4e3c
     or http://127.0.0.1:8888/?token=9b0f6d6918f238c0e8543257a842b65cd4671ee1b55a4e3c

The server is running and listening on port 8888. We cannot access that port directly from our local computer so we set up the an ssh tunnnel to the login node then then from there to the compute node, where the Jupyter notebook server is actually running. You can do this with one command locally on your computer:

remember to change ‘node112’ to the node you are running Jupyter from
$ ssh -L 8888:localhost:8889 viking.york.ac.uk ssh -N -L 8889:localhost:8888 node112

The above command opens up one ssh tunnel, forwarding your local port 8888 to the Viking login node port 8889. Then it opens up another ssh tunnel from the login node’s port 8889 to the compute node112’s port 8888 - where the Jupyter server is listening.

Finally Ctrl + left mouse click on the link from the first terminal session on node112, highlighted above. Either the http://localhost:8888/?token=... or the http://127.0.0.1:8888/?token=... links. Your browser should open and connected to the Jupyter server running on Viking.

Jupyter notebooks using VSCode

Using some of the above guide as reference, another way to so this is with VSCode. You do it all in VSCode and the inbuilt terminals in VSCode. If you’re interested in this method it’s similar to the above in many ways:

  1. Install the Jupyter extension in VSCode

  2. Remote ssh connect to Viking from VSCode’s terminal

  3. Start an interactive session with srun eg srun -N 1 -n 1 -c 10 -t 4:0:0 --pty /bin/bash in the terminal of VSCode

  4. Once the interactive session is running, load the Jupyter module and run the notebook, like above

  5. In a new remote terminal on Viking, in VSCode, set up the ssh forwarding, like above (noting the node number from step 4.)

  6. In VSCode, open a new Jupyter notebook: (Ctrl+Shift+P) and type Jupyter: Create New Jupyter Notebook.

  7. In VSCode, press select kernel in the top right then select Existing Jupyter server

  8. Paste in the URL of the notebook, just like the guide above, follow the prompts in VSCode to name the notebook and select the available kernel

VSCode

VSCode is a modern text editor from Microsoft. It’s possible to use many extensions with it to extend its capabilities but here we’ll explain how to use it to connect to Viking over ssh and allow you to remotely edit text files, which can be very handy.

  1. Install VSCode for your operating system

  2. Follow the instructions to install the Remote - SSH extension in VSCode

  3. Follow the instructions to getting started to connect to viking, the address is viking.york.ac.uk

  4. Enter your password when prompted and you should soon be connected and be able to edit and save files through VSCode.

Tip

Ctrl + ` (backtick) Opens a terminal in VSCode

Tip

If at some point VSCode refuses to connect and it’s not obvious why, sometimes the files it installs on Viking have become corrupted. You can log into Viking in your usual way without VSCode, and delete the following directory ~/.vscode-server. The next time VSCode connects to Viking it will reinstall the files and hopefully will fix the problem.