Manual Reprocessing

You can connect into Diamond’s clusters to manually reprocess data. This is a multi-step process!

Starts the same as general connecting to Diamond (e.g. if doing a remote collection)

Step 1: Get NoMachine/NX Client. Tip: Search for “NoMachine Enterprise Client” then get without server

Step 2: Configure connection to Diamond

Step 3: Double-click on your new connection. It will ask for your username and password the first time.

Step 4: If you are part of a remote collection, the beamline machines will be listed to connect to. Do not connect to these outside of your beamtime! Instead, click “Create new desktop or custom session”

Step 5: Click “Create new virtual desktop”. You may get something like this; ”Automatically Select a Node”. Should see desktop looking something like this.This is a machine on the cluster but should not be used for processing – the memory and CPU are quite restricted. Open the launch menu (top-left), and click on the “Terminal”

Step 6: This opens a terminal window on this hostYou must change hosts to the Slurm submission controller, type: `ssh wilson`. You will then be prompted for your password. Please do not try to process data on the login NX node, or the Wilson submission node. This will either work slowly, start and then crash, or be liable to be killed if it takes too much resource on the shared infrastructure.

Therefore, please navigate to the folder that you want your script to run in - generally in your visit folder, as your home area is too small for most jobs. Use your visit processing folder to process data manually e.g.

This directory will be available for 40 days after your visit ends. Alternatively, you can work in:

and data stored here is kept for 30 days after creation.

Any questions on using these facilities? Please feel free to email [email protected] – no questions are stupid!

Once in the right place, you can submit jobs using SLURM. When your job is running, you can type exit to log out - because it'll run happily in the background, and all the output files will appear where you expect. The output text will go to slurm-jobnumber.out.

More Slurm commands can be found here.