Reprocessing Data in ISPyB

‚Äč‚ÄčThere are tools built into the synchweb interface to the ISPyB database to use Diamond's data reduction pipelines on our cluster to reprocess your data. 

Note this can only run on data still on disk which is only guaranteed for data collected in the last 30 days and available on disk as per our data policy. If you wish to reprocess older stat you may need to do this manually after accessing it via Topcat.

To reprocess data click the Gear icon on the associated data set:

gear icon from dataset



This raises the reprocessing interface:

Program choice

Here you can select

  • to use XDS 3dii or DIALS processing software run via xia2, FAST_DP or autoPROC
  • set a high resolution cut off
  • clicking space/group cell provides fields to provide input values

Note that the process individual check box is currently not working, and reprocessing via autoproc is in development.

You can also set start and end image numbers via the fields Start and End.

start end

Alternatively you can left mouse drag on the spot analysis interface to select fields. Such as in case where there is radiation damage:

selection of images

Selected images will be shown on the spot analysis interface:

selecting spots

Clicking Integrate will start the reprocessing task on our cluster. Results will appear as another tab on the relevant data collection autoprocessing view:


Which opens:

reprocessing window

All reprocessing files can be found in the .ispyb folder of your visit folder followed the processed data directory structure for autoprocessed results eg /dls/i04/data/2017/mx12345/.ispyb/processed/20161208/therm/data3/s_10_1/

Each reprocessing task you start will be run in a run folder, eg run_1, run_2, run_3, etc.

Multiple Sweep or Multiple crystal reprocessing

Multiple crystals, or multiple sweeps of the same crystal can be processed together using the multi-xia2 pipeline.

For example, to process two sweeps of Ferritin of the same crystal together. First find the appropriate datasets in ISPyB.

Ferritin sweeps of same crystal

Then select the gear icon to bring up the reprocess tab:

selecting one crystal

Selecting a second dataset using the gear icon:

multiple sweep

Then select the images either via the textboxes or slide across the spot analysis interface:

select images

Once this has finished reprocessing it can be opening from the reprocessing  tab:

reprocessing selection

Which brings up:

Multi crystal processing window

By clicking the dataset we can see the multi-xia2 results:

multi crystal reprocessing

These results can be downloaded via the archive or found on the filesystem  in the .ispyb folder of your visit folder followed the processed data directory structure for autoprocessed results eg /dls/i04/data/2017/mx12345/.ispyb/processed/20161208/therm/data3/s_10_1/

Within the 30 days data policy.

Diamond Light Source

Diamond Light Source is the UK's national synchrotron science facility, located at the Harwell Science and Innovation Campus in Oxfordshire.

Copyright © 2022 Diamond Light Source


Diamond Light Source Ltd
Diamond House
Harwell Science & Innovation Campus
OX11 0DE

See on Google Maps

Diamond Light Source® and the Diamond logo are registered trademarks of Diamond Light Source Ltd

Registered in England and Wales at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom. Company number: 4375679. VAT number: 287 461 957. Economic Operators Registration and Identification (EORI) number: GB287461957003.