Reprocessing Data in ISPyB

​​There are tools built into the synchweb interface to the ISPyB database to use Diamond's data reduction pipelines on our cluster to reprocess your data. 

Note this can only run on data still on disk which is only guaranteed for data collected in the last 30 days and available on disk as per our data policy. If you wish to reprocess older stat you may need to do this manually after accessing it via Topcat.

To reprocess data click the Gear icon on the associated data set:

 

This raises the reprocessing interface:

Here you can select

• to use XDS 3dii or DIALS processing software run via xia2, FAST_DP or autoPROC
• set a high resolution cut off
• clicking space/group cell provides fields to provide input values

Note that the process individual check box is currently not working, and reprocessing via autoproc is in development.

You can also set start and end image numbers via the fields Start and End.

Alternatively you can left mouse drag on the spot analysis interface to select fields. Such as in case where there is radiation damage:

Selected images will be shown on the spot analysis interface:

Clicking Integrate will start the reprocessing task on our cluster. Results will appear as another tab on the relevant data collection autoprocessing view:

Which opens:

All reprocessing files can be found in the .ispyb folder of your visit folder followed the processed data directory structure for autoprocessed results eg /dls/i04/data/2017/mx12345/.ispyb/processed/20161208/therm/data3/s_10_1/

Each reprocessing task you start will be run in a run folder, eg run_1, run_2, run_3, etc.

Multiple crystals, or multiple sweeps of the same crystal can be processed together using the multi-xia2 pipeline.

For example, to process two sweeps of Ferritin of the same crystal together. First find the appropriate datasets in ISPyB.

Then select the gear icon to bring up the reprocess tab:

Selecting a second dataset using the gear icon:

Then select the images either via the textboxes or slide across the spot analysis interface:

Once this has finished reprocessing it can be opening from the reprocessing  tab:

Which brings up:

By clicking the dataset we can see the multi-xia2 results:

These results can be downloaded via the archive or found on the filesystem  in the .ispyb folder of your visit folder followed the processed data directory structure for autoprocessed results eg /dls/i04/data/2017/mx12345/.ispyb/processed/20161208/therm/data3/s_10_1/

Within the 30 days data policy.