Reprocessing Data in ISPyB

‚Äč‚ÄčThere are tools built into the synchweb interface to the ISPyB database to use Diamond's data reduction pipelines on our cluster to reprocess your data. Here you can select frames to use, define space group and cell and set high resolution.

Note this can only run on data collected in the last 30days and available on disk as per our data policy.

To reprocess data click the Gear icon on the associated data set:

gear

 

This raises the reprocessing interface:

Program choice

Here you can select

  • to use XDS 3dii or DIALS processing software run via xia2, or FAST DP
  • set a high resolution cut off
  • clicking space/group cell provides fields to provide input values

To select reprocessing of a single dataset, the processing individually box should be selected:

reprocessing individually

You can also set start and end image numbers via the fields Start and End.

start end

Alternatively you can left mouse drag on the spot analysis interface to select fields. Such as in case where there is radiation damage:

selection of images

Selected images will be shown on the spot analysis interface:

selecting spots

Clicking Integrate will start the reprocessing task on our cluster. Results will appear as another tab on the relevant data collection autoprocessing view:

reprocessing_results

Which opens:

reprocessing window

All reprocessing files can be found in the .ispyb folder of your visit folder followed the processed data directory structure for autoprocessed results eg /dls/i04/data/2017/mx12345/.ispyb/processed/20161208/therm/data3/s_10_1/

Each reprocessing task you start will be run in a run folder, eg run_1, run_2, run_3, etc.

Multiple Sweep or Multiple crystal reprocessing

Multiple crystals, or multiple sweeps of the same crystal can be processed together using the multi-xia2 pipeline.

For example, to process two sweeps of Ferritin of the same crystal together. First find the appropriate datasets in ISPyB.

Ferritin sweeps of same crystal

Then select the gear icon to bring up the reprocess tab:

selecting one crystal

Selecting a second dataset using the gear icon:

multiple sweep

Then select the images either via the textboxes or slide across the spot analysis interface:

select images

Once this has finished reprocessing it can be opening from the reprocessing  tab:

reprocessing selection

Which brings up:

Multi crystal processing window

By clicking the dataset we can see the multi-xia2 results:

multi crystal reprocessing

These results can be downloaded via the archive or found on the filesystem  in the .ispyb folder of your visit folder followed the processed data directory structure for autoprocessed results eg /dls/i04/data/2017/mx12345/.ispyb/processed/20161208/therm/data3/s_10_1/

Within the 30 days data policy.

Diamond Light Source

Diamond Light Source is the UK's national synchrotron science facility, located at the Harwell Science and Innovation Campus in Oxfordshire.

Copyright © 2020 Diamond Light Source

 

Diamond Light Source Ltd
Diamond House
Harwell Science & Innovation Campus
Didcot
Oxfordshire
OX11 0DE

See on Google Maps

Diamond Light Source® and the Diamond logo are registered trademarks of Diamond Light Source Ltd