Reprocessing Data in ISPyB
There are tools built into the synchweb interface to the ISPyB database to use Diamond's data reduction pipelines on our cluster to reprocess your data. Here you can select frames to use, define space group and cell and set high resolution.
Note this can only run on data collected in the last 30days and available on disk as per our data policy.
To reprocess data click the Gear icon on the associated data set:
This raises the reprocessing interface:
Here you can select
- to use XDS 3dii or DIALS processing software run via xia2, or FAST DP
- set a high resolution cut off
- clicking space/group cell provides fields to provide input values
To select reprocessing of a single dataset, the processing individually box should be selected:
You can also set start and end image numbers via the fields Start and End.
Alternatively you can left mouse drag on the spot analysis interface to select fields. Such as in case where there is radiation damage:
Selected images will be shown on the spot analysis interface:
Clicking Integrate will start the reprocessing task on our cluster. Results will appear as another tab on the relevant data collection autoprocessing view:
All reprocessing files can be found in the .ispyb folder of your visit folder followed the processed data directory structure for autoprocessed results eg /dls/i04/data/2017/mx12345/.ispyb/processed/20161208/therm/data3/s_10_1/
Each reprocessing task you start will be run in a run folder, eg run_1, run_2, run_3, etc.
Multiple Sweep or Multiple crystal reprocessing
Multiple crystals, or multiple sweeps of the same crystal can be processed together using the multi-xia2 pipeline.
For example, to process two sweeps of Ferritin of the same crystal together. First find the appropriate datasets in ISPyB.
Then select the gear icon to bring up the reprocess tab:
Selecting a second dataset using the gear icon:
Then select the images either via the textboxes or slide across the spot analysis interface:
Once this has finished reprocessing it can be opening from the reprocessing tab:
Which brings up:
By clicking the dataset we can see the multi-xia2 results:
These results can be downloaded via the archive or found on the filesystem in the .ispyb folder of your visit folder followed the processed data directory structure for autoprocessed results eg /dls/i04/data/2017/mx12345/.ispyb/processed/20161208/therm/data3/s_10_1/
Within the 30 days data policy.