Immediately Following Data Collection

Click here to find out how data processing is run.

As soon as data collection is complete a second script is started, which stops the per-image analysis service and initiates data processing with fast_dp and xia2. The aim of the data processing with fast_dp is to deliver an indication of the quality of the data and reasonable intensities for subsequent analysis in the shortest possible time. To this end it employs both multi-core and multi-node parallelism on the MX cluster, the flow for which is shown in Figure 1. Typically fast_dp will deliver the results of processing, specifically the scaled and merged intensities and the merging statistics (Table 1) within two minutes of the end of data collection, even for fine sliced Pilatus data. Three xia2 jobs are also started, one using Mosflm / Aimless and two using XDS / XSCALE, with the intention of producing higher quality data for downstream analysis. Each of these xia2 jobs generates a report on the data quality which is linked from the HTML pages (Figure 2). At the end of the fast_dp data processing there are a number of downstream analysis steps added, with the intention of giving more focussed experimental feedback.

Figure 1 Workflow for fast_dp

Table 1 Example results from fast_dp

Figure 2 HTML reports on data processing generated by xia2