Multiplex data merging

Please see the multi-crystal pages in the manual here. This page gives an extremely brief introduction running the relatively new multiplex tool for merging  datasets manually. As it evolves this info will probably also move to the manual but is here for now to increase visability

Multiplex is part of Dials and is written by Richard Gildea. For detailed info see

https://dials.github.io/documentation/tutorials/multi_crystal_symmetry_and_scaling.html

It is possible to optionally pick up space group/unit cell information for autoprocessing from a text file in the $visit/processing directory that you can use in the absence of sample information in ISPyB:

• *.yml file should be in the $visit/processing/ directory (yml == yaml format)
• the filename (excl .yml) should match part of the image path downstream of visit path
•  space group and unit cell from ISPyB take precedence

Example configuration file:

$ cat /dls/i24/data/2020/mx12345-6/processing/therm.yml

ispyb_unit_cell: [57.7, 57.7, 149.8, 90, 90, 90]

ispyb_space_group:  P 41 21 2

The main thing to stress is if you are systematic in the way you collect data (i.e. in naming directories and so on) then this will make your life a *lot* easier.

Basic input

In the terminal, navigate to your processing directory and make a directory for the multiplex run. You need to use dials 3 so type

     module load dials/latest

The basic command is

     xia2.multiplex inputdata options

Input data are the refl and expt files produced when integrating data in dials. There should be a .expt and a .refl from each crystal/data collection run. They can be from an automatically run dials job or one you have run yourself

Some of the options we use most frequently are below. For a full list see the multiplex pages on github linked to above.

Worked example

In this example we will merge all datasets (35) that have a filename starting "mpro_B" collected in the visit directory below that were processed automatically in dials.

/dls/i24/data/2020/lb26986-8/mpro_tray1_batch_4/ligand-abc/

Automatically processed data can be found in the processed directory but apart from that the path is the same. Point at the refl and expt files. Multiplex can be run with

xia2.multiplex /dls/i24/data/2020/lb26986-8/processed/mpro_tray1_batch_1/ligand-abc/mpro_B*/xia2-dials/DataFiles/AUTOMATIC_DEFAULT_SAD_SWEEP1.{refl,expt}

If you like you can add more and more runs by adding more paths to more data. 

That's it. Multiplex will run. Notes on output below but some options are also useful.

Options

Just append these to the above command. 

Define the space group

    symmetry.space_group=C2

Use filtering to remove outlying datasets

    filtering.method=deltacchalf filtering.deltacchalf.stdcutoff=1.5

Manually define the resolution

   resolution.d_min=2.7

Select an image range to, for example, exclude data affected by radiation damage.

   dose=1,200

Output

A first indication is given in the terminal. You should see the merging stats. The key columns for assessing data quality are probably Rpim and CChalf. 

For a full overview of the data look at the html page that is produced

    firefox xia2.multiplex.html

Here you can navigate between the tabs and view plots of scaling stats vs. resolution, variations in unit cell and crystal orientation etc.