3D imaging advances with Python

Tomography processing enhanced by new modular pipeline

Tomography users at Diamond are set to get a boost with new open-source software that makes 3D reconstruction quicker, easier, and more customisable than ever before. Automated processing steps will allow users to get higher-quality 3D reconstructions faster by reducing manual intervention, and make the most of their beamtime by optimising experimental techniques as they go.

Novice and experienced users will now have the opportunity to customise how their data are processed by adding Python plugins to their processing pipeline. Analysis and correction of experimental problems will be possible shortly after each data collection with the application of specific modules. This rapid feedback loop has not previously been available in synchrotron-based X-ray tomography at Diamond.

Aptly named ‘Savu’, after a python subspecies that is incredibly adaptable in terms of habitat, this new modular processing pipeline is open licenced, facility independent, able to operate on any standard cluster infrastructure, and freely available on Github in alpha release. The Savu system has been developed to drive the high-throughput of high-quality tomographic reconstructions, as detailed in a recent Philosophical Transactions A publication1.

 
Tomography experiments can create huge datasets, with one recent user group at Diamond collecting 69 terabytes of data in a single visit. The processing and transfer of such datasets can take a long time. With beamtime a very precious commodity for academic and industrial users, a group of software and beamline scientists at Diamond have been taking steps to reduce the time and complexity associated with data processing by developing the new tomographic reconstruction software, Savu.
 
Removing the technical burden
 
The motivation for Savu came from the need to make life easier for users. Dr Michael Drakopoulos, Principal Beamline Scientist on the Joint Engineering and Environment Processing (JEEP) beamline, was the lead theoretical designer for the pipeline. Users on JEEP had spoken about wanting to have more automation in processing their data: “The pipeline takes away the burden of understanding the details of tomography reconstruction from the users. High-quality reconstructions need a lot of computer power and a well-designed set of sophisticated software algorithms. They’ll now get immediate feed-back on their scientific problem and therefore are able to draw scientific conclusions without delay.”
 
More time for science
 
The modular nature of Savu takes away the need for a low level of input from beamline staff to manage the parallel processing steps. The pipeline is designed to allow users and staff to write bespoke scripts in Python to shape their data processing without affecting the software’s core functioning. This in turn frees up the beamline staff to focus their expertise on supporting the science rather than the data handling tasks.
 
The JEEP beamline (I12) specialises in engineering and material science, and increasingly the bio-medical field. Structural integrity or compositional changes in 3D are revealed from the reconstructions of attenuation and phase contrast scans. Senior Support Scientist Dr Robert Atwood believes users on I12 will greatly benefit from the increased processing speed: “With up to 20 scans per second acquired at the beamline, Savu will allow easy automation of the processing, and state-of-the-art developments in time series tomography – such as those developed as part of the Collaborative Computation Project in Tomographic Imaging (CCPi)2 – can be integrated. This is a major improvement in our high-speed time-resolved tomography, and gives users a real-time response to their experiments.”

Optimising experiments
 
With a growing tomography community at Diamond, there has been a drive to produce common tools for use across all beamlines. For users of the X-ray Imaging and Coherence beamline (I13), one of the major benefits of Savu will be in helping to optimise experiments. Dr Andrew Bodey, Support Scientist on I13 explained: “With the bespoke nature of synchrotron tomography experiments, doing tomography well requires experimental optimisation and the identification of experimental problems. Competing needs must often be balanced. Savu will help to facilitate this by providing quick feedback on experimental problems and data quality.” In the Phil Trans A paper on Savu, the authors describe the various ways in which software can be used to identify and resolve experimental problems3, with supplementary material that illustrates sample deformation and misalignment issues4.
 
Lead developer on the Savu project, Mark Basham (left), with Andrew Bodey from beamline I13.
 
Multimodal data compatibility
 
Chemical tomography on the microfocus spectroscopy beamline (I18) has already been able to benefit from the introduction of Savu. This new technique is used to investigate the active sites of catalysts, and relies on multimodal data to read information during data collection. Dr Stephen Price, PDRA on I18, has developed chemical tomography methodologies at Diamond and written a number of scripts that are now to be bundled up with Savu. “There was no software for multimodal tomography before, and it’s really been a bit of a challenge to adapt standard processing to include the extra dimensions of this data. However, Savu speeds things along by plugging in these scripts and makes chemical tomography much more accessible to the wider scientific community.”
 
Continuing development
 
Lead developer Dr Mark Basham is keen to manage user expectation at this stage, but is excited about what Savu will bring to users at Diamond: “Savu is still under development, and there are still teething troubles and glitches, but we do have a standard pipeline for the full XRD/XRF mapping and full-field tomography. Users have the ability to plug in elements to make the process more automated, but in the beginning they’ll still need to pick what parameters to use. However, once that chain has been built, it’ll be very easy to apply it to any other datasets. These are exciting times for the world of tomography, and with Diamond’s computing power we’re in a unique position be able to break new ground.”

Savu is also in the process of being adopted by the Collaborative Computation Project in Tomographic Imaging (CCPi), and by the pulsed muon and neutron source, ISIS. For any enquiries about Savu and its different applications, please contact Mark Basham in the first instance: Mark.Basham@diamond.ac.uk

 

 

 
Schematic of the data-processing pipeline, in context of other steps required for a complete experiment (Atwood et al. 2015, Fig. 1. Reused courtesy of the authors under CC-BY 4.0 licence).

~~~

References

  1. Atwood RC, Bodey AJ, Price SWT, Basham M, Drakopoulos M. A high-throughput system for high-quality tomographic reconstruction of large datasets at Diamond Light Source. Phil Trans A 373 (20140398), (2015). DOI: 10.1098/rsta.2014.0398
  2. Kazantsev D, Van Eyndhoven G, Lionheart WRB, Withers PJ, Dobson KJ, McDonald SA, Atwood R, Lee PD. Employing temporal self-similarity across the entire time domain in computed tomography reconstruction. Phil Trans A 373 (20140389), (2015). DOI: 10.1098/rsta.2014.0389
  3. Atwood et al., p. 6-7.
  4. Atwood et al. Electronic Supplementary Material – Figures S1-S3: http://rsta.royalsocietypublishing.org/content/373/2043/20140398.figures-only