Diamond Annual Review 2020/21

124 125 D I A M O N D L I G H T S O U R C E A N N U A L R E V I E W 2 0 2 0 / 2 1 D I A M O N D L I G H T S O U R C E A N N U A L R E V I E W 2 0 2 0 / 2 1 Scientific Software, Controls and Computation Mark Heron, Head of Scientific Software Controls and Computation T he Scientific Software, Controls and Computation (SSCC) department manages all software, computing and control systems to facilitate and support the science programme of Diamond. It functions as eight groups: Scientific Computing, Data Analysis, Data Acquisition, Beamline Controls, Accelerator Controls, Electronic Systems, Business Applications and Cyber Security. The overall structure and function of these areas recognises the importance of, and is optimised to provide, the best possible delivery and support for software, computing and control systems. In March 2020, Diamond responded to societal restrictions resulting from the national lockdown, a consequence of the COVID-19 pandemic. For SSCC staff this meant transitioning to developing software, managing systems and providing support to the facility, while working remotely. In a matter of days, the department transitioned to home working successfully, and established new communication channels and ways of working to support the science programme effectively. This was doubly important given that the science programme during the initial part of the lockdown was focused on understanding the structure and function of SARS-CoV-2; the very virus that was shutting down society. Diamond has a long track record of pioneering remote operation of synchrotron beamlines. While well-developed in some science domains this was the opportunity, and a driver to expand, to opening up remote operation of nearly all beamline instruments. Additional IT hardware was delivered and commissioned to facilitate the performant network access required to support all instruments. Remote operation meant a distributed management team. Maintaining effective and timely communication was seen as imperative to be able to support operations effectively. A routine of thrice weekly video calls kept the SSCC Management team in touch with each other and the department’s business function. Whilst maintaining operations during the pandemic was challenging, SSCC was busy with new developments in many areas. A Strategy for the Scientific Software, Controls and Computation was published, and the first of two technical roadmaps, detailing implementation, developed. New software and control systems were delivered in line with requirements determined by the science programme, and new computing hardware was commissioned to address the challenges of processing increasingly big and complex data as a consequence of new detectors. Finally, the year saw detailed planning of implementation of new capabilities as part of the Diamond-II proposed upgrade programme. The following presents selected highlights from exciting developments across a broad range of activities that have taken place within the SSCC department during the past year. Strategy for the Scientific Software, Controls and Computation The Strategy was the first published by SSCC. It is designed to present a vision for how the department will meet forthcoming challenges in software, controls and computing. Its development included extensive consultation within Diamond to explore and understand future science needs and drivers. From this a Vision and a set of Goals were derived, and these in turn informed development of a series of critical objectives to define future direction. Implementation of the objectives will be realised through a series of technical research and development roadmaps. Following an internal review of the Strategy at the end of 2019, an overview was presented to Diamond’s Science Advisory Committee (SAC) in 2020, and a programme to develop the technical R&D roadmaps is underway. The result of this strategic exercise forms a foundation on which to build and deliver essential services to support Diamond’s science programme. Vision for SSCC SSCC’s vision is encapsulated as follows: • The SSCC Department will maximise knowledge generated by Diamond across every science discipline , through a modern, integrated approach to delivery and management of software and computing. • SSCC will enhance science capabilities through: more intuitive user applications, greater automation of analysis, increased remote capabilities, greater automation, faster detector readout and real-time visualisation. • SSCC will enable greater data usability through Findable, Accessible, Interoperable and Reusable (FAIR) and Open Data, and through new capabilities in data science (Machine Learning and Artificial Intelligence), modelling and analysis. Goals for SSCC SSCC’s goals are to: • Deliver high quality software, computing and controls provision and support in order to enable Diamond’s science programme: from experiment approval to publication . • Establish strong partnerships , internally and externally the Science and Technology Facilities Council (STFC), other facilities, universities, etc); to deliver best in class software and computing in a collaborative, effective and sustainable way. • Deliver a step change in experiment capabilities and efficiencies , across the science programme, through: intuitive useable applications; automated analysis of data; remote access capabilities; automation of instrument and sample management; high speed readout of detectors; and near real-time data process for visualisation feedback. • Widen experimental data exploitation through delivering: FAIR and Open Data, and through new capabilities in data science (Machine Learning (ML) and Artificial Intelligence (AI)), modelling and analysis. • Realise a hybrid computing model , which delivers on-site computing services to those applications for which low latency and high bandwidth are essential, with all other services delivered through cloud computing. Scientific Computing for remote operations Diamond has led internationally in providing remote access for its users to synchrotron beamlines and electron microscopes. Remote operation is delivered through a NoMachine NX software service, which provides users with a remote desktop experience, equivalent to being physically at the instrument, and so enable sophisticated experiments to be performed completely remotely. In response to the COVID-19 induced changes to working practices, remote operation capability has been significantly expanded. The suite of instruments that can offer users remote operation now includes 12 Electron Microscopes and 29 photon beamlines and end stations; with remote operation enabled on 17 of those beamlines in the past year. Remote operation of instruments relies on more than just software to provide secure and easy access, it relies on software applications and control protocols that ensure the experiment can be conducted safely and reliably. Diamond continues to innovate to improve scientists’experience of remote operations and to ensure scientific exploration can continue efficiently and effectively. In the past year this has enabled Diamond to deliver science that otherwise would not have been possible. Data Analysis to support COVID-19 science programme Timely and effective development in data processing and analysis capabilities, in response to the pandemic, has been a significant feature of data analysis developments undertaken over the past year. The Macromolecular Crystallography (MX) analysis team has both contributed directly to the COVID-19 science programme at Diamond, and alsoworked to ensure Diamond can operate as normal a service as possible for external users of the facility. Some of Diamond’s academic and industrial users have programmes of scientific research that directly address understanding of the function of the SARS-CoV-2 virus. The MX instruments were already strongly placed following historical emphasis on remote operation of data collection, and recent initiatives to enable Unattended Data Collection (UDC). Social restrictions resulting from COVID-19 accelerated developments, as follows: • UDC was subject to continuous improvements related to experimental decision making, including an extension to be able to screen samples to inform experimental parameters. This significantly increased capacity for the number of data collections per day on beamline I03 where it is fully implemented. The extension is being deployed and tested on three further beamlines. • Processing prioritisation was addressed, to enable best utilisation of computing resources. By automating data collection, it is possible to support analysis of large data sets, enabling timely data analysis. • Significant improvements in multi-crystal data analysis (Fig. 2) were realised through close collaboration between the MX data analysis team and beamline staff. The improvements were applied to room temperature in situ ligand-screening experiments on SARS-CoV-2 Main Protease (MPro). • Automatic data uploads to Zenodo (https://zenodo.org/record/3730917) were implemented for the early phase of MPro drug binding studies, in order to make this data accessible to the scientific community as early as possible. Figure 1: Full remote operation and data acquisition on a Diamond beamline is possible using the infrastructure provided by the Scientific Computing team. Image courtesy Juan Sanchez- Weatherby. Figure 2: Automated multi-crystal analysis for SARS-COV-2 on beamlines I24 and VMXi. X-ray Technologies