Diamond Annual Review 2023/24

71 D I A M O N D L I G H T S O U R C E A N N U A L R E V I E W 2 0 2 3 / 2 4 Malcolm which only allows time or spatially resolved detector triggering. Completion of the I22 project was seen as an opportunity to progress the replacement of Malcolm with Athena and an updated Ophyd library, Ophyd Async, to facilitate the asynchronous control of devices. Key to the success of this software architecture application opportunity, and other opportunities already planned and scheduled, is that the new Athena services can be deployed alongside GDA on a beamline controls server. This approach enables client requests to be directed to either GDA or Athena, allowing some beamline operations to be delivered using the new software architecture. The scheme also facilitates the phased migration of beamline experiment orchestration capabilities from GDA to Athena over time. One of the aims of the I22 project was to test this Athena deployment approach and this was successfully achieved. In November of 2023, a first user experiment was conducted using a baseline Athena release comprising BlueAPI, Bluesky NeXus Writer and RabbitMQ[10] along with a first version of Ophyd Async. This experiment was focused on the collection of timed frame data whilst interacting with a sample environment observing a polymer melting and re-solidifying in a thermally controllable capillary. A Bluesky plan for the experiment was written to trigger detectors at variable multiples of a base rate while ramping temperature, and then the sample was imaged while it was cooling. The plan needed to control a Linkam temperature controller, Tetramms for incident beam flux (I0) and transmitted flux (IT), and detectors for WAXS[11] and SAXS[12] data. The capability was very favourably received by beamline staff and users. The success of this work was a testament to the hard work of the Beamline Controls and DAQ Core teams, the SSCC support staff for I22 and I22’s beamline scientists. Since November, the core software development teams have been refining the core services and components such that the I22 software support team can address further hardware synchronised user experiments. The first of these, planned for June, is to collect a timed frame dataset where the sample environment triggers data acquisition by acquiring results from a multi-syringe stop flow system where multiple components are mixed for reacting and acquisition commences after the dispensation of a particular volume of liquid. The second, in November, addresses the collection of a timed frame dataset where an external hardware trigger commences both an entire measurement sequence, only a single frame in a measurement sequence (in an overall timed frame series), or a group of frames from within a measurement sequence. These measurements will be acquired from a high- pressure cell loaded with proteinaceous samples or self-assembled systems, such as lipids, with triggers sent out at the beginning of a pressure sequence, at certain pressure thresholds and after a sudden pressure jump. Universal laboratory information management systems In November 2023, a team was formed to deliver a new Laboratory Information Management System (LIMS) for Diamond. Currently at Diamond such a system exists for the structural biology beamlines, comprising of the ISPyB database and the SynchWeb web front end. This system has been successful in providing a complete user workflow from sample shipping to viewing analysed data, as well as supporting unattended data collection, but is closely tied to data structures and workflows for structural biology, limiting its flexibility. A new LIMS system will be delivered as part of the Diamond-II modernised beamline software architecture, to support those beamlines not using ISPyB and SynchWeb. This will be called Universal LIMS, as it will be designed as a flexible system that can be easily adapted to the needs of different beamlines and technique areas. Universal LIMS will comprise several individual software services with corresponding web front ends. These will be a shipping service; a container logistics service; a sample service; an experiment definition service; a beamline information search and view service; and Electronic Lab Notebooks (ELNs). [Reference figure 5] The first five of these are where the main effort is currently focused, with ELNs being in scope for a later phase. Together these will provide a complete workflow to facilitate user experiments at Diamond. Workflows for structural biology are well defined and are consistent across the beamlines that use it. This means the workflows are tightly aligned with the database structure, making it more difficult to adapt to other beamlines. The architecture for Universal LIMS is driven by the need to create a flexible system, that can easily be adapted to the needs of the different beamlines and technique areas, whilst minimising code changes required by the development team. To provide the required flexibility a systemusing templates to define what metadata is stored will be used. The templates can be created by a defined set of super-users, and then metadata will be stored against specific versions of these. Similarly, the users can update the templates to new versions, and downstream applications will be able to understand how to interpret data depending on the template version. The templating approach will also be used to define how the stored metadata is displayed to the user. Users will be able to create customisable screens for their scientific metadata, which can be selected based on the underlying data template. Re-usable components will be created too, for example for displaying a series of images in a slideshow. Currently two services are deployed as part of Universal LIMS. There is a prototype instance of SciCat, used widely as a metadata catalogue by the ESS, MAX IV, PSI and others, running on the B24 beamline. A number of changes have been made to SciCat to support Diamond’s needs, most notably including changes to support the use of templates for both the stored datasets and the UI display of the scientific metadata. This is currently being rolled out to more beamlines, and serves as a way of both testing out the approach of using templates and refining the requirements for a beamline information search and view service. The second deployed service is the Shipping Service. Functionality that had existed in SynchWeb has been broken out into a standalone service, which serves as an exemplar service for how the further Universal LIMS services can be built and work together. The Shipping Service has recently been made externally accessible, and to date has shipped over 1,000 Dewars. While it is currently being used within workflows for structural biology, is has been created in such a way that it will work seamlessly with other services at Diamond, such as Universal LIMS. Over the next year the team will roll out more of the Universal LIMS services, starting with a sample service. During the roll outs, opportunities will be identified to both test out the new Diamond-II software architecture and deliver new functionality to beamlines. Containerisation infrastructure and provisioning - cloud native Diamond is transitioning to a cloud native architecture across SSCC, utilising key Cloud Native enabling technologies such as containerisation and microservicesdesignpatterns.Thesesoftwareencapsulationanddevelopment methods drive higher speed and agility in software development, deliver higher application reliability, and provide more portable code. These methods allow Diamond to develop and deploy code more quickly, decrease time to science, and increase ability to respond to scientific drivers, opportunities, and collaborations.