Additional data centre capacity will be added to support the first-level data storage and initial processing needs of Diamond-II. In addition to placing on-site hosting provision on an environmentally sustainable and extensible footing, Diamond’s scientific computing infrastructure will be reviewed, and modernisation and improvement undertaken. This will be to ensure that it is fit to manage the increased data rates from Diamond-II beamlines on day one, and flexible enough to accommodate additional flagship and competitive upgrades. This modernisation and improvement activity will need to explore the role that cloud computing infrastructure (private, public and hybrid) and deployment approaches (Kubernetes) will play in providing the platforms for Diamond-II, as well as the application of novel architectures (DPU/FPGA/GPU) to provide optimised solutions for real/near-real time data analysis and experimental steering at the beamline (aka edge computing).
As well as compute considerations, Diamond’s current approach to storage and networking will need to be re-evaluated and solutions developed to deal with the torrents of data anticipated from Diamond-II. In the case of storage for instance this will necessitate exploration and potential adoption of caching through increased use of ``in memory” computation and adoption of RDMA type hardware (DPUs) and technologies (e.g. GPU Direct) to improve data throughput to NVMe type storage and reduce computational overhead or these intense data rate experiments on beamlines. This will need to be supported by additional, performant storage solutions (such as the current GPFS) to support post collection (but during visit) data analysis on compute cluster resources, and adoptions of less performant (more persistent and extensible) approaches to support post visit data retrieval and analysis as well as internal research; approaches such as Object Storage could also be used as a future approach to archive requirements. Networking will also require similar treatment to evolve or replace the current infrastructure.
Diamond Light Source is the UK's national synchrotron science facility, located at the Harwell Science and Innovation Campus in Oxfordshire.
Copyright © 2022 Diamond Light Source
Diamond Light Source Ltd
Diamond House
Harwell Science & Innovation Campus
Didcot
Oxfordshire
OX11 0DE
Diamond Light Source® and the Diamond logo are registered trademarks of Diamond Light Source Ltd
Registered in England and Wales at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom. Company number: 4375679. VAT number: 287 461 957. Economic Operators Registration and Identification (EORI) number: GB287461957003.