Annual Review 2024-2025

S C I E N T I F I C S O F T WA R E , C O N T R O L S A N D C OM P U TAT I O N A N N U A L R E V I E W 2 0 2 4 / 2 5 42 technical changes with the existing IT infrastructure. Diamond’s project team built strong relationships with those managing Scientific IT and Corporate IT services, and coordinating planning between them. The final rollout had the potential to significantly impact Diamond’s operations, as facility operations depend on IAM’s functionality and reliability. Diamond was comfortable with the level of testing undertaken, and so final migration from CDR to the new product was able to proceed with an acceptable risk. The result was a smooth transition from the old to new systems. Feedback from a lessons-learnt process indicated satisfaction with Diamond’s management of the project. The project’s success from Diamond’s perspective was due to the team’s hard work, a good communication plan, strong project governance, and a shared commitment to delivery. Despite being a demanding project, it was highly satisfying, showcasing commitment and perseverance from all parties involved. Development of a roadmap for Scientific Computing In today’s data-driven research landscape, the pace of scientific discovery is closely tied to the effectiveness of computational methods and infrastructure. At Diamond, we are seeing an exponential increase in scientific data production, exceeding 12 PBs (petabytes) annually, due to increased automation, experiment complexity, and the use of photon imaging techniques and electron microscopy. The arrival of Diamond-II and the launch of upgraded beamlines will significantly boost data production and scientific discovery. Some experiments, like beamline K04, could generate hundreds of terabytes of data per day. This necessitates evaluating our current computing model to handle the increased data volume and velocity. A review of our scientific computing infrastructure began in 2024, leading to a strategic technology and delivery roadmap for scientific computing infrastructure, ensuring support for current operations and new Diamond-II capabilities. Key areas of focus include: • data centres: developing new data centre capacity to replace end-of-life provisions and accommodate next-generation of computer hardware. The 1 MW Data Centre project will start construction in late 2025; • storage: moving from monolithic high- performance file systems to hybrid storage tiers to meet performance and capacity needs; • networking: upgrading network infrastructure with high-bandwidth links and advanced routing protocols to support seamless data transfer; • compute and cloud: transitioning from monolithic compute infrastructure to cloud-based research. This roadmap will provide a framework to deliver the scientific computing infrastructure to realise Diamond-II’s potential and enhance current facility performance. Figure 1: Output of a deep learning network trained to detect the location of drops and crystals in images. Figure 2: Development of the design for 1MW Data Centre project for first-level data storage and initial processing.