Software Engineer (Data Management)

This vacancy is now closed
Job Title Software Engineer (Data Management)
Job Reference DIA1084/NH
Post Type Full time / Permanent
Division Technical
Salary information Circa £35,000
Application deadline 03/03/2016

Job Description

Diamond Light Source is the UK’s largest scientific investment in the past 40 years. Located in rural South Oxfordshire, it works like a giant microscope, harnessing the power of electrons to produce brilliant light that scientists use to study anything from fossils to jet engines to viruses and vaccines.

The Science Computing team is a group of enthusiastic problem solvers dedicated to enabling the highest quality science. They develop and support the best computer, storage and networking hardware available today to provide the world-class computing infrastructure the science demands.

The Diamond experiments generate 1-2PB/year, and this is doubling every year. This exciting role is to work with the scientific, computing and software teams to generate better tools to manage this data deluge so the scientific priorities are met whilst efficiently making the best use of our state-of-the art computing infrastructure.

Most of the data is kept on high speed parallel file systems for a limited time, and then moved to the long term tape archive managed by the Science and Technology Facilities Council (STFC) on the same Harwell campus. Once there, data often has to be restored from tape for reprocessing on Diamond and other HPC systems.

We are seeking a good people person who has both programming and system administrator’s skills. You will work with both the STFC Science Computing Division (SCD) Research Data Group (who provide the software that manages Diamond’s data archive) and the Science Computing team (who provide Diamond’s high performance storage) to ensure our software and systems work together efficiently and reliably. Whilst you will be part of the Science Computing team, you will spend much of your time with SCD) Research Data Group testing and developing new software across a range of different hardware systems. It is likely you will have a background in scientific software, with a strong IT interest.

The aim of this role is to work with existing computing, software and scientific staff at Diamond and in the STFC Science Computing Division Research Data Group to ensure that data generated at the Diamond facility is well managed and archived.

Key Responsibilities:

  • Interact with scientists to understand their requirements and expectations of the data management and archive system;
  • Develop Java and Python software to move data from the high speed storage to the archive and from the archive to re-processing systems. This will leverage existing technologies, including the HSM functionality of GPFS and Lustre, as well as working with the STFC archive team to develop new functionality and interfaces for the archive;
  • Work closely with the archive teams in STFC, to develop and manage the SLA we have with them for provision of archive services. This includes:   

                         o Monitoring archive loads and problems;
                         o Managing, developing and testing new software for data access;
                         o Setting the vision for the archive by anticipating new requirements as demands develop.

Qualification & Experience - Essential

  • Significant experience in developing Python and Java software in a Linux environment;
  • Significant experience of using Linux systems, including some system management expertise, ideally in a HPC/HTC environment;
  • Experience or understanding of data management;
  • Honours degree or BTEC HNC/HND + relevant experience;
  • Ability and initiative to get to the heart of the problem and take it effectively through to completion;
  • Good interpersonal, communication  and presentational skills;
  • Enjoys team work;
  • Self motivated, interested in keeping pace with software testing best practises;
  • Ability to interact effectively with staff at all levels;
  • Friendly and helpful with users and able to communicate effectively.

Qualification & Experience - Desirable

  • Practical experience in a professional, scientific or technical field;
  • Experience of software project management;
  • Experience of software development frameworks and applications such as Eclipse;
  • Higher Degree in an appropriate discipline;
  • Ability to supervise or mentor small teams of more junior technical or administrative roles.

Further Information

Available to travel occasionally in UK and overseas including overnight absences.

Diamond Light Source is:

  • is an equal opportunities employer;
  • encourages career development and training;
  • provides easy access to extra-curricular activities such as clubs, sports and lectures.

Applying for employment

For further details on applying for employment at Diamond, please visit our 'Application Form' page.

Appointments will be made depending on the skills and experience of the candidate. Electronic Diamond application forms in MS-Word are preferred. These should be emailed to

Other information