The Xspress4 Digital Pulse Processor

Figure 1 – Xspress4 Digital Pulse Processor configured as a 70-channel readout now installed on Beamline I20-Scanning to readout a 64-pixel HPGe Spectroscopy Detector.

Diamond has used monolithic multi-channel HPGe detectors on Absorption Spectroscopy beam lines for many years. The Xspress2 Digital Pulse Processor has been used to read out these detectors from the beginning of DLS operations in 2007 until 2016. Xspress2 was developed for DLS by STFC and was an upgrade of the original Xspress Digital Pulse Processor developed by STFC for the SRS, the UK’s first synchrotron facility in operation at Daresbury until 2008.

The experience gained with the operation of HPGe detector systems in high photon flux environments showed that the energy resolution at high counting rate was limited by the cross-talk between pads (channels) in the detector. Detailed studies of this effect showed that the effect of the cross-talk in a “victim” channel was entirely determined by the pulse height of an event in the “aggressor” channel. It was thus concluded that a digital pulse processor with advanced algorithms could correct the effects of cross-talk in real-time. Simulations of the envisaged correction algorithms by post-processing real data taken with Xspress2 showed that such a correction could  lead to more than a doubling of the counting rate per channel for a given energy resolution. In order to implement these algorithms in real-time, dedicated hardware architecture was necessary which facilitated communication between many channels and provided sufficient processing power for sustained high count rate capability. Diamond decided to fund a development project to meet these requirements and provide all X-ray Absorption Spectroscopy users at DLS with enhanced detector system performance.

The output of the aforementioned development project is Xspress4* which is an advanced digital pulse processor developed to read-out monolithic multi-element HPGe detectors and the first to implement real-time cross–talk correction. The first results have been published [1] and the real-time cross-talk correction algorithm patented [2]. A general view of the pulse processor is shown in figure 1.

*The system has been called Xspress4 because Xspress3 is a commercial digital pulse processor developed by Quantum Detectors which is the licensee of Diamond’s and STFC’s Xspress algorithms.

Figure 2 - Internal view of Xspress4 showing a 10-channel ADC Board (right) connected to the FEM-2 FPGA Card (blue) and two Mid-planes - Active (top) and Passive (bottom).

Xspress4 is an integrated but modular digital pulse processing system comprising of a number of major components. Key parts include the digitizer cards which route data to high performance digital processor cards which in turn transmit processed data to a Linux server.

The in-house designed 10 channel digitizer cards have a 9U x 220mm Eurocard standard footprint and are mounted in a dedicated card-frame (crate) which can host up to 10 cards for a total of 100 channels per crate. The card is shown in figure 2. Each channel includes a variable gain, variable offset analogue front-end which enables the detector head output to be precisely matched to the dynamic range of the digitizers analogue to digital converters (ADC). The front-end electronics also integrates an 18MHz linear phase anti-alias filter, whose analogue output is then digitized by a 16-bit 100MHz ADC. The input of the digitizer card can be either single ended or differential to suit different detector output signal architectures. The digitizer cards plug directly onto the digital processor cards and simultaneously into two backplanes inside the crate. The first backplane provides power to the digitisers and routes low skew clock and system trigger signals between cards for synchronised operation in a multi-card configuration. The second backplane routes high speed serial digital signals between the digital processor cards to facilitate inter-card communication.

The input of the digitizer cards are analogue signals produced by the charge sensitive preamplifier of each pad (channel) in the detector head. Each channels signal approximates to a series of random staircases superimposed on a background ramp caused by detector dark current. Each step of the staircase represents an X-ray impinging on the corresponding channel sensor in the detector head. The digitizer cards continuously sample the waveforms and stream data into the digital processor cards for signal processing (X-ray energy measurement).

Each digital processor card is a high performance Field Programmable Gate Array (FPGA) card based on the Xilinx Virtex7 FPGA. This card was developed by STFC’s Centre for Instrumentation [3] as a generic FPGA card offering a very high number of input/output (I/O) lines, a very compact profile to be stacked in modular detectors, and high processing power allowing it to simultaneously handle a large number of data channels. The headline features of this card showed it was a good fit for the Xspress4 project. The 192 LVDS I/O lines can easily handle the data generated by the 10 channels of a single digitizer card and the Virtex7 FPGA has sufficient logic resources to implement all the required processing functions for 10 channels and still have capacity for future additional features. Furthermore, the six 10Gbps high speed-serial links across the I/O connector neatly met the requirements for implementation of real-time inter-card communication.

The Virtex7 FPGA implements the digital processing algorithms which identify the occurrence of steps in the input waveform, remove the slope due to the dark current, and calculate the step (pulse) height of the identified events. The pulse height is proportional to the charge collected (and therefore energy) on the related detector channel. In order to be able to correct for the cross–talk, three key pieces of information are required for each and every event, which, once acquired, are passed to all the potential victim channels, who then use them to make the corrections on a per event basis. The data pipeline of the Xspress4 digital pulse processor is thus the following for each channel:

1. a) Identify an event has occurred; b) Timestamps the events time of arrival; c) Calculates its coarse pulse height (energy); [Each event is a real event on its channel but constitutes an aggressor event to all others];
2. Initiates channel-to-channel communication by transmitting the [aggressor] event channel ID, timestamp and coarse energy to all potential victim channels;
3. On receipt of aggressor event information, the victim channels remove the cross-talk artefacts from their raw unprocessed waveforms by subtracting an equivalent cross-talk replica constructed in real-time. The replica is formed using the received information together with the unique aggressor-to-victim cross-talk signature stored by the victim channel as determined during initial system calibration. This process effectively “cleans” the waveforms of the victim channels removing the effects of the aggressor channels from them;
4. Repetition of the pulse height calculation on the cleaned waveforms resulting in a more accurate energy measurement than was obtained through the coarse measurement of step 1.

If the aggressor and victim channels are processed by the same FPGA card then channel-to-channel communication takes place within the FPGA of that card. If however the channels are processed by different cards channel-to-channel communication is achieved by routing high speed serial data along dedicated lanes in the second backplane mentioned previously.

The end result of the processing pipeline is a stream of accurately measured event pulse heights which are then packetized and sent to the Linux server over a 10Gbps fibre-optic link.

The Linux server hosts a sufficient number of 10Gbps fibre-optic links to read the data coming from a fully populated 100-channel crate. The stream of event data from all channels is then histogrammed to make a series of energy spectra with user defined time intervals. The series of histograms are then sent to data storage where dedicated beam-line analysis software can then conveniently access the data and extract the salient XAS data which beam line users are ultimately most interested in.

Xspress4 was designed such that two crates can be operated together in a master-slave configuration to create a pulse processor capable of reading out up to 200 channels.

Xspress4 is currently installed in I20 and B18 to read-out a 64 channel and 36 channel monolithic HPGe detectors respectively. Thus, the number of required digitizer-FPGA card pairs is 7 in the case of I20 and 4 in the case of B18.

Xspress4 resulted in a considerable improvement in the performance of the detector systems at the beam lines. Counting rates per channel of the order of 500kHz were previously unusable because of the degradation in energy resolution and are now routinely exploited. Figure 3a shows two multichannel analyser spectra of Mo fluorescence produced in the same conditions by channel 45 of the detector at I20 with the Xspress2 and Xspress4 at 550kHz of output counting rate. The FWHM of the Kα peak reduces from 1323eV to 393eV. In figure 3b the values of FWHM of the Mo Kα peak for all the channels at different input counting rates taken with Xspress2 and Xspress4 are plotted. In addition to the much improved resolution it can be noticed that Xspress4 provides also better consistency in the values of FWHM between all the channels.

The better consistency of the data as a function of counting rate proved to be extremely useful to also eliminate some artefacts due to the considerable change of counting rate during the scan resulting from glitches in the monochromator. Figure 3c shows an artefact in the absorption coefficient of Ni produced by the Xspress2 (peak between 8.6keV and 8.65keV in the top plot of figure 3c) when the fluorescence is taken from a sample contaminated by Fe. Because of a monochromator glitch the counting rate is considerably reduced which in the case of the Xspress2 causes a change in the shape of the multichannel analyser spectra (bottom plot in figure 3c). The overlap of Fe Kβ line and Ni Kα line and the change of shape lead to an incorrect estimation of the Ni Kα line intensity. This artefact is almost completely eliminated by the Xspress4.

In conclusion the adoption of the Xspress4 pulse processor enabled the Absorption Spectroscopy beam lines to at least double the useful counting rate, to reduce the time it takes to make a scan, and to get more stable and consistent data less prone to producing artefacts.

Figure 3a – Multichannel analyser spectra of Mo fluorescence (K-alpha 17.5keV) taken with Xspress2 (red) and Xspress4 (blue) in the same conditions for channel 45 of the 64 element HPGe detector.

Figure3b – Energy resolution vs input counting rate for all the channels of the 64 element HPGe detector as measured by Xspress2 (red) and Xspress4 (blue).

Figure3c – Absorption coefficient of Ni in a sample contaminated by Fe (top). The absorption coefficient measured by the Xspress2 is in red and the one measured by Xspress4 is in blue. The bottom plot shows the multichannel analyser spectra of the X-ray fluorescence of the sample with Fe k-beta line overlapping Ni K-alpha line both in the Xspress2 (red) and Xspress4 (blue) case.

The versatility of the Xspress4 hardware and firmware architecture enabled other advanced processing schemes to be envisaged, in addition to the cross-talk correction already implemented.

A common issue with monolithic multi-element detectors is that when X-ray photons land too close to the edge of a pad the freed electrical charge may be shared between that pad and its immediate neighbour at that edge. This issue is normally addressed by adding a collimator in front of the X-ray sensor which prevents the photons from landing in the critical boundary region. However, this solution has some drawbacks. If the angle of incidence of the photons is not normal, the shadow of the collimator can considerably reduce the effective area of the sensing pads. Furthermore, the material the collimator is made out of emits fluorescence photons which also appear in the energy spectra in addition to those from the user sample. In the case of a dilute user sample where the same element of the collimator’s material is under study, the measurement becomes unviable.

Leveraging the time stamping and channel-to-channel communication capabilities of the Xspress4, it has been possible to implement a coincidence method to reject charge shared events. When the freed electrical charge from a single photon is shared among neighbouring pads, the resultant shared events simultaneously appear in all the related channels. Extensions to the standard Xspress4 firmware detect time-coincident events and test to see if they meet a “shared event” selection criteria. If the criteria are met, the system either marks those events as ‘shared’ for later post processing, or deletes them from the final histogram - in the latter case removing partial events which would otherwise spoil the quality of the energy spectra. This scheme was implemented in a monolithic HPGe multi-channel demonstrator and preliminary results are shown in figure 4. The full validation and complete characterization of this method still need to be done.

Figure 4 – Multichannel analyser spectra of an Fe55 radioactive source taken with a monolithic multielement HPGe detector and Xspress4 with the real time charge sharing rejection algorithm (red) and without it (blue).

Another operational mode which can advantageously use Xspress4’s time-stamping capabilities is with very quick XAFS experiments. When histogramming events captured at a fast energy scan rate (e.g. 20kHz equivalent frame rate, 50ms time slice) it is possible that some events will not be assigned to the right time slot because of the variable latency of the steps in the processing pipeline. To overcome this, instead of histogramming the events as they are processed, their salient information can be streamed directly to the storage – the so-called “list-mode” form of operation. The stored information for each event includes the pulse height of the event, the channel which generated it, and the events timestamp. Post-processing of the stored data can thus use the accurate timestamp information to correctly assign an event to arbitrary time slices at the discretion of the user, leading to slice-accurate quick XAFS scan results.