Capturing frontier science, as it happens.

Real-time management matters

Scientists and ICT experts involved in BrightnESS Work Package on Real Time Management of ESS data (WP5) met at PSI to discuss latest progress and to formally start the works related to the development of software to aggregate data from different sources into one data stream and to make it available for processing and analyses (WP5.3).
“Not only will PSI contribute to the development of the neutron event streaming info structure but will also test the new software with real neutrons in real instruments, albeit at a lower event rate then ESS”, says Mark Koennecke from PSI.
WP5.3 is a joint effort between ESS Data Management and Software Centre (DMSC) in Copenhagen, PSI and Elettra Sincrotrone in Trieste, Italy. As Roberto Pugliese explains: ”Elettra will contribute mainly to the design of the scientific data structures and to the study of a fast and reliable data communication and transfer architecture and methods.”
WP5 also includes the University of Copenhagen (KU), which contributes to the creation of a standard neutron event data stream for different detector types (WP 5.1).
An experiment at ESS can produce up to 1 Gigabyte of event data per second. To perform meaningful analysis, event data must include an accurate location as well as a time stamp for the detection of the neutron after scattering. In order to determine the velocity, energy or wavelength of the neutrons, the ESS experiments will use the time of flight (ToF) technique, which is given by the time at which the neutrons are detected after scattering from the object (sample) under investigation minus the time at which the relevant neutron pulse was produced.
BrightnESS WP5 has recently started the development of the software necessary to capture the wide variety of detection parameters (for example the temperature of the sample) and to combine them with the time stamp. If this data, called meta-data, is aggregated with the neutron event data from the detectors in a live (data) stream on a pulse-by-pulse basis, then it can be used as a filter in the further data reduction and analysis software. The live analysed data can also be used to control and/or steer the experiment, thereby optimising the data collection.

A work in progress

Launched in September 2015, the first task of WP5 has been to evaluate how best to live-stream data coming out of the neutrons scattering events. “We will perform a close comparison of the benefits and drawbacks of the two main messaging system candidates: EPICS v4 and ZeroMQ”, says Tobias Richter, group leader for Data Management at ESS DMSC. “First results from event generator tests done recently at PSI were discussed in our meeting on the 31 March and have provided some clear guidelines for setting the evaluation criteria for both systems.” Other activities currently underway in WP5 concern the structure of transferring data through the network, also called wire serialisation format. Options under consideration at this time include: Google Protocol Buffers, BSON as well as defining specialised formats in the collaboration. To test potential choices and allow future decision-making on how to store and maintain all of the captured data, the collaboration of partners in WP5 will make use of 10GB fibre network test equipment.

A collaborative project

One special challenge for Richter and his project partners will lie in the alignment of their progress with so-called in-kind efforts provided by the other institutes around Europe who are in the process of building the control technologies and instruments that will ultimately make up the European Spallation Source. “This requires close interaction with BrightnESS resources dedicated to the strengthening in-kind contribution coordination (WP 2) and to the development of Detectors and Moderators (WP4)”, says Roy Pennings, BrightnESS project coordinator.
The software developed in WP5.3 to enhance the capabilities of ESS scientific instruments could also be used in ESS partner countries - and BrightnESS partner institutes - such as at neutron science facilities like ISIS in the UK, SINQ at PSI in Switzerland, the Institut Laue Langevin (ILL) in France and the Heinz Maier-Leibnitz Zentrum (MLZ) in Germany.
“In the spirit of open-knowledge fostered by the EU, the software developed in this WP will be made available through an open-source software licence”, concludes Richter.