5.3: Beta-version data aggregator software

This document concerns BrightnESS Deliverable 5.3: “Beta-version data aggregator software”. It marks the delivery of a working software prototype of the main component of the data streaming infrastructure, that is needed to determine the roadmap to the full completion of the task (D5.5 due July 2018). The development of the data aggregation software system is progressing ahead of the WP schedule. With the choice of the open source third party project Apache Kafka as the central component for aggregation and streaming, the WP5 participants could focus on the domain specific software components to retrieve and aggregate the data, and to write them to file. This maximises the impact of work package 5 for ESS and other neutron facilities as it provides better functionality, flexibility and fault tolerance of the data acquisition chain. In current integration tests, neutron event data - together with simulated instrument metadata - automatically traverse the readout pipeline from event formation to file writing through Kafka.

DOI: https://doi.org/10.17199/BRIGHTNESS.D5.3