Option to save scans in separate files
After testing (NFS and GPFS) with this script it seems that a reader that runs on a different machine than the writer rarely manages to read the file. Even though the writer flushes regularly, which should put the HDF5 file in a readable state on disk, it might be the case that this is rarely propagated to the disk due to other levels of caching (like the GPFS caching). In other words, the file on disk is never in a readable state during a scan. This also depends on the rate at which the writer is changing the file (which is very high in the test so in reality it might be not as bad).
After discussion it seems we can add an option in Bliss to save 1 file per scan and the dataset file will contain external links to those files.