Skip to content

Resolve "Blissdata: h5py-like API on dynamic file"

Closes #3544 (closed)

Needs !5025 (merged) (RUNNING status reached only after the PREPARED_SCAN event)

Short description

Abstract API

The h5py-like API mainly has the classes Group(Mapping) and Dataset(Sequence). The values of Group are of type Group or Dataset. Note that a Mapping mainly provides item getting and iteration. Sequence mainly provides slicing and iteration.

Static HDF5

We already had static_hdf5.py (static means the file content is not changing). It behaves exactly like h5py.

Dynamic HDF5

This MR provides dynamic_hdf5.py (dynamic means the file content can change any time). The API is exactly the same but the runtime behavior is different:

Static HDF5 Dynamic HDF5
group[name] Return immediately Block until the key is present or the scan is marked as "PREPARED"
dataset[idx] Return immediately Block until the entire slice is available or until the scan is marked as "FINISHED"
for name in group Stops when all names are yielded Stops when the scan is marked as "PREPARED"
for data in dataset Stops when all data points are yielded Stops when the scan is marked as "FINISHED"

The behavior for dynamic HDF5 will be the same when we add memory support (Redis+Lima).

Edited by Wout De Nolf

Merge request reports