Resolve "Blissdata: h5py-like API on dynamic file"
Closes #3544 (closed)
Needs !5025 (merged) (RUNNING status reached only after the PREPARED_SCAN event)
Short description
Abstract API
The h5py-like API mainly has the classes Group(Mapping)
and Dataset(Sequence)
. The values of Group
are of type Group
or Dataset
. Note that a Mapping
mainly provides item getting and iteration. Sequence
mainly provides slicing and iteration.
Static HDF5
We already had static_hdf5.py
(static means the file content is not changing). It behaves exactly like h5py
.
Dynamic HDF5
This MR provides dynamic_hdf5.py
(dynamic means the file content can change any time). The API is exactly the same but the runtime behavior is different:
Static HDF5 | Dynamic HDF5 | |
---|---|---|
group[name] |
Return immediately | Block until the key is present or the scan is marked as "PREPARED" |
dataset[idx] |
Return immediately | Block until the entire slice is available or until the scan is marked as "FINISHED" |
for name in group |
Stops when all names are yielded | Stops when the scan is marked as "PREPARED" |
for data in dataset |
Stops when all data points are yielded | Stops when the scan is marked as "FINISHED" |
The behavior for dynamic HDF5 will be the same when we add memory support (Redis+Lima).
Edited by Wout De Nolf