Nexus writer: wrong chunk shape for oscilloscopes
Reported by @claustre at ID09
This is how Bliss >= 1.10
calculates the chunk/compression configuration
from nexus_writer_service.io.h5_config import guess_dataset_config
scan_shape = (None,) # timescan
detector_shape = (250000,) # MCA size
dtype = numpy.float64
pprint(
guess_dataset_config(
scan_shape=scan_shape, detector_shape=detector_shape, dtype=dtype
)
)
The default scan shape is (512,)
: https://gitlab.esrf.fr/bliss/bliss/-/blob/master/nexus_writer_service/io/h5_config.py#L28
Bliss 1.9 does it differently: https://gitlab.esrf.fr/bliss/bliss/-/blob/1.9.x/nexus_writer_service/subscribers/dataset_proxy.py#L58
So the general problem here is: if we don't know dataset dimension (e.g. timescan or detector with variable shape) the chunk shape may be too large and the file becomes very large for small scans.
Edited by Wout De Nolf