Skip to content

Resolve "Revision HDF5 chunk size"

Wout De Nolf requested to merge 3052-revision-hdf5-chunk-size into master

Closes #3052 (closed)

This MR improves HDF5 chunking and allows configuring it from the Bliss side through SCAN_SAVING.writer_object. The parameters for the Nexus writer are

  • chunk_size: the maximal chunk size in MB. Smaller datasets are not chunked unless they require compression. Default: 1 MB
  • compression_limit: datasets larger than this limit will be compressed. Default: 1 MB
  • compression_scheme: used in case the dataset size is larger than compression_limit. Default: "gzip-byteshuffle"
  • chunk_split: in case the dataset size is larger than chunk_size, the inner dataset dimensions are split in this many parts. Default: 4.
Edited by Wout De Nolf

Merge request reports