Skip to content

Compressed datasets can be very slow to fetch

I don't know if it comes from h5grove, h5web or a combination of the two.

This needs further investigation. It can be probably reproduced outside of Hibou by using a dataset with the same metadata:

Dataset
Type Integer (unsigned), 32-bit, little-endian
Shape 2000 x 2167 x 2070 = 8971380000
Chunk shape 1 x 2167 x 2070 = 4485690
Compression filters
32008 bitshuffle; see https://github.com/kiyo-masui/bitshuffle

Also observed in jupyterlab-h5web

Edited by Loic Huder
To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information