Nexus writer: Unable to create external links in the master files
Reported by @debionne at ID13 (it happened several months ago at another beamline as well)
Traceback (most recent call last):
File "/users/blissadm/local/bliss.git/nexus_writer_service/subscribers/base_subscriber.py", line 450, in _listen_event_loop
self._process_event(event_type, node, event_data)
File "/users/blissadm/local/bliss.git/nexus_writer_service/subscribers/scan_writer_base.py", line 1087, in _process_event
self._event_new_node(node)
File "/users/blissadm/local/bliss.git/nexus_writer_service/subscribers/scan_writer_base.py", line 1113, in _event_new_node
self._event_new_subscan(node)
File "/users/blissadm/local/bliss.git/nexus_writer_service/subscribers/scan_writer_base.py", line 1128, in _event_new_subscan
self._init_subscan(subscan)
File "/users/blissadm/local/bliss.git/nexus_writer_service/subscribers/scan_writer_config.py", line 160, in _init_subscan
self._create_master_links(subscan)
File "/users/blissadm/local/bliss.git/nexus_writer_service/subscribers/scan_writer_config.py", line 590, in _create_master_links
nexus.createLink(nxroot, linkname, nxentry)
File "/users/blissadm/local/bliss.git/nexus_writer_service/io/nexus.py", line 1387, in createLink
h5group[name] = lnk
File "/users/blissadm/conda/miniconda/envs/bliss_dev/lib/python3.7/site-packages/h5py/_hl/group.py", line 438, in __setitem__
lcpl=lcpl, lapl=self._lapl)
File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
File "h5py/h5l.pyx", line 157, in h5py.h5l.LinkProxy.create_external
RuntimeError: Unable to create external link (addr overflow, addr = 41531, size = 512, eoa = 41531)
Solution: repack the master file (you can read it but not write to it)
h5repack /data/visitor/sc5404/id13/20230303/sc5404_id13.h5 /data/visitor/sc5404/id13/20230303/sc5404_id13_repack.h5
mv /data/visitor/sc5404/id13/20230303/sc5404_id13_repack.h5 /data/visitor/sc5404/id13/20230303/sc5404_id13.h5
Issue posted on the HDF5 forum: https://forum.hdfgroup.org/t/h5lcreate-external-fails-but-the-link-is-still-created/10463
Maybe the Nexus writer should do the repack when it gets that error.