Skip to content

Draft: Integrate NXTomoReader in pipeline

Pierre Paleo requested to merge nxreader_integration into master

About

!439 (merged) must be merged first

Replace ChunkReader with NXTomoReader in reconstruction pipeline.

To do

  • Check that excluding projections works in NXTomoReader
  • Update "exclude projections" to be used with NXTomoReader (in DatasetAnalyzer)
  • Integrate in pipeline
  • Update changelog/documentation
  • End-to-end reconstruction test

Notes

This will have quite some impact on chunked.py, as configuring subsampling was also quite complicated (subsample the list of DataUrls, save a copy somewhere, and then restore after loading is done).

To avoid mixing the old mechanism (ChunkReader) and the new one, it's best to first add the EDF Stack reader.

Notes (obsolete)

For now pipeline.fullfield.chunked does something like

if dataset_info.kind == "edf":
    # use ChunkReader with DataUrls built by EDFTomoScan

elif dataset_info.kind == "hdf5":
    # use NXTomoReader

Which is not ideal as the goal is to get rid of ChunkReader and the cumbersome list of DataUrl. A EDF-stack reader will have to be created with the same logic as NXTomoReader.

Generally speaking, when it comes to browsing a dataset and reading the data, there are two kind of classes to consider:

  • A "dataset information holder": (HDF5TomoScan, EDFTomoScan, FScan)
  • A reader (one for each type)

Perhaps the two could be merged, though it's probably good to keep readers simple so that they can be used in a non-ESRF-centric way.

Edited by Pierre Paleo

Merge request reports