Handle config file + dark, ref, proj and return names

Merged payno requested to merge handle_config_file into master

Description / Goal

  • define dark, ref, projection and returns by there group names (in this case 'titles' will be ignore) For this feature we will handle urls. But we expect to have path pointing to HDF5Group and no dataset. So (of course) slices will be rejected.
  • add an option to provide a configuration name to nxtomomill h52nx. In this case it 'overwrite' the settings.py file (for the run) and no other options should be provided
  • create some utils command to generate configuration file

input example files

TODO list

  • create the io module
    • create HDFConfig class: used to store the configuration for the conversion
    • create HDF5ConfigHandler: define a set of function to filter inputs from the HDF5config: for example when titles are provided and some node names for dark, flat... are provided too. Insure also the validity of a configuration
    • integrate HDFConfig with acquisition module and the hdf5converter.
  • handle a configuration file (basically overwrite the settings.py file).
    • nxtomomill h52nx --config [config_file]
  • create a quick start for h5 to generate a configuration file
    • nxtomomill h5-quick-start output_file.cfg
    • add options --from-title-names and --from-urls
  • move entries and ignore_sub_entries to DataUrl in order to handle input file as well.
  • should handle input file from different files. Urls can be provided or just group files.
  • doc
    • html doc
    • update training to present this
  • add test when using configuration with ̀FrameGroup
    • test the Standard acquisition
    • test zseries
  • deprecate the API of h5tonx function
  • add a "initialization" entry which could contain start_time and end_time for example
  • check slices are handled or raise an error
  • handle copy parameter
  • add an example copying dataset and not and check if a copy is made
  • updating design: for now design is: we open a file and we browse it and create the .nx at the same time. it might be wiser to create an uri during preprocessing for each parameter of the AcquisitionBase like rotation_uri, frame_uri, x_translation_uri... and used those during the write step. This would be more convenient for copying frame dataset (we could only copy and update the uri) which would bring visiblity to the existing code. This could also add flexibility to the AcquisitionBase. But as nxtomomill might integrate other acquisition style way different this is not the time to go for this kind of refactoring I guess.

INFO (how to use...)

This PR add an io module close #53 (closed)

Any warning ?

the "data_scans" values from the FRAME_TYPE_SECTION section should be provided with indentation (at least one empty space. Otherwise they will be ignored by configparser.

associated materials (screenshot, test scripts...)

Frame type

user can now provide a list of "FrameGroup" that refer to a scan and which should fit a type. Those can only be provided by the configuration file. The dedicated section looks like:

# optional section
# allows to define scan to be used for nxtomo conversion
# the sequence order will follow the order provided.

# list of scans to be converted. frame type should be provided for each scan.
# expected format is:

# data_scans = (
#     (frame_type=projections, entry=silx:///path/to/file?/path/to/scan/node, copy=true),
#     (frame_type=projections, entry=/path_relative_to_file),
# 	)
# * `frame_type` (mandatory): values can be `projection`, `flat`, `dark`, `alignment` or `init`. 
# * `entry` (mandatory): dataurl with path to the scan to integrate. if the scan is contained in the input_file then you can only provide path/name of the scan. 
# * copy (optional): you can provide a different behavior for the this scan (should we duplicate data or not) 
# more details are available here: todo
data_scans = (
		(frame_type=init, entry=1.1),
		(frame_type=dark, entry=2.1),
		(frame_type=flat, entry=3.1),
		(frame_type=projection, entry=4.1),
		(frame_type=flat, entry=5.1),
		(frame_type=projection, entry=6.1),
		(frame_type=alignment, entry=7.1),
# you can duplicate data inside the input file or create a link to the original frames. in this case you should keep the relative position of the files
default_data_copy = False
Edited by payno

Merge request reports