edf2nx: Add output check and delete source files.
TODO
-
add an option to remove EDF source file -
add a method to check the output volume. for this we have several ways to handle it - get a boolean
check_output_volume
-> for me this does not bring enough option to fine tune. Indeed I think in most of the cases we want to do a quick check (checking Virtual dataset) as in thetomoscan validators
but sometimes (when removing source files - ID16A) we probably want to do more advanced stuff like comparing frame by frame the final vs raw volumes. - get an integer to define a level (0: no check, 1: check for broken/invalid links on the final volume or frame at zero, 2: compare frame by frame)
- get a list of possible checks like -> this is the option selected
- empty: no check done
-
check_vds_links
: check for broken virtual source -
compare_volumes
: compare input volume and output volume -
check_vds_links, compare_volumes
: do both
- get a boolean
-
add unit test -
test on a real dataset -
add CLI option for checking volume and removing input because I think ID16a script are using directly the CLI -
dedicated handling for multi edf files
limitations
- the .info will not be removed. This can be added but not so sure about the interest (especially in the case this is not the default one ("{basename}.info") but another one. This could lead to important technical mishap from users and with no ways to go back.
- For now the two options to access it are only available on 'advanced'
edf2nx
configuration file. But looks like to be the usage. If not we can still add it later to the CLI directly. - I am not sure this test is enough ? But the core of the processing from my point if view is more on the algorithm behind
edf2nx
.
usage
from the configuration file
this PR adds two parameters to the edf2nx
configuration file under the general section
:
-
delete_edf_source_file
: as a boolean. False is the default, if True then the .edf used during the conversion will be removed. -
output_checks
: as a tuple of str. For now the only str available iscompare-output-volume
which test frame by frame the url used and the final HDF5 volume
from the command line
nxtomomill edf2nx {} --output-checks compare-output-volume --delete-edf
Edited by payno