est merge requestshttps://gitlab.esrf.fr/workflow/ewoksapps/est/-/merge_requests2021-05-07T13:59:18+02:00https://gitlab.esrf.fr/workflow/ewoksapps/est/-/merge_requests/43Include wout hdf5 reading2021-05-07T13:59:18+02:00paynoInclude wout hdf5 reading0.3https://gitlab.esrf.fr/workflow/ewoksapps/est/-/merge_requests/42Test with zocallo2021-04-12T14:05:44+02:00paynoTest with zocallofix issues coming from testing with zocallofix issues coming from testing with zocallo0.3https://gitlab.esrf.fr/workflow/ewoksapps/est/-/merge_requests/38Bm23 proto2021-01-29T08:02:38+01:00paynoBm23 protoTODO
====
- [x] insure data can be read
- [x] add noise operation
- [x] add settings update from the script generated from `est canvas`
- [ ] ~~normalization range end: `E_end – 100`~~
- [ ] ~~Spline range end: `E_end – 50`~~
-...TODO
====
- [x] insure data can be read
- [x] add noise operation
- [x] add settings update from the script generated from `est canvas`
- [ ] ~~normalization range end: `E_end – 100`~~
- [ ] ~~Spline range end: `E_end – 50`~~
- [ ] ~~FT k-range end: `etok(E_end-100)`~~
- [x] in this can be handled by the "caller". What we need to an option to provide parameters from the command line. This can be done providing an url for now but we could also have one option per widget ? `--set-xftf-params param1:value` ... ?
- [x] reproduce data treatment
- [x] update larch plot made from est to fit the one from the slides. Maybe we could use Nexus attributes to define plots ?
- [x] add option to display flat if larch `pre_edge` has been call
- [x] add plot group in process
- [x] define a plot group per process
- [x] define silx style
- [x] define unit that can be used on title
- [x] add title to plots
- [x] add some 'latex attributes'
- [x] add unit test to insure plot, "_NexusDatasetDef" and "_NexusSpectrumDef" are correctly used
- [x] add possible inputs for I0, I1, I2 and mu_ref
- [x] update plot on est
- [x] add plot of noise with Y axis 'log'
- [x] handle HDF5 file
- [x] create a release version to insure we will be able to easily retrieve this version
- [x] enforce h5py3
- [x] insure qt is not requested to execute this script
- [x] add an energy ROI and rename current ROI "position roi"
BM23 datasets
=============
- "100_NO_NH3_scan_at_199 C.dat" -> silx spec file fails to read this one directly.
- can also be accessible from Cu_CHA_68E_Proc5b.h5
display spectrum:
```
est xas-viewer --input-spectra 110.1/measurement/mu_trans@Cu_CHA_68E_Proc5b.h5 --input-channel 110.1/measurement/musst_eneenc@Cu_CHA_68E_Proc5b.h5 --input-energy-unit keV
```
![image](/uploads/4192d4a5635aaf3e00bb1253cdd32620/image.png)
Noise operation
===============
![image](/uploads/da8549d446a26ea23a70235c7f455bd9/image.png)
Results are a bit different but this probably come from
Example of an output file
=========================
[script_test_larch_bm23_output.h5](/uploads/7c5c3fac52d5882d7eb0e33caa4474f6/script_test_larch_bm23_output.h5)
For now there is some attributes from "SILX_STYLE" which are not handled by the viewer as the right axis for plot.
1st generated from: [script_test_larch_bm23.py](/uploads/a1577fd9f74459f8a47652c653a5e6f2/script_test_larch_bm23.py)
![image](/uploads/f3659e13835a12ac079d1ae3ceb447a7/image.png)
2nd generated from [test_bm23.py](/uploads/f5eddff0df30df33f630feb7908a6c23/test_bm23.py): [output_bm23.h5](/uploads/271436cb718239127c190cb7f0107759/output_bm23.h5)
3rd generated from [[test_bm23.py](/uploads/429fe6cb5322e0e8045b73f89054b1c8/test_bm23.py)] : [bm_test_output2.h5](/uploads/1d1f84ae821f46089ea3c298fd2d28e2/bm_test_output2.h5)
final .ows file: [test_bm23.ows](/uploads/1ba09706d9212023dd0ea30262c9e135/test_bm23.ows)0.3https://gitlab.esrf.fr/workflow/ewoksapps/est/-/merge_requests/36look at specfile_data.dat file2020-12-21T13:33:35+01:00paynolook at specfile_data.dat fileDescription / Goal
------------------
Allow the user to provide column names and scan title for spec files.
Add also an option to define 'monitor'
TODO list
---------
- [x] split .xmu and .spec treatment at GUI level
- [x] add column ...Description / Goal
------------------
Allow the user to provide column names and scan title for spec files.
Add also an option to define 'monitor'
TODO list
---------
- [x] split .xmu and .spec treatment at GUI level
- [x] add column name handling for spec file
- [x] add scan selection using scan "title"
- [x] add option to provide monitor.
in this case mu = mu / monitor
- [x] insure data can be load from .h5, .dat and .xmu
- [x] insure 'process' application can still be launched
- [x] rework orange 'xas-input' widget settings to handle the new settings
- [x] update application options to handle column names and scan title (process and xas-viewer)
- [x] handle 'data_path' for DataUrl with spec&PyMca
INFO (how to use...)
--------------------
options have been added to specify the column names:
``` bash
clear; est xas-viewer --input specfiledata_tests.dat --abs-col-name zap_det_dtc --energy-col-name arr_hdh_ene --scan-title "fscan2 7.105 7.18 60 5e-05" --monitor-col-name arr_I02sum
```
close #27
Any warning ?
-------------
The plot window could take more space I guess. Might be done in another PR.
associated materials (screenshot, test scripts...)
--------------------------------------------------
![image](/uploads/7214f91773e4111bc76056c38a37cf8d/image.png)0.3https://gitlab.esrf.fr/workflow/ewoksapps/est/-/merge_requests/35Rework results save2020-12-17T11:37:43+01:00paynoRework results savegroup all results in a common h5py.Group per processgroup all results in a common h5py.Group per process0.3https://gitlab.esrf.fr/workflow/ewoksapps/est/-/merge_requests/34[core][dim] rename energy / channel dim Z2020-12-16T14:31:31+01:00payno[core][dim] rename energy / channel dim Z0.3https://gitlab.esrf.fr/workflow/ewoksapps/est/-/merge_requests/33Add ows to script2020-12-15T16:57:06+01:00paynoAdd ows to scriptDescription / Goal
------------------
Add a new application to est: `ows-to-script`: convert a workflow defined in a .ows file to a python script executing this workflow.
TODO list
---------
- [x] add ows-to-script function (close #3...Description / Goal
------------------
Add a new application to est: `ows-to-script`: convert a workflow defined in a .ows file to a python script executing this workflow.
TODO list
---------
- [x] add ows-to-script function (close #35 )
- [x] doc
INFO (how to use...)
--------------------
- xas_input is ignored
- Can define the input `XasObject` from script parameters. This is why xas_input widget are ignored. We only consider one input for now. Current default parameter handled by the input script:
``` python
-h, --help show this help message and exit
-i INPUT_, --input INPUT_
Input of the workflow. Should be a path to a file
--input-spectra INPUT_SPECTRA
Input spectra url
--input-spectra-dims INPUT_SPECTRA_DIMS
Input spectra dimension. Should be a tuple of three
values: (X,Y,channel). If None will take the default
dimension according to the input type.
--input-channel INPUT_CHANNEL
Input channel url (usually energy)
--input-configuration INPUT_CONFIGURATION
Input configuration url
--input-energy-unit INPUT_ENERGY_UNIT
energy unit
--input-dimensions INPUT_DIMENSIONS
dimension of the input as (channel, Y, X) for
example.If None will take default unit according to
the input type
```
- output file is not handled and should be defined in the xas_output parameters (as relative) for now. Could be an improvement.
usage example
-------------
call ows:
``` bash
est ows-to-script workflow.ows workflow_as_script.py --overwrite
```
execute the file
``` bash
python script_test_pymca.py --input EXAFS_Cu.dat
```
Tested with the following workflow:
![image](/uploads/9cf9bb3eb9a4f56254ecdd9d1adfc9f9/image.png)
Any warning ?
-------------
defining on the fly the output file is not handled for now.0.3https://gitlab.esrf.fr/workflow/ewoksapps/est/-/merge_requests/32[refactoring] split types.py class2020-12-15T09:15:17+01:00payno[refactoring] split types.py classDescription / Goal
------------------
TODO list
---------
- [x] add Spectra class with API
- [x] split types into a submodule
- [x] clean API / call from xas_object
- [ ] ~~add offset information~~ -> must be done in another PR. There ...Description / Goal
------------------
TODO list
---------
- [x] add Spectra class with API
- [x] split types into a submodule
- [x] clean API / call from xas_object
- [ ] ~~add offset information~~ -> must be done in another PR. There is some GUI to be updated too.
- [x] doc
- [x] add test
- [x] add test on the spectra API (using python slices...)
INFO (how to use...)
--------------------
close #24
Any warning ?
-------------
associated materials (screenshot, test scripts...)
--------------------------------------------------0.3https://gitlab.esrf.fr/workflow/ewoksapps/est/-/merge_requests/31update xas input - add xas-viewer2020-12-16T07:41:45+01:00paynoupdate xas input - add xas-viewerDescription / Goal
------------------
upgrade the xas input
TODO list
---------
- [x] plot data for xas_input acording to url, dimensions...
- [x] add dimensions for 2D data (.dat files for example)
- [x] use `SpectrumViewer` to...Description / Goal
------------------
upgrade the xas input
TODO list
---------
- [x] plot data for xas_input acording to url, dimensions...
- [x] add dimensions for 2D data (.dat files for example)
- [x] use `SpectrumViewer` to display spectrum loaded
- [x] add a raw data viewer for spectrum
- [x] update automatically plot / raw data when data path change or dimension
- [x] add a `xas-viewer` application able to load and display input data (from command line)
INFO (how to use...)
--------------------
Most of the modifications are visible from the xas_input.
The new application xas-viewer can be used from command line:
``` bash
# from a simple ASCII file:
est xas-viewer --input EXAFS_Cu.dat
# from a hdf5 file using the '@' syntax
est xas-viewer --input-spectra data@B33_test_1.h5 --input-channel energy@B33_test_1.h5
# from a hdf5 file using silx DataUrl
est xas-viewer --input-spectra "silx:///[...]B33_test_1.h5?path=/data" --input-channel "silx:///[...]B33_test_1.h5?path=/energy"
```
Any warning ?
-------------
For now we provide meaning of each dimension (dimension 0 is channel, dimension1 is X...) when it looks like for x-ray larch this is the other way around for x, y... user should provide the dimension (X is dimension1, Y on dimension 2...).
Not sure to know what is the best.
More info
---------
- close #29
- close #20
- close #28
associated materials (screenshot, test scripts...)
--------------------------------------------------
![Screenshot_from_2020-12-11_11-08-14](/uploads/56a04b69073a76ad35e20b76c19672c8/Screenshot_from_2020-12-11_11-08-14.png)
![Screenshot_from_2020-12-11_11-08-23](/uploads/051dfa7be00ba2f1ef0d25630b9f43e4/Screenshot_from_2020-12-11_11-08-23.png)
![Screenshot_from_2020-12-11_11-10-55](/uploads/4fca1ab1d347e59bec2b7583d8c27a21/Screenshot_from_2020-12-11_11-10-55.png)0.3https://gitlab.esrf.fr/workflow/ewoksapps/est/-/merge_requests/30Fix exec pre 0 32020-12-09T17:10:19+01:00paynoFix exec pre 0 3Description / Goal
------------------
fix and rework exec since using pypushflow and project has been leave for a while
TODO list
---------
- [x] fix exec application
- [x] handle url from the exec function
INFO (how to use...)
-----...Description / Goal
------------------
fix and rework exec since using pypushflow and project has been leave for a while
TODO list
---------
- [x] fix exec application
- [x] handle url from the exec function
INFO (how to use...)
--------------------
An example on how to call exec with url (in this case using the `@` but [silx url](http://www.silx.org/doc/silx/latest/modules/io/url.html) should be more commonly used)
``` bash
est process ~/Documents/test_pymca.ows --input-spectra data@B33_test_1.h5 --input-channel energy@B33_test_1.h5 --output B33_output.h5
```0.3https://gitlab.esrf.fr/workflow/ewoksapps/est/-/merge_requests/29project update for 0.32020-12-09T08:17:53+01:00paynoproject update for 0.3TODO
====
- [x] add pages. close #26
- [x] add gitlab issue / PR pattern
- [x] rework canvas launcher
- [x] add some python typing on core classes close #16
- [x] set plot data margins. close #17TODO
====
- [x] add pages. close #26
- [x] add gitlab issue / PR pattern
- [x] rework canvas launcher
- [x] add some python typing on core classes close #16
- [x] set plot data margins. close #170.3https://gitlab.esrf.fr/workflow/ewoksapps/est/-/merge_requests/28replace pushworkflow by pypushflow2020-12-08T10:08:31+01:00paynoreplace pushworkflow by pypushflowTODO
====
- [x] remove pushworkflow
- [x] add pypushflow
- [x] add pypushflow add on and aliases
- [x] adapt application
- [x] adapt unittest
- [x] move dependency to h5py3
- [x] update handle python versionTODO
====
- [x] remove pushworkflow
- [x] add pypushflow
- [x] add pypushflow add on and aliases
- [x] adapt application
- [x] adapt unittest
- [x] move dependency to h5py3
- [x] update handle python version0.3https://gitlab.esrf.fr/workflow/ewoksapps/est/-/merge_requests/26Add unitsystem2020-12-08T16:17:30+01:00paynoAdd unitsystemGOAL
====
Allows load of dataset with an energy not defined in eV.
extra
-----
- close #31
- close #30
- xas input: drag/from handling
update utils.input
------------------
![Screenshot_from_2020-03-31_16-24-28](/uploads/55ffe28c98...GOAL
====
Allows load of dataset with an energy not defined in eV.
extra
-----
- close #31
- close #30
- xas input: drag/from handling
update utils.input
------------------
![Screenshot_from_2020-03-31_16-24-28](/uploads/55ffe28c98ac956aa0aaaf973456e59e/Screenshot_from_2020-03-31_16-24-28.png)
![Screenshot_from_2020-03-31_16-24-19](/uploads/366bc3b2e25fea801551be471def4e6e/Screenshot_from_2020-03-31_16-24-19.png)0.3https://gitlab.esrf.fr/workflow/ewoksapps/est/-/merge_requests/22Fix 202020-01-13T11:26:10+01:00paynoFix 20* [x] manage different dimensions for 3D spectra
* [x] load and save dimensions from the configuration
* [x] add an interface to estimate E0 (from pymca and larch process) ?
* [x] integrate the E0 calculator in the existing interfaces...* [x] manage different dimensions for 3D spectra
* [x] load and save dimensions from the configuration
* [x] add an interface to estimate E0 (from pymca and larch process) ?
* [x] integrate the E0 calculator in the existing interfaces (pymca & larch)
* [x] add a E0 calculator widget that is compatible with larch and pymca processes.
* [x] test with B33_30mu_1f_noEXAFS dataset
* [ ] ~~unable image ROI selection. should be always active~~ -> to be discuss later. Maybe not showing the RoiManager is simpler and better for the users. ?
* [x] Input widget
* [ ] ~~should run the load without freezing the gui.~~ we will see this later
* [x] should be a Dialog with when start is selected it should hide the widget
* [x] remove copy of hdf5 files
* [x] add a widget to make a copy - saving point
* [x] error log message should specify the spectrum position when process fails.
For now we are patching the XASNormalizationParameters with the E0CalculationDialog. This part could be moved to pymca ?0.3https://gitlab.esrf.fr/workflow/ewoksapps/est/-/merge_requests/20[refactoring] rename project est instead of xas2019-09-23T16:52:49+02:00payno[refactoring] rename project est instead of xas![icon.svg](/uploads/b4b64990ad6b64b904f06dce858d593b/icon.svg)![icon.svg](/uploads/b4b64990ad6b64b904f06dce858d593b/icon.svg)0.3