est issueshttps://gitlab.esrf.fr/workflow/ewoksapps/est/-/issues2022-07-19T08:29:16+02:00https://gitlab.esrf.fr/workflow/ewoksapps/est/-/issues/69remove saving processes2022-07-19T08:29:16+02:00paynoremove saving processessaving all the processing is clearly not adapted with launch an ewoks workflow each 2 seconds.
This should be either turn off definitely or set as an option.saving all the processing is clearly not adapted with launch an ewoks workflow each 2 seconds.
This should be either turn off definitely or set as an option.paynopayno2022-07-20https://gitlab.esrf.fr/workflow/ewoksapps/est/-/issues/81I cannot select the mu column from EXAFS_Cu.dat2024-02-12T16:34:06+01:00Wout De NolfI cannot select the mu column from EXAFS_Cu.dat![image](/uploads/d40ed66640f1315aea61e61002df6249/image.png)
The second column is always equal to the first column, despite the fact that I selected the second column.![image](/uploads/d40ed66640f1315aea61e61002df6249/image.png)
The second column is always equal to the first column, despite the fact that I selected the second column.https://gitlab.esrf.fr/workflow/ewoksapps/est/-/issues/79Add CONTRIBUTING guide2024-02-08T16:26:48+01:00Loic HuderAdd CONTRIBUTING guideFor:
- Description of the install for dev
- Description of linting/testing
- Put doc generation instructions: https://gitlab.esrf.fr/workflow/ewoksapps/est/-/merge_requests/128#note_296969For:
- Description of the install for dev
- Description of linting/testing
- Put doc generation instructions: https://gitlab.esrf.fr/workflow/ewoksapps/est/-/merge_requests/128#note_296969https://gitlab.esrf.fr/workflow/ewoksapps/est/-/issues/78CI: test_get_data is falky2023-10-16T10:48:33+02:00Wout De NolfCI: test_get_data is falkyhttps://gitlab.esrf.fr/workflow/ewoksapps/est/-/jobs/819171
https://gitlab.esrf.fr/workflow/ewoksapps/est/-/jobs/816122
```_____________________________ test_get_data[0.01] ______________________________
period = 0.01
spectrum_cu_from...https://gitlab.esrf.fr/workflow/ewoksapps/est/-/jobs/819171
https://gitlab.esrf.fr/workflow/ewoksapps/est/-/jobs/816122
```_____________________________ test_get_data[0.01] ______________________________
period = 0.01
spectrum_cu_from_larch = <est.core.types.spectrum.Spectrum object at 0x7f97f3c63ee0>
tmpdir = local('/tmp/pytest-of-root/pytest-0/test_get_data_0_01_0')
@pytest.mark.parametrize("period", [0.01, 0.1, 0.5])
def test_get_data(period, spectrum_cu_from_larch, tmpdir):
filename = str(tmpdir / "data.h5")
scan = "1.1"
npoints = len(spectrum_cu_from_larch.energy)
positioners = {"energy": spectrum_cu_from_larch.energy}
detectors = {"mu": spectrum_cu_from_larch.mu}
npoints = spectrum_cu_from_larch.energy.size
blocksize = max(int(npoints / 10), 1)
est_time = npoints / blocksize * period
tmax = time.time() + est_time + 5
print("Estimated scan time", est_time, "s")
urls = [
f"silx://{filename}::/{scan}/measurement/energy",
f"silx://{filename}::/{scan}/measurement/mu",
]
with nxwriter_process(filename, scan, positioners, detectors, blocksize, period):
nprogress = 0
while nprogress != npoints:
> data = [get_data(url, retry_timeout=3) for url in urls]
src/est/tests/test_acquisition.py:34:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
src/est/tests/test_acquisition.py:34: in <listcomp>
data = [get_data(url, retry_timeout=3) for url in urls]
/usr/local/lib/python3.8/site-packages/silx/utils/retry.py:165: in wrapper
return method(*args, **kw)
src/est/io/utils/read.py:58: in get_data
dset = h5[data_path]
h5py/_objects.pyx:54: in h5py._objects.with_phil.wrapper
???
h5py/_objects.pyx:55: in h5py._objects.with_phil.wrapper
???
/usr/local/lib/python3.8/site-packages/h5py/_hl/group.py:357: in __getitem__
oid = h5o.open(self.id, self._e(name), lapl=self._lapl)
h5py/_objects.pyx:54: in h5py._objects.with_phil.wrapper
???
h5py/_objects.pyx:55: in h5py._objects.with_phil.wrapper
???
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> ???
E KeyError: 'Unable to synchronously open object (addr overflow, addr = 6416, size = 328, eoa = 2048)'
h5py/h5o.pyx:189: KeyError
----------------------------- Captured stdout call -----------------------------
```paynopaynohttps://gitlab.esrf.fr/workflow/ewoksapps/est/-/issues/76xraylarch cannot handle data with NaN's2023-10-02T17:22:04+02:00Wout De Nolfxraylarch cannot handle data with NaN's```python
Traceback (most recent call last):
File "/users/blissadm/conda/miniconda/envs/ewoksworker/lib/python3.9/site-packages/ewokscore/task.py", line 401, in execute
self.run()
File "/users/blissadm/conda/miniconda/envs/ewoksw...```python
Traceback (most recent call last):
File "/users/blissadm/conda/miniconda/envs/ewoksworker/lib/python3.9/site-packages/ewokscore/task.py", line 401, in execute
self.run()
File "/users/blissadm/conda/miniconda/envs/ewoksworker/lib/python3.9/site-packages/est/core/process/larch/autobk.py", line 122, in run
self._pool_process(xas_obj=_xas_obj)
File "/users/blissadm/conda/miniconda/envs/ewoksworker/lib/python3.9/site-packages/est/core/process/larch/autobk.py", line 131, in _pool_process
process_spectr_autobk(
File "/users/blissadm/conda/miniconda/envs/ewoksworker/lib/python3.9/site-packages/est/core/process/larch/autobk.py", line 77, in process_spectr_autobk
autobk(
File "/users/blissadm/conda/miniconda/envs/ewoksworker/lib/python3.9/site-packages/larch/larchlib.py", line 531, in wrapper
result = fcn(*args, **kwargs)
File "/users/blissadm/conda/miniconda/envs/ewoksworker/lib/python3.9/site-packages/larch/xafs/autobk.py", line 194, in autobk
result = minimize(__resid, params, method='leastsq',
File "/users/blissadm/conda/miniconda/envs/ewoksworker/lib/python3.9/site-packages/lmfit/minimizer.py", line 2601, in minimize
return fitter.minimize(method=method)
File "/users/blissadm/conda/miniconda/envs/ewoksworker/lib/python3.9/site-packages/lmfit/minimizer.py", line 2345, in minimize
return function(**kwargs)
File "/users/blissadm/conda/miniconda/envs/ewoksworker/lib/python3.9/site-packages/lmfit/minimizer.py", line 1651, in leastsq
lsout = scipy_leastsq(self.__residual, variables, **lskws)
File "/users/blissadm/conda/miniconda/envs/ewoksworker/lib/python3.9/site-packages/scipy/optimize/_minpack_py.py", line 415, in leastsq
shape, dtype = _check_func('leastsq', 'func', func, x0, args, n)
File "/users/blissadm/conda/miniconda/envs/ewoksworker/lib/python3.9/site-packages/scipy/optimize/_minpack_py.py", line 25, in _check_func
res = atleast_1d(thefunc(*((x0[:numinputs],) + args)))
File "/users/blissadm/conda/miniconda/envs/ewoksworker/lib/python3.9/site-packages/lmfit/minimizer.py", line 562, in __residual
return coerce_float64(out, nan_policy=self.nan_policy)
File "/users/blissadm/conda/miniconda/envs/ewoksworker/lib/python3.9/site-packages/lmfit/minimizer.py", line 2445, in coerce_float64
raise ValueError(msg)
ValueError: NaN values detected in your input data or the output of your objective/model function - fitting algorithms cannot handle this! Please read https://lmfit.github.io/lmfit-py/faq.html#i-get-errors-from-nan-in-my-fit-what-can-i-do for more information.
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/users/blissadm/conda/miniconda/envs/ewoksworker/lib/python3.9/site-packages/celery/app/trace.py", line 477, in trace_task
R = retval = fun(*args, **kwargs)
File "/users/blissadm/conda/miniconda/envs/ewoksworker/lib/python3.9/site-packages/celery/app/trace.py", line 760, in __protected_call__
return self.run(*args, **kwargs)
File "/users/blissadm/conda/miniconda/envs/ewoksworker/lib/python3.9/site-packages/ewoksjob/apps/ewoks.py", line 25, in wrapper
return method(self, *args, **kwargs)
File "/users/blissadm/conda/miniconda/envs/ewoksworker/lib/python3.9/site-packages/ewoksjob/worker/task.py", line 18, in wrapper
return func(*args, **kwargs)
File "/users/blissadm/conda/miniconda/envs/ewoksworker/lib/python3.9/site-packages/ewoksjob/apps/ewoks.py", line 34, in execute_graph
return ewoks.execute_graph(*args, **kwargs)
File "/users/blissadm/conda/miniconda/envs/ewoksworker/lib/python3.9/site-packages/ewoks/bindings.py", line 78, in execute_graph
result = mod.execute_graph(graph, execinfo=execinfo, **execute_options)
File "/users/blissadm/conda/miniconda/envs/ewoksworker/lib/python3.9/site-packages/ewokscore/events/contexts.py", line 25, in wrapper
return method(*args, execinfo=execinfo, **kw)
File "/users/blissadm/conda/miniconda/envs/ewoksworker/lib/python3.9/site-packages/ewokscore/bindings.py", line 56, in execute_graph
return sequential.execute_graph(taskgraph.graph, **execute_options)
File "/users/blissadm/conda/miniconda/envs/ewoksworker/lib/python3.9/site-packages/ewokscore/graph/execute/sequential.py", line 122, in execute_graph
task.execute(
File "/users/blissadm/conda/miniconda/envs/ewoksworker/lib/python3.9/site-packages/ewokscore/task.py", line 407, in execute
raise RuntimeError(f"Task '{self.label}' failed") from e
RuntimeError: Task 'autobk' failed
```https://gitlab.esrf.fr/workflow/ewoksapps/est/-/issues/74Typing a URL in the input widget is slow2022-09-26T14:15:55+02:00Wout De NolfTyping a URL in the input widget is slowI think this is because we have an event after every letter while we want and event and the END of input (change focus or press enter).I think this is because we have an event after every letter while we want and event and the END of input (change focus or press enter).paynopaynohttps://gitlab.esrf.fr/workflow/ewoksapps/est/-/issues/73CI fails due to new flake8-nb version2022-08-17T13:22:20+02:00Wout De NolfCI fails due to new flake8-nb versionCI fails: https://gitlab.esrf.fr/workflow/ewoksapps/est/-/jobs/418908
Not sure what's going on because E402 is ignored in setup.cfg:
```
# E402 (module import not at top of file) ignored
[flake8_nb]
ignore = E501, E203, W503, E402
max-...CI fails: https://gitlab.esrf.fr/workflow/ewoksapps/est/-/jobs/418908
Not sure what's going on because E402 is ignored in setup.cfg:
```
# E402 (module import not at top of file) ignored
[flake8_nb]
ignore = E501, E203, W503, E402
max-line-length = 88
```paynopaynohttps://gitlab.esrf.fr/workflow/ewoksapps/est/-/issues/72Plot “flat” group instead of “norm” if “make flat” is ticked2022-07-19T15:25:36+02:00Wout De NolfPlot “flat” group instead of “norm” if “make flat” is tickedWout De NolfWout De Nolfhttps://gitlab.esrf.fr/workflow/ewoksapps/est/-/issues/71Feedback on parameters2022-07-21T10:24:06+02:00Wout De NolfFeedback on parametersWidgets:
- Autobk widget: Rbkg and kmin must be float
- XFTF widget: Kmin, kmax, dk, dk2 must be float
Re-define parameters:
- Noise: Make Estart and Eend relative to the Edge energy. Default values can be the same as for norm range in ...Widgets:
- Autobk widget: Rbkg and kmin must be float
- XFTF widget: Kmin, kmax, dk, dk2 must be float
Re-define parameters:
- Noise: Make Estart and Eend relative to the Edge energy. Default values can be the same as for norm range in autobk.
- XFTF and Autobk: k_min defaults to 3, k_max defaults to 0.9*max(kvalues)paynopaynohttps://gitlab.esrf.fr/workflow/ewoksapps/est/-/issues/70Plot chi(k)*k^n instead of chi(k)2022-07-21T16:52:53+02:00Wout De NolfPlot chi(k)*k^n instead of chi(k)Wout De NolfWout De Nolfhttps://gitlab.esrf.fr/workflow/ewoksapps/est/-/issues/68bm23 feedback2022-07-21T10:24:06+02:00paynobm23 feedback* feedbacks on preedge:
* plots of `pre_edge` and `post_edge` seems to provide the same curve. Is it a display issue or something else ? should be fixed from online
* insure parameters are explains somewhere: with tooltip or from a...* feedbacks on preedge:
* plots of `pre_edge` and `post_edge` seems to provide the same curve. Is it a display issue or something else ? should be fixed from online
* insure parameters are explains somewhere: with tooltip or from an interrogataion mark. Is a tooltip enought ?
* add plot of `spectrum.flatten_mu`
* autobk and rbkg should be floating and not integers
* rename exponent to `nvict` factor (give link to the original parameter name)
* autobk: replace the `spectrum` plot by the `normalized mu`
* xrcf
* replace k_max by percentage. k_min to 3. K can be get from previous computing.
* `k_min` and `k_max` must be floats
* kweight is present in xfcf and autobk but are two differentes elements. Precisions should be provided for it.https://gitlab.esrf.fr/workflow/ewoksapps/est/-/issues/67The ows properties should only ewoks Settings2022-07-05T11:00:03+02:00Wout De NolfThe ows properties should only ewoks SettingsWhen saving the ows file, the node properties should only contain these settings:
```python
namespace["_ewoks_default_inputs"] = Setting(dict(), schema_only=schema_only)
namespace["_ewoks_varinfo"] = Setting(dict(), schema_only=schema_o...When saving the ows file, the node properties should only contain these settings:
```python
namespace["_ewoks_default_inputs"] = Setting(dict(), schema_only=schema_only)
namespace["_ewoks_varinfo"] = Setting(dict(), schema_only=schema_only)
namespace["_ewoks_execinfo"] = Setting(dict(), schema_only=schema_only)
# Deprecated:
namespace["default_inputs"] = Setting(dict(), schema_only=schema_only)
```
All the other Settings will be est widgets add will be discarded when executing headless.paynopaynohttps://gitlab.esrf.fr/workflow/ewoksapps/est/-/issues/66XAS input: use HDF5 as the default2022-07-05T11:41:39+02:00Wout De NolfXAS input: use HDF5 as the defaulthttps://gitlab.esrf.fr/workflow/ewoksapps/est/-/issues/65Input widget: HDF5 selection start from what is in text box2022-07-05T12:04:49+02:00Wout De NolfInput widget: HDF5 selection start from what is in text boxWout De NolfWout De Nolfhttps://gitlab.esrf.fr/workflow/ewoksapps/est/-/issues/64BM23: execute example_bm23 when saved before2022-07-05T11:42:58+02:00Wout De NolfBM23: execute example_bm23 when saved before```
Traceback (most recent call last):
File "/users/blissadm/conda/miniconda/envs/ewoksworker/lib/python3.7/site-packages/ewokscore/task.py", line 337, in execute
self.run()
File "/users/blissadm/local/est/est/core/process/noise....```
Traceback (most recent call last):
File "/users/blissadm/conda/miniconda/envs/ewoksworker/lib/python3.7/site-packages/ewokscore/task.py", line 337, in execute
self.run()
File "/users/blissadm/local/est/est/core/process/noise.py", line 109, in run
self._pool_process(xas_obj=_xas_obj)
File "/users/blissadm/local/est/est/core/process/noise.py", line 131, in _pool_process
overwrite=True,
File "/users/blissadm/local/est/est/core/process/noise.py", line 71, in process_noise_savgol
spectrum.norm_noise_savgol = numpy.mean(noise) / spectrum.edge_step
TypeError: unsupported operand type(s) for /: 'float' and 'NoneType'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/users/blissadm/conda/miniconda/envs/ewoksworker/lib/python3.7/site-packages/ewoksorange/bindings/taskexecutor.py", line 31, in execute_task
self.__task.execute()
File "/users/blissadm/conda/miniconda/envs/ewoksworker/lib/python3.7/site-packages/ewokscore/task.py", line 343, in execute
raise RuntimeError(f"Task '{self.label}' failed") from e
RuntimeError: Task 'noise' failed
```paynopaynohttps://gitlab.esrf.fr/workflow/ewoksapps/est/-/issues/61orangecontrig: remove control area2022-06-28T18:28:58+02:00paynoorangecontrig: remove control areaThe control area takes too much space from my point of view. We should removed it.The control area takes too much space from my point of view. We should removed it.https://gitlab.esrf.fr/workflow/ewoksapps/est/-/issues/59Input widget: plot doesn't update when loading an xmu file2022-07-01T20:51:46+02:00Wout De NolfInput widget: plot doesn't update when loading an xmu filepaynopaynohttps://gitlab.esrf.fr/workflow/ewoksapps/est/-/issues/58Task that creates input for Flint plot2022-06-29T13:51:13+02:00Wout De NolfTask that creates input for Flint plotUse case in Bliss+Flint (see `execute_workflow` for what the workflow needs to return):
```python
from time import sleep
import numpy
from bliss.common.plot import get_flint
def execute_workflow():
return {
"plot1": {
...Use case in Bliss+Flint (see `execute_workflow` for what the workflow needs to return):
```python
from time import sleep
import numpy
from bliss.common.plot import get_flint
def execute_workflow():
return {
"plot1": {
"x": numpy.arange(100),
"y": numpy.random.random(100),
"xlabel": "Energy (eV)",
"ylabel": "mu",
},
"plot2": {
"x": numpy.arange(100),
"y": numpy.random.random(100) + 10,
"xlabel": "Wavenumber (Å)",
"ylabel": "k^2 χ(k) (Å^2)",
},
"plot3": {
"x": numpy.arange(100),
"y": numpy.random.random(100) + 20,
"xlabel": "Radial Distance (Å)",
"ylabel": "|χ(R)| (Å^3)",
},
"plot4": {
"x": numpy.arange(100),
"y": numpy.random.random(100) + 30,
"xlabel": "Energy (eV)",
"ylabel": "Noise",
},
}
def prepare_scan_plot(plots, scan, data):
for name, p in plots.items():
with p.transaction():
p.add_curve_item("x", "y", legend=scan)
p.add_curve_item("x", "y", legend=scan)
p.xlabel = data[name]["xlabel"]
p.ylabel = data[name]["ylabel"]
def refresh_scan_plot(plots, data):
for name, p in plots.items():
with p.transaction():
d = data[name]
p.set_data(x=d["x"], y=d["y"])
def plot_exafs(scan="1.1"):
f = get_flint()
plots = dict()
plots["plot1"] = f.get_plot("curve", unique_name="plot1")
plots["plot2"] = f.get_plot("curve", unique_name="plot2")
plots["plot3"] = f.get_plot("curve", unique_name="plot3")
plots["plot4"] = f.get_plot("curve", unique_name="plot4")
# loop until the end of the scan (interval of n seconds)
for i in range(10):
data = execute_workflow()
if i == 0:
prepare_scan_plot(plots, scan, data)
refresh_scan_plot(plots, data)
sleep(1)
return f, plots
```
Or with a grid https://gitlab.esrf.fr/bliss/bliss/-/merge_requests/4744Wout De NolfWout De Nolfhttps://gitlab.esrf.fr/workflow/ewoksapps/est/-/issues/57Est tasks should not absorb exceptions2022-07-05T11:41:39+02:00Wout De NolfEst tasks should not absorb exceptionsFor example
https://gitlab.esrf.fr/workflow/ewoksapps/est/-/blob/main/est/core/process/larch/autobk.py#L86
`ewoksorange` makes sure the exception is captured for the widgets
https://gitlab.esrf.fr/workflow/ewoks/ewoksorange/-/blob/mai...For example
https://gitlab.esrf.fr/workflow/ewoksapps/est/-/blob/main/est/core/process/larch/autobk.py#L86
`ewoksorange` makes sure the exception is captured for the widgets
https://gitlab.esrf.fr/workflow/ewoks/ewoksorange/-/blob/main/ewoksorange/bindings/owwidgets.py#L405paynopaynohttps://gitlab.esrf.fr/workflow/ewoksapps/est/-/issues/56Test workflows for data produced during a scan2022-06-27T15:39:35+02:00Wout De NolfTest workflows for data produced during a scanWout De NolfWout De Nolf