darfix issueshttps://gitlab.esrf.fr/XRD/darfix/-/issues2024-03-26T12:41:50+01:00https://gitlab.esrf.fr/XRD/darfix/-/issues/102On windows: orange-canvas returns immediately2024-03-26T12:41:50+01:00Wout De NolfOn windows: orange-canvas returns immediately```
(py3.11) C:\> python -m Orange.canvas
(py3.11) C:\> # prompt is available only when exiting orange
```
```
(py3.11) C:\> ewoks-canvas # console_scripts entry point
(py3.11) C:\> # prompt is available only when exiting orange
```
`...```
(py3.11) C:\> python -m Orange.canvas
(py3.11) C:\> # prompt is available only when exiting orange
```
```
(py3.11) C:\> ewoks-canvas # console_scripts entry point
(py3.11) C:\> # prompt is available only when exiting orange
```
```
(py3.11) C:\> orange-canvas # console_scripts entry point
(py3.11) C:\> # prompt is available IMMEDIATELY and NO LOGS are available!!! We do not want that.
```
Happens for python 3.8 and 3.11
FYI silx does not detach either
```
(py3.11) C:\> silx view # console_scripts entry point
(py3.11) C:\> # prompt is available only when exiting orange
```https://gitlab.esrf.fr/XRD/darfix/-/issues/99Hercules comments2024-03-26T12:42:42+01:00Wout De NolfHercules commentsInstallation on windows issues I had to do 3 different things on 3 different laptops:
- downgrade pyqtgraph Fix https://gitlab.esrf.fr/XRD/darfix/-/issues/103
- install opencv-python-headless Fix https://gitlab.esrf.fr/XRD/darfix/-/issu...Installation on windows issues I had to do 3 different things on 3 different laptops:
- downgrade pyqtgraph Fix https://gitlab.esrf.fr/XRD/darfix/-/issues/103
- install opencv-python-headless Fix https://gitlab.esrf.fr/XRD/darfix/-/issues/103
- enable windows path lengths Fix https://gitlab.esrf.fr/XRD/darfix/-/issues/103
- orange-canvas did not work. python -m Orange.canvas did work. Fix https://gitlab.esrf.fr/XRD/darfix/-/issues/102
- rounding error in the dimensions widgets (step scan). Fix https://gitlab.esrf.fr/XRD/darfix/-/merge_requests/263
- ~noise removal: step widget in background removal should not be there~ It is there when "Keep data on disk" is selected in the load widget.
- ~crash on noise removal (first compute button)~ This is just a disk full error
![error_darfix](/uploads/2208126652daa502820e0a3b8dcb37e0/error_darfix.png)
We also need to fix the installation instructions in the README and in the docs. Fix https://gitlab.esrf.fr/XRD/darfix/-/issues/103https://gitlab.esrf.fr/XRD/darfix/-/issues/98Support silx 2.02024-02-20T15:33:09+01:00Wout De NolfSupport silx 2.0Bug report:
> One is that the ROI widget does only work with Silx 1.1.2 not with Silx 2.0.0 (the current version)
Second, the reinstalled darfix crashes when I try to correct the beamshift.Bug report:
> One is that the ROI widget does only work with Silx 1.1.2 not with Silx 2.0.0 (the current version)
Second, the reinstalled darfix crashes when I try to correct the beamshift.https://gitlab.esrf.fr/XRD/darfix/-/issues/97transformation widget - rotate RSM2024-02-08T06:41:29+01:00paynotransformation widget - rotate RSMrelated to #96
in the transformation widget there is a “Rotate RSM” option which, when checked, should compensate for the 90 degree rotation given by the mounting of the basler camera at the beamline. This dataset doesn’t seem to exhib...related to #96
in the transformation widget there is a “Rotate RSM” option which, when checked, should compensate for the 90 degree rotation given by the mounting of the basler camera at the beamline. This dataset doesn’t seem to exhibit that rotation for some reason (not sure if the detector was remounted or not), but I am unsure on whether the flip is the right way (maybe @Raquel Rodriguez Lamas can speak more on that if she has a test dataset).https://gitlab.esrf.fr/XRD/darfix/-/issues/96missing reciprocal space flip (in trasformation widget ?)2024-02-09T07:17:36+01:00paynomissing reciprocal space flip (in trasformation widget ?)lack of flip for reciprocal space data (showcased with HRRSM) which, under our current understanding, is connected to the transformation widget (when RSM is selected). A reference raw detector image (from an unrelated dataset) looks like...lack of flip for reciprocal space data (showcased with HRRSM) which, under our current understanding, is connected to the transformation widget (when RSM is selected). A reference raw detector image (from an unrelated dataset) looks like this:
![image](/uploads/4a4ee41b9b5c9a4bc32d24729c55dce7/image.png)
In that image two clear lines are visible, one on the left and one on the top (which should correspond to higher 2theta values). After using the transformation widget with RSM as in this simple workflow:
![image](/uploads/40dff8c9575eb2a734eb8f9fcb939d20/image.png)
The z sum looks like this:
![image](/uploads/5a019f07beae3d7c2546f6b157a7db03/image.png)
With the horizontal edge being at the bottom (lower 2theta) instead of the top. This means that the resulting line profile looks like this:
![image](/uploads/529e041f6bfb82031913f3785deae6c6/image.png)
Instead of this:
![image](/uploads/46a135e5667b10862bcca712bada3bf1/image.png)https://gitlab.esrf.fr/XRD/darfix/-/issues/94Grain plot widget: add color maps2024-01-16T11:22:29+01:00Wout De NolfGrain plot widget: add color mapsRequested by @rarodrig
One of our users has developed a wonderful package with colormaps that we would love to have available directly in darfix (mostly for the mosaicity map).
Here goes the link to the page where he's made them avail...Requested by @rarodrig
One of our users has developed a wonderful package with colormaps that we would love to have available directly in darfix (mostly for the mosaicity map).
Here goes the link to the page where he's made them available: https://trygvrad.github.io/colorstamps-2d-colormaps/#more-67https://gitlab.esrf.fr/XRD/darfix/-/issues/93CI hangs on test_sdist-3.6_glx2024-01-14T11:58:33+01:00Wout De NolfCI hangs on test_sdist-3.6_glxhttps://gitlab.esrf.fr/XRD/darfix/-/jobs/911258
```
Running with gitlab-runner 16.0.1 (79704081)
on DAU GitLab Docker runner da54e541, system ID: r_y8bSgmFDXejC
Preparing the "docker" executor 00:02
Using Docker executor with image do...https://gitlab.esrf.fr/XRD/darfix/-/jobs/911258
```
Running with gitlab-runner 16.0.1 (79704081)
on DAU GitLab Docker runner da54e541, system ID: r_y8bSgmFDXejC
Preparing the "docker" executor 00:02
Using Docker executor with image docker-registry.esrf.fr/dau/ewoks:python_3.6_glx ...
Pulling docker image docker-registry.esrf.fr/dau/ewoks:python_3.6_glx ...
Using docker image sha256:11727346b36e2a97f4e167a53e2f2b79cdb5ffa550bdd68a0f2611a8d7cb87b5 for docker-registry.esrf.fr/dau/ewoks:python_3.6_glx with digest docker-registry.esrf.fr/dau/ewoks@sha256:404b697e256fbc746c4f2dc67db218c0aa101a7d22019ab6c45976236a842868 ...
Preparing environment 00:01
Running on runner-da54e541-project-1791-concurrent-0 via 8aa80ad7f198...
Getting source from Git repository 00:01
Fetching changes with git depth set to 50...
Reinitialized existing Git repository in /builds/XRD/darfix/.git/ >>>> LAST MESSAGE WHEN IT HANGS
Checking out 25be9eae as detached HEAD (ref is main)...
...
```https://gitlab.esrf.fr/XRD/darfix/-/issues/92Tasks with cumulative operations need to keep the history2024-01-16T10:48:59+01:00Wout De NolfTasks with cumulative operations need to keep the historyThose tasks need something like this: [operations.py](/uploads/e7e395b279576aeec250712b91a3c49a/operations.py)
[operationstask.py](/uploads/fa4c4b7bec2d895f29eaf64a1626076e/operationstask.py)Those tasks need something like this: [operations.py](/uploads/e7e395b279576aeec250712b91a3c49a/operations.py)
[operationstask.py](/uploads/fa4c4b7bec2d895f29eaf64a1626076e/operationstask.py)https://gitlab.esrf.fr/XRD/darfix/-/issues/91transformation widget : independent magnification for vertical and horizontal...2024-03-26T10:56:09+01:00Wout De Nolftransformation widget : independent magnification for vertical and horizontal axisRequested by @rarodrig
Transformation widget: when OBpitch clicked -> provide two parameters instead of one always.
![image](/uploads/d0a1a01cdbcd3e3c99bab8d737005e6e/image.png)Requested by @rarodrig
Transformation widget: when OBpitch clicked -> provide two parameters instead of one always.
![image](/uploads/d0a1a01cdbcd3e3c99bab8d737005e6e/image.png)https://gitlab.esrf.fr/XRD/darfix/-/issues/90OperationThread exception: do not try to fetch the result2023-08-03T16:57:03+02:00Wout De NolfOperationThread exception: do not try to fetch the result`OperationThread.data` will be `None` when `OperationThread.run` fails. Handle this case properly in all tasks.`OperationThread.data` will be `None` when `OperationThread.run` fails. Handle this case properly in all tasks.https://gitlab.esrf.fr/XRD/darfix/-/issues/88Rocking curve widget: select cursor position as pixel coordinates (like the ROI)2024-01-16T10:29:59+01:00Raquel Rodriguez LamasRocking curve widget: select cursor position as pixel coordinates (like the ROI)Select pixel coordinates with QLineEdit boxes (or whatever text boxes) instead of cursor.Select pixel coordinates with QLineEdit boxes (or whatever text boxes) instead of cursor.https://gitlab.esrf.fr/XRD/darfix/-/issues/87Workflow arguments from CLI2024-03-26T10:57:07+01:00Raquel Rodriguez LamasWorkflow arguments from CLIExamples:
- Noise: median/threshold/hotpixel (needs orders, number of repeats and different values (e.g. threshold))
- shift correction: provide manual shift
- `DimensionDefinition`: provide dimensions manually or do a "fit"
- ...Examples:
- Noise: median/threshold/hotpixel (needs orders, number of repeats and different values (e.g. threshold))
- shift correction: provide manual shift
- `DimensionDefinition`: provide dimensions manually or do a "fit"
- ...https://gitlab.esrf.fr/XRD/darfix/-/issues/81Data convert_to_hdf5 bug?2024-01-16T10:34:19+01:00Wout De NolfData convert_to_hdf5 bug?https://gitlab.esrf.fr/XRD/darfix/-/blob/main/src/darfix/core/dataset.py#L2341
```python
data = self.flatten() # flattened copy of the data in memory
# Return the dataset when it exists and has the correct shape
shape = data[0].shape[...https://gitlab.esrf.fr/XRD/darfix/-/blob/main/src/darfix/core/dataset.py#L2341
```python
data = self.flatten() # flattened copy of the data in memory
# Return the dataset when it exists and has the correct shape
shape = data[0].shape[0] * data[0].shape[1]
if "dataset" in self._file:
if self._file["dataset"].shape == (len(data), shape):
return self._file["dataset"] # The values may NOT be equal to self.flatten(). Is this what we want?
else:
del self._file["dataset"]
# Create the dataset otherwise
self._file.create_dataset("dataset", (len(data), shape))
# Fill the dataset values
for i in range(len(data)):
self._file["dataset"][i] = data[i].flatten()
return self._file["dataset"]
```
@julia.garriga Is this a bug or done intentionally?https://gitlab.esrf.fr/XRD/darfix/-/issues/79Does not work without a full orange installation2023-03-02T13:23:55+01:00Wout De NolfDoes not work without a full orange installationNever import from Orange directly
```
from Orange.widgets.settings import Setting
from Orange.widgets.widget import OWWidget, Output
```
Instead use `from ewoksorange.gui.orange_imports import ...`Never import from Orange directly
```
from Orange.widgets.settings import Setting
from Orange.widgets.widget import OWWidget, Output
```
Instead use `from ewoksorange.gui.orange_imports import ...`https://gitlab.esrf.fr/XRD/darfix/-/issues/77Pre-defined conditions to export grain plot, PCA, Zsum, and with a pre define...2024-01-16T10:40:45+01:00Wout De NolfPre-defined conditions to export grain plot, PCA, Zsum, and with a pre defined name followed by a _nameofthewidgetThe way the batch processing works now, it opens the GUI to ask you about the number of levels on orientation distribution and the format you what to export the data. So, if we could do it in advance by choosing the format of the file we...The way the batch processing works now, it opens the GUI to ask you about the number of levels on orientation distribution and the format you what to export the data. So, if we could do it in advance by choosing the format of the file we want and the conditions, we could even set a script to run the analysis in series (considering the 3D scenario). Also, save the exported file name followed by a _zsum (for example) considering the widget you are using.
Widgets that need saving:
- Grain Pot (.h5)
- Z sum (.edf)
- PCA (ASCII)
- Rocking Curve Imaging (.h5)
- BSS (.h5)
- Weak Beam (.h5/.edf)
[batch_mosa_NNSC_210peak.ows](/uploads/30bf581b418dea300787d66899b60502/batch_mosa_NNSC_210peak.ows)https://gitlab.esrf.fr/XRD/darfix/-/issues/76Dimension definition using the exported h5file of the treated data2024-01-16T10:47:48+01:00Wout De NolfDimension definition using the exported h5file of the treated dataThis will be great because, from what i know, we can only use the batch processing considering a "single string" of widgets.
But after the saving the processed data, we could save time by running other widget with the "cleaned" data. Oth...This will be great because, from what i know, we can only use the batch processing considering a "single string" of widgets.
But after the saving the processed data, we could save time by running other widget with the "cleaned" data. Otherwise i will have to run the same analysis to use a new widget.
[batch_mosa_NNSC_210peak.ows](/uploads/4d287eebebdaf2e4df6181188ff5a046/batch_mosa_NNSC_210peak.ows)https://gitlab.esrf.fr/XRD/darfix/-/issues/73CLI without full installation does not work2023-01-16T17:34:19+01:00Wout De NolfCLI without full installation does not work```
(py38) denolf@lindenolf:~$ darfix -h
Traceback (most recent call last):
File "/users/denolf/virtualenvs/pybox_njRFFJ/ubuntu_20_04/py38/bin/darfix", line 5, in <module>
from darfix.app.__main__ import main
File "/users/denolf...```
(py38) denolf@lindenolf:~$ darfix -h
Traceback (most recent call last):
File "/users/denolf/virtualenvs/pybox_njRFFJ/ubuntu_20_04/py38/bin/darfix", line 5, in <module>
from darfix.app.__main__ import main
File "/users/denolf/virtualenvs/pybox_njRFFJ/ubuntu_20_04/py38/lib/python3.8/site-packages/darfix/app/__main__.py", line 7, in <module>
from darfix.core.process import graph_data_selection
File "/users/denolf/virtualenvs/pybox_njRFFJ/ubuntu_20_04/py38/lib/python3.8/site-packages/darfix/core/process.py", line 19, in <module>
from silx.gui import qt
File "/users/denolf/virtualenvs/pybox_njRFFJ/ubuntu_20_04/py38/lib/python3.8/site-packages/silx/gui/qt/__init__.py", line 50, in <module>
from ._qt import * # noqa
File "/users/denolf/virtualenvs/pybox_njRFFJ/ubuntu_20_04/py38/lib/python3.8/site-packages/silx/gui/qt/_qt.py", line 79, in <module>
raise ImportError(
ImportError: No Qt wrapper found. Install PyQt5, PySide2, PySide6, PyQt6.
```
```
Traceback (most recent call last):
File "/users/denolf/virtualenvs/pybox_njRFFJ/ubuntu_20_04/py38/bin/darfix", line 5, in <module>
from darfix.app.__main__ import main
File "/users/denolf/virtualenvs/pybox_njRFFJ/ubuntu_20_04/py38/lib/python3.8/site-packages/darfix/app/__main__.py", line 7, in <module>
from darfix.core.process import graph_data_selection
File "/users/denolf/virtualenvs/pybox_njRFFJ/ubuntu_20_04/py38/lib/python3.8/site-packages/darfix/core/process.py", line 21, in <module>
from darfix.gui.blindSourceSeparationWidget import Method
File "/users/denolf/virtualenvs/pybox_njRFFJ/ubuntu_20_04/py38/lib/python3.8/site-packages/darfix/gui/blindSourceSeparationWidget.py", line 11, in <module>
from .displayComponentsWidget import DisplayComponentsWidget
File "/users/denolf/virtualenvs/pybox_njRFFJ/ubuntu_20_04/py38/lib/python3.8/site-packages/darfix/gui/displayComponentsWidget.py", line 10, in <module>
from silx.gui.plot import ScatterView, StackView, Plot1D
File "/users/denolf/virtualenvs/pybox_njRFFJ/ubuntu_20_04/py38/lib/python3.8/site-packages/silx/gui/plot/__init__.py", line 62, in <module>
from .PlotWidget import PlotWidget # noqa
File "/users/denolf/virtualenvs/pybox_njRFFJ/ubuntu_20_04/py38/lib/python3.8/site-packages/silx/gui/plot/PlotWidget.py", line 59, in <module>
from . import PlotInteraction
File "/users/denolf/virtualenvs/pybox_njRFFJ/ubuntu_20_04/py38/lib/python3.8/site-packages/silx/gui/plot/PlotInteraction.py", line 38, in <module>
from . import items
File "/users/denolf/virtualenvs/pybox_njRFFJ/ubuntu_20_04/py38/lib/python3.8/site-packages/silx/gui/plot/items/__init__.py", line 43, in <module>
from .image_aggregated import ImageDataAggregated # noqa
File "/users/denolf/virtualenvs/pybox_njRFFJ/ubuntu_20_04/py38/lib/python3.8/site-packages/silx/gui/plot/items/image_aggregated.py", line 39, in <module>
from .axis import Axis
File "/users/denolf/virtualenvs/pybox_njRFFJ/ubuntu_20_04/py38/lib/python3.8/site-packages/silx/gui/plot/items/axis.py", line 35, in <module>
import dateutil.tz
ModuleNotFoundError: No module named 'dateutil'
```https://gitlab.esrf.fr/XRD/darfix/-/issues/72Noise widget operations2024-03-26T10:56:59+01:00Wout De NolfNoise widget operations![image](/uploads/2c915a6cf9c3c337301777d6d72651cb/image.png)
Operations can be executed in any order and multiple times. Make sure this is saved in the .ows file.![image](/uploads/2c915a6cf9c3c337301777d6d72651cb/image.png)
Operations can be executed in any order and multiple times. Make sure this is saved in the .ows file.https://gitlab.esrf.fr/XRD/darfix/-/issues/67Better error message for headless execution without files2024-01-16T10:52:31+01:00Wout De NolfBetter error message for headless execution without fileshttps://gitlab.esrf.fr/XRD/darfix/-/issues/66Load a workflow with files that are already gone2024-01-16T10:51:34+01:00Wout De NolfLoad a workflow with files that are already goneThe data selection widget never displays because of
```python
Traceback (most recent call last):
File "/usr/local/lib/python3.8/dist-packages/orangecanvas/scheme/widgetmanager.py", line 236, in __add_widget_for_node
w = self.creat...The data selection widget never displays because of
```python
Traceback (most recent call last):
File "/usr/local/lib/python3.8/dist-packages/orangecanvas/scheme/widgetmanager.py", line 236, in __add_widget_for_node
w = self.create_widget_for_node(node)
File "/usr/local/lib/python3.8/dist-packages/orangewidget/workflow/widgetsscheme.py", line 300, in create_widget_for_node
widget = self.create_widget_instance(node)
File "/usr/local/lib/python3.8/dist-packages/orangewidget/workflow/widgetsscheme.py", line 413, in create_widget_instance
widget.__init__()
File "/usr/local/lib/python3.8/dist-packages/orangecontrib/darfix/widgets/dataselection.py", line 78, in __init__
self.setDataset()
File "/usr/local/lib/python3.8/dist-packages/orangecontrib/darfix/widgets/dataselection.py", line 81, in setDataset
self._widget.setRawFilenames(self.filenames)
File "/usr/local/lib/python3.8/dist-packages/darfix/gui/datasetSelectionWidget.py", line 235, in setFiles
self.addFile(file)
File "/usr/local/lib/python3.8/dist-packages/darfix/gui/datasetSelectionWidget.py", line 217, in addFile
assert os.path.isfile(file)
AssertionError
```