Skip to content

GitLab

  • Menu
Projects Groups Snippets
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in / Register
  • est est
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
  • Issues 12
    • Issues 12
    • List
    • Boards
    • Service Desk
    • Milestones
  • Jira
    • Jira
  • Merge requests 0
    • Merge requests 0
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Schedules
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Monitor
    • Monitor
    • Incidents
  • Packages & Registries
    • Packages & Registries
    • Container Registry
  • Analytics
    • Analytics
    • Value stream
    • CI/CD
    • Repository
  • Wiki
    • Wiki
  • Snippets
    • Snippets
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar
  • workflow
  • ewoksapps
  • estest
  • Merge requests
  • !40

Merged
Created Mar 11, 2021 by payno@paynoOwner18 of 19 tasks completed18/19 tasks

Move Process to Ewoks `Task` and update setup

  • Overview 1
  • Commits 57
  • Pipelines 24
  • Changes 105

TODO list

  • should be merge first:
    • workflow/ewoks/ewoksorange!9 (diffs)
  • replace old requirements.txt... by a setup.cfg
  • insure example_larch.ows and example_pymca.ows can be converted to graph
  • insure those graphs can be converted back to orange widgets
    • handle graph and qualified name
    • insure we can load back the .ows generated
    • add inheritance from OWEwoksWidget
    • handle properties (if we wan't to handle those)
  • insure processing of those workflows can be done using pypushflow
  • update process, reprocess... functions
  • add unit tests
  • remove "ows_to_script" -> process of the .ows should be powerfull enought
  • replace est process application
  • remove est reprocessing application: this will be included with process + persistency.
  • update and fix CI
  • clean: process(self, xas_obj=None) becomes run(self, xas_obj=None) but should be run() at the end.
  • test processing from the orange canvas is still working
    • test pymca standard workflow
    • test larch standard workflow
  • add an example / test on ewoksorange to save the design used by est (not inheriting from OWEwoksWidget but directly from Registered). Might be an issue later with consistency: workflow/ewoks/ewoksorange!16 (merged) ??? !!!

Details / extra information

depends on workflow/ewoks/ewoksorange!9 (merged)

insure example_larch.ows and example_pymca.ows can be converted to graph

For now there is some processes that we want to ignore. To this purpose I added a IgnoreTask class.

input_ows_file = "/home/payno/dev/spectro/est/doc/tutorials/example_larch.ows"

from esrf2orange3.owsconvert import ows_to_ewoks, ewoks_to_ows
from esrftaskgraph.graph import TaskGraph
graph = ows_to_ewoks(input_ows_file)
assert isinstance(graph, TaskGraph)

larch workflow input

larch workflow as a esrf task graph

pymca workflow input

pymca workflow as a esrf task graph

insure those graphs can be converted back to orange widgets

One unit test created for both use cases (larch and pymca) in est.test.test_ewoks.py file

insure processing of those workflows can be done (process application)

update the process application

Example:

est process doc/tutorials/example_pymca.ows -i EXAFS_Cu.dat -o output.h5 --input-energy-col-name "Column 1" --input-abs-col-name "Column 2"

remove the reprocess application

In fact this will be automatically managed by the process application + ewoks task persistence when will be based by HDF5 files.

update the ows_to_script function ?

Simplification: remove it. process should be enough and modifying a process flow with some users functions should be simpler now with ewoks.

Edited Jul 27, 2021 by payno
Assignee
Assign to
Reviewer
Request review from
Time tracking
Source branch: esrftaskgraph_prototype