Commit 6aa8c883 authored by Wout De Nolf's avatar Wout De Nolf
Browse files

add documentation

parent dc7d5828
Pipeline #52155 passed with stages
in 1 minute and 11 seconds
......@@ -15,6 +15,7 @@ __pycache__/
/dist/
*.egg-info/
.eggs/
/doc/_generated
# Unittest / coverage
*.py,cover
......
......@@ -3,3 +3,9 @@ include:
test-3.6:
extends: ".test-3.6"
deploy_docs:
extends: ".sphinx_and_artifacts"
pages:
extends: ".pages"
# EwoksPpf: Pypushflow binding for Ewoks
This task scheduler allows the execution of cyclic and acyclic graphs and uses python's *multiprocessing* for task distribution.
ewoksppf provides task scheduling for cyclic Ewoks workflows.
## Graph of actors
An Ewoks task graph is converted to a graph of *actors* before execution.
## Install
### Source
A task is wrapped by an *EwoksPythonActor* followed by some binder actors, depending on the type of link between source and target. For each link in the task graph, the final actor of the source is a *NameMapperActor* which is connected to the target.
#### Unconditional link
```mermaid
graph LR;
EwoksPythonActor-->NameMapperActor;
```
#### Conditional link with one condition
```mermaid
graph LR;
EwoksPythonActor-->DecodeRouterActor;
DecodeRouterActor-->NameMapperActor;
```
#### Conditional link with multiple conditions
```mermaid
graph LR;
EwoksPythonActor-->DecodeRouterActor1;
EwoksPythonActor-->DecodeRouterActor2;
DecodeRouterActor1-->JoinActor;
DecodeRouterActor2-->JoinActor;
JoinActor-->NameMapperActor;
```bash
python -m pip install ewoksppf[test]
```
#### On-Error conditional link
```mermaid
graph LR;
EwoksPythonActor-->NameMapperActor;
```
## Test
### Target
```mermaid
graph LR;
InputMergeActor-->EwoksPythonActor;
```bash
pytest --pyargs ewoksppf.tests
```
The *NameMapperActor* of a source for a specific target is connected to the *InputMergeActor* before that target.
### Start actor
The *StartActor* is connected to all tasks that are the start of a task scheduling thread.
### Stop actor
Tasks without successors are connected to the *StopActor*. This actor manages a dictionary of outputs which is considered to be the "output of the graph".
### Error handler
All actors that do not have an _on_error_ conditional link are connected to the *ErrorHandler* which directly connected to the *StopActor*.
### Custom actors
These actors are not provided by *pypushflow*
## Documentation
* *EwoksythonActor*: like *PythonActor* but it passes node name and attributes to the next actor.
* *InputMergeActor*: like *Joinactor* (merges the input data dictionaries) triggers the downstream actors when all required input has been provided at least once. Only one non-required input passed.
* *DecodeRouterActor*: line *RouterActor* but it dereferences thet input hashes to get the values.
* *NameMapperActor*: before triggering the next task it applies filtering and name mapping to the input data.
https://workflow.gitlab-pages.esrf.fr/ewoks/ewoksppf/
{{ fullname | escape | underline}}
.. currentmodule:: {{ module }}
.. autoclass:: {{ objname }}
:members:
:show-inheritance:
:inherited-members:
{% block methods %}
.. automethod:: __init__
{% if methods %}
.. rubric:: {{ _('Methods') }}
.. autosummary::
{% for item in methods %}
~{{ name }}.{{ item }}
{%- endfor %}
{% endif %}
{% endblock %}
{% block attributes %}
{% if attributes %}
.. rubric:: {{ _('Attributes') }}
.. autosummary::
{% for item in attributes %}
~{{ name }}.{{ item }}
{%- endfor %}
{% endif %}
{% endblock %}
\ No newline at end of file
{{ fullname | escape | underline}}
.. automodule:: {{ fullname }}
{% block attributes %}
{% if attributes %}
.. rubric:: Module Attributes
.. autosummary::
:toctree:
{% for item in attributes %}
{{ item }}
{%- endfor %}
{% endif %}
{% endblock %}
{% block functions %}
{% if functions %}
.. rubric:: {{ _('Functions') }}
.. autosummary::
:toctree:
{% for item in functions %}
{{ item }}
{%- endfor %}
{% endif %}
{% endblock %}
{% block classes %}
{% if classes %}
.. rubric:: {{ _('Classes') }}
.. autosummary::
:toctree:
{% for item in classes %}
{{ item }}
{%- endfor %}
{% endif %}
{% endblock %}
{% block exceptions %}
{% if exceptions %}
.. rubric:: {{ _('Exceptions') }}
.. autosummary::
:toctree:
{% for item in exceptions %}
{{ item }}
{%- endfor %}
{% endif %}
{% endblock %}
{% block modules %}
{% if modules %}
.. rubric:: Modules
.. autosummary::
:toctree:
:recursive:
{% for item in modules %}
{{ item }}
{%- endfor %}
{% endif %}
{% endblock %}
\ No newline at end of file
Implementation notes
====================
Graph of actors
---------------
An Ewoks task graph is converted to a graph of *actors* before execution.
Source
^^^^^^
A task is wrapped by an *EwoksPythonActor* followed by some binder actors, depending on the type of link between source and target. For each link in the task graph, the final actor of the source is a *NameMapperActor* which is connected to the target.
Unconditional link
.. mermaid::
graph LR;
EwoksPythonActor-->NameMapperActor;
Conditional link with one condition
.. mermaid::
graph LR;
EwoksPythonActor-->DecodeRouterActor;
DecodeRouterActor-->NameMapperActor;
Conditional link with multiple conditions
.. mermaid::
graph LR;
EwoksPythonActor-->DecodeRouterActor1;
EwoksPythonActor-->DecodeRouterActor2;
DecodeRouterActor1-->JoinActor;
DecodeRouterActor2-->JoinActor;
JoinActor-->NameMapperActor;
On-Error conditional link
.. mermaid::
graph LR;
EwoksPythonActor-->NameMapperActor;
Target
^^^^^^
.. mermaid::
graph LR;
InputMergeActor-->EwoksPythonActor;
The *NameMapperActor* of a source for a specific target is connected to the *InputMergeActor* before that target.
Start actor
^^^^^^^^^^^
The *StartActor* is connected to all tasks that are the start of a task scheduling thread.
Stop actor
^^^^^^^^^^
Tasks without successors are connected to the *StopActor*. This actor manages a dictionary of outputs which is considered to be the "output of the graph".
Error handler
^^^^^^^^^^^^^
All actors that do not have an _on_error_ conditional link are connected to the *ErrorHandler* which directly connected to the *StopActor*.
Custom actors
^^^^^^^^^^^^^
These actors are not provided by *pypushflow*
* *EwoksythonActor*: like *PythonActor* but it passes node name and attributes to the next actor.
* *InputMergeActor*: like *Joinactor* (merges the input data dictionaries) triggers the downstream actors when all required input has been provided at least once. Only one non-required input passed.
* *DecodeRouterActor*: line *RouterActor* but it dereferences thet input hashes to get the values.
* *NameMapperActor*: before triggering the next task it applies filtering and name mapping to the input data.
API documentation
=================
.. autosummary::
:toctree: _generated
:recursive:
ewoksppf
"""rm -rf doc/_generated/; python setup.py build_sphinx -E -a
"""
copyright = "2021, ESRF"
author = "ESRF"
extensions = ["sphinx.ext.autodoc", "sphinx.ext.autosummary", "sphinxcontrib.mermaid"]
templates_path = ["_templates"]
exclude_patterns = []
html_theme = "alabaster"
html_static_path = []
autosummary_generate = True
autodoc_default_flags = [
"members",
"undoc-members",
"show-inheritance",
]
ewoskppf |release|
==================
ewoksppf provides task scheduling for cyclic Ewoks workflows.
ewoskppf has been developed by the `Software group <http://www.esrf.eu/Instrumentation/software>`_ of the `European Synchrotron <https://www.esrf.eu/>`_.
.. toctree::
:maxdepth: 2
actors
api
......@@ -30,6 +30,9 @@ dev =
%(test)s
black
flake8
doc =
sphinx
sphinxcontrib-mermaid
# E501 (line too long) ignored for now
# E203 and W503 incompatible with black formatting (https://black.readthedocs.io/en/stable/compatible_configs.html#flake8)
......@@ -38,3 +41,9 @@ ignore = E501, E203, W503
max-line-length = 88
exclude =
.eggs
[build_sphinx]
project = ewoksppf
version = attr: ewoksppf.__version__
release = 0.1
source-dir = ./doc
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment