Skip to content

GitLab

Commit 1daf16ca authored by Wout De Nolf's avatar Wout De Nolf
Browse files

docs: replace icat tango server setup with icat client configuration

parent af9e3446
Pipeline #50436 failed with stages
in 130 minutes and 22 seconds
Hide whitespace changes
Inline Side-by-side
doc/docs/installation_esrf.md
View file @ 1daf16ca
......@@ -124,8 +124,6 @@ installing this server.
There must be **one Nexus writer device** per BLISS session. Do not
forget to add a device when a new BLISS session is created.
### ESRF data policy
The ESRF data policy allows users to access their data and electronic logbook at
......@@ -133,184 +131,15 @@ https://data.esrf.fr Data is registered with [ICAT](https://data.esrf.fr) and
the data written in [Nexus compliant](https://www.nexusformat.org/) HDF5 files
in a specific directory structure.
Two additional TANGO devices, installed automatically with BLISS (as
dependencies), need to be running and enabled for this. BLISS also needs to be
configured to use ESRF data policy.
!!! warning
There must be **two ICAT server** per BLISS session. Do not
forget to add the two devices when a new BLISS session is created.
#### ICAT Tango severs configuration
* `MetaExperiment` server handles the proposal and the sample
* `MetadataManager` server handles the dataset
These are referred to as the ICAT servers. They will inform the ICAT database
about the collected datasets during an experiment and they allow BLISS to
communicate with the **electronic logbook**.
On a beamline there can be multiple `MetadataManager` servers, each serving a
specific technique that needs a specific set of metadata parameters to be
registered with the ICAT database.
#### Using BEACON as the tango database
The registration can be done by defining servers and devices properties in the
beamline configuration.
```yaml
- class: MetaExperiment
properties:
queueName: "/queue/icatIngest"
queueURLs:
- bcu-mq-01.esrf.fr:61613
- bcu-mq-02.esrf.fr:61613
- class: MetadataManager
properties:
queueName: "/queue/icatIngest"
queueURLs:
- bcu-mq-01.esrf.fr:61613
- bcu-mq-02.esrf.fr:61613
API_KEY: elogbook-be70ac55-fd08-4840-9b29-b73262958ca8
icatplus_server: "https://icatplus.esrf.fr"
server: "icat.esrf.fr"
port: 443
username: reader
- server: MetadataManager
personal_name: icatservers
device:
- tango_name: id00/metadata/test
class: MetadataManager
properties:
beamlineID: id00
dataFolderPattern: "{dataRoot}"
metaExperimentDevice: "id00/metaexp/test"
- server: MetaExperiment
personal_name: icatservers
device:
- tango_name: id00/metaexp/test
class: MetaExperiment
properties:
beamlineID: id00
```
#### Using the traditional tango database
The registration can be done by defining servers and devices properties in Jive.
A kind way to proceed is:
* to get templates files `manager.tango` and `experiment.tango`
* to adapt templates files:
- replace `idXX` by bemaline name
- replace `session_WWZ` by the session name
* to load templates in Jive
* to create device for other sessions, just copy devices using Jive:
- in **Server** tab
- unfold `MetadataManager` ; `idxx` ; `MetadataManager` ; `idxx/metadata/sessionXXX`
- right clic on existing device and then : **rename** or **create device** / **copy**
`manager.tango`:
```
MetadataManager/idXX/DEVICE/MetadataManager: "idXX/metadata/session_WWZ"
idXX/metadata/session_WWZ->beamlineID: idXX
idXX/metadata/session_WWZ->dataFolderPattern: {dataRoot}
idXX/metadata/session_WWZ->metaExperimentDevice: "idXX/metaexp/idXX"
idXX/metadata/session_WWZ/dataFolder->__value: ""
idXX/metadata/session_WWZ/datasetName->__value: ""
idXX/metadata/session_WWZ/metadataFile->__value: ""
CLASS/MetadataManager->API_key: elogbook-be70ac55-fd08-4840-9b29-b73262958ca8
CLASS/MetadataManager->icatplus_server: "https://icatplus.esrf.fr"
CLASS/MetadataManager->queueName: "/queue/icatIngest"
CLASS/MetadataManager->queueURLs: bcu-mq-01.esrf.fr:61613,\
bcu-mq-02.esrf.fr:61613
```
`experiment.tango`:
```
MetaExperiment/idXX/DEVICE/MetaExperiment: "idXX/metaexp/session_WWZ"
idXX/metaexp/session_WWZ->beamlineID: idXX
idXX/metaexp/session_WWZ/dataRoot->__value: ""
idXX/metaexp/session_WWZ/proposal->__value: ""
idXX/metaexp/session_WWZ/sample->__value: ""
CLASS/MetaExperiment->queueName: "/queue/icatIngest"
CLASS/MetaExperiment->queueURLs: bcu-mq-01.esrf.fr:61613,\
bcu-mq-02.esrf.fr:61613
In order for BLISS to communicate with the ESRF data policy servers, the following
configuration should be added to file:`__init__.yml` located at beamline configuration
root:
```
##### MetaExperiment server
Class properties:
![Meta experiment class properties](img/data_policy/jive_metaexp_classprops.jpg)
These properties are used to register [datasets](data_policy.md#change-dataset).
They are the same for all the beamlines.
* `queueName`: "/queue/icatIngest"
* `queueURLs`:
- `bcu-mq-01.esrf.fr:61613`
- `bcu-mq-02.esrf.fr:61613`
Devices:
![Meta experiment device properties](img/data_policy/jive_metaexp_props.jpg)
The devices have only one property to be adapted for the beamline:
* `beamlineID`: `id00`
##### MetadataManager server
These properties are used to send messages to the [electronic
logbook](data_metadata.md#electronic-logbook).
They are the same for all the beamlines.
* `API_KEY`:`elogbook-be70ac55-fd08-4840-9b29-b73262958ca8`
- security key to connect to elogbook
* `icatplus_server`: `https://icatplus.esrf.fr`
* `queueName`: "/queue/icatIngest"
* `queueURLs`:
- `bcu-mq-01.esrf.fr:61613`
- `bcu-mq-02.esrf.fr:61613`
icat_servers:
metadata_urls: [URL1, URL2]
elogbook_url: URL3
elogbook_token: elogbook-00000000-0000-0000-0000-000000000000
```
Not present in jive ???
server: "icat.esrf.fr"
port: 443
username: reader
```
Class properties:
![Metadata manager class properties](img/data_policy/jive_metadata_manager_classprops.jpg)
Devices:
![Metadata manager device properties](img/data_policy/jive_metadata_manager_props.jpg)
!!! note
`MetaExperiment` must be started **before** `MetadataManager`.
#### Enable in BLISS
......@@ -335,15 +164,6 @@ scan_saving:
...
```
!!! note
The beamline name specified under `scan_saving:` will be used to find
the metadata servers: `id00/metadata/<session_name>` and
`id00/metaexp/<session_name>`. So it must correspond to the Tango
device domain name.
**There must be 2 metadata Tango devices running per BLISS session.**
(+1 nexus writer) Do not forget to add them for each new BLISS session.
##### Multiple mount points
Multiple mount points can be configured for each proposal type (visitor, inhouse and temp) and optionally for the icat servers (`MetadataManager` and `MetaExperiment`)
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment