Data produced among sessions is published into [Redis](1)(RAM storage) and written to disk in a hdf5 file ([see data saving](scan_saving.md)).
Data produced among sessions is published into [Redis](1)(RAM storage) and in the same time written to disk in a hdf5 file ([see data saving](scan_saving.md)).
On Redis, data is stored for a limited period of time (1 day by default) and for a limited amount (1GB by default).
## Experiment and data files structure
...
...
@@ -8,7 +8,7 @@ In a Bliss session, the experiment can be seen as a tree, where the trunk is the
As an example, imagine that we create a session named '**test_session**' and that we want to perform several measurements with two different samples.
The measurements will consist in scanning the samples {'**sample1**', '**sample2**'} by moving a motor '**roby**' and measuring the value of a counter '**diode**' along the scan.
The sample scan is performed with the command '**ascan(roby, 0, 10, 10, 0.1, diode)**' ([see scan commands](scan_default.md)).
The sample scan is performed with the command `ascan(roby, 0, 9, 10, 0.1, diode)` ([see scan commands](scan_default.md)).
* Start the session:
...
...
@@ -23,21 +23,29 @@ While starting a session for the first time, a directory with the same name as t
>>>SCAN_SAVING.template='{session}/{sample}/'# modify the data saving path template
>>>SCAN_SAVING.sample='sample1'# set the value of the parameter {sample}
```
* Perform a first measurement:
```py
>>>ascan(roby,0,10,10,0.1,diode)
>>>ascan(roby,0,9,10,0.1,diode)
```
* Perform a second measurement:
```py
>>>ascan(roby,0,10,10,0.1,diode)
>>>ascan(roby,0,9,10,0.1,diode)
```
* Change the data saving path for measurements on sample2:
```py
>>>SCAN_SAVING.sample='sample2'
```
* Perform a measurement:
```py
>>>ascan(roby,0,10,10,0.1,diode)
>>>ascan(roby,0,9,10,0.1,diode)
```
For this experiment the files structure inside the session main folder is described by the following tree:
...
...
@@ -50,7 +58,7 @@ The measurements data can be accessed by reading the content of the hdf5 files (
## Experiment and Redis data structure
Redis stores the data in RAM memory as soon as it is produced. Therefore retrieving data from **Redis allows a fast and live access to the data**.
In parallel of the in-file data storage describe above, Redis stores the data in RAM memory as soon as it is produced. Therefore retrieving data from **Redis allows a fast and live access to the data**.
Redis stores the data as a flatten list of (key:value) pairs but hopefully the **bliss.data.node** module provides a simple interface to access the data in a structured way that reflects the session structure.
With the function **n = get_node("node_name")***node_name* is your entry point and *n* is the associated [DataNodeContainer]().
With the function **n.iterator.walk(wait=False)** you can iterate over all the children nodes of the node *n*.
Among the children nodes, we can distinguish two other types of node, the [ChannelDataNode]() and the [scan](). They both inherit from the *DataNodeContainer* class.
With the function `n = get_node("node_db_name")`*node_db_name* is your entry point and *n* is the associated [DataNodeContainer](scan_data_node.md#datanodecontainer).
With the function `n.iterator.walk(wait=False)` you can iterate over all the child nodes of the node *n* ([see DataNodeIterator](scan_data_node.md#datanodeiterator)).
Among the child nodes, we can distinguish two other types of node, the [ChannelDataNode](scan_data_node.md#channeldatanode) and the [scan](scan_data_node.md#scan). They both inherit from the `DataNodeContainer` class.
The experimental measures are associated to the *ChannelDataNodes*, and the scan object to the *Scan* nodes.
You can access any node using its full name (`db_name`):
The classes inheriting from the `DataNode` class provide the `iterator` method which returns a `DataNodeIterator` object.
The DataNodeIterator provides the best methods to monitor the events happening during the experiment and follow the data production.
The method `walk(filter=None, wait=True)` iterates over existing child nodes that match the `filter` argument. If `wait` is True (default), the function blocks until a new node appears. It returns the new node when it appears and then waits again for a new node.
The method `walk_events(filter=None)` walks through child nodes, just like `walk` function but waits for node events (like `EVENTS.NEW_CHILD` or `EVENTS.NEW_DATA_IN_CHANNEL`). It returns the event type and the node then waits again for next event.
```py
>>>session=get_node("test_session")
>>>deff(filter=None):
>>>"""wait for any new node in the session"""
>>>fornodeinsession.iterator.walk(filter=filter):
>>>print(node.name,node.type)
>>>defg(filter='channel'):
>>>"""wait for a new event happening in any node of the type 'channel' (ChannelDataNode) """
Note: In the example above we use `node.get(-1)` to retrieve the last data value produced on this node. For more details see method `get(from_index, to_index=None)` of the [ChannelDataNode](scan_data_node.md#channeldatanode).
### Scanning & experiment sequences
In the context of an experiment or a scan procedure it exist a more convenient way to obtain the data produced by a scan.
At the shell level, it exist a global variable `SCANS` that stores all the scan objects that have been launched in this shell session.
For example `SCANS[-1]` returns the last scan object and `SCANS[-1].get_data()` returns the data of that scan.
