Definition of a Dataset object in Bliss (that is in line with ICAT)
Currently there is no object behind an ICAT dataset in bliss. With @matias.guijarro we came to the conclusion that this could be helpful to have one, especially to get the conceptual behavior foreseen in the data policy correctly implemented in Bliss.
So, imagine you had a class Dataset
in Bliss that should map the ICAT dataset, what are the properties are the methodes this object should have and how should it behave (during init and finalization). This is what I see so far, but it might not be complete:
edit: I (@pithan) try to collect the answers from below from time to time in this description that seem to be finalized
Properties:
-
name -
proposal -
session (in data policy sense) -
sample -
path (how is the filepath called in the data policy language?) -
✔ folder_path
in hdf5-master-config
-
-
metadata (a dict like structure with predefined keys that come from hdf5-master-config XML definition) -
scans (a list of scans belong to the dataset) -
technique (?? something related to the Experiment definition in ICAT or is this just part of metadata?) - does this mandatory need to be one of
XANES
,XRF
,KMAP
,HOLO
,FLUO
,TOMO
or is there also something likeCUSTOM
orNONE
for all other experiments?
- does this mandatory need to be one of
- what else is missing?
Methods:
-
__init__
what should happen here?- collect certain metadata from devices
-
push_to_MetaDataManager
(pushes metadata to the tango server, does not finalize the dataset) -
finalize
(closes the Dataset)- collect metadata again
- call
push_to_MetaDataManager
- tell (the MetaDataManager?) that the Dataset is closed
- what else?
Behavior:
-
is there only one instance of this Dataset objects at a time? ... a sort of singleton that represents the active dataset in icat? -
🔩 there can be multiple Datasets running at the same time, so no singleton! -
🔺 Can there be multiple open datasets on the same Metadata tango server?
-
-
when should the Dataset be finalized? - Whenever a new one is opened?
- When the user closes it manually?
- When the bliss session terminates?
-
🔺 Should acquisition be possible without having a Dataset?
Other questions:
- who can / where is the ICAT Dataset closed? ... How can the object in Bliss know if the Dataset has been closed in ICAT
- is it possible to read back a status and data that has already been pushed about the open dataset from somewhere?
- What is the difference between the keys in XML and the properties of the tango server (e.g. tailing '_start', '_end') and is a dict like structure sufficient to gather the metadata?
- Should it be possible to re-enter an already existing (not yet closed) dataset?
- How / Where is defined which metadata fields have to be collected when?
-
🔺 what are the needed actions in order make sure data that has been acquired (e.g. in a scan) is recorded in the proper Dataset (especially if there are muliple once open)?
I guess @denolf and @demariaa have most of the answers to these questions... could you help?