Skip to content
GitLab
Projects Groups Snippets
  • /
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in
  • bliss bliss
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
  • Issues 544
    • Issues 544
    • List
    • Boards
    • Service Desk
    • Milestones
  • Jira
    • Jira
  • Merge requests 147
    • Merge requests 147
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Schedules
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Monitor
    • Monitor
    • Incidents
  • Analytics
    • Analytics
    • Value stream
    • CI/CD
    • Repository
  • Wiki
    • Wiki
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar
  • BlissBliss
  • blissbliss
  • Merge requests
  • !3462

Resolve "Adding additional data to hdf5 - improvement proposition"

  • Review changes

  • Download
  • Email patches
  • Plain diff
Merged Wout De Nolf requested to merge 2566-adding-additional-data-to-hdf5-improvement-proposition into master Feb 08, 2021
  • Overview 0
  • Commits 3
  • Pipelines 7
  • Changes 9

Closes #2566 (closed)

Extend the API for adding scan info to the Nexus file:

# Static information (specific to 1 scan)
scan_info = {
    "instrument": {"diode3": {"@myattr": 1, "mydset1": [1, 2, 3]}},
    # Links support is added in another MR:
    # https://gitlab.esrf.fr/bliss/bliss/-/merge_requests/3482
    # "measurement": {">mylink": "../instrument/diode3/mydset"},
    "measurement": {"mylink": [1, 2, 3]},
    "scan_meta_categories": ["measurement"],
}

# Dynamic information (applies to all scans, although we could
# inspect `scan` and return `None` if needed)
def dynamic_gen1(scan):
    return {"@myattr": 2}

def dynamic_gen2(scan):
    return {"mydset2": [4, 5, 6]}

usm = get_user_scan_meta()
usm.add_categories(["mygroup"])
usm.mygroup.set("dynamic_gen1", dynamic_gen1)
usm.mygroup.set("dynamic_gen2", dynamic_gen2)

# Scan
scan = scans.loopscan(3, .1, diode3, scan_info=scan_info)

with this result

{'1.1': {'@NX_class': 'NXentry',
         'instrument': {'@NX_class': 'NXinstrument',
                        'diode3': {'@NX_class': 'NXdetector',
                                   '@myattr': 1,
                                   'data': array([51.46790903, 48.43374378, 51.08758888]),
                                   'mode': 'MEAN',
                                   'mydset1': array([1, 2, 3]),
                                   'type': 'samplingcounter'},
                       },
         'measurement': {'@NX_class': 'NXcollection',
                         'diode3': array([51.46790903, 48.43374378, 51.08758888]),
                         'mylink': array([1, 2, 3])},
         'mygroup': {'@NX_class': 'NXcollection',
                     '@myattr': 2,
                     'mydset2': array([4, 5, 6])},
        }
}

What's new about this is the ability to add metadata categories (in the example we added measurement). Before you only had a fixed list of categories.

  • add static category: scan_info["scan_meta_categories"] = [...]
  • add dynamic category: get_user_scan_meta().add_categories([...])

Both are merged in Redis as scan_node.info.get("scan_meta_categories") which is basically a list of scan_info keys that will be dict-dumped to the Nexus file.

Unrelated to this MR: link support will be added in !3482 (merged).

Edited Feb 20, 2021 by Wout De Nolf
Assignee
Assign to
Reviewers
Request review from
Time tracking
Source branch: 2566-adding-additional-data-to-hdf5-improvement-proposition