optimize device preparation of scans

Closes #2318 (closed)

Issue

Scan prepare is slow (#2327 (closed)): simple scan profiling shows that Scan._prepare_device takes the longest. Despite the name, the time is mostly spend on creating nodes in Redis, not on preparing the hardware.

Solution

Create all Redis nodes within a single pipeline in Scan._prepare_device. Following the way we handle asynchronous Redis calls in Bliss, I have no other option than replace all connections of all settings of all nodes (and restore them afterwards). I introduced a helper class for this:

from bliss.data.node import DataNodeAsyncHelper

with DataNodeAsyncHelper(sync_proxy) as helper:
    helper.replace_connection(node1)
    helper.replace_connection(node2)
    # ... all Redis calls of node1 and node2 are asynchronous

# ... all Redis calls of node1 and node2 are synchronous

So that's the equivalent of bliss.config.settings.pipeline but for DataNode's and the nodes are added in the context (because you need to use it also when creating the nodes).

Thanks too !3257 (merged) this works also with the CachingRedisDbProxy (which is used for the scan and channel nodes). Multiple connections with client tracking can now be created, which is needed for pipelines.

Test case

Using the NEXUS_WRITER_SESSION from the test suite (which has many diodes and MCA's so lots of nodes in Redis).

NEXUS_WRITER_SESSION [1]: ct(0.1, xrfMG)

After ~1000 ct's (~200.000 KEYS in Redis):

• before optimization: "scan.node.prepare" = 1.3 s

• after optimization: "scan.node.prepare" = 200 ms

The next bottleneck is "scan.events.device" (see #2373 (closed))

Side effects of this MR:

• To make creating the scan nodes push-only (because it needs to run in a pipeline), I had to move the name patching of ChannelDataNode to the publishing side. In fact it seems that ChannelDataNode.name is alway equal to AcquisitionChannel.fullname. In any case, this is not decided by ChannelDataNode anymore but by the Scan which creates the node (passes an argument channel_name which becomes ChannelDataNode.name but as before has no effect on ChannelDataNode.db_name).

• Using DataNodeAsyncHelper makes DataNode greenlet-unsafe during its context. So that's the same as bliss.config.settings.pipeline. In Scan._prepare_device the nodes are only referenced locally until all is done. So that's fine.