DataNode get and block_size
Reported by @sfisher . We had channels with a Redis event block size of 15 (meaning each Redis event contains 15 scan points):
{'sampz_data': [1, 16, 31, 46, 61, 76, 91, 106, 121, 136, 151, 166, 181, 196, 211, 226, 241],
'sampy_data': [1, 16, 31, 46, 61, 76, 91, 106, 121, 136, 151, 166, 181, 196, 211, 226, 241]}
In this case apparently only node.get_as_array(n*15)
works, any other starting index gives:
2020-07-13 21:42:19,759 daiquiri.core.hardware.bliss.scans ERROR Couldnt get paged scan data for session_sxm:data:id21:inhouse:20Jul:blc12195:id21:TO_4KB:TO_4KB_roi53_30:1_l2scan sampy 5.124000000000932 85.12400000000095 15 sampz 1.4160000000004658 86.41600000000048 16 0.1:axis:sampz. Requested 230 to 255, node length 255
Traceback (most recent call last):
File "/users/blissadm/local/daiquiri/daiquiri/daiquiri/core/hardware/bliss/scans.py", line 192, in get_scan_data
data = node.get_as_array(paging["st"], paging["en"])
File "/users/blissadm/local/wout.bliss.git/bliss/data/nodes/channel.py", line 233, in get_as_array
raw_data = self.get(from_index, to_index)
File "/users/blissadm/local/wout.bliss.git/bliss/data/nodes/channel.py", line 162, in get
return self.raw_to_data(from_index, raw_data)
File "/users/blissadm/local/wout.bliss.git/bliss/data/nodes/channel.py", line 182, in raw_to_data
"Data is not anymore available first_index:"
RuntimeError: Data is not anymore available first_index:240 request_index:230
The node walking is done in Daiquiri:
for node in scan.iterator.walk(filter="channel", wait=False):
parts = node.db_name.split(":")
name = parts[-2] + ":" + parts[-1]
sobj["data"][name] = {"name": node.name, "shape": node.info["shape"]}
data = np.array([])
if len(node.info["shape"]) == 0 and paging["en"] > paging["st"]:
try:
data = node.get_as_array(paging["st"], paging["en"])
except RuntimeError:
logger.exception(
f"Couldnt get paged scan data for {node.db_name}. Requested {paging['st']} to {paging['en']}, node length {len(node)}"
)