Lima simulator LOADER_PREFETCH issues
@debionne @pguillou @matias.guijarro
I have 3 issues when testing the lima simulator LOADER_PREFETCH mode.
lima-camera-simulator 1.9.1 py37h6bb024c_0 esrf-bcu
lima-camera-simulator-tango 1.9.1 0 esrf-bcu
lima-core 1.9.4 py37h6bb024c_0 esrf-bcu
lima-tango-server 1.9.5 0 esrf-bcu
Prepare camera for scan (using images from /tmp_14_days/debionne/data/):
lima_simulator.saving.initialize()
lima_simulator.saving.file_format = "HDF5"
lima_simulator.saving.mode = "ONE_FILE_PER_N_FRAMES"
lima_simulator.saving.frames_per_file = 10
simulator = lima_simulator._get_proxy('simulator')
simulator.mode = "LOADER_PREFETCH"
simulator.nb_prefetched_frames = 10
simulator.file_pattern = '/data/id21/inhouse/wout/dev/blissscripts/limapreloaddata/*.edf'
- No HDF5 support for LOADER_PREFETCH?
simulator.file_pattern = '/data/id21/inhouse/wout/dev/blissscripts/limapreloaddata/*.h5'
!!! === DevFailed: DevFailed[
DevError[
desc = limacore.Exception: Common: getImageTypeDepth(/opt/conda/conda-bld/lima-core_1590425612490/work/common/src/SizeUtils.cpp, 214): InvalidValue: Invalid image type
origin = File "/data/id21/inhouse/wout/dev/virtualenvs/lid21data/bliss_env/lib/python3.7/site-packages/Lima/Server/AttrHelper.py", line 132, in __call__
self.__func2Call(value)
reason = PyDs_PythonError
severity = ERR]
DevError[
desc = Failed to write_attribute on device id00/simulator/simulator1, attribute file_pattern
origin = DeviceProxy::write_attribute()
reason = API_AttributeFailed
severity = ERR]
] === !!! ( for more details type cmd 'last_error' )
- Simulator in LOADER_PREFETCH mode saves empty images
loopscan(31, 1e-6, lima_simulator)
The data is there but all pixel values are zero (not the case for the loaded images)
- Can run only one scan:
loopscan(31, 1e-6, lima_simulator) # runs fine
loopscan(31, 1e-6, lima_simulator) # exceptions and lima process hangs
ERROR 2020-06-10 19:33:12,897 bliss.scans: Exception caught in lima_simulator.trigger
!!! === CommunicationFailed: DevFailed[
DevError[
desc = TRANSIENT CORBA system exception: TRANSIENT_CallTimedout
origin = Connection::command_inout()
reason = API_CorbaException
severity = ERR]
DevError[
desc = Timeout (3000 mS) exceeded on device id00/limaccds/simulator1, command startAcq
origin = Connection::command_inout()
reason = API_DeviceTimedOut
severity = ERR]
] === !!! ( for more details type cmd 'last_error' )
ERROR 2020-06-10 19:33:12,931 bliss.scans: Exception caught in timer.wait_slaves
ERROR 2020-06-10 19:33:12,934 bliss.scans: Exception caught in timer.start
!!! === CommunicationFailed: DevFailed[
DevError[
desc = TRANSIENT CORBA system exception: TRANSIENT_CallTimedout
origin = Connection::command_inout()
reason = API_CorbaException
severity = ERR]
DevError[
desc = Timeout (3000 mS) exceeded on device id00/limaccds/simulator1, command startAcq
origin = Connection::command_inout()
reason = API_DeviceTimedOut
severity = ERR]
] === !!! ( for more details type cmd 'last_error' )
ERROR 2020-06-10 19:33:15,953 bliss.scans: Exception caught in lima_simulator.stop
!!! === CommunicationFailed: DevFailed[
DevError[
desc = TRANSIENT CORBA system exception: TRANSIENT_CallTimedout
origin = Connection::command_inout()
reason = API_CorbaException
severity = ERR]
DevError[
desc = Timeout (3000 mS) exceeded on device id00/limaccds/simulator1, command stopAcq
origin = Connection::command_inout()
reason = API_DeviceTimedOut
severity = ERR]
] === !!! ( for more details type cmd 'last_error' )
ERROR 2020-06-10 19:33:15,962 bliss.scans: Exception caught in timer.wait_slaves
!!! === CommunicationFailed: DevFailed[
DevError[
desc = TRANSIENT CORBA system exception: TRANSIENT_CallTimedout
origin = Connection::command_inout()
reason = API_CorbaException
severity = ERR]
DevError[
desc = Timeout (3000 mS) exceeded on device id00/limaccds/simulator1, command startAcq
origin = Connection::command_inout()
reason = API_DeviceTimedOut
severity = ERR]
] === !!! ( for more details type cmd 'last_error' )
!!! === CommunicationFailed: DevFailed[
DevError[
desc = TRANSIENT CORBA system exception: TRANSIENT_CallTimedout
origin = Connection::command_inout()
reason = API_CorbaException
severity = ERR]
DevError[
desc = Timeout (3000 mS) exceeded on device id00/limaccds/simulator1, command stopAcq
origin = Connection::command_inout()
reason = API_DeviceTimedOut
severity = ERR]
] === !!! ( for more details type cmd 'last_error' )
!!! === CommunicationFailed: DevFailed[
DevError[
desc = TRANSIENT CORBA system exception: TRANSIENT_CallTimedout
origin = Connection::command_inout()
reason = API_CorbaException
severity = ERR]
DevError[
desc = Timeout (3000 mS) exceeded on device id00/limaccds/simulator1, command stopAcq
origin = Connection::command_inout()
reason = API_DeviceTimedOut
severity = ERR]
] === !!! ( for more details type cmd 'last_error' )
Took 0:00:06.269651[s]
!!! === CommunicationFailed: DevFailed[
DevError[
desc = TRANSIENT CORBA system exception: TRANSIENT_CallTimedout
origin = Connection::command_inout()
reason = API_CorbaException
severity = ERR]
DevError[
desc = Timeout (3000 mS) exceeded on device id00/limaccds/simulator1, command startAcq
origin = Connection::command_inout()
reason = API_DeviceTimedOut
severity = ERR]
] === !!! ( for more details type cmd 'last_error' )
Tail of the Lima error log:
...
/lima-camera-simulator_1584962343211/work/src/SimulatorFrameLoader.cpp:279)-Trace: Open file /data/id21/inhouse/wout/dev/blissscripts/limapreloaddata/ferrierite_bc_fast_0107.edf
[2020/06/10 19:36:04.529414] 7f4c62763700 *Camera*Simulator::FrameLoader::getNextFrame (/opt/conda/conda-bld/lima-camera-simulator_1584962343211/work/src/SimulatorFrameLoader.cpp:306)-Trace: frame_dim=<2048x2048x2-Bpp16>, mem_size=8388608
[2020/06/10 19:36:04.533268] 7f4c62763700 *Camera*Simulator::FrameLoader::getNextFrame (/opt/conda/conda-bld/lima-camera-simulator_1584962343211/work/src/SimulatorFrameLoader.cpp:321)-Trace: data: 64 5b 68 5b
[2020/06/10 19:36:04.533281] 7f4c62763700 *Camera*Simulator::FrameLoader::getNextFrame (/opt/conda/conda-bld/lima-camera-simulator_1584962343211/work/src/SimulatorFrameLoader.cpp:247)-Funct: Exit
[2020/06/10 19:36:04.533287] 7f4c62763700 *Camera*Simulator::FrameLoader::getNextFrame (/opt/conda/conda-bld/lima-camera-simulator_1584962343211/work/src/SimulatorFrameLoader.cpp:247)-Funct: Enter
[2020/06/10 19:36:04.533506] 7f4c62763700 *Camera*Simulator::FrameLoader::getNextFrame (/opt/conda/conda-bld/lima-camera-simulator_1584962343211/work/src/SimulatorFrameLoader.cpp:279)-Trace: Open file /data/id21/inhouse/wout/dev/blissscripts/limapreloaddata/ferrierite_bc_fast_0108.edf
[2020/06/10 19:36:04.533532] 7f4c62763700 *Camera*Simulator::FrameLoader::getNextFrame (/opt/conda/conda-bld/lima-camera-simulator_1584962343211/work/src/SimulatorFrameLoader.cpp:306)-Trace: frame_dim=<2048x2048x2-Bpp16>, mem_size=8388608
[2020/06/10 19:36:04.536277] 7f4c62763700 *Camera*Simulator::FrameLoader::getNextFrame (/opt/conda/conda-bld/lima-camera-simulator_1584962343211/work/src/SimulatorFrameLoader.cpp:321)-Trace: data: 61 5d 52 56
[2020/06/10 19:36:04.536288] 7f4c62763700 *Camera*Simulator::FrameLoader::getNextFrame (/opt/conda/conda-bld/lima-camera-simulator_1584962343211/work/src/SimulatorFrameLoader.cpp:247)-Funct: Exit
[2020/06/10 19:36:04.536292] 7f4c62763700 *Camera*Simulator::FrameLoader::getNextFrame (/opt/conda/conda-bld/lima-camera-simulator_1584962343211/work/src/SimulatorFrameLoader.cpp:247)-Funct: Enter
[2020/06/10 19:36:04.536509] 7f4c62763700 *Camera*Simulator::FrameLoader::getNextFrame (/opt/conda/conda-bld/lima-camera-simulator_1584962343211/work/src/SimulatorFrameLoader.cpp:279)-Trace: Open file /data/id21/inhouse/wout/dev/blissscripts/limapreloaddata/ferrierite_bc_fast_0109.edf
[2020/06/10 19:36:04.536537] 7f4c62763700 *Camera*Simulator::FrameLoader::getNextFrame (/opt/conda/conda-bld/lima-camera-simulator_1584962343211/work/src/SimulatorFrameLoader.cpp:306)-Trace: frame_dim=<2048x2048x2-Bpp16>, mem_size=8388608
[2020/06/10 19:36:04.539914] 7f4c62763700 *Camera*Simulator::FrameLoader::getNextFrame (/opt/conda/conda-bld/lima-camera-simulator_1584962343211/work/src/SimulatorFrameLoader.cpp:321)-Trace: data: 64 52 5c 5c
[2020/06/10 19:36:04.539926] 7f4c62763700 *Camera*Simulator::FrameLoader::getNextFrame (/opt/conda/conda-bld/lima-camera-simulator_1584962343211/work/src/SimulatorFrameLoader.cpp:247)-Funct: Exit
[2020/06/10 19:36:04.539930] 7f4c62763700 *Camera*Simulator::FrameLoader::getNextFrame (/opt/conda/conda-bld/lima-camera-simulator_1584962343211/work/src/SimulatorFrameLoader.cpp:247)-Funct: Enter
[2020/06/10 19:36:04.540047] 7f4c62763700 *Camera*Simulator::FrameLoader::getNextFrame (/opt/conda/conda-bld/lima-camera-simulator_1584962343211/work/src/SimulatorFrameLoader.cpp:247)-Funct: Exit
[2020/06/10 19:36:04.540058] 7f4c62763700 *Camera*SimuThread::Camera::execPrepareAcq (/opt/conda/conda-bld/lima-camera-simulator_1584962343211/work/src/SimulatorCamera.cpp:83)-Funct: Exit
[2020/06/10 19:36:04.540067] 7f4c62763700 *Camera*SimuThread::Camera::execCmd (/opt/conda/conda-bld/lima-camera-simulator_1584962343211/work/src/SimulatorCamera.cpp:66)-Funct: Exit
***** Thread 7f4c62763700 function exited due to an exception without clean-up! *****