Skip to content
GitLab
Projects Groups Snippets
  • /
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in
  • L Lima-camera-eiger
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
  • Issues 4
    • Issues 4
    • List
    • Boards
    • Service Desk
    • Milestones
  • Jira
    • Jira
  • Merge requests 0
    • Merge requests 0
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Schedules
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Monitor
    • Monitor
    • Incidents
  • Analytics
    • Analytics
    • Value stream
    • CI/CD
    • Repository
  • Wiki
    • Wiki
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar
  • LimaGroupLimaGroup
  • Lima-camera-eiger
  • Issues
  • #12
Closed
Open
Issue created Aug 12, 2021 by Jonathan Wright@wright

Out of memory crashes during prepareAcq

I understood this is a known bug. Here are the steps to get it to reproduce at ID11, I just verified it still crashes since upgrading to bliss 1.8. Here is a plot of the percent memory + cpu versus time from top:

wrght

This needs to be run a few times from when the bliss nscope session is restarted to kill the Lima server:

# /users/opid11/bliss_scripts/eigertimeout.py                                                                                                                                                                                                                                                                    
def eigertimeout1():                                                                                                                                        
    # go between 32/16 bit and a couple vs many frames                                                                                                              
    time.sleep(5)                                                                                                                                           
    eiger.camera.auto_summation = 'OFF'                                                                                                                     
    limatake(0.1,2)                                                                                                                                         
    t = 0.002 # go to 16 bit to 32 bit                                                                                                                      
    limatake(t, 120*1024*1024*1024//(2162*2068*2),save=False)                                                                                               
    time.sleep(5)                                                                                                                                           
    eiger.camera.auto_summation = 'ON'                                                                                                                      
    limatake(0.1,2)                                                                                                                                         
    print("!"*80)                                                                                                                                           
    print("####### control-c the next scan please, when collecting frames")                                                                                 
    print("!"*80)                                                                                                                                           
    limatake(t, 120*1024*1024*1024//(2162*2068*4),save=False)              
  1. on lid11eh3 start nscope session, run the test script and control-c when it is collecting frames in the scan that is tagged:
NSCOPE [1]: user_script_load('eigertimeout.py')
Loading [/users/opid11/bliss_scripts/eigertimeout.py]
Merged [user] namespace in session.
NSCOPE [2]: user.eigertimeout1()
acquisition chain
\u2514\u2500\u2500 eiger

Scan 18 Thu Aug 12 11:11:29 2021  nscope user = opid11
limatake 0.1000 2
Preparing eiger ...
Running ...
eiger acq #2  
Finished (took 0:00:05.300922)

acquisition chain
\u2514\u2500\u2500 eiger

Scan 19 Thu Aug 12 11:11:35 2021  nscope user = opid11
limatake 0.0020 14409
Preparing eiger ...
Running ...
eiger acq #14409  
Finished (took 0:01:15.016661)

acquisition chain
\u2514\u2500\u2500 eiger

Scan 20 Thu Aug 12 11:12:56 2021  nscope user = opid11
limatake 0.1000 2
Preparing eiger ...
Running ...
eiger acq #2  
Finished (took 0:00:02.628463)

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
####### control-c the next scan please, when collecting frames
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
acquisition chain
\u2514\u2500\u2500 eiger

Scan 21 Thu Aug 12 11:12:58 2021  nscope user = opid11
limatake 0.0020 7204
Preparing eiger ...
Running ...
^Ceiger acq #4436  
Finished (took 0:00:47.220614)

!!! === ScanAbort:  === !!! ( for more details type cmd 'last_error' )
  1. exit bliss and repeat step 1, it should crash on the second or third run.

Over on lid11eiger2lima you can watch the memory grow with: top -d 1 -b | grep -e 1430951 -e ^top

tagging @papillon. This bug was previously being chased on the bliss side bliss/bliss!3888 (merged), I am moving it here as it seems to be the lima eiger server that is crashing.

Assignee
Assign to
Time tracking