Resolve "Cached connections owned by the Beacon connection"
Closes #2396 (closed)
This MR changes the management of the Redis cache. I kept the current implementation (apart from some small bugs) and moved things around in preparation for optimizations that involve Redis IO (e.g. !3167 (merged)).
I will give the complete overview of Redis connection and cache management in this description. Most of this already existed, but some stuff is managed differently.
Beacon connection
A Beacon connection manages all Redis proxies, Redis connection pools and Redis caches. Creating any Redis connection should go through the Beacon connection:
beacon_conn.get_redis_proxy(db=0)
beacon_conn.get_caching_redis_proxy(db=0, shared_cache=True)
Currently all db=0
caching proxies share the same cache (so they use shared_cache=True
which is the default):
-
BasicScanSaving
wardrobe -
ScanDisplay
wardrobe -
AxisSettings
hashsetting - some of the
MeasurementGroup
settings
There is only one place where a db=1
caching proxy is created: the Scan.root_node
and its children (scan node, controller nodes, channel nodes). We create a new cache for each scan (so get_caching_redis_proxy(db=1, shared_cache=False)
). Otherwise we would accumulating db=1
cache over time. In addition the cache is destroyed at the end of the scan and the node proxies start behaving like normal (uncached) proxies.
Redis connection without beacon
You can also use the entire Redis API without a beacon connection but then you are in charge of cleaning up resources if you can't wait for garbage collection (especially pool.disconnect()
when you need to close cache and/or sockets)
pool = create_connection_pool("redis://localhost:25002", db=1, caching=True)
proxy = pool.create_proxy()
Redis connection pools
Each pool only talks to one Redis database. Think of a connection as a socket, except that it is not greenlet-safe to execute transactions (transaction: send Redis command followed by receiving the response).
graph TD;
redis.connection.ConnectionPool-->RedisDbConnectionPool;
RedisDbConnectionPool-->CachingRedisDbConnectionPool;
-
RedisDbConnectionPool
makesConnectionPool
greenlet-safe and ensures all connections set their client name - Each
CachingRedisDbConnectionPool
instance owns oneRedisCache
instance (dictionary +CacheInvalidationGreenlet
). -
RedisCache
has a dict interface and also owns theCacheInvalidationGreenlet
. When the greenlet is not running, any dictionary-like access to the cache will raise aRedisCacheException
. -
CacheInvalidationGreenlet
is a greenlet that listens to the key invalidation pubsub connection. -
CachingRedisDbConnectionPool
registers new connections with theCacheInvalidationGreenlet
pubsub connection of it'sRedisCache
. This is needed to ensure that keys changed in the Redis database are removed from the local cache.
Redis synchronous proxies
They should be instantiated from the connection pools. Redis commands send through these proxies are executed immediately.
graph TD;
redis.client.Redis-->RedisDbProxy;
RedisDbProxy-->SafeRedisDbProxy;
RedisDbProxy-->SingleRedisConnectionProxy;
SafeRedisDbProxy-->CachingRedisDbProxy;
-
SafeRedisDbProxy
can be created fromRedisDbConnectionPool.create_proxy
-
SingleRedisConnectionProxy
can be created fromRedisDbConnectionPool.create_single_connection_proxy
-
SingleRedisConnectionProxy
in not greenlet-safe and is currently only used inCacheInvalidationGreenlet
. -
CachingRedisDbProxy
(which used to be calledCacheConnection
) is no longer a wrapper. It IS a proxy which can be created fromCachingRedisDbConnectionPool.create_proxy
.
Redis asynchronous proxies
They should be instantiated from a proxy (synchronous or asynchronous). Redis commands send through these proxies are are executed asynchronously (postponed until you call syncproxy.execute()
or canceled with syncproxy.reset()
)
graph TD;
redis.client.Pipeline-->AsyncRedisDbProxy;
AsyncRedisDbProxy-->CachingAsyncRedisDbProxy
An asynchronous proxy holds a fixed connection from the Redis connection pool until execute
or reset
. It is therefore not greenlet-safe.