Max size for the worker threadpool
On huge machines, the worker threadpool looks to be setup by default to be able to use every CPU available.
It would be good to provide something to limit that.
Celery provides some options we could expose in sidecar
- https://docs.celeryq.dev/en/stable/userguide/optimizing.html
- https://docs.celeryq.dev/en/stable/userguide/configuration.html#std-setting-worker_concurrency
At iccid191
there is about 64 theads pre allocated, but we will process a single task at a time.
5677 5677 5677 ? 00:07:33 supervisord
3058720 3058720 5677 ? 00:06:07 sidecar.worker
3058955 3058720 5677 ? 00:00:00 sidecar.worker
3058957 3058720 5677 ? 00:00:00 sidecar.worker
3058959 3058720 5677 ? 00:00:00 sidecar.worker
3058961 3058720 5677 ? 00:00:00 sidecar.worker
3058963 3058720 5677 ? 00:00:00 sidecar.worker
3058965 3058720 5677 ? 00:00:00 sidecar.worker
3058967 3058720 5677 ? 00:00:00 sidecar.worker
3058969 3058720 5677 ? 00:00:00 sidecar.worker
3058971 3058720 5677 ? 00:00:00 sidecar.worker
3058973 3058720 5677 ? 00:00:00 sidecar.worker
3058976 3058720 5677 ? 00:00:00 sidecar.worker
3058978 3058720 5677 ? 00:00:00 sidecar.worker
3058980 3058720 5677 ? 00:00:00 sidecar.worker
3058982 3058720 5677 ? 00:00:00 sidecar.worker
3058984 3058720 5677 ? 00:00:00 sidecar.worker
3058986 3058720 5677 ? 00:00:00 sidecar.worker
3058988 3058720 5677 ? 00:00:00 sidecar.worker
3058990 3058720 5677 ? 00:00:00 sidecar.worker
3058992 3058720 5677 ? 00:00:00 sidecar.worker
3058994 3058720 5677 ? 00:00:00 sidecar.worker
3058996 3058720 5677 ? 00:00:00 sidecar.worker
3058998 3058720 5677 ? 00:00:00 sidecar.worker
3059001 3058720 5677 ? 00:00:00 sidecar.worker
3059003 3058720 5677 ? 00:00:00 sidecar.worker
3059005 3058720 5677 ? 00:00:00 sidecar.worker
3059007 3058720 5677 ? 00:00:00 sidecar.worker
3059009 3058720 5677 ? 00:00:00 sidecar.worker
3059011 3058720 5677 ? 00:00:00 sidecar.worker
3059013 3058720 5677 ? 00:00:00 sidecar.worker
3059015 3058720 5677 ? 00:00:00 sidecar.worker
3059017 3058720 5677 ? 00:00:00 sidecar.worker
3059019 3058720 5677 ? 00:00:00 sidecar.worker
3059021 3058720 5677 ? 00:00:00 sidecar.worker
3059024 3058720 5677 ? 00:00:00 sidecar.worker
3059026 3058720 5677 ? 00:00:00 sidecar.worker
3059028 3058720 5677 ? 00:00:00 sidecar.worker
3059030 3058720 5677 ? 00:00:00 sidecar.worker
3059032 3058720 5677 ? 00:00:00 sidecar.worker
3059034 3058720 5677 ? 00:00:00 sidecar.worker
3059036 3058720 5677 ? 00:00:00 sidecar.worker
3059038 3058720 5677 ? 00:00:00 sidecar.worker
3059040 3058720 5677 ? 00:00:00 sidecar.worker
3059042 3058720 5677 ? 00:00:00 sidecar.worker
3059044 3058720 5677 ? 00:00:00 sidecar.worker
3059046 3058720 5677 ? 00:00:00 sidecar.worker
3059049 3058720 5677 ? 00:00:00 sidecar.worker
3059051 3058720 5677 ? 00:00:00 sidecar.worker
3059053 3058720 5677 ? 00:00:00 sidecar.worker
3059055 3058720 5677 ? 00:00:00 sidecar.worker
3059057 3058720 5677 ? 00:00:00 sidecar.worker
3059059 3058720 5677 ? 00:00:00 sidecar.worker
3059061 3058720 5677 ? 00:00:00 sidecar.worker
3059063 3058720 5677 ? 00:00:00 sidecar.worker
3059065 3058720 5677 ? 00:00:00 sidecar.worker
3059067 3058720 5677 ? 00:00:00 sidecar.worker
3059069 3058720 5677 ? 00:00:00 sidecar.worker
3059071 3058720 5677 ? 00:00:00 sidecar.worker
3059073 3058720 5677 ? 00:00:00 sidecar.worker
3059074 3058720 5677 ? 00:00:00 sidecar.worker
3059076 3058720 5677 ? 00:00:00 sidecar.worker
3059078 3058720 5677 ? 00:00:00 sidecar.worker
3059079 3058720 5677 ? 00:00:00 sidecar.worker
3059080 3058720 5677 ? 00:00:00 sidecar.worker
3059081 3058720 5677 ? 00:00:00 sidecar.worker
3544029 3544029 5677 ? 00:05:29 sidecar.worker
3544179 3544029 5677 ? 00:00:00 sidecar.worker
3544180 3544029 5677 ? 00:00:00 sidecar.worker
3544183 3544029 5677 ? 00:00:00 sidecar.worker
3544189 3544029 5677 ? 00:00:00 sidecar.worker
3544190 3544029 5677 ? 00:00:00 sidecar.worker
3544198 3544029 5677 ? 00:00:00 sidecar.worker
3544199 3544029 5677 ? 00:00:00 sidecar.worker
3544201 3544029 5677 ? 00:00:00 sidecar.worker
3544208 3544029 5677 ? 00:00:00 sidecar.worker
3544209 3544029 5677 ? 00:00:00 sidecar.worker
3544219 3544029 5677 ? 00:00:00 sidecar.worker
3544220 3544029 5677 ? 00:00:00 sidecar.worker
3544222 3544029 5677 ? 00:00:00 sidecar.worker
3544229 3544029 5677 ? 00:00:00 sidecar.worker
3544230 3544029 5677 ? 00:00:00 sidecar.worker
3544238 3544029 5677 ? 00:00:00 sidecar.worker
3544239 3544029 5677 ? 00:00:00 sidecar.worker
3544241 3544029 5677 ? 00:00:00 sidecar.worker
3544248 3544029 5677 ? 00:00:00 sidecar.worker
3544250 3544029 5677 ? 00:00:00 sidecar.worker
3544257 3544029 5677 ? 00:00:00 sidecar.worker
3544258 3544029 5677 ? 00:00:00 sidecar.worker
3544261 3544029 5677 ? 00:00:00 sidecar.worker
3544267 3544029 5677 ? 00:00:00 sidecar.worker
3544268 3544029 5677 ? 00:00:00 sidecar.worker
3544276 3544029 5677 ? 00:00:00 sidecar.worker
3544277 3544029 5677 ? 00:00:00 sidecar.worker
3544280 3544029 5677 ? 00:00:00 sidecar.worker
3544287 3544029 5677 ? 00:00:00 sidecar.worker
3544288 3544029 5677 ? 00:00:00 sidecar.worker
3544291 3544029 5677 ? 00:00:00 sidecar.worker
3544297 3544029 5677 ? 00:00:00 sidecar.worker
3544298 3544029 5677 ? 00:00:00 sidecar.worker
3544302 3544029 5677 ? 00:00:00 sidecar.worker
3544307 3544029 5677 ? 00:00:00 sidecar.worker
3544309 3544029 5677 ? 00:00:00 sidecar.worker
3544316 3544029 5677 ? 00:00:00 sidecar.worker
3544317 3544029 5677 ? 00:00:00 sidecar.worker
3544319 3544029 5677 ? 00:00:00 sidecar.worker
3544326 3544029 5677 ? 00:00:00 sidecar.worker
3544327 3544029 5677 ? 00:00:00 sidecar.worker
3544330 3544029 5677 ? 00:00:00 sidecar.worker
3544336 3544029 5677 ? 00:00:00 sidecar.worker
3544337 3544029 5677 ? 00:00:00 sidecar.worker
3544345 3544029 5677 ? 00:00:00 sidecar.worker
3544346 3544029 5677 ? 00:00:00 sidecar.worker
3544348 3544029 5677 ? 00:00:00 sidecar.worker
3544355 3544029 5677 ? 00:00:00 sidecar.worker
3544356 3544029 5677 ? 00:00:00 sidecar.worker
3544359 3544029 5677 ? 00:00:00 sidecar.worker
3544365 3544029 5677 ? 00:00:00 sidecar.worker
3544366 3544029 5677 ? 00:00:00 sidecar.worker
3544373 3544029 5677 ? 00:00:00 sidecar.worker
3544375 3544029 5677 ? 00:00:00 sidecar.worker
3544377 3544029 5677 ? 00:00:00 sidecar.worker
3544384 3544029 5677 ? 00:00:00 sidecar.worker
3544385 3544029 5677 ? 00:00:00 sidecar.worker
3544387 3544029 5677 ? 00:00:00 sidecar.worker
3544394 3544029 5677 ? 00:00:00 sidecar.worker
3544395 3544029 5677 ? 00:00:00 sidecar.worker
3544398 3544029 5677 ? 00:00:00 sidecar.worker
3544404 3544029 5677 ? 00:00:00 sidecar.worker
3544406 3544029 5677 ? 00:00:50 sidecar.worker
3544413 3544029 5677 ? 00:00:20 sidecar.worker
Edited by Valentin Valls