Make the test suite clean up after itself even in most tricky failure modes (#17216) · Issues · tails / sysadmin

Make the test suite clean up after itself even in most tricky failure modes

Originally created by @intrigeri on #17216 (Redmine)

Goals:

avoid leaving zombie processes behind (e.g. Xvfb, unclutter, tor processes started by Chutney, avconv, local services run by cucumber, etc.)
drop the whole “reboot before running the test suite” dance in our CI (#10601 (closed), #11295 (closed))
have CI nodes that can run both builds & tests, which would provide great performance improvements to our feedback loop: we often have build jobs in the queue while isotesters are idling, or test jobs in the queue while builders are idling

How to do so? Wrap the test suite with something that will track resources and clean them up, such as:

systemd-run, which could a not-super-expensive incremental step forward: #11295 (comment 61201)
Docker, which would allow us to have multiple Jenkins executors on one single big node (instead of a number of isotesterN VMs with one executor each), resulting in better usage of our hardware resources:

Scope:

Initially: our CI.
Ideally, the solution would also work for developers who run the test suite locally.
Ideally, the solution would not depend on Jenkins much, if at all, so we benefit from it even if we migrate to another kind of CI some day.

Edited May 15, 2020 by intrigeri

To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information