jenkins: all test suite runs on base branches are considered failures
(Just in case it isn't clear outside of FT, base branches are considered stable, testing and devel, but until tails/puppet-tails!140 is merged this actually affects all branches that doesn't start with a number.)
On jenkins,
* [test_Tails_ISO_stable \#4565](https://jenkins.tails.boum.org/job/test_Tails_ISO_stable/4565/) from Dec 12 was the last run where all scenarios passed and the job was correclty marked as a success, and
* [test_Tails_ISO_stable \#4576](https://jenkins.tails.boum.org/job/test_Tails_ISO_stable/4576/) from Dec 18 is (AFAICT) the first run after that where all scenario passed but the job was incorrectly considered a failure.
So what happened between Dec 12 and 18? On Dec 14 https://gitlab.tails.boum.org/tails/puppet-tails/-/commit/ba5ba04f95a81cd6d0f1ff17dd87acda9d3c93f0 started running `sys.exit("...")` for these branches, and it will exit with 1 (so error) when passed a string. I am not 100% sure this is the cause, but it looks promising. The fix would be to just `print()` the message and then `sys.exit(0)`. I added this potential fix to my tails/puppet-tails!140 MR.
issue