title: Automated builds and tests
An automated build and test environment would be pretty useful to ensure a few facts:
- new code does not break anything
- new build tools (
live-build) and included software (
live-boot) don't break anything
- updates pushed to the APT repositories we don't control (e.g. Debian's) don't break anything
- the quality of our releases is good enough
- power users get to test early images built from feature branches,
- we can ask bug reporters to test an image built from a bugfix branch, and see if it fixes their problem
What we have
- A Jenkins instance running on lizard, that builds a few branches whenever they are pushed. Results are available publicly.
- A partially automated test suite:
- Complete phase one (build and publish ISOs automatically): fix all child tickets of tails#5324 (closed).
- Run the test suite automatically on autobuilt ISOs: tails#5288 (closed)
- See other tickets in the Continuous Integration category, and organize the next steps.
What we need
Ideally, every automatically built ISO should be automatically tested, and the results displayed online.
Putting automatic builds and automatic tests on the same physical machine (while using virtualization to separate concerns) saves hosting and bandwidth costs: fresh images can be tested without being transferred.
In order to be able to build Tails in memory (which greatly speeds up builds) and to properly test HIGHMEM memory wipes, the test server should have at least 32 GB. Using KVM inside a KVM virtual machine is supported by the "nested KVM" feature. VirtualBox (Vagrant) in KVM is not supported as of March 2013.
We could set a threshold of tests that have to be successful before publishing one daily build image. This would prevent too-broken images to spread out and potentially harm users.
The setup of the automated build and test server should be done with public Puppet modules.