Migrate Jenkins to Dragon

In our CI jobs, we do a lot of copying artifacts back and forth between Jenkins and its workers. The performance of all such operations (possibly happening concurrently) is bound by the speed of 1 single CPU core on the Jenkins orchestrator. lizard's CPU cores are individually slow, so it would help quite a bit if we migrated it to Dragon.

Preparation:

• set up nginx locally on jenkins.lizard to serve jenkins.tails.boum.org and nightly.tails.boum.org
• manually copy letsencrypt certificates from www to jenkins
• coordinate downtime
• prepare a base install jenkins.dragon VM on dragon
• prepare firewalling on dragon to forward port 80/443 to jenkins.dragon
• prepare static html on www to serve during downtime (a simple maintenance page)

Migration steps:

• shut down both jenkins.lizard and jenkins.dragon
• shut down all isotesters/builders/workers
• let www serve the maintenance page
• rsync all data except puppet keys/certs from jenkins.lizard to jenkins.dragon
• replace jenkins.lizard with jenkins.dragon everywhere in hiera, puppet-tails, and jenkins-jobs
• rename the node entry in manifests/nodes.pp from jenkins.lizard to jenkins.dragon
• remove the NFS service from jenkins.dragon
• fix networking on jenkins.dragon (both manually and in puppet)
• spin up jenkins.dragon and fix whatever is broken / breaks during puppet run
• spin up all isotesters/builders/workers
• change DNS entries for jenkins.tails.boum.org and nightly.tails.boum.org
• test/debug/fix jenkins
• set up backups for jenkins.dragon
• update documentation

Cleanup:

• remove the NFS mount on www
• remove the jenkins.tails.boum.org and nightly.tails.boum.org vhosts on www
• remove the jenkins.lizard VM
• remove the backup job for jenkins.lizard
• remove monitoring for jenkins.lizard