Adjust for ikiwiki → GitLab wiki authored by intrigeri's avatar intrigeri
[[!tag archived]]
This is about [[!tails_ticket 5630]].
[[!toc levels=2]] This is about tails/tails#5630.
[[_TOC_]]
<a id="why"></a> <a id="why"></a>
...@@ -87,16 +89,16 @@ that some of these advantages are incompatible with each other. ...@@ -87,16 +89,16 @@ that some of these advantages are incompatible with each other.
- decreases the risk that our build systems are all affected by the - decreases the risk that our build systems are all affected by the
same architecture — or platform — specific security issue; same architecture — or platform — specific security issue;
- may allow us to support more architectures (e.g. - may allow us to support more architectures (e.g.
[[!tails_ticket 10972 desc="ARM"]]), because we could do most our ARM (tails/tails#10972)), because we could do most our
CI and QA on the fastest one. CI and QA on the fastest one.
* Detect bugs, for example the ones provided on the * Detect bugs, for example the ones provided on the
[reproducible builds project's buy-in page](https://reproducible-builds.org/docs/buy-in/) [reproducible builds project's buy-in page](https://reproducible-builds.org/docs/buy-in/)
(retrieved on 2015-02-16): (retrieved on 2015-02-16):
"[[!debbug 773916 desc="a library had a different application binary interface for every build"]], "[a library had a different application binary interface for every build](https://bugs.debian.org/773916),
[[!debbug 801855 desc="garbled strings due to encoding mismatch"]], [garbled strings due to encoding mismatch](https://bugs.debian.org/801855),
[[!debbug 778486 desc="missing translations"]], [missing translations](https://bugs.debian.org/778486),
or [[!debbug 778707 desc="changing dependencies"]]". or [changing dependencies](https://bugs.debian.org/778707)".
* "The constraint of having to reflect about the build environment * "The constraint of having to reflect about the build environment
also helps developers to think about the relationship with external also helps developers to think about the relationship with external
...@@ -115,7 +117,7 @@ that some of these advantages are incompatible with each other. ...@@ -115,7 +117,7 @@ that some of these advantages are incompatible with each other.
This draft needs to be updated. It was written in 2014, way before we This draft needs to be updated. It was written in 2014, way before we
came up with the previous list of advantages, or discussed this topic came up with the previous list of advantages, or discussed this topic
much. It was primarily meant to explain why we needed much. It was primarily meant to explain why we needed
a [[!tails_ticket 5926 desc="freezable APT repository"]], as a first a freezable APT repository (tails/tails#5926), as a first
step towards reproducible builds. step towards reproducible builds.
</div> </div>
...@@ -235,12 +237,12 @@ following implementation for our first iteration: ...@@ -235,12 +237,12 @@ following implementation for our first iteration:
* To freeze the build environment, we use APT snapshots in the same way * To freeze the build environment, we use APT snapshots in the same way
we do in the Tails build system, by storing the serials for the various we do in the Tails build system, by storing the serials for the various
APT repositories in APT repositories in
[[!tails_gitweb_dir vagrant/definitions/tails-builder/config/APT_snapshots.d/]]. [vagrant/definitions/tails-builder/config/APT snapshots.d/](https://gitlab.tails.boum.org/tails/tails/-/tree/master/vagrant/definitions/tails-builder/config/APT_snapshots.d/).
* Only the debian-security APT source uses Debian's APT repository, so * Only the debian-security APT source uses Debian's APT repository, so
that we get security fixes. This will probably not influence the that we get security fixes. This will probably not influence the
reproducibility of the ISO. This is done in the reproducibility of the ISO. This is done in the
[[!tails_gitweb vagrant/provision/setup-tails-builder desc="Vagrant provisioning [Vagrant provisioning
script"]]. script](https://gitlab.tails.boum.org/tails/tails/-/blob/master/vagrant/provision/setup-tails-builder).
* To ensure that changes in the Vagrant build system are still taken into account when * To ensure that changes in the Vagrant build system are still taken into account when
using a basebox, we dynamically set the name of the basebox by using a basebox, we dynamically set the name of the basebox by
including the short ID of the last commit in the `vagrant` directory in including the short ID of the last commit in the `vagrant` directory in
...@@ -248,7 +250,7 @@ following implementation for our first iteration: ...@@ -248,7 +250,7 @@ following implementation for our first iteration:
* We update the basebox APT snapshots serials when preparing a new Tails * We update the basebox APT snapshots serials when preparing a new Tails
major release. major release.
* A new VM is created from the basebox for each build. After the build, * A new VM is created from the basebox for each build. After the build,
the VM is destroyed ([[!tails_ticket 11980]] and [[!tails_ticket 11981]]). the VM is destroyed (tails/tails#11980 and tails/tails#11981).
* The `keeprunning` build option can be used so that the VM is kept running * The `keeprunning` build option can be used so that the VM is kept running
and reused for subsequent builds of the same branch. and reused for subsequent builds of the same branch.
* The VM encodes (in `/var/lib/vagrant_box_build_from`) the branch for * The VM encodes (in `/var/lib/vagrant_box_build_from`) the branch for
...@@ -261,7 +263,7 @@ following implementation for our first iteration: ...@@ -261,7 +263,7 @@ following implementation for our first iteration:
build VM. build VM.
In our Jenkins setup we instead use an existing, external In our Jenkins setup we instead use an existing, external
`apt-cacher-ng` in order to share the APT cache between build VMs `apt-cacher-ng` in order to share the APT cache between build VMs
and save disk space ([[!tails_ticket 11979]]). and save disk space (tails/tails#11979).
So we don't publish Vagrant baseboxes anymore: everyone building So we don't publish Vagrant baseboxes anymore: everyone building
a Tails ISO image will instead build their own. Their resulting a Tails ISO image will instead build their own. Their resulting
...@@ -270,7 +272,7 @@ similar enough to produce ISO images that are identical to the ones ...@@ -270,7 +272,7 @@ similar enough to produce ISO images that are identical to the ones
published by the Tails project. published by the Tails project.
**Edit:** It has now been released and is documented in the page about Tails **Edit:** It has now been released and is documented in the page about Tails
[[Vagrant setup|contribute/build/vagrant-setup]]. [Vagrant setup](https://tails.boum.org/contribute/build/vagrant-setup).
## Make Tails ISO build in a reproducible manner ## Make Tails ISO build in a reproducible manner
...@@ -400,14 +402,14 @@ through a few additional steps, such as: ...@@ -400,14 +402,14 @@ through a few additional steps, such as:
throughput, we might have to purchase and allocate more hardware throughput, we might have to purchase and allocate more hardware
to them. to them.
For estimates on hardware cost for Lizard ([[!tails_ticket 12002]]), see [[the For estimates on hardware cost for Lizard (tails/tails#12002), see [the
dedicated page|blueprint/reproducible_builds/hardware]]. dedicated page](reproducible_builds/hardware).
**Edit:** We already adapted our server infrastructure to support this project, **Edit:** We already adapted our server infrastructure to support this project,
integrating it in our [[Vagrant setup|contribute/build/vagrant-setup]] integrating it in our [Vagrant setup](https://tails.boum.org/contribute/build/vagrant-setup)
mentioned above. Specifics about how we deployed that in our Jenkins infra are mentioned above. Specifics about how we deployed that in our Jenkins infra are
[[documented [documented
here|contribute/working_together/roles/sysadmins/automated_builds_in_Jenkins]]. here](https://tails.boum.org/contribute/working_together/roles/sysadmins/automated_builds_in_Jenkins).
# Various ideas # Various ideas
...@@ -490,7 +492,7 @@ See "Adjust our infrastructure accordingly" above. ...@@ -490,7 +492,7 @@ See "Adjust our infrastructure accordingly" above.
* <https://wiki.debian.org/ReproducibleInstalls> * <https://wiki.debian.org/ReproducibleInstalls>
* [[!tor_bug 3688]] * [bug #3688 on Tor Project's Trac](https://bugs.torproject.org/3688)
* Making POT files more stable: debconf 1.5.58's changelog entry reads * Making POT files more stable: debconf 1.5.58's changelog entry reads
"Don't update po/debconf.pot unless doing so changes something other than "Don't update po/debconf.pot unless doing so changes something other than
...@@ -501,16 +503,16 @@ See "Adjust our infrastructure accordingly" above. ...@@ -501,16 +503,16 @@ See "Adjust our infrastructure accordingly" above.
- see commits authored by "lamby" in - see commits authored by "lamby" in
[Webconverger's Debian live configuration](https://github.com/Webconverger/Debian-Live-config/commits/master), [Webconverger's Debian live configuration](https://github.com/Webconverger/Debian-Live-config/commits/master),
that include a script to clamp mtimes that include a script to clamp mtimes
- [[!debbug 832998]] / [[!debbug 833118]] - [Debian bug #832998](https://bugs.debian.org/832998) / [Debian bug #833118](https://bugs.debian.org/833118)
* Making ISO filesystem reproducible: * Making ISO filesystem reproducible:
- ask Chris Lamb <lamby@debian.org> (keywords: libisofs, - ask Chris Lamb <lamby@debian.org> (keywords: libisofs,
libisoburn, xorriso) libisoburn, xorriso)
- [[!debbug 831379]] / [[!debbug 832689]] - [Debian bug #831379](https://bugs.debian.org/831379) / [Debian bug #832689](https://bugs.debian.org/832689)
# Progress # Progress
See [[our May report|news/report_2017_05]]. See [our May report](https://tails.boum.org/news/report_2017_05).
# Future work # Future work
...@@ -519,7 +521,7 @@ See [[our May report|news/report_2017_05]]. ...@@ -519,7 +521,7 @@ See [[our May report|news/report_2017_05]].
## Integrate custom Debian package builds in the automated ISO builds ## Integrate custom Debian package builds in the automated ISO builds
<div class="note"> <div class="note">
See discussion on [[!tails_ticket 6220]]. See discussion on tails/tails#6220.
</div> </div>
In our first iteration of reproducible ISO builds, we treat the In our first iteration of reproducible ISO builds, we treat the
...@@ -533,7 +535,7 @@ process as trusted input. These repositories are of two kinds: ...@@ -533,7 +535,7 @@ process as trusted input. These repositories are of two kinds:
improved later, and is outside of the scope of the work described improved later, and is outside of the scope of the work described
in this section; in this section;
* our [[custom APT repository|contribute/APT_repository/custom]], * our [custom APT repository](https://tails.boum.org/contribute/APT_repository/custom),
that stores our custom Debian packages; in addition to the people that stores our custom Debian packages; in addition to the people
listed above (system administrators, successful attackers), Tails listed above (system administrators, successful attackers), Tails
developers with commit rights can modify the content of this developers with commit rights can modify the content of this
...@@ -552,7 +554,7 @@ Debian packages: ...@@ -552,7 +554,7 @@ Debian packages:
different developers' systems. So, one can't reproduce the build of different developers' systems. So, one can't reproduce the build of
a given ISO unless they use the exact packages that were uploaded a given ISO unless they use the exact packages that were uploaded
to our custom APT repository. This brings the same to our custom APT repository. This brings the same
[[set of problems|reproducible_builds#why]] that lead us to make [set of problems](reproducible_builds#why) that lead us to make
our ISO image build reproducibly. For example, the state of our Git our ISO image build reproducibly. For example, the state of our Git
tree does not fully define what an ISO built from it will be, which tree does not fully define what an ISO built from it will be, which
makes reviewing and auditing harder than it should be. makes reviewing and auditing harder than it should be.
...@@ -593,3 +595,4 @@ There are a number of open questions: ...@@ -593,3 +595,4 @@ There are a number of open questions:
Part of this project will therefore be to research and discuss these Part of this project will therefore be to research and discuss these
topics with the affected parties, and come up with user stories and topics with the affected parties, and come up with user stories and
with a fitting design. with a fitting design.