Commit 212d6a83 authored by sajolida's avatar sajolida
Browse files

Merge remote-tracking branch 'origin/master' into devel

parents 8a5d4cef 68904c7b
......@@ -128,7 +128,7 @@ if [ -n "$JENKINS_URL" ] && [ -z "$GIT_TAG" ] \
fi
# build the doc wiki
./build-wiki
./build-website
# refresh translations of our programs
./refresh-translations || fatal "refresh-translations failed ($?)."
......
......@@ -83,7 +83,7 @@ if [ "$TAILS_CLEAN_BUILD" ]; then
fi
if [ -z "$JENKINS_URL" ]; then
./build-wiki
./build-website
fi
BUILD_DIR=$(mktemp -d /tmp/tails-build.XXXXXXXX)
......
**Ticket**: [[!tails_ticket 7161]]
The idea I had was to let the server(s) send a reduced list of hosts. Not
only it would allow to work-around Tor DNS limitations, but also to have
some weighted round robin, in order to prioritize some high bandwidth
mirrors, if we choose to.
If I had to mention the ideal design goals for such changes, I would say
that the more straightforward would be the better for implementation and
also for maintainability.
[[!toc levels=3]]
# The plan
......@@ -42,6 +33,15 @@ We decided to implement a two-way strategy for this feature:
# Initial research
The idea I had was to let the server(s) send a reduced list of hosts. Not
only it would allow to work-around Tor DNS limitations, but also to have
some weighted round robin, in order to prioritize some high bandwidth
mirrors, if we choose to.
If I had to mention the ideal design goals for such changes, I would say
that the more straightforward would be the better for implementation and
also for maintainability.
## Using DNS
Using DNS seems to be an easy way to do some round robin in low level. It
......
......@@ -14,28 +14,308 @@ This reports covers the activity of Tails in September 2015.
Everything in this report can be made public.
Note: the numbers preceded with a `#` correspond to tickets in our bug
tracker which contains more technical details and timeline. For example,
ticket #6938 can been seed on https://labs.riseup.net/code/issues/6938.
tracker which contain more technical details and timeline. For example,
ticket #6938 can be seen on <https://labs.riseup.net/code/issues/6938>.
# A. Replace Claws Mail with Icedove
## A.n. description of subsection
- A.1.3. Integrate Icedove into Tails
- A.n.m. description of deliverable: ticket numbers
We amended the strategy we had in mind initially and made it more
incremental. If time allows, a first stage will be to include
Icedove in Tails 1.7 (2015-11-03), but without the Icedove account
setup wizard. During this first stage, Torbirdy's own account setup
wizard will be used. And the second stage will be about securing
Icedove's wizard (#6154). If this works out as we hope, Tails users
will be able to start using Icedove two months earlier than what we
planned initially, and the transition period from Claws Mail will be
longer, and thus smoother, for users.
status summary:
We started implementing this plan (#10285), set up team coordination
tools, and triaged what is a blocker for the first stage, from what is
not.
* what was done
* what is the outcome (how it makes Tails better)
* what was not done, and why
Then we worked on the "Torbirdy uses Arabic as a default locale"
bug, submitted a pull request upstream, that was accepted (#9821).
- A.1.4. Provide a migration path for our users from Claws Mail to Icedove
We decided how long to keep Claws Mails once we have Icedove
(#10010), and initiated work on the migration for users of Tails
persistence feature (#9498).
# B. Improve our quality assurance process
## B.2. Continuously run our entire test suite on all those ISO images once they are built
- B.2.1. Adjust our infrastructure to run tests in parallel
Great progress was made on this front, and more specifically on the
last remaining chunk of it: firing up a clean test runner virtual
machine before each test suite run. It was tricky to implement this in
a way that prevented race conditions, but we now have a working
prototype that we are confident fixes the issues we have seen earlier
(#9486, #10215). Along the way we encouraged a few Debian developers
to take care of a package we rely on (`jenkins-job-builder`), and one
of them promptly took it over and updated it to the version we need
(#9646).
- B.2.2. Decide what kind of ISO images qualify for testing and when,
how to process, advertise, and store the results (#8667)
Early in September, we reached an agreement on all discussion topics
that were left pending in August, such as how to archive videos from
test suite runs. As can be seen in the "Help Jenkins integration"
section below, our test suite team promptly started adjusting their
code to match what we will need on the Jenkins deployment.
<https://tails.boum.org/blueprint/automated_builds_and_tests/automated_tests_specs/>
- B.2.3. Research and design a solution to generate a set of Jenkins ISO test jobs
Some research and experiments were done for sending ISO images that
shall be tested to the test suite runner virtual machines (#9597), and
major blockers were removed in the underlying infrastructure. Some of
our Puppet code saw a nice refactoring in this process. On the same
topic, we reached a consensus regarding what "old" ISO image to use
for tests that require two images, such as upgrade tests (#10117). And
here as well, test suite developers promptly implemented what we
needed (#10147).
<https://tails.boum.org/blueprint/automated_builds_and_tests/jenkins/>
## B.3 Extend the coverage of our test suite
### General improvements
- Filesystem shares are incompatible with QEMU snapshots: #5571
We have come up with a short-term strategy that will work well with
our current workflow. In addition we have our eyes on a long-term
technical solution that will require adding a small feature into
upstream QEMU. This is, however, out of the scope of this
deliverable.
### Help Jenkins integration
These changes, on the test suite side, were prompted by the ongoing
work on "B.2. Continuously run our entire test suite on all those ISO
images once they are built".
It should be noted that many of the things mentioned here also greatly
assist developers when debugging the automated test suite.
- Leverage Cucumber's formatter system for debug logging: #9491
This makes it easier to have clean console logging while still
keeping the full debug log in a separate file. Consequently it will
be easier to get an overview of how a test is currently running.
- Capture individual videos for failed scenarios only: #10148
This will both make these video artifacts more manageable and useful
for the developers (more focused, better granularity), and will save
a lot of disk space on our servers by excluding videos of tests that
succeeded and hence aren't very interesting.
- Make the old Tails ISO default to the "new" ISO: #10147
This compromise will test 90% of what we want to test, and simplify
the Jenkins setup by eliminating the need to share multiple ISO
artifacts to the test suite context.
### B.3.6. Fix newly identified issues to make our test suite more robust and faster
#### Performance improvements
- Snapshot improvements: #6094, #8008
The proof-of-concept that was written as part of B.3.4 has matured
into a reliable implementation and
it is basically done; only fine-tuning and style improvements
remain. It is expected to be ready for the Jenkins deployment in
Milestone III, and will allow us to run 33% more tests. It will add
less overhead for new tests using persistence in the future, and
thus complete "B.3.9. Optimize tests that need a persistent volume"
three months in advance.
- Use the more efficient x264 encoding when capturing videos: #10001
This will reduce the CPU load on the host running the automated test
suite, as well as reduce its runtime with a few percent.
- Optimize IRC test using waitAny: #9653
In case there are connection issues, this may save several minutes
per instance by waiting for both the failure and success condition
in parallel, instead of serially.
#### Robustness improvements
Some of what follows was part of a project we have with
another sponsor.
- Avoid nested FindFailed exceptions in waitAny()/findAny(): #9633
This works around a race condition due to a bug in Rjb that made
these helpers fail with some probability depending on the host
hardware.
- Import logging module in otr-bot.py: #9375
Without this fix, the bot may occasionally fail due to it wanting to
use the logging facility when it is not in place.
- Force new Tor circuit and reload web site on browser
timeouts: #10116
Given the inherent instability of Tor circuits, this will
drastically improve the robustness of all Tor Browser tests.
- Pidgin's multi-window GUI sometimes causes unexpected behaviour
(e.g. one window covering the window we want to interact with):
* Focus Pidgin's buddy list before trying to access the tools
menu: #10217
* Wait for (and focus if necessary) Pidgin's Certificate windows: #10222
- Develop a strategy for dealing with newly discovered fragile tests: #10288
By leveraging our Jenkins instance, following this strategy will
isolate individual robustness issues into individual branches while
keeping all other branches functional. Consequently it will be
easier to track and deal with future robustness issues.
- Escape regexp used to match nick in CTCP replies: #10219
Due to how we randomize the nick name for the default Pidgin
accounts, there was a 10% chance to generate one with characters
that would have a special meaning when used inside regular
expressions, causing failures.
### Writing more automated tests
- B.3.8. Automatically test that udev-watchdog is monitoring the
right device: #9890.
This was completed and merged almost four months ahead of schedule.
# C. Scale our infrastructure
## C.1. Change in depth the infrastructure of our pool of mirrors
We started working on this project, and decided to handle the
redirection on the client's side (for the record, the original plan
was to do it server-side). We quickly put together a very rough
proof-of-concept, and then moved on to update our plans for the next
steps, accordingly to our new technological choice.
The big picture is described on the corresponding blueprint:
<https://tails.boum.org/blueprint/HTTP_mirror_pool/>
- C.1.1. Specify a way of describing the pool of mirrors
We picked a serialization format (JSON) that matches our
implementation choices, and started researching what would be the
best naming scheme for mirrors, taking into account future HTTPS
hardening we have in mind, and support in various popular web
servers (#10294).
- C.1.3. Design and implement the mirrors pool administration process and tools
We settled on ikiwiki overlays for integration into our website, and
on using Git and SSH to store and convey the configuration (#8637).
- C.1.2. Write & audit the code that makes the redirection decision
We did some prototyping work (#8639), and then started refactoring
it so that the code can be reused by other components that will need
to implement the same redirection scheme client-side (#10284).
## C.2. Be able to detect within hours failures and malfunction on our services
This deliverable is technically due for January 15, but we kept on
working on it.
- C.2.1. Research and decide what monitoring solution to use: #8645
We completed experiments and comparisons between monitoring systems,
and settled on Icinga 2. We started looking for solutions regarding
the single requirement of ours that it does not satisfy.
<https://tails.boum.org/blueprint/monitor_servers/>
- C.2.2. Set up the monitoring software and the underlying
infrastructure: #8646, #8647
We found hosting for our monitoring setup, got access to the
machine, and installed an operating system on it.
## C.4. Maintain our already existing services
This covers "C.4.3. Administer our services upto milestone III" until
the end of September.
Aside of the usual security updates and taking care of daily requests
coming from the Tails development community, we did some resources
planning, and updated the system requirements for the VM that will be
used as a failover for our critical services (#10243) and looked for
hosting that would meet our needs (#10244). We have an initial
agreement with a hosting organization, and will follow-up on
this shortly.
# D. Migration to Debian Jessie
## D.1. Adjust to the change of desktop environment to GNOME Shell
- D.1.1. Adjust to the change of desktop environment to GNOME Shell
We completed the work started on our "Shutdown helper" applet for
Jessie (#8302): visually impaired users can now use it, and we made
sure it is integrated with our translation system.
We cleaned up the desktop Applications menu (#8505).
## D.6. Upgrade Tails-specific tools to Debian Jessie technologies
- D.6.1. Port Tails-specific tools from udisks 1 to udisks 2
We followed up on the persistent volume assistant's porting to
udisks 2, and made sure it does not trigger spurious GNOME
notifications that could confuse users (#9280).
- D.6.3. Port WhisperBack, our integrated bug reporting tool, to Python 3
Native SOCKS support was completed, which was the only missing piece
to make WhisperBack work great on Jessie and Python 3 (#9412).
## Additional improvements that were not planned
- When starting Tails in a virtual machine that runs with non-free
technology (and does not hide this fact), users are now warned about
the risks (#5315).
- Simplify printers administration: it can now be done without having
to set an administration password, just like it was back when Tails
was based on Debian Squeeze (#8443). This removes a usability
pain-point, namely the need to restart Tails when one realizes too
late they need to print a document, and should have set an
administration password. In passing, we noticed that AppArmor
blocked adding a printer on Jessie, and fixed it (#10210).
# E. Release management
- Tails 1.6 was released on 2015-09-22 [1]:
* Upgrade Tor Browser to version 5.0.3 (based on Firefox 38.3.0 ESR).
* Upgrade I2P to version 0.9.22 and enable its AppArmor profile.
* Fix several issues related to MAC address spoofing:
- If MAC address spoofing fails on a network interface and this
interface cannot be disabled, then all networking is now
completely disabled.
- A notification is displayed if MAC address spoofing causes
network issues, for example if a network only allows
connections from a list of authorized MAC addresses.
[1] <https://tails.boum.org/news/version_1.6/>
......@@ -15,7 +15,7 @@ Everything in this report can be made public.
Note: the numbers preceded with a `#` correspond to tickets in our bug
tracker which contains more technical details and timeline. For example,
ticket #6938 can been seed on https://labs.riseup.net/code/issues/6938.
ticket #6938 can be seen on <https://labs.riseup.net/code/issues/6938>.
# A. Replace Claws Mail with Icedove
......
......@@ -15,7 +15,7 @@ Everything in this report can be made public.
Note: the numbers preceded with a `#` correspond to tickets in our bug
tracker which contains more technical details and timeline. For example,
ticket #6938 can been seed on https://labs.riseup.net/code/issues/6938.
ticket #6938 can be seen on <https://labs.riseup.net/code/issues/6938>.
# A. Replace Claws Mail with Icedove
......
......@@ -15,7 +15,7 @@ Everything in this report can be made public.
Note: the numbers preceded with a `#` correspond to tickets in our bug
tracker which contains more technical details and timeline. For example,
ticket #6938 can been seed on https://labs.riseup.net/code/issues/6938.
ticket #6938 can be seen on <https://labs.riseup.net/code/issues/6938>.
# A. Replace Claws Mail with Icedove
......
......@@ -15,7 +15,7 @@ Everything in this report can be made public.
Note: the numbers preceded with a `#` correspond to tickets in our bug
tracker which contains more technical details and timeline. For example,
ticket #6938 can been seed on https://labs.riseup.net/code/issues/6938.
ticket #6938 can be seen on <https://labs.riseup.net/code/issues/6938>.
# A. Replace Claws Mail with Icedove
......
......@@ -15,7 +15,7 @@ Everything in this report can be made public.
Note: the numbers preceded with a `#` correspond to tickets in our bug
tracker which contains more technical details and timeline. For example,
ticket #6938 can been seed on https://labs.riseup.net/code/issues/6938.
ticket #6938 can be seen on <https://labs.riseup.net/code/issues/6938>.
# A. Replace Claws Mail with Icedove
......
......@@ -15,7 +15,7 @@ Everything in this report can be made public.
Note: the numbers preceded with a `#` correspond to tickets in our bug
tracker which contains more technical details and timeline. For example,
ticket #6938 can been seed on https://labs.riseup.net/code/issues/6938.
ticket #6938 can be seen on <https://labs.riseup.net/code/issues/6938>.
# A. Replace Claws Mail with Icedove
......
......@@ -15,7 +15,7 @@ Everything in this report can be made public.
Note: the numbers preceded with a `#` correspond to tickets in our bug
tracker which contains more technical details and timeline. For example,
ticket #6938 can been seed on https://labs.riseup.net/code/issues/6938.
ticket #6938 can be seen on <https://labs.riseup.net/code/issues/6938>.
# A. Replace Claws Mail with Icedove
......
......@@ -15,7 +15,7 @@ Everything in this report can be made public.
Note: the numbers preceded with a `#` correspond to tickets in our bug
tracker which contains more technical details and timeline. For example,
ticket #6938 can been seed on https://labs.riseup.net/code/issues/6938.
ticket #6938 can be seen on <https://labs.riseup.net/code/issues/6938>.
# A. Replace Claws Mail with Icedove
......
......@@ -15,7 +15,7 @@ Everything in this report can be made public.
Note: the numbers preceded with a `#` correspond to tickets in our bug
tracker which contains more technical details and timeline. For example,
ticket #6938 can been seed on https://labs.riseup.net/code/issues/6938.
ticket #6938 can be seen on <https://labs.riseup.net/code/issues/6938>.
# A. Replace Claws Mail with Icedove
......
......@@ -15,7 +15,7 @@ Everything in this report can be made public.
Note: the numbers preceded with a `#` correspond to tickets in our bug
tracker which contains more technical details and timeline. For example,
ticket #6938 can been seed on https://labs.riseup.net/code/issues/6938.
ticket #6938 can be seen on <https://labs.riseup.net/code/issues/6938>.
# A. Replace Claws Mail with Icedove
......
......@@ -128,26 +128,19 @@ The test suite produces different kind of artifacts: logfiles, screen
captures for failing steps, snapshots of the test VM, and also videos of
the running test session.
Videos may be a bit too much to keep, given they slow down the test
suite and might take quite a bit of disk space to store. If we want to
keep them, we may want to do so only for failing test suite runs. If we
decide to still use them, then we probably have to wait for
[[!tails_ticket 10001]] too be resolved.
We can keep the video captures in the build artifacts, now that
[[!tails_ticket 10001]] is resolved.
Proposal for a first iteration:
Decision:
* For green test suite run: keep the test logs (Jenkins natively do
that).
* For red test suite run: keep the screen and video captures, the
* For red test suite run: keep the screenshots and video captures, the
logs and the pcap files.
On the second iteration, we will keep video capture only for the red
tests.
The retention strategy should be the same than for the automatically
built ISOs. In particular, we will have to pay attention to the rotation
of videos capture (given they'll quickly bloat our storage space).
Keeping them only for 7 days sounds reasonnable.
In [[!tails_ticket 10155]] we calculated that we can probably keep the
video captures for a full release cycle. This will be refine is reality
claims the contrary after an evaluation.
# Scenarios
......
[[!meta title="Jenkins"]]
[[!meta title="Automated tests implementation details"]]
For Jenkins resources, see [[blueprint/automated_builds_and_tests/resources]].
[[!toc levels=2]]
Resources
=========
Miscellaneous
-------------
- [Jenkins Best
Practices](https://wiki.jenkins-ci.org/display/JENKINS/Jenkins+Best+Practices)
- [plugins](https://wiki.jenkins-ci.org/display/JENKINS/Plugins)
* [Git plugin](https://wiki.jenkins-ci.org/display/JENKINS/Git+Plugin)
* [Copy Artifact
plugin](https://wiki.jenkins-ci.org/display/JENKINS/Copy+Artifact+Plugin)
can be used to run a test job against the result of a build job,
e.g. for Debian packages (think Lintian) or Tails ISO images; see
[grml's setup
documentation](http://jenkins-debian-glue.org/getting_started/manual/)
that uses it.
- the [jenkins](http://jujucharms.com/charms/precise/jenkins) and
[jenkins-slave](http://jujucharms.com/charms/precise/jenkins-slave)
JuJu charms may be good sources of inspiration for deployment
- [[!cpan Net-Jenkins]] (not in Debian) allows to interact with
a Jenkins server: create and start jobs, get information about
builds etc.
Jobs management
---------------
- [Job builder](http://ci.openstack.org/jenkins-job-builder/) provides
one-way (Git to Jenkins) jobs synchronization; it's in Debian sid.
* [configuration documentation](http://ci.openstack.org/jenkins-job-builder/configuration.html)
* Debian uses it in their `update_jdn.sh`: it runs `jenkins-jobs
update $config` after importing updated YAML job config files
from Git.
* Tor [use
it](https://gitweb.torproject.org/project/jenkins/jobs.git/tree) too.
- jenkins.debian.net uses the [SCM
Sync](https://wiki.jenkins-ci.org/display/JENKINS/SCM+Sync+configuration+plugin)
plugin, that apparently handles committing to the VCS on
configuration changes done in the web interface, and maybe more.
- [jenkins-yaml](https://github.com/varnish/jenkins-yaml) might make
it easy to generate a large number of similar Jenkins jobs, e.g.
one per branch
- [jenkins_jobs puppet module](http://tradeshift.com/blog/tstech-managing-jenkins-job-configurations-by-puppet/)
Web setup
---------
### Visible read-only on the web
We'd like our Jenkins instance to be visible read-only on the web.
We'd rather not rely on Jenkins authentication / authorization to
enforce this read-only policy. We'd rather see the frontend reverse
proxy take care of this.
The
[`getUnprotectedRootActions()`](http://javadoc.jenkins-ci.org/jenkins/model/Jenkins.html#getUnprotectedRootActions())
method should return the list of URL prefixes that we want to allow.
And we could forbid anything else.
The [Reverse Proxy
Auth](https://wiki.jenkins-ci.org/display/JENKINS/Reverse+Proxy+Auth+Plugin)
Jenkins plugin can be useful to display [an example
usage](https://github.com/jenkinsci/reverse-proxy-auth-plugin/commit/72567a974960be2363107614ba3f705ec6e9b695)
of this method.
### Miscellaneous
- [sample nginx configuration](https://wiki.jenkins-ci.org/display/JENKINS/Installing+Jenkins+on+Ubuntu)
Notifications
-------------
- [IRC plugin](https://wiki.jenkins-ci.org/display/JENKINS/IRC+Plugin),
but I'm told that the jenkins email notifications are way nicer
than what this plugin can do, so see [a better way to do
it](http://jenkins.debian.net/userContent/setup.html#_installing_kgb_client)
- [[!cpan Jenkins-NotificationListener]] is a server that listens to
messages from Jenkins [Notification
plugin](https://wiki.jenkins-ci.org/display/JENKINS/Notification+Plugin).
### Notifying different people depending on what triggered the build
At least the obvious candidate (Email-ext plugin) doesn't seem able to
email different recipients depending on what triggered the build
out-of-the-box. But apparently, one can set up two 'Script - After
Build' email triggers in the Email-ext configuration: one emails the
culprit, the other emails the RM. And then, they do something or not