Commit 175f21a5 authored by anonym's avatar anonym
Browse files

Merge branch 'devel' into test/6302-mac-spoofing

parents 0590e6e9 4c64182b
......@@ -128,7 +128,7 @@ if [ -n "$JENKINS_URL" ] && [ -z "$GIT_TAG" ] \
fi
# build the doc wiki
./build-wiki
./build-website
# refresh translations of our programs
./refresh-translations || fatal "refresh-translations failed ($?)."
......
......@@ -66,6 +66,10 @@ Package: poedit
Pin: release o=Debian Backports,n=wheezy-backports
Pin-Priority: 999
Package: firmware-amd-graphics
Pin: release o=Debian,a=unstable
Pin-Priority: 999
Package: firmware-atheros
Pin: release o=Debian,a=unstable
Pin-Priority: 999
......@@ -106,7 +110,7 @@ Package: firmware-linux-nonfree
Pin: release o=Debian,a=unstable
Pin-Priority: 999
Package: firmware-ralink
Package: firmware-misc-nonfree
Pin: release o=Debian,a=unstable
Pin-Priority: 999
......
......@@ -250,6 +250,7 @@ xserver-xorg-input-vmmouse
#endif
### Firmwares
firmware-amd-graphics
firmware-atheros
firmware-brcm80211
firmware-ipw2x00
......@@ -258,7 +259,7 @@ firmware-libertas
firmware-linux
firmware-linux-free
firmware-linux-nonfree
firmware-ralink
firmware-misc-nonfree
firmware-realtek
firmware-zd1211
b43-fwcutter
......
......@@ -21,6 +21,7 @@ Feature: Various checks
When Tails has booted a 64-bit kernel
Then the VirtualBox guest modules are available
@fragile
Scenario: The shipped Tails OpenPGP keys are up-to-date
Given I have started Tails from DVD without network and logged in
Then the OpenPGP keys shipped with Tails will be valid for the next 3 months
......
......@@ -51,7 +51,7 @@ Feature: Chatting anonymously using Pidgin
When I say something to my friend
Then I receive a response from my friend
@check_tor_leaks
@check_tor_leaks @fragile
Scenario: Connecting to the #tails IRC channel with the pre-configured account
Given I have started Tails from DVD and logged in and the network is connected
And Pidgin has the expected accounts configured with random nicknames
......
......@@ -3,6 +3,12 @@ require "#{Dir.pwd}/features/support/extra_hooks.rb"
require 'time'
require 'rspec'
# Force UTF-8. Ruby will default to the system locale, and if it is
# non-UTF-8, String-methods will fail when operating on non-ASCII
# strings.
Encoding.default_external = Encoding::UTF_8
Encoding.default_internal = Encoding::UTF_8
def fatal_system(str)
unless system(str)
raise StandardError.new("Command exited with #{$?}")
......
......@@ -16,6 +16,7 @@ Feature: The Tor enforcement is effective
And the firewall's NAT rules only redirect traffic for Tor's TransPort and DNSPort
And the firewall is configured to block all IPv6 traffic
@fragile
Scenario: Anti test: Detecting IPv4 TCP leaks from the Unsafe Browser with the firewall leak detector
Given I have started Tails from DVD and logged in and the network is connected
And I capture all network traffic
......
......@@ -27,6 +27,7 @@ Feature: Tor stream isolation is effective
And the Tor Browser has started and loaded the startup page
Then I see that Tor Browser is properly stream isolated
@fragile
Scenario: Gobby is using the default SocksPort
When I monitor the network connections of Gobby
And I start "Gobby" via the GNOME "Internet" applications menu
......@@ -45,12 +46,14 @@ Feature: Tor stream isolation is effective
And the whois command is successful
Then I see that whois is properly stream isolated
@fragile
Scenario: Explicitly torify-wrapped applications are using the default SocksPort
When I monitor the network connections of Gobby
And I run "torify /usr/bin/gobby-0.5" in GNOME Terminal
And I connect Gobby to "gobby.debian.org"
Then I see that Gobby is properly stream isolated
@fragile
Scenario: Explicitly torsocks-wrapped applications are using the default SocksPort
When I monitor the network connections of Gobby
And I run "torsocks /usr/bin/gobby-0.5" in GNOME Terminal
......
@product
@product @fragile
Feature: Browsing the web using the Tor Browser
As a Tails user
when I browse the web using the Tor Browser
......@@ -25,7 +25,7 @@ Feature: Browsing the web using the Tor Browser
Then I can save the current page as "index.html" to the default downloads directory
And I can print the current page as "output.pdf" to the default downloads directory
@check_tor_leaks
@check_tor_leaks @fragile
Scenario: Importing an OpenPGP key from a website
Given I have started Tails from DVD and logged in and the network is connected
When I start the Tor Browser
......@@ -35,7 +35,7 @@ Feature: Browsing the web using the Tor Browser
When I accept to import the key with Seahorse
Then I see "KeyImportedNotification.png" after at most 10 seconds
@check_tor_leaks
@check_tor_leaks @fragile
Scenario: Playing HTML5 audio
Given I have started Tails from DVD and logged in and the network is connected
When I start the Tor Browser
......@@ -45,7 +45,7 @@ Feature: Browsing the web using the Tor Browser
And I click the HTML5 play button
And 1 application is playing audio after 10 seconds
@check_tor_leaks
@check_tor_leaks @fragile
Scenario: Watching a WebM video
Given I have started Tails from DVD and logged in and the network is connected
When I start the Tor Browser
......@@ -106,7 +106,7 @@ Feature: Browsing the web using the Tor Browser
And the Tor Browser has started
Then the Tor Browser uses all expected TBB shared libraries
@check_tor_leaks
@check_tor_leaks @fragile
Scenario: Opening check.torproject.org in the Tor Browser shows the green onion and the congratulations message
Given I have started Tails from DVD and logged in and the network is connected
When I start the Tor Browser
......@@ -114,7 +114,7 @@ Feature: Browsing the web using the Tor Browser
And I open the address "https://check.torproject.org" in the Tor Browser
Then I see "TorBrowserTorCheck.png" after at most 180 seconds
@check_tor_leaks
@check_tor_leaks @fragile
Scenario: The Tor Browser's "New identity" feature works as expected
Given I have started Tails from DVD and logged in and the network is connected
When I start the Tor Browser
......
......@@ -36,6 +36,7 @@ Feature: Browsing the web using the Unsafe Browser
And I start the Unsafe Browser
Then I see a warning about another instance already running
@fragile
Scenario: Opening check.torproject.org in the Unsafe Browser shows the red onion and a warning message.
Given I have started Tails from DVD and logged in and the network is connected
When I successfully start the Unsafe Browser
......
pythonlib @ ee7d6a33
Subproject commit 4cea598050b217267e1489413aaf1e416abc25ee
Subproject commit ee7d6a337ba6df1fa32c42766192525b3f37d36d
......@@ -83,7 +83,7 @@ if [ "$TAILS_CLEAN_BUILD" ]; then
fi
if [ -z "$JENKINS_URL" ]; then
./build-wiki
./build-website
fi
BUILD_DIR=$(mktemp -d /tmp/tails-build.XXXXXXXX)
......
**Ticket**: [[!tails_ticket 7161]]
The idea I had was to let the server(s) send a reduced list of hosts. Not
only it would allow to work-around Tor DNS limitations, but also to have
some weighted round robin, in order to prioritize some high bandwidth
mirrors, if we choose to.
If I had to mention the ideal design goals for such changes, I would say
that the more straightforward would be the better for implementation and
also for maintainability.
[[!toc levels=3]]
# The plan
......@@ -42,6 +33,15 @@ We decided to implement a two-way strategy for this feature:
# Initial research
The idea I had was to let the server(s) send a reduced list of hosts. Not
only it would allow to work-around Tor DNS limitations, but also to have
some weighted round robin, in order to prioritize some high bandwidth
mirrors, if we choose to.
If I had to mention the ideal design goals for such changes, I would say
that the more straightforward would be the better for implementation and
also for maintainability.
## Using DNS
Using DNS seems to be an easy way to do some round robin in low level. It
......
......@@ -14,28 +14,308 @@ This reports covers the activity of Tails in September 2015.
Everything in this report can be made public.
Note: the numbers preceded with a `#` correspond to tickets in our bug
tracker which contains more technical details and timeline. For example,
ticket #6938 can been seed on https://labs.riseup.net/code/issues/6938.
tracker which contain more technical details and timeline. For example,
ticket #6938 can be seen on <https://labs.riseup.net/code/issues/6938>.
# A. Replace Claws Mail with Icedove
## A.n. description of subsection
- A.1.3. Integrate Icedove into Tails
- A.n.m. description of deliverable: ticket numbers
We amended the strategy we had in mind initially and made it more
incremental. If time allows, a first stage will be to include
Icedove in Tails 1.7 (2015-11-03), but without the Icedove account
setup wizard. During this first stage, Torbirdy's own account setup
wizard will be used. And the second stage will be about securing
Icedove's wizard (#6154). If this works out as we hope, Tails users
will be able to start using Icedove two months earlier than what we
planned initially, and the transition period from Claws Mail will be
longer, and thus smoother, for users.
status summary:
We started implementing this plan (#10285), set up team coordination
tools, and triaged what is a blocker for the first stage, from what is
not.
* what was done
* what is the outcome (how it makes Tails better)
* what was not done, and why
Then we worked on the "Torbirdy uses Arabic as a default locale"
bug, submitted a pull request upstream, that was accepted (#9821).
- A.1.4. Provide a migration path for our users from Claws Mail to Icedove
We decided how long to keep Claws Mails once we have Icedove
(#10010), and initiated work on the migration for users of Tails
persistence feature (#9498).
# B. Improve our quality assurance process
## B.2. Continuously run our entire test suite on all those ISO images once they are built
- B.2.1. Adjust our infrastructure to run tests in parallel
Great progress was made on this front, and more specifically on the
last remaining chunk of it: firing up a clean test runner virtual
machine before each test suite run. It was tricky to implement this in
a way that prevented race conditions, but we now have a working
prototype that we are confident fixes the issues we have seen earlier
(#9486, #10215). Along the way we encouraged a few Debian developers
to take care of a package we rely on (`jenkins-job-builder`), and one
of them promptly took it over and updated it to the version we need
(#9646).
- B.2.2. Decide what kind of ISO images qualify for testing and when,
how to process, advertise, and store the results (#8667)
Early in September, we reached an agreement on all discussion topics
that were left pending in August, such as how to archive videos from
test suite runs. As can be seen in the "Help Jenkins integration"
section below, our test suite team promptly started adjusting their
code to match what we will need on the Jenkins deployment.
<https://tails.boum.org/blueprint/automated_builds_and_tests/automated_tests_specs/>
- B.2.3. Research and design a solution to generate a set of Jenkins ISO test jobs
Some research and experiments were done for sending ISO images that
shall be tested to the test suite runner virtual machines (#9597), and
major blockers were removed in the underlying infrastructure. Some of
our Puppet code saw a nice refactoring in this process. On the same
topic, we reached a consensus regarding what "old" ISO image to use
for tests that require two images, such as upgrade tests (#10117). And
here as well, test suite developers promptly implemented what we
needed (#10147).
<https://tails.boum.org/blueprint/automated_builds_and_tests/jenkins/>
## B.3 Extend the coverage of our test suite
### General improvements
- Filesystem shares are incompatible with QEMU snapshots: #5571
We have come up with a short-term strategy that will work well with
our current workflow. In addition we have our eyes on a long-term
technical solution that will require adding a small feature into
upstream QEMU. This is, however, out of the scope of this
deliverable.
### Help Jenkins integration
These changes, on the test suite side, were prompted by the ongoing
work on "B.2. Continuously run our entire test suite on all those ISO
images once they are built".
It should be noted that many of the things mentioned here also greatly
assist developers when debugging the automated test suite.
- Leverage Cucumber's formatter system for debug logging: #9491
This makes it easier to have clean console logging while still
keeping the full debug log in a separate file. Consequently it will
be easier to get an overview of how a test is currently running.
- Capture individual videos for failed scenarios only: #10148
This will both make these video artifacts more manageable and useful
for the developers (more focused, better granularity), and will save
a lot of disk space on our servers by excluding videos of tests that
succeeded and hence aren't very interesting.
- Make the old Tails ISO default to the "new" ISO: #10147
This compromise will test 90% of what we want to test, and simplify
the Jenkins setup by eliminating the need to share multiple ISO
artifacts to the test suite context.
### B.3.6. Fix newly identified issues to make our test suite more robust and faster
#### Performance improvements
- Snapshot improvements: #6094, #8008
The proof-of-concept that was written as part of B.3.4 has matured
into a reliable implementation and
it is basically done; only fine-tuning and style improvements
remain. It is expected to be ready for the Jenkins deployment in
Milestone III, and will allow us to run 33% more tests. It will add
less overhead for new tests using persistence in the future, and
thus complete "B.3.9. Optimize tests that need a persistent volume"
three months in advance.
- Use the more efficient x264 encoding when capturing videos: #10001
This will reduce the CPU load on the host running the automated test
suite, as well as reduce its runtime with a few percent.
- Optimize IRC test using waitAny: #9653
In case there are connection issues, this may save several minutes
per instance by waiting for both the failure and success condition
in parallel, instead of serially.
#### Robustness improvements
Some of what follows was part of a project we have with
another sponsor.
- Avoid nested FindFailed exceptions in waitAny()/findAny(): #9633
This works around a race condition due to a bug in Rjb that made
these helpers fail with some probability depending on the host
hardware.
- Import logging module in otr-bot.py: #9375
Without this fix, the bot may occasionally fail due to it wanting to
use the logging facility when it is not in place.
- Force new Tor circuit and reload web site on browser
timeouts: #10116
Given the inherent instability of Tor circuits, this will
drastically improve the robustness of all Tor Browser tests.
- Pidgin's multi-window GUI sometimes causes unexpected behaviour
(e.g. one window covering the window we want to interact with):
* Focus Pidgin's buddy list before trying to access the tools
menu: #10217
* Wait for (and focus if necessary) Pidgin's Certificate windows: #10222
- Develop a strategy for dealing with newly discovered fragile tests: #10288
By leveraging our Jenkins instance, following this strategy will
isolate individual robustness issues into individual branches while
keeping all other branches functional. Consequently it will be
easier to track and deal with future robustness issues.
- Escape regexp used to match nick in CTCP replies: #10219
Due to how we randomize the nick name for the default Pidgin
accounts, there was a 10% chance to generate one with characters
that would have a special meaning when used inside regular
expressions, causing failures.
### Writing more automated tests
- B.3.8. Automatically test that udev-watchdog is monitoring the
right device: #9890.
This was completed and merged almost four months ahead of schedule.
# C. Scale our infrastructure
## C.1. Change in depth the infrastructure of our pool of mirrors
We started working on this project, and decided to handle the
redirection on the client's side (for the record, the original plan
was to do it server-side). We quickly put together a very rough
proof-of-concept, and then moved on to update our plans for the next
steps, accordingly to our new technological choice.
The big picture is described on the corresponding blueprint:
<https://tails.boum.org/blueprint/HTTP_mirror_pool/>
- C.1.1. Specify a way of describing the pool of mirrors
We picked a serialization format (JSON) that matches our
implementation choices, and started researching what would be the
best naming scheme for mirrors, taking into account future HTTPS
hardening we have in mind, and support in various popular web
servers (#10294).
- C.1.3. Design and implement the mirrors pool administration process and tools
We settled on ikiwiki overlays for integration into our website, and
on using Git and SSH to store and convey the configuration (#8637).
- C.1.2. Write & audit the code that makes the redirection decision
We did some prototyping work (#8639), and then started refactoring
it so that the code can be reused by other components that will need
to implement the same redirection scheme client-side (#10284).
## C.2. Be able to detect within hours failures and malfunction on our services
This deliverable is technically due for January 15, but we kept on
working on it.
- C.2.1. Research and decide what monitoring solution to use: #8645
We completed experiments and comparisons between monitoring systems,
and settled on Icinga 2. We started looking for solutions regarding
the single requirement of ours that it does not satisfy.
<https://tails.boum.org/blueprint/monitor_servers/>
- C.2.2. Set up the monitoring software and the underlying
infrastructure: #8646, #8647
We found hosting for our monitoring setup, got access to the
machine, and installed an operating system on it.
## C.4. Maintain our already existing services
This covers "C.4.3. Administer our services upto milestone III" until
the end of September.
Aside of the usual security updates and taking care of daily requests
coming from the Tails development community, we did some resources
planning, and updated the system requirements for the VM that will be
used as a failover for our critical services (#10243) and looked for
hosting that would meet our needs (#10244). We have an initial
agreement with a hosting organization, and will follow-up on
this shortly.
# D. Migration to Debian Jessie
## D.1. Adjust to the change of desktop environment to GNOME Shell
- D.1.1. Adjust to the change of desktop environment to GNOME Shell
We completed the work started on our "Shutdown helper" applet for
Jessie (#8302): visually impaired users can now use it, and we made
sure it is integrated with our translation system.
We cleaned up the desktop Applications menu (#8505).
## D.6. Upgrade Tails-specific tools to Debian Jessie technologies
- D.6.1. Port Tails-specific tools from udisks 1 to udisks 2
We followed up on the persistent volume assistant's porting to
udisks 2, and made sure it does not trigger spurious GNOME
notifications that could confuse users (#9280).
- D.6.3. Port WhisperBack, our integrated bug reporting tool, to Python 3
Native SOCKS support was completed, which was the only missing piece
to make WhisperBack work great on Jessie and Python 3 (#9412).
## Additional improvements that were not planned
- When starting Tails in a virtual machine that runs with non-free
technology (and does not hide this fact), users are now warned about
the risks (#5315).
- Simplify printers administration: it can now be done without having
to set an administration password, just like it was back when Tails
was based on Debian Squeeze (#8443). This removes a usability
pain-point, namely the need to restart Tails when one realizes too
late they need to print a document, and should have set an
administration password. In passing, we noticed that AppArmor
blocked adding a printer on Jessie, and fixed it (#10210).
# E. Release management
- Tails 1.6 was released on 2015-09-22 [1]:
* Upgrade Tor Browser to version 5.0.3 (based on Firefox 38.3.0 ESR).
* Upgrade I2P to version 0.9.22 and enable its AppArmor profile.
* Fix several issues related to MAC address spoofing:
- If MAC address spoofing fails on a network interface and this
interface cannot be disabled, then all networking is now
completely disabled.
- A notification is displayed if MAC address spoofing causes
network issues, for example if a network only allows
connections from a list of authorized MAC addresses.
[1] <https://tails.boum.org/news/version_1.6/>
......@@ -15,7 +15,7 @@ Everything in this report can be made public.
Note: the numbers preceded with a `#` correspond to tickets in our bug
tracker which contains more technical details and timeline. For example,
ticket #6938 can been seed on https://labs.riseup.net/code/issues/6938.
ticket #6938 can be seen on <https://labs.riseup.net/code/issues/6938>.
# A. Replace Claws Mail with Icedove
......
......@@ -15,7 +15,7 @@ Everything in this report can be made public.
Note: the numbers preceded with a `#` correspond to tickets in our bug
tracker which contains more technical details and timeline. For example,
ticket #6938 can been seed on https://labs.riseup.net/code/issues/6938.
ticket #6938 can be seen on <https://labs.riseup.net/code/issues/6938>.
# A. Replace Claws Mail with Icedove
......
......@@ -15,7 +15,7 @@ Everything in this report can be made public.
Note: the numbers preceded with a `#` correspond to tickets in our bug
tracker which contains more technical details and timeline. For example,
ticket #6938 can been seed on https://labs.riseup.net/code/issues/6938.
ticket #6938 can be seen on <https://labs.riseup.net/code/issues/6938>.
# A. Replace Claws Mail with Icedove
......
......@@ -15,7 +15,7 @@ Everything in this report can be made public.
Note: the numbers preceded with a `#` correspond to tickets in our bug
tracker which contains more technical details and timeline. For example,
ticket #6938 can been seed on https://labs.riseup.net/code/issues/6938.
ticket #6938 can be seen on <https://labs.riseup.net/code/issues/6938>.
# A. Replace Claws Mail with Icedove
......
......@@ -15,7 +15,7 @@ Everything in this report can be made public.
Note: the numbers preceded with a `#` correspond to tickets in our bug
tracker which contains more technical details and timeline. For example,
ticket #6938 can been seed on https://labs.riseup.net/code/issues/6938.
ticket #6938 can be seen on <https://labs.riseup.net/code/issues/6938>.
# A. Replace Claws Mail with Icedove
......
......@@ -15,7 +15,7 @@ Everything in this report can be made public.
Note: the numbers preceded with a `#` correspond to tickets in our bug
tracker which contains more technical details and timeline. For example,
ticket #6938 can been seed on https://labs.riseup.net/code/issues/6938.
ticket #6938 can be seen on <https://labs.riseup.net/code/issues/6938>.
# A. Replace Claws Mail with Icedove
......
......@@ -15,7 +15,7 @@ Everything in this report can be made public.
Note: the numbers preceded with a `#` correspond to tickets in our bug
tracker which contains more technical details and timeline. For example,
ticket #6938 can been seed on https://labs.riseup.net/code/issues/6938.
ticket #6938 can be seen on <https://labs.riseup.net/code/issues/6938>.
# A. Replace Claws Mail with Icedove
......