Commit f8e5fb99 authored by bertagaz's avatar bertagaz
Browse files

Flag online scenarios as fragile.

They all rely on the `wait_until_tor_is_working` helper which has lately
proven to be still buggy. That's tracked by ticket #10497.
parent fc28e0be
...@@ -30,7 +30,7 @@ Feature: Various checks ...@@ -30,7 +30,7 @@ Feature: Various checks
Given I have started Tails from DVD without network and logged in Given I have started Tails from DVD without network and logged in
Then the shipped Debian repository key will be valid for the next 3 months Then the shipped Debian repository key will be valid for the next 3 months
@doc @doc @fragile
Scenario: The "Report an Error" launcher will open the support documentation Scenario: The "Report an Error" launcher will open the support documentation
Given I have started Tails from DVD without network and logged in Given I have started Tails from DVD without network and logged in
And the network is plugged And the network is plugged
...@@ -45,6 +45,7 @@ Feature: Various checks ...@@ -45,6 +45,7 @@ Feature: Various checks
And the live user is a member of only its own group and "audio cdrom dialout floppy video plugdev netdev fuse scanner lp lpadmin vboxsf" And the live user is a member of only its own group and "audio cdrom dialout floppy video plugdev netdev fuse scanner lp lpadmin vboxsf"
And the live user owns its home dir and it has normal permissions And the live user owns its home dir and it has normal permissions
@fragile
Scenario: No initial network Scenario: No initial network
Given I have started Tails from DVD without network and logged in Given I have started Tails from DVD without network and logged in
And I wait between 30 and 60 seconds And I wait between 30 and 60 seconds
...@@ -54,18 +55,21 @@ Feature: Various checks ...@@ -54,18 +55,21 @@ Feature: Various checks
And the time has synced And the time has synced
And process "vidalia" is running within 30 seconds And process "vidalia" is running within 30 seconds
@fragile
Scenario: The 'Tor is ready' notification is shown when Tor has bootstrapped Scenario: The 'Tor is ready' notification is shown when Tor has bootstrapped
Given I have started Tails from DVD without network and logged in Given I have started Tails from DVD without network and logged in
And the network is plugged And the network is plugged
When I see the 'Tor is ready' notification When I see the 'Tor is ready' notification
Then Tor is ready Then Tor is ready
@fragile
Scenario: The tor process should be confined with Seccomp Scenario: The tor process should be confined with Seccomp
Given I have started Tails from DVD without network and logged in Given I have started Tails from DVD without network and logged in
And the network is plugged And the network is plugged
And Tor is ready And Tor is ready
Then the running process "tor" is confined with Seccomp in filter mode Then the running process "tor" is confined with Seccomp in filter mode
@fragile
Scenario: No unexpected network services Scenario: No unexpected network services
Given I have started Tails from DVD without network and logged in Given I have started Tails from DVD without network and logged in
When the network is plugged When the network is plugged
......
@product @product @fragile
Feature: Getting a DHCP lease without leaking too much information Feature: Getting a DHCP lease without leaking too much information
As a Tails user As a Tails user
when I connect to a network with a DHCP server when I connect to a network with a DHCP server
......
#10497: wait_until_tor_is_working
@product @check_tor_leaks @fragile @product @check_tor_leaks @fragile
Feature: Electrum Bitcoin client Feature: Electrum Bitcoin client
As a Tails user As a Tails user
......
...@@ -15,11 +15,13 @@ Feature: I2P ...@@ -15,11 +15,13 @@ Feature: I2P
And the I2P Browser sudo rules are present And the I2P Browser sudo rules are present
And the I2P firewall rules are enabled And the I2P firewall rules are enabled
@fragile
Scenario: I2P's AppArmor profile is in enforce mode Scenario: I2P's AppArmor profile is in enforce mode
Given I have started Tails from DVD with I2P enabled and logged in and the network is connected Given I have started Tails from DVD with I2P enabled and logged in and the network is connected
When I2P is running When I2P is running
Then the running process "i2p" is confined with AppArmor in enforce mode Then the running process "i2p" is confined with AppArmor in enforce mode
@fragile
Scenario: The I2P Browser works as it should Scenario: The I2P Browser works as it should
Given I have started Tails from DVD with I2P enabled and logged in and the network is connected Given I have started Tails from DVD with I2P enabled and logged in and the network is connected
And the I2P router console is ready And the I2P router console is ready
...@@ -27,6 +29,7 @@ Feature: I2P ...@@ -27,6 +29,7 @@ Feature: I2P
Then the I2P router console is displayed in I2P Browser Then the I2P router console is displayed in I2P Browser
And the I2P Browser uses all expected TBB shared libraries And the I2P Browser uses all expected TBB shared libraries
@fragile
Scenario: Closing the I2P Browser shows a stop notification and properly tears down the chroot. Scenario: Closing the I2P Browser shows a stop notification and properly tears down the chroot.
Given I have started Tails from DVD with I2P enabled and logged in and the network is connected Given I have started Tails from DVD with I2P enabled and logged in and the network is connected
And the I2P router console is ready And the I2P router console is ready
...@@ -35,6 +38,7 @@ Feature: I2P ...@@ -35,6 +38,7 @@ Feature: I2P
Then I see the I2P Browser stop notification Then I see the I2P Browser stop notification
And the I2P Browser chroot is torn down And the I2P Browser chroot is torn down
@fragile
Scenario: The I2P internal websites can be viewed in I2P Browser Scenario: The I2P internal websites can be viewed in I2P Browser
Given I have started Tails from DVD with I2P enabled and logged in and the network is connected Given I have started Tails from DVD with I2P enabled and logged in and the network is connected
And the I2P router console is ready And the I2P router console is ready
...@@ -44,7 +48,8 @@ Feature: I2P ...@@ -44,7 +48,8 @@ Feature: I2P
When I open the address "http://i2p-projekt.i2p" in the I2P Browser When I open the address "http://i2p-projekt.i2p" in the I2P Browser
Then the I2P homepage loads in I2P Browser Then the I2P homepage loads in I2P Browser
Scenario: I2P is configured to run in Hidden mode @fragile
Scenario: I2P is configured to run in Hidden mode
Given I have started Tails from DVD with I2P enabled and logged in and the network is connected Given I have started Tails from DVD with I2P enabled and logged in and the network is connected
And the I2P router console is ready And the I2P router console is ready
When I start the I2P Browser through the GNOME menu When I start the I2P Browser through the GNOME menu
...@@ -63,6 +68,7 @@ Feature: I2P ...@@ -63,6 +68,7 @@ Feature: I2P
Then Pidgin successfully connects to the "I2P" account Then Pidgin successfully connects to the "I2P" account
And I can join the "#i2p" channel on "I2P" And I can join the "#i2p" channel on "I2P"
@fragile
Scenario: I2P displays a notice when bootstrapping fails Scenario: I2P displays a notice when bootstrapping fails
Given I have started Tails from DVD with I2P enabled and logged in Given I have started Tails from DVD with I2P enabled and logged in
And I2P is not running And I2P is not running
...@@ -77,6 +83,7 @@ Feature: I2P ...@@ -77,6 +83,7 @@ Feature: I2P
When I start the I2P Browser through the GNOME menu When I start the I2P Browser through the GNOME menu
Then the I2P router console is displayed in I2P Browser Then the I2P router console is displayed in I2P Browser
@fragile
Scenario: I2P displays a notice when it fails to start Scenario: I2P displays a notice when it fails to start
Given I have started Tails from DVD with I2P enabled and logged in Given I have started Tails from DVD with I2P enabled and logged in
And I2P is not running And I2P is not running
......
@product @check_tor_leaks @product @check_tor_leaks @fragile
Feature: Icedove email client Feature: Icedove email client
As a Tails user As a Tails user
I may want to use an email client I may want to use an email client
......
@product @product @fragile
Feature: Localization Feature: Localization
As a Tails user As a Tails user
I want Tails to be localized in my native language I want Tails to be localized in my native language
......
...@@ -10,6 +10,7 @@ Feature: Spoofing MAC addresses ...@@ -10,6 +10,7 @@ Feature: Spoofing MAC addresses
And I capture all network traffic And I capture all network traffic
And the network is plugged And the network is plugged
@fragile
Scenario: MAC address spoofing is disabled Scenario: MAC address spoofing is disabled
When I enable more Tails Greeter options When I enable more Tails Greeter options
And I disable MAC spoofing in Tails Greeter And I disable MAC spoofing in Tails Greeter
...@@ -20,6 +21,7 @@ Feature: Spoofing MAC addresses ...@@ -20,6 +21,7 @@ Feature: Spoofing MAC addresses
And the network device has its default MAC address configured And the network device has its default MAC address configured
And the real MAC address was leaked And the real MAC address was leaked
@fragile
Scenario: MAC address spoofing is successful Scenario: MAC address spoofing is successful
When I log in to a new session When I log in to a new session
And the Tails desktop is ready And the Tails desktop is ready
......
@product #10497: wait_until_tor_is_working
@product @fragile
Feature: Chatting anonymously using Pidgin Feature: Chatting anonymously using Pidgin
As a Tails user As a Tails user
when I chat using Pidgin when I chat using Pidgin
...@@ -90,6 +91,7 @@ Feature: Chatting anonymously using Pidgin ...@@ -90,6 +91,7 @@ Feature: Chatting anonymously using Pidgin
And I close Pidgin's certificate manager And I close Pidgin's certificate manager
Then I cannot add a certificate from the "/live/overlay/home/amnesia/.gnupg" directory to Pidgin Then I cannot add a certificate from the "/live/overlay/home/amnesia/.gnupg" directory to Pidgin
#10443 - OFTC tests are fragile
@check_tor_leaks @fragile @check_tor_leaks @fragile
Scenario: Using a persistent Pidgin configuration Scenario: Using a persistent Pidgin configuration
Given I have started Tails without network from a USB drive with a persistent partition enabled and logged in Given I have started Tails without network from a USB drive with a persistent partition enabled and logged in
......
#10497: wait_until_tor_is_working
#10498: SSH tests are fragile
@product @fragile @product @fragile
Feature: Logging in via SSH Feature: Logging in via SSH
As a Tails user As a Tails user
......
...@@ -4,12 +4,16 @@ Feature: Time syncing ...@@ -4,12 +4,16 @@ Feature: Time syncing
I want Tor to work properly I want Tor to work properly
And for that I need a reasonably accurate system clock And for that I need a reasonably accurate system clock
#10497: wait_until_tor_is_working
@fragile
Scenario: Clock with host's time Scenario: Clock with host's time
Given I have started Tails from DVD without network and logged in Given I have started Tails from DVD without network and logged in
When the network is plugged When the network is plugged
And Tor is ready And Tor is ready
Then Tails clock is less than 5 minutes incorrect Then Tails clock is less than 5 minutes incorrect
#10497: wait_until_tor_is_working
@fragile
Scenario: Clock with host's time in bridge mode Scenario: Clock with host's time in bridge mode
Given I have started Tails from DVD without network and logged in with bridge mode enabled Given I have started Tails from DVD without network and logged in with bridge mode enabled
When the network is plugged When the network is plugged
...@@ -18,6 +22,8 @@ Feature: Time syncing ...@@ -18,6 +22,8 @@ Feature: Time syncing
And Tor is ready And Tor is ready
Then Tails clock is less than 5 minutes incorrect Then Tails clock is less than 5 minutes incorrect
#10497: wait_until_tor_is_working
@fragile
Scenario: Clock is one day in the past Scenario: Clock is one day in the past
Given I have started Tails from DVD without network and logged in Given I have started Tails from DVD without network and logged in
When I bump the system time with "-1 day" When I bump the system time with "-1 day"
...@@ -25,6 +31,8 @@ Feature: Time syncing ...@@ -25,6 +31,8 @@ Feature: Time syncing
And Tor is ready And Tor is ready
Then Tails clock is less than 5 minutes incorrect Then Tails clock is less than 5 minutes incorrect
#10497: wait_until_tor_is_working
@fragile
Scenario: Clock is one day in the past in bridge mode Scenario: Clock is one day in the past in bridge mode
Given I have started Tails from DVD without network and logged in with bridge mode enabled Given I have started Tails from DVD without network and logged in with bridge mode enabled
When I bump the system time with "-1 day" When I bump the system time with "-1 day"
...@@ -34,6 +42,8 @@ Feature: Time syncing ...@@ -34,6 +42,8 @@ Feature: Time syncing
And Tor is ready And Tor is ready
Then Tails clock is less than 5 minutes incorrect Then Tails clock is less than 5 minutes incorrect
#10497: wait_until_tor_is_working
@fragile
Scenario: Clock is way in the past Scenario: Clock is way in the past
Given I have started Tails from DVD without network and logged in Given I have started Tails from DVD without network and logged in
# 13 weeks will span over two Tails release cycles. # 13 weeks will span over two Tails release cycles.
...@@ -42,7 +52,8 @@ Feature: Time syncing ...@@ -42,7 +52,8 @@ Feature: Time syncing
And Tor is ready And Tor is ready
Then Tails clock is less than 5 minutes incorrect Then Tails clock is less than 5 minutes incorrect
@fragile #10497: wait_until_tor_is_working
@fragile
Scenario: Clock way in the past in bridge mode Scenario: Clock way in the past in bridge mode
Given I have started Tails from DVD without network and logged in with bridge mode enabled Given I have started Tails from DVD without network and logged in with bridge mode enabled
When I bump the system time with "-6 weeks" When I bump the system time with "-6 weeks"
...@@ -52,6 +63,8 @@ Feature: Time syncing ...@@ -52,6 +63,8 @@ Feature: Time syncing
And Tor is ready And Tor is ready
Then Tails clock is less than 5 minutes incorrect Then Tails clock is less than 5 minutes incorrect
#10497: wait_until_tor_is_working
#10440: Time syncing tests are fragile
@fragile @fragile
Scenario: Clock is one day in the future Scenario: Clock is one day in the future
Given I have started Tails from DVD without network and logged in Given I have started Tails from DVD without network and logged in
...@@ -60,6 +73,8 @@ Feature: Time syncing ...@@ -60,6 +73,8 @@ Feature: Time syncing
And Tor is ready And Tor is ready
Then Tails clock is less than 5 minutes incorrect Then Tails clock is less than 5 minutes incorrect
#10497: wait_until_tor_is_working
@fragile
Scenario: Clock is one day in the future in bridge mode Scenario: Clock is one day in the future in bridge mode
Given I have started Tails from DVD without network and logged in with bridge mode enabled Given I have started Tails from DVD without network and logged in with bridge mode enabled
When I bump the system time with "+1 day" When I bump the system time with "+1 day"
...@@ -69,6 +84,8 @@ Feature: Time syncing ...@@ -69,6 +84,8 @@ Feature: Time syncing
And Tor is ready And Tor is ready
Then Tails clock is less than 5 minutes incorrect Then Tails clock is less than 5 minutes incorrect
#10497: wait_until_tor_is_working
#10440: Time syncing tests are fragile
@fragile @fragile
Scenario: Clock way in the future Scenario: Clock way in the future
Given I have started Tails from DVD without network and logged in Given I have started Tails from DVD without network and logged in
...@@ -77,6 +94,8 @@ Feature: Time syncing ...@@ -77,6 +94,8 @@ Feature: Time syncing
And Tor is ready And Tor is ready
Then Tails clock is less than 5 minutes incorrect Then Tails clock is less than 5 minutes incorrect
#10497: wait_until_tor_is_working
#10440: Time syncing tests are fragile
@fragile @fragile
Scenario: Clock way in the future in bridge mode Scenario: Clock way in the future in bridge mode
Given I have started Tails from DVD without network and logged in with bridge mode enabled Given I have started Tails from DVD without network and logged in with bridge mode enabled
......
@product @product @fragile
Feature: Using Tails with Tor pluggable transports Feature: Using Tails with Tor pluggable transports
As a Tails user As a Tails user
I want to circumvent censorship of Tor by using Tor pluggable transports I want to circumvent censorship of Tor by using Tor pluggable transports
......
@product #10497: wait_until_tor_is_working
@product @fragile
Feature: The Tor enforcement is effective Feature: The Tor enforcement is effective
As a Tails user As a Tails user
I want all direct Internet connections I do by mistake or applications do by misconfiguration or buggy leaks to be blocked I want all direct Internet connections I do by mistake or applications do by misconfiguration or buggy leaks to be blocked
......
@product @check_tor_leaks #10497: wait_until_tor_is_working
@product @check_tor_leaks @fragile
Feature: Tor stream isolation is effective Feature: Tor stream isolation is effective
As a Tails user As a Tails user
I want my Torified sessions to be sensibly isolated from each other to prevent identity correlation I want my Torified sessions to be sensibly isolated from each other to prevent identity correlation
......
#10376: The "the Tor Browser loads the (startup page|Tails roadmap)" step is fragile
#10497: wait_until_tor_is_working
@product @fragile @product @fragile
Feature: Browsing the web using the Tor Browser Feature: Browsing the web using the Tor Browser
As a Tails user As a Tails user
...@@ -132,6 +134,7 @@ Feature: Browsing the web using the Tor Browser ...@@ -132,6 +134,7 @@ Feature: Browsing the web using the Tor Browser
And the Tor Browser has started and loaded the startup page And the Tor Browser has started and loaded the startup page
Then the Tor Browser has no plugins installed Then the Tor Browser has no plugins installed
@fragile
Scenario: The persistent Tor Browser directory is usable Scenario: The persistent Tor Browser directory is usable
Given I have started Tails without network from a USB drive with a persistent partition enabled and logged in Given I have started Tails without network from a USB drive with a persistent partition enabled and logged in
And the network is plugged And the network is plugged
......
#10497: wait_until_tor_is_working
#10444: Git tests are fragile
@product @check_tor_leaks @fragile @product @check_tor_leaks @fragile
Feature: Cloning a Git repository Feature: Cloning a Git repository
As a Tails user As a Tails user
......
@product @check_tor_leaks @product @check_tor_leaks @fragile
Feature: Keyserver interaction with GnuPG Feature: Keyserver interaction with GnuPG
As a Tails user As a Tails user
when I interact with keyservers using various GnuPG tools when I interact with keyservers using various GnuPG tools
......
@product @check_tor_leaks @product @check_tor_leaks @fragile
Feature: Various checks for torified software Feature: Various checks for torified software
Background: Background:
......
...@@ -44,6 +44,7 @@ Feature: Using Totem ...@@ -44,6 +44,7 @@ Feature: Using Totem
# as /lib/live/mount/overlay. # as /lib/live/mount/overlay.
And AppArmor has denied "/usr/bin/totem" from opening "/lib/live/mount/overlay/home/amnesia/.gnupg/video.mp4" And AppArmor has denied "/usr/bin/totem" from opening "/lib/live/mount/overlay/home/amnesia/.gnupg/video.mp4"
#10497: wait_until_tor_is_working
@check_tor_leaks @fragile @check_tor_leaks @fragile
Scenario: Watching a WebM video over HTTPS, with and without the command-line Scenario: Watching a WebM video over HTTPS, with and without the command-line
Given a computer Given a computer
......
...@@ -4,6 +4,7 @@ Feature: Browsing the web using the Unsafe Browser ...@@ -4,6 +4,7 @@ Feature: Browsing the web using the Unsafe Browser
when I browse the web using the Unsafe Browser when I browse the web using the Unsafe Browser
I should have direct access to the web I should have direct access to the web
@fragile
Scenario: The Unsafe Browser can access the LAN Scenario: The Unsafe Browser can access the LAN
Given I have started Tails from DVD and logged in and the network is connected Given I have started Tails from DVD and logged in and the network is connected
And a web server is running on the LAN And a web server is running on the LAN
...@@ -11,6 +12,7 @@ Feature: Browsing the web using the Unsafe Browser ...@@ -11,6 +12,7 @@ Feature: Browsing the web using the Unsafe Browser
And I open a page on the LAN web server in the Unsafe Browser And I open a page on the LAN web server in the Unsafe Browser
Then I see "UnsafeBrowserHelloLANWebServer.png" after at most 20 seconds Then I see "UnsafeBrowserHelloLANWebServer.png" after at most 20 seconds
@fragile
Scenario: Starting the Unsafe Browser works as it should. Scenario: Starting the Unsafe Browser works as it should.
Given I have started Tails from DVD and logged in and the network is connected Given I have started Tails from DVD and logged in and the network is connected
When I successfully start the Unsafe Browser When I successfully start the Unsafe Browser
...@@ -23,6 +25,7 @@ Feature: Browsing the web using the Unsafe Browser ...@@ -23,6 +25,7 @@ Feature: Browsing the web using the Unsafe Browser
And the Unsafe Browser has no proxy configured And the Unsafe Browser has no proxy configured
And the Unsafe Browser uses all expected TBB shared libraries And the Unsafe Browser uses all expected TBB shared libraries
@fragile
Scenario: Closing the Unsafe Browser shows a stop notification and properly tears down the chroot. Scenario: Closing the Unsafe Browser shows a stop notification and properly tears down the chroot.
Given I have started Tails from DVD and logged in and the network is connected Given I have started Tails from DVD and logged in and the network is connected
When I successfully start the Unsafe Browser When I successfully start the Unsafe Browser
...@@ -30,6 +33,7 @@ Feature: Browsing the web using the Unsafe Browser ...@@ -30,6 +33,7 @@ Feature: Browsing the web using the Unsafe Browser
Then I see the Unsafe Browser stop notification Then I see the Unsafe Browser stop notification
And the Unsafe Browser chroot is torn down And the Unsafe Browser chroot is torn down
@fragile
Scenario: Starting a second instance of the Unsafe Browser results in an error message being shown. Scenario: Starting a second instance of the Unsafe Browser results in an error message being shown.
Given I have started Tails from DVD and logged in and the network is connected Given I have started Tails from DVD and logged in and the network is connected
When I successfully start the Unsafe Browser When I successfully start the Unsafe Browser
...@@ -44,11 +48,13 @@ Feature: Browsing the web using the Unsafe Browser ...@@ -44,11 +48,13 @@ Feature: Browsing the web using the Unsafe Browser
Then I see "UnsafeBrowserTorCheckFail.png" after at most 60 seconds Then I see "UnsafeBrowserTorCheckFail.png" after at most 60 seconds
And the clearnet user has sent packets out to the Internet And the clearnet user has sent packets out to the Internet
@fragile
Scenario: The Unsafe Browser cannot be configured to use Tor and other local proxies. Scenario: The Unsafe Browser cannot be configured to use Tor and other local proxies.
Given I have started Tails from DVD and logged in and the network is connected Given I have started Tails from DVD and logged in and the network is connected
When I successfully start the Unsafe Browser When I successfully start the Unsafe Browser
Then I cannot configure the Unsafe Browser to use any local proxies Then I cannot configure the Unsafe Browser to use any local proxies
@fragile
Scenario: The Unsafe Browser will not make any connections to the Internet which are not user initiated Scenario: The Unsafe Browser will not make any connections to the Internet which are not user initiated
Given I have started Tails from DVD and logged in and the network is connected Given I have started Tails from DVD and logged in and the network is connected
And I capture all network traffic And I capture all network traffic
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment