Additional Software is often broken after initial persistence setup and when recovering old APT lists (#17278) · Issues · tails / tails

Additional Software is often broken after initial persistence setup and when recovering old APT lists

Originally created by @intrigeri on #17278 (Redmine)

The test failure is:

12:03:29   Scenario: I set up Additional Software when installing a package without persistent partition and the package is installed next time I start Tails # features/additional_software_packages.feature:25
12:07:42     Given I start Tails from a freshly installed USB drive with an administration password and the network is plugged and I login                    # features/step_definitions/common_steps.rb:163
12:07:42     And I update APT using apt                                                                                                                       # features/step_definitions/apt.rb:46
12:07:42     And I install "sslh" using apt                                                                                                                   # features/step_definitions/apt.rb:65
12:07:42     Then I am proposed to add the "sslh" package to my Additional Software                                                                           # features/step_definitions/additional_software_packages.rb:33
12:07:42     And I create a persistent storage and activate the Additional Software feature                                                                   # features/step_definitions/additional_software_packages.rb:38
12:07:42     And I shutdown Tails and wait for the computer to power off                                                                                      # features/step_definitions/common_steps.rb:542
12:20:21     And I start Tails from USB drive "__internal" with network unplugged and I login with persistence enabled                                        # features/step_definitions/common_steps.rb:139
12:20:21       try_for() timeout expired (Timeout::Error)
12:20:21       ./features/support/helpers/misc_helpers.rb:89:in `rescue in try_for'
12:20:21       ./features/support/helpers/misc_helpers.rb:36:in `try_for'
12:20:21       ./features/step_definitions/additional_software_packages.rb:14:in `/^the Additional Software (upgrade|installation) service has started$/'
12:20:21       ./features/step_definitions/common_steps.rb:152:in `/^I start Tails from (.+?) drive "(.+?)"( with network unplugged)?( and I login( with persistence enabled)?( (?:and|with) an administration password)?)?$/'
12:20:21       features/additional_software_packages.feature:32:in `And I start Tails from USB drive "__internal" with network unplugged and I login with persistence enabled'
12:21:50     And Additional Software is correctly configured for package "sslh"                                                                               # features/step_definitions/additional_software_packages.rb:50
12:21:50     And the package "sslh" is installed after Additional Software has been started                                                                   # features/step_definitions/common_steps.rb:571
12:21:50       Scenario failed at time 02:09:22

That is, step "the Additional Software installation service has started" fails when called from Given /^I start Tails from (.+?) drive "(.+?)"( with network unplugged)?( and I login( with persistence enabled)?( (?:and|with) an administration password)?)?$/.

The Journal looks like this:

Nov 30 12:10:03 amnesia systemd[5150]: Starting Trigger installation of Additional Software Packages...
Nov 30 12:10:03 amnesia systemd[5150]: Starting Wait for Tor to Have Bootstrapped...
Nov 30 12:10:03 amnesia systemd[5150]: Started Tracker metadata database store and lookup manager.
Nov 30 12:10:03 amnesia sudo[8393]:  amnesia : TTY=unknown ; PWD=/home/amnesia ; USER=root ; COMMAND=/bin/systemctl start tails-additional-software-install.service
Nov 30 12:10:03 amnesia sudo[8393]: pam_unix(sudo:session): session opened for user root by (uid=0)
[…]
Nov 30 12:10:04 amnesia systemd[1]: Starting Install Additional Software Packages...
[…]
Nov 30 12:10:09 amnesia tails-additional-software[8409]: [INFO] Starting to install additional software...
Nov 30 12:10:09 amnesia tails-additional-software[8409]: [INFO] Found additional packages list.
[…]
Nov 30 12:10:09 amnesia sudo[8461]:     root : TTY=unknown ; PWD=/ ; USER=amnesia ; COMMAND=/usr/local/lib/tails-additional-software-notify Installing your additional software from persistent storage... This can take several minutes.
[…]
Nov 30 12:10:11 amnesia tails-additional-software[8409]: [INFO] Will install the following packages: sslh
[…]
Nov 30 12:10:24 amnesia tails-additional-software[8409]: [INFO] Reading package lists...
Nov 30 12:10:24 amnesia tails-additional-software[8409]: [INFO] Building dependency tree...
Nov 30 12:10:24 amnesia tails-additional-software[8409]: [INFO] Reading state information...
Nov 30 12:10:25 amnesia tails-additional-software[8409]: [INFO] Package sslh is not available, but is referred to by another package.
Nov 30 12:10:25 amnesia tails-additional-software[8409]: [INFO] This may mean that the package is missing, has been obsoleted, or
Nov 30 12:10:25 amnesia tails-additional-software[8409]: [INFO] is only available from another source
Nov 30 12:10:25 amnesia tails-additional-software[8409]: [INFO]
Nov 30 12:10:25 amnesia tails-additional-software[8409]: [INFO] E: Package 'sslh' has no installation candidate
Nov 30 12:10:25 amnesia tails-additional-software[8409]: [WARNING] apt-get exited with returncode 100
Nov 30 12:10:25 amnesia tails-additional-software[8409]: [WARNING] Warning: installation of sslh failed

It looks like this started around Nov 13. This matches the time when the fix for #17203 (closed) was merged, but I don’t understand why changing the dependencies of the ASP upgrade service, which we don’t even try to start in the failing scenarios, can possibly affect whether installation of ASP works or not.

For now I’m not classifying this as a test suite -only issue: there’s a chance this affects real users.

Feature Branch: bugfix/17278-fix-additional-software-intrigeri

Attachments

Related issues

Related to #17203 (closed)
Related to #17392

_Originally created by @intrigeri on [#17278 (Redmine)](https://public-redmine-archive.tails.boum.org/code/issues/17278)_

The test failure
    is:

12:03:29   Scenario: I set up Additional Software when installing a package without persistent partition and the package is installed next time I start Tails # features/additional_software_packages.feature:25
    12:07:42     Given I start Tails from a freshly installed USB drive with an administration password and the network is plugged and I login                    # features/step_definitions/common_steps.rb:163
    12:07:42     And I update APT using apt                                                                                                                       # features/step_definitions/apt.rb:46
    12:07:42     And I install "sslh" using apt                                                                                                                   # features/step_definitions/apt.rb:65
    12:07:42     Then I am proposed to add the "sslh" package to my Additional Software                                                                           # features/step_definitions/additional_software_packages.rb:33
    12:07:42     And I create a persistent storage and activate the Additional Software feature                                                                   # features/step_definitions/additional_software_packages.rb:38
    12:07:42     And I shutdown Tails and wait for the computer to power off                                                                                      # features/step_definitions/common_steps.rb:542
    12:20:21     And I start Tails from USB drive "__internal" with network unplugged and I login with persistence enabled                                        # features/step_definitions/common_steps.rb:139
    12:20:21       try_for() timeout expired (Timeout::Error)
    12:20:21       ./features/support/helpers/misc_helpers.rb:89:in `rescue in try_for'
    12:20:21       ./features/support/helpers/misc_helpers.rb:36:in `try_for'
    12:20:21       ./features/step_definitions/additional_software_packages.rb:14:in `/^the Additional Software (upgrade|installation) service has started$/'
    12:20:21       ./features/step_definitions/common_steps.rb:152:in `/^I start Tails from (.+?) drive "(.+?)"( with network unplugged)?( and I login( with persistence enabled)?( (?:and|with) an administration password)?)?$/'
    12:20:21       features/additional_software_packages.feature:32:in `And I start Tails from USB drive "__internal" with network unplugged and I login with persistence enabled'
    12:21:50     And Additional Software is correctly configured for package "sslh"                                                                               # features/step_definitions/additional_software_packages.rb:50
    12:21:50     And the package "sslh" is installed after Additional Software has been started                                                                   # features/step_definitions/common_steps.rb:571
    12:21:50       Scenario failed at time 02:09:22

That is, `step "the Additional Software installation service has
started"` fails when called from `Given /^I start Tails from (.+?) drive
"(.+?)"( with network unplugged)?( and I login( with persistence
enabled)?( (?:and|with) an administration password)?)?$/`.

The Journal looks like
    this:

Nov 30 12:10:03 amnesia systemd[5150]: Starting Trigger installation of Additional Software Packages...
    Nov 30 12:10:03 amnesia systemd[5150]: Starting Wait for Tor to Have Bootstrapped...
    Nov 30 12:10:03 amnesia systemd[5150]: Started Tracker metadata database store and lookup manager.
    Nov 30 12:10:03 amnesia sudo[8393]:  amnesia : TTY=unknown ; PWD=/home/amnesia ; USER=root ; COMMAND=/bin/systemctl start tails-additional-software-install.service
    Nov 30 12:10:03 amnesia sudo[8393]: pam_unix(sudo:session): session opened for user root by (uid=0)
    […]
    Nov 30 12:10:04 amnesia systemd[1]: Starting Install Additional Software Packages...
    […]
    Nov 30 12:10:09 amnesia tails-additional-software[8409]: [INFO] Starting to install additional software...
    Nov 30 12:10:09 amnesia tails-additional-software[8409]: [INFO] Found additional packages list.
    […]
    Nov 30 12:10:09 amnesia sudo[8461]:     root : TTY=unknown ; PWD=/ ; USER=amnesia ; COMMAND=/usr/local/lib/tails-additional-software-notify Installing your additional software from persistent storage... This can take several minutes.
    […]
    Nov 30 12:10:11 amnesia tails-additional-software[8409]: [INFO] Will install the following packages: sslh
    […]
    Nov 30 12:10:24 amnesia tails-additional-software[8409]: [INFO] Reading package lists...
    Nov 30 12:10:24 amnesia tails-additional-software[8409]: [INFO] Building dependency tree...
    Nov 30 12:10:24 amnesia tails-additional-software[8409]: [INFO] Reading state information...
    Nov 30 12:10:25 amnesia tails-additional-software[8409]: [INFO] Package sslh is not available, but is referred to by another package.
    Nov 30 12:10:25 amnesia tails-additional-software[8409]: [INFO] This may mean that the package is missing, has been obsoleted, or
    Nov 30 12:10:25 amnesia tails-additional-software[8409]: [INFO] is only available from another source
    Nov 30 12:10:25 amnesia tails-additional-software[8409]: [INFO]
    Nov 30 12:10:25 amnesia tails-additional-software[8409]: [INFO] E: Package 'sslh' has no installation candidate
    Nov 30 12:10:25 amnesia tails-additional-software[8409]: [WARNING] apt-get exited with returncode 100
    Nov 30 12:10:25 amnesia tails-additional-software[8409]: [WARNING] Warning: installation of sslh failed

It looks like this started around Nov 13. This matches the time when the
fix for tails/tails#17203 was merged, but I don’t understand why changing the
dependencies of the ASP upgrade service, which we don’t even try to
start in the failing scenarios, can possibly affect whether
*installation* of ASP works or not.

For now I’m not classifying this as a test suite -only issue: there’s a
chance this affects real users.

Feature Branch: bugfix/17278-fix-additional-software-intrigeri

### Attachments

* [02_18_54_My_Additional_Software_list_is_configurable_through_a_GUI_or_through_notifications_when_I_install_or_remove_packages_with_APT_or_Synaptic.png](https://redmine.tails.boum.org/code/attachments/download/2572/02_18_54_My_Additional_Software_list_is_configurable_through_a_GUI_or_through_notifications_when_I_install_or_remove_packages_with_APT_or_Synaptic.png)
  * [02_18_54_My_Additional_Software_list_is_configurable_through_a_GUI_or_through_notifications_when_I_install_or_remove_packages_with_APT_or_Synaptic.journal](https://redmine.tails.boum.org/code/attachments/download/2573/02_18_54_My_Additional_Software_list_is_configurable_through_a_GUI_or_through_notifications_when_I_install_or_remove_packages_with_APT_or_Synaptic.journal)

### Related issues

- **Related to** tails/tails#17203
  - **Related to** tails/tails#17392

Edited May 21, 2020 by intrigeri

To upload designs, you'll need to enable LFS. More information