Fix-committed: #17278

Merge remote-tracking branch 'origin/stable' into feature/17646-ruby-coding-standards-baseline+force-all-tests

This reverts commit 794bb06d.

The reverted change might be what makes try_for() sometimes fail to
notice when it should time out, resulting in crazy things like:

    try_for: attempt 288 (3586.91s elapsed of 300s)...

where Jenkins will abort the test due to inactivity.

Merge remote-tracking branch 'origin/feature/17621-keep-latest-screenshot' into stable (Closes: #17621)

The default configuration lives in /usr/share/tor/tor-service-defaults-torrc
nowadays.

sajolida authored May 11, 2020 and intrigeri committed May 15, 2020

- Our torrc is not outdated
- Previous links were both broken
- Our torrc is not for a "typical Tor user"
- Our torrc still works for much newer versions of Tor
- Our users are not supposed to edit this file

These variables have not been used
since 463388f4.

I'd like to keep saving this extra debugging data for a while,
but let's minimize the impact when scenarios in other features fail.

We currently set DefaultTimeoutStopSec=5s. I assume that in most cases, deleting
overlayfs directories from memory should be super fast, but I'd rather not bet
on this for security matters.

Give tails-synchronize-data-to-new-persistent-volume-on-shutdown.service more time to do its job (refs: #17278)

We currently set DefaultTimeoutStopSec=5s, which means copying just the APT
lists (215MB currently), not even taking into account the actual additional
packages installed by the user, would need to happen at no less than 43MB/s for
the ExecStop= to succeed.

Most real-world bare metal USB sticks don't provide anything close
to this write rate.

I can't reproduce the problems caused by this bug when running the test suite on
my laptop, but it does happen regularly on lizard: even virtual USB stick backed
by tmpfs are not always fast enough there.

I've observed cases when that service started, copied part of the data,
but after the next boot:

 - Not all the data can be found on the persistent volume.
 - The flag file that indicates that this service has finished its job
   is not present.

At this point I don't know whether this service was killed before
it could finish its job, or this service did copy the data but
it was not sync'ed to disk. Let's eliminate the second possibility
and make it easier to reason on this problem.

I guess we could revert this commit once we're confident we have fixed
the main bugs in this area.

I guess we could revert this commit once we're confident we have fixed
the main bugs in this area.

The Journal saved on failure has been called *.journal for years.

Let's not require developers to change their habits,
and let's not break tooling that may rely on this.

I'll need this to debug #17278.

I'll need something similar to debug #17278.

We have already masked apt-daily.timer (#12390), but apt-daily-upgrade.timer was
left enabled. AFAICT it's a no-op by default, but better safe than sorry.

Finally, drop masking of apt-daily.timer: APT::Periodic::Enable effectively
makes apt-daily*.timer no-ops.

Fixup against 4efe5283.

Interestingly, I did not have this problem with Ruby 2.7.1 on my system,
but on Jenkins (Ruby 2.3.3) I see this error:

  wrong argument type String (expected Symbol) (TypeError)
  ./features/step_definitions/pidgin.rb:113:in `/^my XMPP friend goes online( and joins the multi-user chat)?$/'

Until we require Buster for running our test suite, the current implementation
is rather ugly, but I figured it's better to have an ugly backwards
compatibility workaround than to stick with old arguments passing style: it
brings us one step closer to Ruby best practices.

When the ordering is wrong, one gets error output with reversed expected vs.
actual, which is confusing and has made me waste time already.

refs: #17278

 - Consistently use present tense

 - While switching scenario titles to present tense, rephrase some to be
   expressed from the perspective of the user, which is what BDD is about (at
   least for situations when the user is the main stakeholder).

The benefit of getting 100% independent results for these 2 scenarios
does not seem to outweigh the performance and resources cost:
the chances that only one of them fails are pretty low.

Mostly dropping the vague "live systems" terminology.

Tails does *not* boot on internal hard drives by default.

The benefit of getting 100% independent results for these 2 scenarios
does not seem to outweigh the performance and resources cost:
the chances that only one of them fails are pretty low.

Here, we test the exact same thing as "Scenario: Upgrading an old Tails USB
installation from another Tails USB drive" and "Scenario: Booting Tails from
a USB drive upgraded from USB with persistence enabled", except we're upgrading
by cloning a Tails running from DVD.

The code path that clones the running Tails is the same, regardless
of the physical boot medium that Tails was started from.

Therefore, IMO testing 2 slightly different pairs of scenarios is not worth the
resources it uses on our CI, the increased feedback loop duration for
developers. Let's focus on upgrading by cloning from a USB stick, which covers
most of our documented upgrade scenarios.

After reading the corresponding issue, my understanding is that
detecting the UEFI firmware screen was causing the problem,
and that code is gone.

Since then a number of other things happened in this department:

 - We've switched from syslinux to GRUB for UEFI boot. This could
   potentially impact the robustness of this test.

 - We've made boot menu and command line handling more robust.

 - UEFI is the common case rather than the exception nowadays,
   so testing it is becoming increasingly more important.

So I think it's worth giving this another try, see what happens,
and be ready to revert this commit.

IIRC, we introduced this in 2013 as a regression test for a Tails Installer bug.

Given:

 - A USB stick that has a MBR partition table but no partition
   is a pretty rare corner case.

 - Nowadays, the only documented user scenarios that rely on Tails Installer are
   about full manual upgrades, in which case the destination drive will have
   a GPT. We're testing this in the "Installing Tails with Tails Installer to
   a used USB drive" scenario and in usb_upgrade.feature.

 - I can't recall this test ever catching a regression on this front
   in the last 7 years. Granted, there's a good chance I would have
   forgotten.

⇒ IMO, at this point this scenario is not worth spending more than 7 minutes
on during every Jenkins test suite run.

This scenario tested 2 things:

 - Starting Tails installed from the (hybrid) ISO image copied bit-by-bit to the
   USB stick: we don't support this since January 2019 anymore.

 - Tails Installer is able to reinstall a USB stick installed from the (hybrid)
   ISO image copied bit-by-bit, by cloning another running Tails. That's
   a useful feature for users who are migrating a USB stick installed in 2018 or
   earlier, but after almost 1.5 years of transition time, IMO we've reached the
   point when it's not worth spending almost 12 minutes (more than 25% of the
   usb_install.feature) on every Jenkins test suite run, just to ensure we did
   not break this scenario. Given the low amount of work we're putting into
   Tails Installer these days, the risk of regression is quite limited anyway.