Changes

Page history
Rename blueprints: .mdwn → .md authored by intrigeri's avatar intrigeri 
Rationale:

 - Right now:

   - These pages' Markdown will be rendered when browsing our
     repository in the GitLab web interface

   - The .md extension is more common nowadays: GitHub, GitLab, and various
     other major players have settled on it. Let's get used to it.

 - If we decide to migrate our blueprints to GitLab (#18079),
   this will be a necessary first step.
Show whitespace changes
Inline Side-by-side
Debian_testing.md 0 → 100644
View page @ 7db7c482
[[!meta title="Basing Tails on quarterly snapshots of Debian Testing"]]
Tracking ticket: [[!tails_ticket 12615]]
[[!toc levels=2]]
# The big picture
We've been thinking for a while of being based on quarterly snapshots
of Debian testing. We've used the "porting Tails to Debian Stretch"
cycle to evaluate how it would feel like. See the
[[initial plan & analysis|blueprint/Debian_Stretch/#rolling]].
## Calendar
A two-years Debian release cycle could look like from our perspective,
if we were based on quarterly snapshots of Debian testing:
1. a new Debian stable is released
2. 6 months during which Tails is based on Debian stable that was just
released, and we can get ready for the next stage
3. 12 months during which Tails is based on quarterly snapshots of
(non-frozen) Debian testing taken at the beginning of each major
Tails release cycle
4. 6 months during which Tails is based on a frozen Debian testing
5. go back to (1)
We would be based on a moving target half of the time; the remaining
half of the time we are based on something that doesn't change much.
## Why?
- We'll need to track 2 Debian versions at the same time (and
continuously forward-port development that was based on the oldest
one) during 6-7 months maximum, compared to 19 months (Jessie cycle)
and 10 months (Stretch cycle). We would save lots of time there,
that could be re-invested in aspects of this proposal that will
require additional work.
- no need to produce and maintain backports of Debian packages anymore
- support new hardware better
- upstream bugfixes trickle down faster and for free to Tails users
⇒ greater incentive for us to fix bugs upstream instead of
implementing Tails-specific workarounds
- we help stabilizing Debian testing
- the Foundations Team prefers doing the "porting to next Debian" work
continuously than as a huge disruptive project every two years
- no huge change every 2 years confusing users and creating huge burts
in the Help Desk and Foundations Team post-release workload
- new features / modern software
This idea is about doing a different kind of work, and doing it in
different places than in the past. It probably won't lower the total
amount of effort we have to do, but it likely will make such efforts
generate less frustration, more happiness, and a warm feeling of being
part of a broader community.
# Fears, concerns & problems we need to address or mitigate
- Security
* How to keep track of security issues that affect us, and whether
their fix has been uploaded and has migrated to testing yet?
See e.g. how security support for Debian testing [used to be
(briefly) done](http://secure-testing-master.debian.net/), and in
particular the
[Vulnerable source packages in the testing suite](https://security-tracker.debian.org/tracker/status/release/testing)
and the
[Candidates for DTSAs](https://security-tracker.debian.org/tracker/status/dtsa-candidates).
At first glance it seems totally doable for Foundations Team
members to run (e.g. locally) a lightweight fork of Debian's
security tracker:
- Problem to solve: list security issues that affect Tails
(current stable and testing branches) but are fixed in
current Debian testing and/or sid
- Point it to our relevant APT snapshots (and perhaps current
Debian testing or sid instead of Tails devel, to avoid having
to resolve the "latest" unfrozen snapshot)
- Add our own annotations about issues we decide to
ignore/postpone, e.g. using the new
[`<ignored>` and `<postponed>` tags](https://security-team.debian.org/security_tracker.html#issues-not-warranting-a-security-advisory)
- Filter packages based on some relevant build manifest so we
don't ever see packages we don't ship in Tails.
- Automatically merge from Debian's repo and report to us in case
of merge conflicts.
* more freeze exceptions in order to address security issues
* need to keep our stable branch in an always releasable state
⇒ the tracking & cherry-picking of security issues must be done
continuously, and not only right before a planned freeze or release
* when the updated package from Debian testing/sid is not
installable on current Tails stable/testing, we'll have to
backport security fixes
- Transitions
* How to deal with
[[!debwiki OngoingTransitions desc="ongoing transitions"]] that
block migration of security updates from sid to testing?
* How to select the right snapshot we shall take, e.g.
during transitions?
* Sometimes we'll need to rebuild on testing some packages we want
to cherry-pick from sid.
- Consequences for users
* too many software regressions and not well tested new features
* confused users due to constant incremental changes
* bigger upgrades on average: according to research done on
[[!tails_ticket 14622]], we should assume ~600MB large IUKs;
there will be consequences in terms of:
- RAM requirements: XXX
- download time: XXX
- download reliability ⇒ add a retry/continue mechanism to Tails Upgrader?
* our users debug Debian testing on top of debugging Tails
- Drawbacks for contributors & the Tails community
* more frequent regressions in ISO reproducibility
* harder to extract info from help desk
* harder for help desk to deal with user requests: having to
re-learn constantly how things work / what's broken
* spend too much mental space dealing with always changing software
* lack of Debian skills internally at the moment ⇒ we'll need to
invest time learning
* need to better study the delta between N & N+1 (for help desk,
release notes, security audits, etc.)
- Communication
* needs new version numbering scheme
* feeling of stalled/slower project for users because no big
all-at-a-time changes
* how do we deal with our (implicit) hardware support promise? i.e.
"The hardware you purchase for Tails N.0 will work with N.x"
* harder to get press write stuff as we'll lack releases with tons
of changes like 2.0 or 3.0
- Additional Software Packages: will they be pulled from current
Debian testing or from our snapshot? Can we tell APT to install by
default from Debian testing, unless it's impossible or problematic
(e.g. would remove tons of packages) and then fall back to
installing from our snapshot?
# The plan
**Update**: this was deferred. We'll come back to this in 2018-04 or
2018-05 and either come up with a new plan, or postpone this entirely.
* From now to the end of 2017-11: the Foundations Team tries to port
the code & test suite during sprints. If the work doesn't fit into
these sprints then we'll need to draw conclusions.
* At the end of 2017-11:
1. [[!tails_ticket 14578 desc="Decide"]] whether we want to release
Tails based on Debian testing
in 2018-01, 2018-04, or give up and rather release it mid-2019.
The following assumes "in 2018-01" and will need to be adjusted
if we decide something else.
2. The Foundations Team tells technical writers what needs to be
updated ([[!tails_ticket 14579]])
* November-January: technical writers update the documentation
* January 16: first Tails release based on Debian testing
# Zooming in on specific problems
<a id="doc-update"></a>
## How can the Foundations Team tell Technical Writers what part of our doc needs to be updated?
The first time we'll have to deal with this problem is
[[!tails_ticket 14579]], but it will come back every quarter.
Current implementation:
- [[!tails_gitweb bin/doc-impacted-by]]
- [[!tails_gitweb doc-source-relationships.yml]]
We want to automate this as much as we can, but let's be clear:
automation will not fully solve this problem unless we switch to
executable documentation (there is research and tools for that but
we're not there yet). So some amount of manual documentation testing
will be needed. Let's start with strategies that will limit the amount
and frequency of the needed manual documentation testing, and further
below we'll discuss manual testing.
### Changes in our code, test suite, and list of included packages
Whenever we have to adjust our code to keep working on Debian testing,
it's plausible that the corresponding documentation needs an update as
well. Developers could either proactively tell technical writers about
it along the way, or look at `git diff` when requested to sum up what
part of Tails is impacted by the changes.
Similarly, whenever we have to update the test suite, it's plausible
that the corresponding documentation needs an update as well.
Test suite maintainers can then add the updated scenario to a list of
things the technical writers should check (again: either incrementally
or upon request).
And finally, developers or technical writers could look at the diff
between the old and new lists of included packages, to identify what
part of Tails is impacted by the changes.
All this is fine but it requires someone to keep a mental index of
what kind of change impacts what part of our documentation. Not only
this is boring work, but nobody is in a good position to do it, as it
requires a good knowledge of our code base _and_ of our documentation.
So I think we should build and maintain this index incrementally, in
Git instead of in our minds, and use it automatically. Read on :)
### Metadata that encodes the relationship between {source code, test suite, packages} and documentation pages
Whenever we notice that changes in a given file in our source code,
automated test suite scenario, or package, have an effect on a given
documentation page, we could encode this information somewhere, in
a machine-readable format. Then, we could write a program whose input
would be:
* this metadata
* 2 Git commitish
* 2 package lists
This program would output the list of documentation pages that are
affected by the changes.
This set of metadata shall be built and garbage-collected incrementally.
Remaining questions:
- Where to store this metadata?
At least the information about included packages lives nowhere in
our Git tree, so encoding it close to the source is impossible
(besides, I'd rather not have to extract metadata from comments in
source code written in N different programming languages). So we
have main options:
1. Store it in the affected documentation pages themselves,
(HTML comments? ikiwiki directives? any other idea?)
Advantages:
- No work required when renaming pages.
- Easier to maintain by tech writers as they can see this info
while working on the doc, which increases the chances they keep
it up-to-date.
- Lightweight: no need to explicitly write the doc page name.
2. Store it externally, in a dedicated file. Advantages:
- Probably nicer format to read and edit, both for humans and
computers, than whatever we would embed in doc pages.
- Does not affect translators: anything we would put in doc pages
will be extracted to PO files; so storing this metadata there
would waste translators' time (either because they uselessly
translate it, or because they paste the English string to reach
100% translation status and keep their dashboard of pages
needing attention usable).
- Works for encoding which sources files impact screenshots.
intrigeri & anonym discussed that and were torn. The advantages of
the first option are very compelling, but the drawback it has wrt.
translators seems to be a deal-breaker. So the second option
appears to be the best one.
XXX: sajolida?
- Defining sets of sources?
Tails user-facing features are built from a not-that-small number
of source files. It could be useful to express the fact that
changes to any of these source files (e.g. the ones that define the
"Tor Browser" feature) may affect a given doc page, without having
to list them all for every affected page.
- Defining sets of doc pages?
Similarly, it might be that some doc sections share most of their
dependencies. It could be useful to define named groups of pages.
- Who shall we optimize the data format for?
In practice, developers are the ones who know where a given
user-visible change comes from, so whenever a needed doc update was
missed, most likely the tech writers will ask the Foundations Team
either to add the missing metadata, or what exactly caused that
user-visible change so they can add it themselves.
I (intrigeri) am really not sure which of these two situations will
be the most common one, so I'm wary of optimizing too much for one
of these two groups at this point.
- Mapping source → doc, or doc → source?
Update: a format where we define a list of dependencies between
X doc pages and Y source files, without explicitly mapping one to
the other, would avoid having to choose too early, which would be
nice. I'll simply do that.
Internally, the code that uses this metadata will care about
"source → doc", but there's no reason why the data format itself
should be optimized for this program. Let's instead optimize it for
the humans who will maintain the metadata :)
"doc → source" expresses a "depends on" / "is affected by"
relationship, which is well suited when fixing the metadata after
a postmortem analysis of "why was that doc page not updated before
the release?". OTOH "source → doc" expresses an "impacts"
relationship, which is well suited to proactive adjustments of the
metadata, e.g. when a developer realizes their proposed branch will
impact the doc. I (intrigeri) am pretty sure that the first
situation will be the most common one for a while until we get used
to this metadata and the second one slowly takes over.
Example (YAML):
# doc → source oriented
- page: doc/first_steps/accessibility
files:
- config/chroot_local-includes/etc/dconf/db/local.d/00_Tails_defaults
packages:
- caribou
- dasher
- orca
- page: doc/first_steps/welcome_screen
package: tails-greeter
files:
- config/binary_local-hooks/*grub*
- config/binary_local-hooks/*syslinux*
- config/chroot_local-includes/etc/dconf/db/local.d/00_Tails_defaults
- page: doc/first_steps/welcome_screen/administration_password
package: tails-greeter
- page: doc/first_steps/welcome_screen/bridge_mode
packages:
- obfs4proxy
- tor
file: config/chroot_local-includes/usr/share/tails/tbb-sha256sums.txt
tests: tor_bridges.feature
# "source → doc" oriented
- package: gnome-disk-utility
pages:
- doc/first_steps/persistence/change_passphrase
- doc/first_steps/persistence/check_file_system
- package: tails-persistence-setup
tests: persistence.feature
pages:
- doc/first_steps/persistence/configure
- doc/first_steps/persistence/delete
- package: openpgp-applet
tests: encryption.feature
- package: tails-greeter
files:
- features/images/TailsGreeter*
pages:
- doc/first_steps/persistence/use
- doc/first_steps/welcome_screen
- doc/first_steps/welcome_screen/*
### Automated testing
As said above, _changes in_ our automated test suite can already help
identify the bits of documentation that need an update. To boost this
capability we could:
- Add steps to our existing tests to validate the screenshots we have
in our documentation. This seems doable but not trivial:
- Screenshots embed context, e.g. window placement and elements
that change all the time; so we would have to extract the
relevant part of each screenshot (for example based on
coordinates encoded somewhere).
- Credentials used when taking screenshots (e.g. Pidgin) must be
the ones used by the isotesters.
- Screenshots in our documentation are resized to 66% so we need
to resize them back. This is quite easy, but the information
lost in the double-resizing process may prevent Sikuli from
finding it on the screen.
- Add automated tests that specifically check things we document.
- Try to make our Gherkin scenarios closer to end-user documentation.
### Manual testing of the documentation
Which part of the documentation shall be tested, when, and by whom?
The sources of information listed above should help focus this effort
on the relevant parts of our documentation.
In many cases, a quick look will be enough, and following the entire
doc page to the letter won't be needed. But likely stricter testing
will be needed from time to time.
Random ideas:
- Aim for automation: feed the aforementioned metadata store with
information gathered during manual testing, so that we need less
and less manual testing as time goes.
- Prioritize: focus on core pages.
- Spread the load between different release, i.e. test some pages for
some release and some others for another release.
### Dismissed options
#### Manual testing of Tails releases
In theory our manual test suite could also help: some of the manual
tests could be performed by following our end-user documentation to
the letter. But I've not been able to find any single candidate test
case for this, so let's forget it.
#### Users and Help Desk
Users and Help Desk are also in a good position to notice outdated
bits and report them back, but they can only do so after the fact,
while ideally we'd rather update the documentation _before_ the
corresponding changes are released to our users. So if we receive such
notifications, e.g. at RC time, fine, but let's not count on it
too much.