Reconsider who does the QA during the release process
For 5.1 I've done the full QA (aka. "manual test suite" + automated test suite). The manual QA only took me 1 hour.
I was very surprised it was this quick. It makes me wonder why we bother delegating this to multiple other Tails contributors who each have little time for Tails.
Drawbacks of the current setup:
- Manual testers are not available full time for Tails, by far, so:
- They are rarely available at the time that would be the most convenient for the release process ⇒ the RM often has to wait for them (or to eventually decide to do the tests themselves).
- Reciprocally, it may cost these manual testers quite some effort to accommodate the needs of the RM, just to clock less than 1 hour of work.
- Each tester duplicates work: downloading, verifying, and installing the image (which they might not do otherwise)
- Coordination overhead
- Administrative & accounting overhead
- These manual testers cannot do some of the QA checks ("Changes", automated test suite), so a developer/RM must chime in anyway.
- Communication overhead, to analyze the problems reported by manual testers, clarify misunderstandings, reproduce, etc.
- Having to run the automated test suite, and analyze the results, adds to the already heavy RM's workload.
- RM'ing is somewhat of an isolated/isolating task relatively to the activities of the FT+RM team.
Advantages of the current setup:
- It saves developers/RMs a little bit of time.
- Perhaps it creates a warm and fuzzy feeling of working as a team, beyond our usual team boundaries. (I used to feel this way but it's been a while since it happened.)
- Other FT+RM folks can mostly ignore the fact a release is ongoing and focus on other aspects of their work.
At this point, I'm tempted to propose that once the FT+RM team capacity allows this, the FT+RM team members who are not the RM for the current release are responsible for doing this QA (both manual & automated; and possibly also for being the TR, so we can schedule this together with the RM shifts, instead of having to find a TR every 4 weeks).
In the worst case scenario, none of them will be around so the RM will have to spend this extra 1h doing manual QA (but compared to the status quo, they won't have to prepare a pad, send emails, and coordinate with manual testers).