Usability tests of first-time use (Round 2)

Ideas of things to test:

• New warnings
• Tor diagram (#18751)

In the end, I didn't test:

• The new Persistent Storage (#18648 (closed)), because I want to test again the whole installation flow, including the verification
• The old Persistent Storage, because we have enough findings from December already
• DuckDuckGo searches on our website, because search is broken right now (#19105 (closed))
• Installation from Windows, because I don't have a spare Windows laptop and it's hard to get people to bring theirs during the tests
• Website on mobile. I didn't set up to test this properly.
• Wi-Fi in GNOME (network-manager-applet#83 and network-manager-applet#84). Everybody goes through Tor Browser and Tor Connection now, so it's not a problem for us anymore.
• The Activities overview vs. the Applications menu. People mostly use the Applications menu. Have both is not a problem.

TODO:

• Ask people about their understanding of the autoconfig: what information is shared with whom?
• Test the UX when time sync fails: how is the UX? what can we do better?
• Make sure that the wording of the Wi-Fi notification works fine
  • "Configure Only Once" was confusing to boyska. He thought that it was persisting only this Wi-Fi network

Scheduled for August 18 to 21.

• Recruiting
  • Schedule slots (3 hours, with actual tests from 00:30 to 02:30)
  • Create screening survey
  • Test screening survey
  • Send link to screening survey
  • Select participants using screening spreadsheet in ux/private
  • Answer to all candidates using email templates
  • Only confirm after sound check
• Preparation
  • Prepare the tasks
  • Prepare a fallback bridge in QR code
  • Adapt research information sheet
  • Adapt consent form
  • Adapt checklist
    • Explicit reset procedure
    • Explicit recording procedure
  • Bring mouse and keyboard (remove from checklist)
  • Test the recording setup
  • Prepare incentives
  • Print
    • Research information sheet
    • Consent forms
    • Tasks
• Analysis
  • Create rainbow table
  • Fill in demographics tab of rainbow table
  • Check issues that were reported in the last round of similar tests
  • Mark new issues
  • Document solved issues
  • Link rainbow table from here: https://gitlab.tails.boum.org/tails/ux/-/blob/master/first-time/rainbow_table_installation_2022_08_Sao_Paulo.fods
  • Document on GitLab
    • Document findings in existing issues
    • Create missing issues
    • Link all issues to this one
    • Link related issues together
    • Verify labels on all issues
  • Prioritize work together with the developers
    • Document benefit
    • Document cost
    • Identify top-priority tasks
    • Identify low-hanging fruits
  • Prepare video clips
  • Share highlights on tails-dev
  • Write public report: https://lists.autistici.org/message/20221012.181235.a673efeb.en.html
• Wrap up
  • Improve template screening survey
  • Improve template rainbow table
  • Save candidates in email database
  • Add to achievements.mdwn
  • Get reimbursed for travel, incentives, etc.
  • Improve checklist
    • a portable operating system that you install on a USB stick and that protects
    • Leave at any time
  • Write notes about organizing
    • 3 hours slots
    • Having 2 tests a day and spread out to fix most obvious issues between them
    • Best days of the week

Top priority issues:

• #19179
• #19180
• #19181
• #19182
• #19184
• #19185
• #19188
• #19195
• #19203
• #19197

Low-hanging fruits:

• #19194 (closed)
• #19196
• #19205 (closed)