Usability tests of first-time use (Round 2)
Ideas of things to test:
-
New warnings -
Tor diagram (#18751)
In the end, I didn't test:
- The new Persistent Storage (#18648), because I want to test again the whole installation flow, including the verification
- The old Persistent Storage, because we have enough findings from December already
- DuckDuckGo searches on our website, because search is broken right now (#19105 (closed))
- Installation from Windows, because I don't have a spare Windows laptop and it's hard to get people to bring theirs during the tests
- Website on mobile. I didn't set up to test this properly.
- Wi-Fi in GNOME (network-manager-applet#83 and network-manager-applet#84). Everybody goes through Tor Browser and Tor Connection now, so it's not a problem for us anymore.
- The Activities overview vs. the Applications menu. People mostly use the Applications menu. Have both is not a problem.
TODO:
- Ask people about their understanding of the autoconfig: what information is shared with whom?
- Test the UX when time sync fails: how is the UX? what can we do better?
- Make sure that the wording of the Wi-Fi notification works fine
- "Configure Only Once" was confusing to boyska. He thought that it was persisting only this Wi-Fi network
Scheduled for August 18 to 21.
-
Recruiting -
Schedule slots (3 hours, with actual tests from 00:30 to 02:30) -
Create screening survey -
Test screening survey -
Send link to screening survey -
Select participants using screening spreadsheet in ux/private -
Answer to all candidates using email templates -
Only confirm after sound check
-
-
Preparation -
Prepare the tasks -
Prepare a fallback bridge in QR code -
Adapt research information sheet -
Adapt consent form -
Adapt checklist -
Explicit reset procedure -
Explicit recording procedure
-
-
Bring mouse and keyboard(remove from checklist) -
Test the recording setup -
Prepare incentives -
Print -
Research information sheet -
Consent forms -
Tasks
-
-
-
Analysis -
Create rainbow table -
Fill in demographics tab of rainbow table -
Check issues that were reported in the last round of similar tests -
Mark new issues -
Document solved issues -
Link rainbow table from here: https://gitlab.tails.boum.org/tails/ux/-/blob/master/first-time/rainbow_table_installation_2022_08_Sao_Paulo.fods -
Document on GitLab -
Document findings in existing issues -
Create missing issues -
Link all issues to this one -
Link related issues together -
Verify labels on all issues
-
-
Prioritize work together with the developers -
Document benefit -
Document cost -
Identify top-priority tasks -
Identify low-hanging fruits
-
-
Prepare video clips -
Share highlights on tails-dev
-
-
Wrap up -
Improve template screening survey -
Improve template rainbow table -
Save candidates in email database -
Add to achievements.mdwn -
Get reimbursed for travel, incentives, etc. -
Improve checklist -
a portable operating system that you install on a USB stick and that protects -
Leave at any time
-
-
Write notes about organizing -
3 hours slots -
Having 2 tests a day and spread out to fix most obvious issues between them -
Best days of the week
-
-
Top priority issues:
Low-hanging fruits: