Changes

Page history
Add Glean authored by intrigeri's avatar intrigeri
Hide whitespace changes
Inline Side-by-side
metrics.md
View page @ c0335c1b
[[_TOC_]]
# Privacy-preserving data collection techniques
- https://divviup.org/
- [LWN's "Divvi Up: privacy-respecting telemetry aggregation"](https://lwn.net/Articles/983843/)
- https://research.swtch.com/telemetry-intro
- https://en.wikipedia.org/wiki/Differential_privacy
- https://lwn.net/ml/fedora-devel/CAJqbrbeOZrHvYjvMCc=qGZD_VXBs3-qReeirr+F8t01Eq1sYhw@mail.gmail.com/
- https://blogs.gnome.org/wjjt/2023/07/05/endless-oss-privacy-preserving-metrics-system/
- https://theevilskeleton.gitlab.io/2023/07/16/opt-in-telemetry-and-asking-users-for-feedback-may-not-work-in-practice.html
# Archive
## Current status
- Metrics are collected:
- Manually
- Only by intrigeri
- Into text files
- In the blueprint wiki
- We are collecting:
- Community metrics
- Subscribers to amnesia-news
- ~~Messages on tails-dev~~
- Subscribers on tails-dev
- Twitter followers
- Help desk
- ~~Messages on tails-bug~~
- ~~WhisperBack reports~~
- Releases
- Date
- ~~Size~~ (already available elsewhere)
- Usage
- Boots
- Downloads of the OpenPGP signature
## Concerns
- The workload is not shared
- ~~The data is not systematically published (some are in the monthly reports)~~
- Manual process
## Taking a step back...
- Why are we collecting these metrics?
- What shall we do with them?
- What metrics are most important to collect?
- What kind of tools and infrastructure shall we use to manage these metrics?
- Where do we store them? Who has access to them?
## Ideas of abstract goals behind the numbers
From a brainstorming session, with no priorities or check on whether
these are possible and worth it to gather.
- Find how many users we do have? What are the trends?
- How many bugs? Per category? Are we carrying around old bugs? Are we adding new bugs?
- Are we reducing errors for users ?
- Are we reducing the time needed for task completion?
- Help desk reports
- How technically skilled do you need to be to use Tails? How technically skilled are our users?
- Donations
- How many donations do we get?
- How much are people donation?
- When do we get donation?
- Incoming links? Who is endorsing Tails?
- Where are the users located?
- How big, diverse and active is our community?
- Statistics on private and public meetings:
- How many people attended?
- How many meetings?
- How many newcomers?
- Conferences
- Attended?
- How many mentions?
- How fast to we solve problems?
- Security issues
- How organized and accountable we are?
- Conversion rate
- User agents
- Heat map of the website usage
- Are people coming to download the image?
- Are people coming to read the documentation?
- Why are people interested in Tails?
## Possible ways to gather this data
- Ticket tracker for help desk
- Web analytic
- Surveys
## References
- [The definition of vanity metrics and how to identify them](https://www.tableau.com/learn/articles/vanity-metrics)
- [Translating UX Goals into Analytics Measurement Plans](https://www.nngroup.com/articles/ux-goals-analytics/)
[[_TOC_]]
# Privacy-preserving data collection techniques
- https://divviup.org/
- [LWN's "Divvi Up: privacy-respecting telemetry aggregation"](https://lwn.net/Articles/983843/)
- https://research.swtch.com/telemetry-intro
- https://en.wikipedia.org/wiki/Differential_privacy
- https://lwn.net/ml/fedora-devel/CAJqbrbeOZrHvYjvMCc=qGZD_VXBs3-qReeirr+F8t01Eq1sYhw@mail.gmail.com/
- https://blogs.gnome.org/wjjt/2023/07/05/endless-oss-privacy-preserving-metrics-system/
- https://theevilskeleton.gitlab.io/2023/07/16/opt-in-telemetry-and-asking-users-for-feedback-may-not-work-in-practice.html
- [Mozilla's Glean](https://github.com/mozilla/glean/) (not sure how privacy-preserving it is)
# Archive
## Current status
- Metrics are collected:
- Manually
- Only by intrigeri
- Into text files
- In the blueprint wiki
- We are collecting:
- Community metrics
- Subscribers to amnesia-news
- ~~Messages on tails-dev~~
- Subscribers on tails-dev
- Twitter followers
- Help desk
- ~~Messages on tails-bug~~
- ~~WhisperBack reports~~
- Releases
- Date
- ~~Size~~ (already available elsewhere)
- Usage
- Boots
- Downloads of the OpenPGP signature
## Concerns
- The workload is not shared
- ~~The data is not systematically published (some are in the monthly reports)~~
- Manual process
## Taking a step back...
- Why are we collecting these metrics?
- What shall we do with them?
- What metrics are most important to collect?
- What kind of tools and infrastructure shall we use to manage these metrics?
- Where do we store them? Who has access to them?
## Ideas of abstract goals behind the numbers
From a brainstorming session, with no priorities or check on whether these are possible and worth it to gather.
- Find how many users we do have? What are the trends?
- How many bugs? Per category? Are we carrying around old bugs? Are we adding new bugs?
- Are we reducing errors for users ?
- Are we reducing the time needed for task completion?
- Help desk reports
- How technically skilled do you need to be to use Tails? How technically skilled are our users?
- Donations
- How many donations do we get?
- How much are people donation?
- When do we get donation?
- Incoming links? Who is endorsing Tails?
- Where are the users located?
- How big, diverse and active is our community?
- Statistics on private and public meetings:
- How many people attended?
- How many meetings?
- How many newcomers?
- Conferences
- Attended?
- How many mentions?
- How fast to we solve problems?
- Security issues
- How organized and accountable we are?
- Conversion rate
- User agents
- Heat map of the website usage
- Are people coming to download the image?
- Are people coming to read the documentation?
- Why are people interested in Tails?
## Possible ways to gather this data
- Ticket tracker for help desk
- Web analytic
- Surveys
## References
- [The definition of vanity metrics and how to identify them](https://www.tableau.com/learn/articles/vanity-metrics)
- [Translating UX Goals into Analytics Measurement Plans](https://www.nngroup.com/articles/ux-goals-analytics/)
\ No newline at end of file