tails issueshttps://gitlab.tails.boum.org/tails/tails/-/issues2024-03-28T16:18:28Zhttps://gitlab.tails.boum.org/tails/tails/-/issues/20122Weblate commits buggy translations to Git for strings with "notranslate" label2024-03-28T16:18:28ZintrigeriWeblate commits buggy translations to Git for strings with "notranslate" labelSee https://gitlab.tails.boum.org/tails/tails/-/jobs/105679. On https://translate.tails.boum.org/translate/tails/mediapart/pl/?checksum=3f90c769f86db251&sort_by=-priority,position I see the "notranslate" label.
- [x] Short-term: revert ...See https://gitlab.tails.boum.org/tails/tails/-/jobs/105679. On https://translate.tails.boum.org/translate/tails/mediapart/pl/?checksum=3f90c769f86db251&sort_by=-priority,position I see the "notranslate" label.
- [x] Short-term: revert these buggy translations
- [ ] Longer-term: can we enforce "notranslate"?
cc @zenTails_6.2https://gitlab.tails.boum.org/tails/tails/-/issues/20202Improve the translation process and documentation2024-02-14T14:50:51ZZen FuImprove the translation process and documentationIn Dec 2023 we ran a small survey to find out what was the current state of translation teams:
- tails#19999+
- [Results and initial analysis](https://gitlab.tails.boum.org/tails/summit/-/wikis/Teams/Translation_Platform/State-of-transl...In Dec 2023 we ran a small survey to find out what was the current state of translation teams:
- tails#19999+
- [Results and initial analysis](https://gitlab.tails.boum.org/tails/summit/-/wikis/Teams/Translation_Platform/State-of-translation-teams-on-Dec-2023)
This issue intends to come up with changes that could be implemented to improve the translation process and documentation, as a follow-up of those findings. We shoud also consider changes to the role of the Translation Platform maintainers (see [this comment](https://gitlab.tails.boum.org/tails/tails/-/issues/20047#note_221680) for context).https://gitlab.tails.boum.org/tails/tails/-/issues/16974Improve Weblate UX for first-time visitors2023-12-31T10:55:39ZintrigeriImprove Weblate UX for first-time visitors_Originally created by @intrigeri on [#16974 (Redmine)](https://public-redmine-archive.tails.boum.org/code/issues/16974)_
I find the Weblate homepage for non-logged-in users quite confusing at
the moment.
Besides, even though some basi..._Originally created by @intrigeri on [#16974 (Redmine)](https://public-redmine-archive.tails.boum.org/code/issues/16974)_
I find the Weblate homepage for non-logged-in users quite confusing at
the moment.
Besides, even though some basic contributions can be done anonymously,
having a user account is the only way to build a work track record,
which is our only way to build trust, and eventually give reviewer
credentials to someone. So IMO we should strongly encourage users to log
in.
So I’d like the homepage of Weblate, for non-logged-in users, to be
mostly a login/register page, possibly with a “Contribute anonymously”
link.
Blueprint: https://tails.boum.org/blueprint/translation_platform/#ux
### Related issues
- **Related to** tails/tails#10037
- **Blocked by** tails/sysadmin#16943https://gitlab.tails.boum.org/tails/tails/-/issues/17655Review Weblate's translation workflow and its integration with Git repos and ...2023-08-15T20:44:17ZZen FuReview Weblate's translation workflow and its integration with Git repos and websites_Originally created by @zen on [#17655 (Redmine)](https://public-redmine-archive.tails.boum.org/code/issues/17655)_
For the integration of the Translation Platform with the main website,
we originally wanted that:
* Anyone can suggest...._Originally created by @zen on [#17655 (Redmine)](https://public-redmine-archive.tails.boum.org/code/issues/17655)_
For the integration of the Translation Platform with the main website,
we originally wanted that:
* Anyone can suggest.
* Logged in users can vote on suggestions.
* Most voted suggestions would appear in the staging website.
* Reviewers can accept translations.
* Accepted translations appear in the main website automatically.
# Current state: Weblate workflow vs Git repository vs website rendering
The following table attempts to address the following questions:
* Which of the possible states of strings (Needs editing, Waiting for review, Approved) gets committed to the repo?
* Which of these states gets committed but is marked as fuzzy?
* Which of these states get rendered in the main website?
* Which of these states get rendered in the staging website?
The Weblate workflow did not change since we first installed ([3.5.1](https://docs.weblate.org/en/weblate-3.5.1/workflows.html)) it up to the latest version ([4.12.2](https://docs.weblate.org/en/weblate-4.12.2/workflows.html) nowadays).
Using the info from [Weblate Doc](https://docs.weblate.org/en/latest/workflows.html?highlight=fuzzy#translation-states) and comparing strings in Weblate, Tails Git repo and the main website, this is the current state of affairs:
| Weblate state | Committed to Git? | Marked as fuzzy? | Rendered in the main website? | Rendered in staging website? |
| ------------------ | ----------------- | ---------------- | ------------------------- | ----------------- |
| Suggestions | ❌ | (doesn't apply) | ❌ | ✅ |
| Untranslated | ❌ | (doesn't apply) | ❌ | ❌ |
| Needs editing /Fuzzy | ✅ | ✅ | ❌ | ❌ |
| Waiting for review / Translated | ✅ | ❌ | ✅ | ✅ (if not committed to git) |
| Approved | ✅ | ❌ | ✅ | ✅ (if not committed to git already) |
# Analysis of the current state
- There's a difference between what Weblate and what Tails understand as "reviewed strings":
- For Weblate, a string "Waiting for review" [is a valid translation](https://docs.weblate.org/en/weblate-4.9.1/workflows.html#translation-states):
- "It is stored in the file as a valid translation" and, because of that, it gets rendered in our main website (as well as in the staging website, unless there are suggestions for it).
- The difference between "Waiting for review" and "Approved" only lives in the Weblate database.
- For Tails, an unreviewed string needs "[to be accepted by reviewers to go live onto the main Tails website](https://tails.boum.org/contribute/how/translate/with_translation_platform/#index3h2)".
- Right now:
- We're only using [3 "kinds" of roles](https://gitlab.tails.boum.org/tails/puppet-tails/-/blob/aa4ed4f4f87e0faae31feea62eeed2a180661ce4/files/weblate/config/weblate_permissions.yaml) in Weblate related to actually translating (not counting other functionalities as managing glossaries, etc): Guests, Users and (global and per-language) Reviewers.
- `Guests` and `Users` can only suggest, and suggestions determine what gets rendered on the staging website.
- `Reviewers` can accept suggestions to turn them into any of the other states described in the table above.
- As emmapeel [wrote](https://gitlab.tails.boum.org/tails/tails/-/issues/17655#note_149467):
- This workflow probably wasn't in place when Weblate started to be evaluated.
- We currently use a bad mapping from Weblate workflow features to Tails translation workflow.
- The current setup causes several usability problems.
## Assumptions made by our current integration code
**Weblate → Canonical:**
1. **Assumption:** All changes made via Weblate web interface that are committed to the Weblate Git repo will eventually be merged into the Canonical repo (unless they conflict with changes made to Canonical in the meantime).
**Consequence:** Canonical gets all translation content and state changes made in the Weblate interface (not counting conflicts and suggestions): "Needs edit" translations are committed as fuzzy (and thus invalidated, rendering the main language instead), and "Needs review" and "Approved" translations are committed as "valid" translations.
2. **Assumption:** Weblate and our custom code only change `.po` files in the Weblate and Integration repos.
**Consequence:** Pushing changes to Canonical fails if anything other than `.po` files is changed.
3. **Assumption:** When there are conflicts, changes from Canonical have precedence over changes from Weblate.
**Consequence:** Conflicting translations made through Weblate are lost.
**Canonical → Weblate:**
1. **Assumption:** The update of the translations in the Weblate backend is not made automatically by Weblate (i.e. `AUTO_UPDATE` is off, so Weblate-initiated Git pulls and component discovery are never triggered automatically).
**Consequences:**
- We need a custom mechanism to update the Weblate backend with changes from Canonical.
- Such mechanism is responsible for maintaining integrity between the Weblate backend and repository, that is, to ensure that when it finishes all translations content and state are the same in both places.
2. **Assumption:** Source string changes are not automatically reflected in the Canonical repo for languages that are not activated in the main website. (The `.po` files for languages that are activated in the main website are automatically updated by the production Ikiwiki.)
**Consequence:** Our custom code needs to propagate relevant changes to source strings to the languages that are not activated in the main website, but that are activated in Weblate and the staging website.
3. **Assumption:** Changes in any source string or translation in a component cause the whole component to be updated in the Weblate backend.
**Consequence:** If the state of a translation in the Weblate backend differs from its state in the `.po` file (for example if we implement only committing of "Approved" strings), then a change in another translation in the same component may cause data loss in the Weblate backend.
# Analysis of the proposals
In terms of UX, @emmapeel proposes:
- `Guests` and unprivileged `Users` can only suggest, and suggestions only live in Weblate.
- `Translators` (which are "approved" `Users`, added to the `Translators` group) can determine what gets rendered in the staging website.
- `Reviewers` can determine what is accepted and included in the main website.
Implementation-wise, because there's no easy way to change how Weblate behaves, we have to face the problem that "Waiting for review" strings are treated as "valid" translations, thus always get committed to the Weblate repository without the fuzzy tag (see the table above).
From the discussion below in this issue, i understand that @emmapeel's idea is to either:
1. Change the logic of our integration scripts and integration repository(ies) in a way that only "Approved" strings are propagated from Weblate to the Tails canonical repository, or
2. Change Weblate behavior upstream, which would mean convince Weblate devs that only strings in "Approved" state should be committed to the repository (effectively changing the first and second columns in the table above).
In any of the above cases:
- The new `Translator` role would be able to add strings directly to the "Waiting for review" state.
- Such strings would be rendered in the staging website only.
- Only `Reviewers` would be able to turn such strings into "Approved" state.
## Proposal 1: Replace merging from Weblate by importing "Approved" strings
- Replace the "Merge Weblate Changes" step by an "Integrate Approved Strings" step.
- Update the "Update Weblate Components" step to account for the fact that the Weblate repo will never converge with the Canonical repo.
- Change Weblate roles/permissions so that Translators can add strings to "Needs review".
- Change the generation of the Staging website to use strings in "Needs review" state instead of suggestions.
Pros: improved UX, satisfies security policy.
Cons: high cost of implementation, needs changes in many places of our integration code.
## Proposal 2: Foster the implementation of a write-policy in Weblate
- Wait or help Weblate to implement a [write-policy](https://github.com/WeblateOrg/weblate/issues/3745).
- Change Weblate config so only "Approved" strings get committed to the Weblate repository.
- Profit.
Pros: improved UX, satisfies security policy, benefits more Weblate users.
Cons: needs knowledge of Weblate codebase, needs acceptance by Weblate, needs currently unknown amount of resources.
## Proposal 3: Stick with the same workflow, improve communication with Reviewers
Train Reviewers to:
- Do their own translations by means of suggestions.
- Periodically check for new suggestions.
- Do not use the "Needs review" state: every accepted suggestion must be reviewed by the time of acceptance and immediately marked and saved as "Approved".
Pros: cheap solution.
Cons: bad UX.
# Next steps
Preparation and evaluation:
- [x] Create a table comparing weblate states, storage in file in Git repo, fuzziness, and rendering in the main and staging websites.
- [x] Compare the current state with emmapeel's proposal
- [x] List what would need to be changed to implement the proposal
- [x] Ask for review, confirmation and input from folks, maybe iterate to improve the proposal
- [x] Write a proposal that only changes workflow and permissions
- [x] Map assumptions that our code/design relies on
- [x] Maybe: Prototype something that commits only "Approved" strings to Git
- [ ] Decide next stepsTranslation Platform in Productionhttps://gitlab.tails.boum.org/tails/tails/-/issues/16872Make it obvious to visitors of the staging website that it is not trustworthy2023-07-20T21:16:51ZAnonymousMake it obvious to visitors of the staging website that it is not trustworthy_Originally created by @Anonymous on [#16872 (Redmine)](https://public-redmine-archive.tails.boum.org/code/issues/16872)_
This was mentioned by hefee as a nice to have.
### Related issues
- **Related to** tails/tails#150..._Originally created by @Anonymous on [#16872 (Redmine)](https://public-redmine-archive.tails.boum.org/code/issues/16872)_
This was mentioned by hefee as a nice to have.
### Related issues
- **Related to** tails/tails#15080
- **Related to** tails/sysadmin#15360
- **Related to** tails/tails#15080Translation Platform in Productionhttps://gitlab.tails.boum.org/tails/tails/-/issues/18988Re-enable i18nspector unknown-message-flag check2023-05-17T14:55:54ZintrigeriRe-enable i18nspector unknown-message-flag checkOnce https://github.com/jwilk/i18nspector/issues/11 is fixed in a version of i18nspector that we can upgrade to on every relevant system (i.e. they all run Bookworm or newer), we should revert tails/jenkins-tools!1.
Relevant systems:
...Once https://github.com/jwilk/i18nspector/issues/11 is fixed in a version of i18nspector that we can upgrade to on every relevant system (i.e. they all run Bookworm or newer), we should revert tails/jenkins-tools!1.
Relevant systems:
- [ ] RMs
- Ratioanle: We run `lint_po` as part of our release process.
- [x] intrigeri
- [ ] boyska
- [x] Jenkins?
- No: we've migrated these checks to GitLab CI.https://gitlab.tails.boum.org/tails/tails/-/issues/18935Ask advice from folks in the translation industry about paying translators2022-08-19T14:20:32ZintrigeriAsk advice from folks in the translation industry about paying translators- [x] Ask Erin from Localization Lab
- [x] Ask some of sajolida's friends in the translation industry
- [x] G from Luxembourg- [x] Ask Erin from Localization Lab
- [x] Ask some of sajolida's friends in the translation industry
- [x] G from Luxembourghttps://gitlab.tails.boum.org/tails/tails/-/issues/18879Improve translation platform scripts documentation2022-03-11T11:03:54ZZen FuImprove translation platform scripts documentationFor [our custom translation platform scripts](https://gitlab.tails.boum.org/tails/puppet-tails/-/tree/master/files/weblate/scripts):
- [ ] Add a README file that briefly explains the relation between them, possibly linking to the design...For [our custom translation platform scripts](https://gitlab.tails.boum.org/tails/puppet-tails/-/tree/master/files/weblate/scripts):
- [ ] Add a README file that briefly explains the relation between them, possibly linking to the design doc
- [ ] Improve in-script comments to describe what each script does and explain more intricate partshttps://gitlab.tails.boum.org/tails/tails/-/issues/15454Improve PO rules: Add check for square bracket count2022-03-11T10:47:51ZemmapeelImprove PO rules: Add check for square bracket count_Originally created by @emmapeel on [#15454 (Redmine)](https://public-redmine-archive.tails.boum.org/code/issues/15454)_
Our beautiful translators are human after all, and sometimes they don’t
copy the ikiwiki syntax quite right.
A sim..._Originally created by @emmapeel on [#15454 (Redmine)](https://public-redmine-archive.tails.boum.org/code/issues/15454)_
Our beautiful translators are human after all, and sometimes they don’t
copy the ikiwiki syntax quite right.
A simple check could be added to make sure the source and target strings
have the same amount of \[ and \] characters.
Reasoning:
\- If there is an error on the number of brackets, ikiwiki will build
without errors but the interface will be broken.https://gitlab.tails.boum.org/tails/tails/-/issues/18872Make it easier for Weblate users to preview suggestions on the staging website2022-03-08T06:24:14ZZen FuMake it easier for Weblate users to preview suggestions on the staging website(This issue merges tails#15080 and sysadmin#15360.)
# Requirements
The [staging website](https://staging.tails.boum.org) is an important tool for translators to self-learn and improves the translation quality:
* allow Translators to p...(This issue merges tails#15080 and sysadmin#15360.)
# Requirements
The [staging website](https://staging.tails.boum.org) is an important tool for translators to self-learn and improves the translation quality:
* allow Translators to preview their suggestions before they are accepted by Reviewers.
* allow all users see the context in which components/strings appear to have a better understanding of how it should be translated.
For that, is must:
- be readily available for a component/string from the Weblate interface, otherwise chances are translators won’t use the
staging website much (if at all).
- not lag behind the changes one is reviewing most of the time, otherwise chances are that translators get used to ignore it because it’s not useful.
# Implementation steps
- [ ] Add to each Weblate component links to its corresponding pages in the staging website (take our multiple uses of `[[!inline]]` into account).
- [ ] Add a link to the staging website in other places in Weblate with an explanatory sentence (eg. login page, banner, etc).
- [ ] Increase the frequency of staging website update (currently takes 1h and runs once a day)
- Consider faster ways (currently takes 1h):
- Make sure IkiWiki is refreshing instead of rebuilding.
- Consider only updating subsets of languages and/or files.
- Use a systemd timer instead of cron (avoids concurrent runs)
- Avoid monopolizing a vCPU permanently (eg. if it takes 1h, run once every 4 hours).
- [ ] Ensure the staging website build is resilient enough (i.e. doesn't break often -- some checks have already been implemented for that).
- [ ] Fix documentation accordingly.https://gitlab.tails.boum.org/tails/tails/-/issues/16862Remove workaround for GitPython incorrectly populating blobs for submodule ch...2022-02-24T15:16:56ZhefeeRemove workaround for GitPython incorrectly populating blobs for submodule changes, when using diff_Originally created by @hefee on [#16862 (Redmine)](https://public-redmine-archive.tails.boum.org/code/issues/16862)_
Our weblate\_update\_git.py script implements an own merge strategy and
so it needs to access the elements within the ..._Originally created by @hefee on [#16862 (Redmine)](https://public-redmine-archive.tails.boum.org/code/issues/16862)_
Our weblate\_update\_git.py script implements an own merge strategy and
so it needs to access the elements within the diff, to decide how to
merge. Currently we need a workaround for submodule changes, done by
remote (main git).
The upstream issue is tracked at:
<https://github.com/gitpython-developers/GitPython/issues/891>
If that is solved, we can remove the workaround.
For puppet-tails@a36884e96ce36d3c5e1c348d9813fffce46e4350
<code class="python">
480 if not isWikiPo(path):
481 try:
482 blob = i.b_blob
483 index.updateFile(path, blob)
484 except ValueError:
[...]
</code>https://gitlab.tails.boum.org/tails/tails/-/issues/18855Consider giving translators the "Define author of translation upload" permission2022-02-18T10:59:09ZintrigeriConsider giving translators the "Define author of translation upload" permissionFrom tails/tails#17649:
To be discussed:
* Add “Define author of translation upload” to translators. It means, when you upload a po file, you should be able to give an author for it. This is good for attribution, because sometimes you ...From tails/tails#17649:
To be discussed:
* Add “Define author of translation upload” to translators. It means, when you upload a po file, you should be able to give an author for it. This is good for attribution, because sometimes you add files somebody else has translated. This permission should be for translators, not for reviewers. You don’t need to upload a translation while you review.
Also, every change decided here would have to be implemented in tails/sysadmin#17338.Translation Platform in Productionhttps://gitlab.tails.boum.org/tails/tails/-/issues/16871Define a strategy to disable non-maintained languages on Weblate2022-02-18T10:46:53ZAnonymousDefine a strategy to disable non-maintained languages on Weblate_Originally created by @Anonymous on [#16871 (Redmine)](https://public-redmine-archive.tails.boum.org/code/issues/16871)_
We need to define a strategy to delete non-maintained languages, meaning
languages which do not see progress on We..._Originally created by @Anonymous on [#16871 (Redmine)](https://public-redmine-archive.tails.boum.org/code/issues/16871)_
We need to define a strategy to delete non-maintained languages, meaning
languages which do not see progress on Weblate for a certain amount of
time and which are not enabled in production.https://gitlab.tails.boum.org/tails/tails/-/issues/16853Research if we could support translating other branches via Weblate2022-02-18T10:29:43ZAnonymousResearch if we could support translating other branches via Weblate_Originally created by @Anonymous on [#16853 (Redmine)](https://public-redmine-archive.tails.boum.org/code/issues/16853)_
currently, only the master branch is translateable. So for upcoming
versions, translators will still have to use g..._Originally created by @Anonymous on [#16853 (Redmine)](https://public-redmine-archive.tails.boum.org/code/issues/16853)_
currently, only the master branch is translateable. So for upcoming
versions, translators will still have to use git. It might be useful to
allow translating other branches.https://gitlab.tails.boum.org/tails/tails/-/issues/17205Consider managing translations of tails.git:po/tails.pot in Weblate2022-02-18T10:28:44ZintrigeriConsider managing translations of tails.git:po/tails.pot in Weblate_Originally created by @intrigeri on [#17205 (Redmine)](https://public-redmine-archive.tails.boum.org/code/issues/17205)_
See the “\[Tails-l10n\] Include languages with 0 reviewed strings”
thread for the initial motivation for creating ..._Originally created by @intrigeri on [#17205 (Redmine)](https://public-redmine-archive.tails.boum.org/code/issues/17205)_
See the “\[Tails-l10n\] Include languages with 0 reviewed strings”
thread for the initial motivation for creating this ticket.
There surely are other pros & cons for this idea, hence “consider”.
### Related issues
- **Related to** tails/tails#17568https://gitlab.tails.boum.org/tails/tails/-/issues/16761Move scripts out of the wiki/src directory2022-02-18T10:26:53ZhefeeMove scripts out of the wiki/src directory_Originally created by @hefee on [#16761 (Redmine)](https://public-redmine-archive.tails.boum.org/code/issues/16761)_
As we want to create a autopush for Weblate, it may be problematic, if
scripts try to load configurations from directo..._Originally created by @hefee on [#16761 (Redmine)](https://public-redmine-archive.tails.boum.org/code/issues/16761)_
As we want to create a autopush for Weblate, it may be problematic, if
scripts try to load configurations from directories and weblate drops a
`*po` file into it.
List of files with executable flag:
find wiki -perm -u+rwx -type f | grep -v ".po$"
wiki/src/lib/js/jquery.min.js
wiki/src/contribute/l10n_tricks/doc-whatchanged
wiki/src/contribute/l10n_tricks/transifex_translators.sh
wiki/src/contribute/l10n_tricks/language_statistics.sh
wiki/src/contribute/l10n_tricks/git-clean-po
wiki/src/contribute/l10n_tricks/unifyPo.py
wiki/src/contribute/how/documentation/compress-image.sh
wiki/src/contribute/how/documentation/qrcode-decode.sh
wiki/src/contribute/how/documentation/qrcode-encode.sh
wiki/src/blueprint/greeter_revamp_UI/mockups/mockup.py
wiki/src/blueprint/mumble-server
### Related issues
- **Related to** tails/sysadmin#15402https://gitlab.tails.boum.org/tails/tails/-/issues/17919[weblate] when pages are moved, the component name stays as the old path, cre...2021-05-11T10:44:52Zemmapeel[weblate] when pages are moved, the component name stays as the old path, creating confusionWith https://gitlab.tails.boum.org/tails/tails/-/issues/15185 we made sure that if a page is moved, it is also moved in weblate.
But it seems since a while the 'component name' in weblate is not being updated to the new path, which crea...With https://gitlab.tails.boum.org/tails/tails/-/issues/15185 we made sure that if a page is moved, it is also moved in weblate.
But it seems since a while the 'component name' in weblate is not being updated to the new path, which creates confusion when trying to look at the page in the staging or main website, for example.
An example is: https://translate.tails.boum.org/projects/tails/wikisrcdocfirst_stepsstartup_optionspo/es/
This file used to be located at https://tails.boum.org/doc/first_steps/startup_options/ but now is at https://tails.boum.org/doc/first_steps/welcome_screen/