sysadmin issueshttps://gitlab.tails.boum.org/tails/sysadmin/-/issues2023-12-07T14:47:30Zhttps://gitlab.tails.boum.org/tails/sysadmin/-/issues/18053Improve the way we monitor mirrors speeds and handle slow mirrors2023-12-07T14:47:30ZZen FuImprove the way we monitor mirrors speeds and handle slow mirrorsThe current approach with `check-mirrors.sh` doesn't give us a good historic overview of mirror speeds (only e-mail notifications), and we would like to improve that.
Initial proposal:
- Use https://exchange.nagios.org/directory/Plugin...The current approach with `check-mirrors.sh` doesn't give us a good historic overview of mirror speeds (only e-mail notifications), and we would like to improve that.
Initial proposal:
- Use https://exchange.nagios.org/directory/Plugins/Network-and-Systems-Management/check_speed-2Esh/details
- Set it up with the current latest image URL
- Iterate from there
- Sysadmins will be notified when a mirror is slow, then we can:
- Check history
- If it's always good, acknowledge for a couple of days
- Elif it's bumpy for a bit longer, adjust priority/weight and contact mirror admin
- Else remove it completely
Note: we could even move the canonical configuration of the mirror pool to hiera and have puppet generate the mirrors.json file so we don't have to keep mirrors.json and icinga config in sync manually.groentegroentehttps://gitlab.tails.boum.org/tails/sysadmin/-/issues/18052Some images in iso-history.t.b.o become temporarily inaccessible2024-02-03T11:24:57ZZen FuSome images in iso-history.t.b.o become temporarily inaccessibleReported by @anonym while RM'ing, experienced 403 while trying to access some Tails images:
- This was the first release after the [switch to use git-annex adjusted branches](https://gitlab.tails.boum.org/tails/sysadmin/-/issues/18040#n...Reported by @anonym while RM'ing, experienced 403 while trying to access some Tails images:
- This was the first release after the [switch to use git-annex adjusted branches](https://gitlab.tails.boum.org/tails/sysadmin/-/issues/18040#note_219652).
- Seems that there were at least 2 instances of the issue:
- One permanent, related to `.git/annex/objects` permissions, as well as the permissions of subdirectories and files.
- One temporary, possibly related to the git rebase dance made by git-annex when updating an adjusted branch.
- The iso-history repo was updated right before the temporary instance, but waited for images to be listed on https://iso-hiustory.t.b.o with full size.
Cc: @foundations-teamZen FuZen Fuhttps://gitlab.tails.boum.org/tails/sysadmin/-/issues/18051start using tails.net for e-mail2024-03-19T12:32:15Zgroentestart using tails.net for e-mailE-mails from boum.org are getting rejected because we don't have DMARC there and it's a pain to set up without access to the domainname. Let's use our fancy tails.net name instead and do mail properly.
Given that we now have the chance ...E-mails from boum.org are getting rejected because we don't have DMARC there and it's a pain to set up without access to the domainname. Let's use our fancy tails.net name instead and do mail properly.
Given that we now have the chance to start fresh without wonky mail routing, I propose we set up a dedicated VM for this on a machine outside of lizard, and preferably outside of that colo, just so mail will still work if nearly everything else fails. We'd need a VM with at least 2GB RAM, its own IP, and possibility to set reverse DNS from a trusted provider.
In terms of functionality, I propose we set up:
- relay/smarthost functionality for nodes inside our VPN using tails.net
- full DMARC shizzle, MTA-STS, and DANE
- schleuder lists running under tails.net
To slowly migrate our schleuder lists, we can for every boum.org schleuder list create a tails.net one, move the subscribers to the new list, and subscribe the old list to the new one.groentegroentehttps://gitlab.tails.boum.org/tails/sysadmin/-/issues/18050sajolida cannot SSH to LimeSurvey VM anymore2023-11-24T14:59:51Zsajolidasajolida@pimienta.orgsajolida cannot SSH to LimeSurvey VM anymoreToday, while sending the November newsletter, I couldn't SSH into the survey VM anymore.
I customized the unsubscription message on survey to be more generic and not talked about "surveys", as it does by default:
![image](/uploads/0829...Today, while sending the November newsletter, I couldn't SSH into the survey VM anymore.
I customized the unsubscription message on survey to be more generic and not talked about "surveys", as it does by default:
![image](/uploads/082941576a131e3513b3dddd25223ce6/image.png)
That's described in Step #11 of https://tails.net/contribute/how/user_experience/limesurvey/.
For some reason, the message was reset this time (and previous time as well). I guess that this step is not handle by the automatic upgrades anymore.
I tried to connect like usual:
```
OpenSSH_9.2p1 Debian-2+deb12u1, OpenSSL 3.0.11 19 Sep 2023
debug1: Reading configuration data /home/xxxxxx/.ssh/config
debug1: /home/xxxxxx/.ssh/config line 19: Applying options for *.onion
debug1: /home/xxxxxx/.ssh/config line 76: Applying options for unlehxtgqn5xg6ccxwnm5w3ddcrll5ctlavo7srjvkkpay67dv3sdzyd.onion
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: include /etc/ssh/ssh_config.d/*.conf matched no files
debug1: /etc/ssh/ssh_config line 21: Applying options for *
debug1: Executing proxy command: exec /bin/nc.openbsd -X 5 -x 127.0.0.1:9050 unlehxtgqn5xg6ccxwnm5w3ddcrll5ctlavo7srjvkkpay67dv3sdzyd.onion 22
debug1: identity file /home/xxxxxx/.ssh/id_rsa_sajolida type 0
debug1: identity file /home/xxxxxx/.ssh/id_rsa_sajolida-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_9.2p1 Debian-2+deb12u1
debug1: Remote protocol version 2.0, remote software version OpenSSH_9.2p1
debug1: compat_banner: match: OpenSSH_9.2p1 pat OpenSSH* compat 0x04000000
debug1: Authenticating to unlehxtgqn5xg6ccxwnm5w3ddcrll5ctlavo7srjvkkpay67dv3sdzyd.onion:22 as 'sajolida'
debug1: load_hostkeys: fopen /home/xxxxxx/.ssh/known_hosts2: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts2: No such file or directory
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: algorithm: sntrup761x25519-sha512@openssh.com
debug1: kex: host key algorithm: ssh-ed25519
debug1: kex: server->client cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug1: kex: client->server cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: SSH2_MSG_KEX_ECDH_REPLY received
debug1: Server host key: ssh-ed25519 SHA256:mG4yyJ1cSvHJj8GDrZS15PeVg4N+Nlfj09xApfPSeAk
debug1: load_hostkeys: fopen /home/xxxxxx/.ssh/known_hosts2: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts2: No such file or directory
debug1: Host 'unlehxtgqn5xg6ccxwnm5w3ddcrll5ctlavo7srjvkkpay67dv3sdzyd.onion' is known and matches the ED25519 host key.
debug1: Found key in /home/xxxxxx/.ssh/known_hosts:159
debug1: rekey out after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: rekey in after 134217728 blocks
debug1: get_agent_identities: bound agent to hostkey
debug1: get_agent_identities: agent returned 2 keys
debug1: Will attempt key: /home/xxxxxx/.ssh/id_rsa_sajolida RSA SHA256:ALq73t8MKPfgfOkE5OhzXIlo55jPVBoYEJEbg1HjFQE explicit agent
debug1: Will attempt key: /home/xxxxxx/.ssh/id_rsa RSA SHA256:l5S3x9mCBGLmrwcxQQJgJSSUoFCRp7c5X4HM4KCjxJk agent
debug1: SSH2_MSG_EXT_INFO received
debug1: kex_input_ext_info: server-sig-algs=<ssh-ed25519,sk-ssh-ed25519@openssh.com,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,sk-ecdsa-sha2-nistp256@openssh.com,webauthn-sk-ecdsa-sha2-nistp256@openssh.com,ssh-dss,ssh-rsa,rsa-sha2-256,rsa-sha2-512>
debug1: kex_input_ext_info: publickey-hostbound@openssh.com=<0>
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey
debug1: Next authentication method: publickey
debug1: Offering public key: /home/xxxxxx/.ssh/id_rsa_sajolida RSA SHA256:ALq73t8MKPfgfOkE5OhzXIlo55jPVBoYEJEbg1HjFQE explicit agent
debug1: Authentications that can continue: publickey
debug1: Offering public key: /home/xxxxxx/.ssh/id_rsa RSA SHA256:l5S3x9mCBGLmrwcxQQJgJSSUoFCRp7c5X4HM4KCjxJk agent
debug1: Authentications that can continue: publickey
debug1: No more authentication methods to try.
sajolida@unlehxtgqn5xg6ccxwnm5w3ddcrll5ctlavo7srjvkkpay67dv3sdzyd.onion: Permission denied (publickey).
```
For some reason my SSH key gets rejected.
I don't necessarily need an SSH access but otherwise I'd like this message to be customized again with 894643ab3752766f46fa3fcbf5634936e9908da2 in the LimeSurvey code.groentegroentehttps://gitlab.tails.boum.org/tails/sysadmin/-/issues/18049Setup infrastructure for containerized services2024-03-12T13:36:20ZZen FuSetup infrastructure for containerized servicesWe want to self-host some container-based services in a place where owning them wouldn't easily escalate to endangering Tails users:
- HedgeDoc
- Errbot
- Maybe LimeSurvey
For that, we need to:
- [x] Design an update mechanism (either...We want to self-host some container-based services in a place where owning them wouldn't easily escalate to endangering Tails users:
- HedgeDoc
- Errbot
- Maybe LimeSurvey
For that, we need to:
- [x] Design an update mechanism (either automated or manual)
- [x] Find out required specs
- [x] Discuss with tails-board about donating to Tachanka!
- [x] Request a VM to Tachanka!
- [x] Add the new VM to our infraZen FuZen Fuhttps://gitlab.tails.boum.org/tails/sysadmin/-/issues/18048Review mirrors workflow and doc2023-11-30T12:31:34ZZen FuReview mirrors workflow and docSome time has passed since mirrors work was handed over to Sysadmins, and now is a good time to review some leftover bits:
- [x] Check why puppet-tails!122 was not merged and make sure to merge it: puppet-tails!135
- [x] Delete `dns.liz...Some time has passed since mirrors work was handed over to Sysadmins, and now is a good time to review some leftover bits:
- [x] Check why puppet-tails!122 was not merged and make sure to merge it: puppet-tails!135
- [x] Delete `dns.lizard:/srv/repositories/mirrors.git`
- [x] Review mirrors documentation in `sysadmin-private.git`
- [x] Review mirrors documentation in the website
- [ ] Review workflow for dealing with slow mirrorsZen FuZen Fuhttps://gitlab.tails.boum.org/tails/sysadmin/-/issues/18047Backups on ecours trip over database files2023-11-22T16:33:10ZgroenteBackups on ecours trip over database filesBorg doesn't like files being changed while being backed up, which makes the backup service fail on ecours. We should exclude the database files and make sure the database is regularly dumped.Borg doesn't like files being changed while being backed up, which makes the backup service fail on ecours. We should exclude the database files and make sure the database is regularly dumped.groentegroentehttps://gitlab.tails.boum.org/tails/sysadmin/-/issues/18046Jenkins building on lizard2023-11-20T12:24:07ZboyskaJenkins building on lizardHi sysadmins!
Today it happened to me that a [build I was waiting](https://jenkins.tails.boum.org/view/Tails%20Build/job/build_Tails_ISO_19351-whisperback-description/5/) was taking so long, and then I noticed it was running on lizard.
...Hi sysadmins!
Today it happened to me that a [build I was waiting](https://jenkins.tails.boum.org/view/Tails%20Build/job/build_Tails_ISO_19351-whisperback-description/5/) was taking so long, and then I noticed it was running on lizard.
Is this intended? I thought our plan was to have lizard isobuilder's only available for building IUKs when the RM explicitly selects this to be the case, but no automatic build otherwise.https://gitlab.tails.boum.org/tails/sysadmin/-/issues/18045IUK builds failing2023-12-27T00:16:34ZgroenteIUK builds failingMany IUK builds were failing with various errors: java errors, segfaults, missing arguments, etc.
On top of that, iguana decided to reboot while running the build (!?)
Something seems pretty effy, we need to investigate.
See: https://je...Many IUK builds were failing with various errors: java errors, segfaults, missing arguments, etc.
On top of that, iguana decided to reboot while running the build (!?)
Something seems pretty effy, we need to investigate.
See: https://jenkins.tails.boum.org/job/build_IUKs/ build 89 - 94.
@intrigeri and @boyska pinging you so you're aware.groentegroentehttps://gitlab.tails.boum.org/tails/sysadmin/-/issues/18044Improve the firewall config and code of the new test email server for Jenkins...2023-11-23T11:07:22ZZen FuImprove the firewall config and code of the new test email server for Jenkins AgentsFirewall improvements:
- [x] Cleanup unused firewall config in Jenkins Agents
- [x] Force Jenkins Agents to connect directly to the VM's IP:
- [x] Export rules so isoworkers-mail.tails.net accepts from Jenkins Agents in mail and http(...Firewall improvements:
- [x] Cleanup unused firewall config in Jenkins Agents
- [x] Force Jenkins Agents to connect directly to the VM's IP:
- [x] Export rules so isoworkers-mail.tails.net accepts from Jenkins Agents in mail and http(s) ports.
- [x] Map isoworkers-mail.tails.net to 192.168.126.20 in Jenkins Agents
- [x] Remove the tirewall::public_service entries for mail (leave port 80)
- [x] Remove custom firewall rules added to Iguana.
Code improvements:
- [x] Move the client email password config to `hieradata/role/isoworker.eyaml`
- [x] Move the Nginx config from node definition to a profile.
- [x] Move the Libvirt hosts firewall config to exported resources in the same profile as above.
- ~~Refactor `tails::tester::support::email*` to a profile (same as above, probably).~~ → Deferred to #16958
Missing bits:
- [x] tails!1268+
- [x] Apply improvements to the firewall config once again.
- [x] Check that Thunderbird tests keep working.Zen FuZen Fuhttps://gitlab.tails.boum.org/tails/sysadmin/-/issues/18043Git push does not trigger Jenkins build anymore2023-10-16T13:44:45ZintrigeriGit push does not trigger Jenkins build anymoreI noticed this regression while doing dev work, and just verified this:
```
$ git push lizard devel
Enumerating objects: 7, done.
Counting objects: 100% (7/7), done.
Delta compression using up to 8 threads
Compressing objects: 100% (3/3...I noticed this regression while doing dev work, and just verified this:
```
$ git push lizard devel
Enumerating objects: 7, done.
Counting objects: 100% (7/7), done.
Delta compression using up to 8 threads
Compressing objects: 100% (3/3), done.
Writing objects: 100% (3/3), 379 bytes | 379.00 KiB/s, done.
Total 3 (delta 2), reused 0 (delta 0), pack-reused 0
remote: .--- Notifying Jenkins... -------------------------------------
remote: |
remote: 11
remote: | Notifying Jenkins of Git updates...
remote: | curl: (6) Could not resolve host: jenkins.lizard
remote: |
remote: `--------------------------------------------------------------
remote: 2023-10-16T10:09:42+00:00 hook-start ./hooks/post-update.d/www_website_ping-post-update.hook
remote: 2023-10-16T10:09:42+00:00 hook-end ./hooks/post-update.d/www_website_ping-post-update.hook
To git.puppet.tails.boum.org:tails
7751d15f12..d2cefa041d devel -> devel
```
It seems that "Could not resolve host: jenkins.lizard" is the culprit. Maybe related to tails/sysadmin#17856?groentegroentehttps://gitlab.tails.boum.org/tails/sysadmin/-/issues/18042Mirroring of tails.git to salsa.debian.org is broken → Automated triggering o...2023-10-13T09:20:03ZZen FuMirroring of tails.git to salsa.debian.org is broken → Automated triggering of website refreshes is brokenThe [command to mirror our Gitolite's tails.git to Salsa](https://gitlab.tails.boum.org/tails/puppet-tails/-/blob/master/files/gitolite/hooks/salsa_mirror-post-update.hook?ref_type=heads#L5) fails because of lack of permissions:
```
zen...The [command to mirror our Gitolite's tails.git to Salsa](https://gitlab.tails.boum.org/tails/puppet-tails/-/blob/master/files/gitolite/hooks/salsa_mirror-post-update.hook?ref_type=heads#L5) fails because of lack of permissions:
```
zen@puppet-git /tmp> sudo -u gitolite3 git -C /var/lib/gitolite3/repositories/tails.git/ push --quiet --all --force --prune salsa
git@salsa.debian.org: Permission denied (publickey).
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
```
Because the [`post-update` hook bails out if any of the underlying hooks fail](https://gitlab.tails.boum.org/tails/puppet-tails/-/blob/master/files/gitolite/hooks/post-update.d.hook?ref_type=heads#L5), it never gets to trigger a refresh of the website.
## To-Do
- [x] Stop mirroring `tails.git` to salsa.debian.org
- [x] Maybe inform users of that repository that it's deprecated
- [x] Maybe delete the Salsa projectTails_5.19intrigeriintrigerihttps://gitlab.tails.boum.org/tails/sysadmin/-/issues/18040Mirroring via git-annex now takes much longer and causes high CPU load2023-11-29T10:29:06ZZen FuMirroring via git-annex now takes much longer and causes high CPU loadSince some days (unfortunately can't specify, logs are gone for some reason), the time taken by mirroring periodic tasks increased a lot:
- `git-annex-mirror-torbrowser-archive.service`: 1min → 6 min
- `git-annex-mirror-iso-history.serv...Since some days (unfortunately can't specify, logs are gone for some reason), the time taken by mirroring periodic tasks increased a lot:
- `git-annex-mirror-torbrowser-archive.service`: 1min → 6 min
- `git-annex-mirror-iso-history.service` 26s → 17min
Also, they now occupy almost 2 full cores each while running (a constant load of 4).
High load is not expected, neither of them should ever be used intensely:
* torbrowser-archive: we upload to it once for each Tor Browser release, and some timer job is copying the stuff out from there to www.lizard <http://www.lizard>. so really the load should be minimal
* iso-history: also used mostly around release time, and should have minimal load normally
(Thanks anonym for that info!)
What changed lately:
- The VM was upgraded to Bookworm.
- FT started using `git annex sync --no-content`: tails@08625e83d4882448826cec85c37460dad4a14199
FTR: I have increased the amount of cores of the VM for now so it can take the load meanwhile.
Cc: @foundations-team, in case they have more input to give. :smile_cat:Zen FuZen Fuhttps://gitlab.tails.boum.org/tails/sysadmin/-/issues/18039Add more schleuder admins for tails-foundations@2023-09-28T17:56:05ZanonymAdd more schleuder admins for tails-foundations@It seems only intrigeri is an admin of the tails-foundations@ schleuder list, but I think we want all FT members that also are CWs to be admins. So can me and boyska be added? And perhaps groente too since he is about to join FT and is a...It seems only intrigeri is an admin of the tails-foundations@ schleuder list, but I think we want all FT members that also are CWs to be admins. So can me and boyska be added? And perhaps groente too since he is about to join FT and is a CW. In fact, this all started because I wanted to add groente to the list but realized only intrigeri could.Zen FuZen Fuhttps://gitlab.tails.boum.org/tails/sysadmin/-/issues/18038Update Jenkins core and plugins (2023Q4 edition)2024-03-13T00:29:51ZZen FuUpdate Jenkins core and plugins (2023Q4 edition)Last time: sysadmin#18019
# Preparation
* [x] Go through [the changelog](https://www.jenkins.io/changelog/), paying attention to changes on how agents connect to controller, config changes that may need update, important changes in plu...Last time: sysadmin#18019
# Preparation
* [x] Go through [the changelog](https://www.jenkins.io/changelog/), paying attention to changes on how agents connect to controller, config changes that may need update, important changes in plugins, etc.
* Current version: 2.414.2
* Available version: 2.426.2
# Deployment
* [x] Deploy Jenkins upgrade to latest version available
* [x] Use the [Groovy](https://tails.boum.org/contribute/working_together/roles/sysadmins/Jenkins/1.generate-plugins-list.groovy) and [Python3](https://tails.boum.org/contribute/working_together/roles/sysadmins/Jenkins/2.generate-puppet-code.py) scripts to generate a list of up-to-date plugins
* [x] Deploy plugin upgrades
* [x] Restart all agents.
* [x] Manually run the Update jobs script (may be needed so XML is valid with current Jenkins):
```
sudo -u jenkins /usr/local/sbin/deploy_jenkins_jobs update
```
* [x] Go through warnings in Jenkins interface.
* [x] Manually uninstall removed plugins via [Jenkins interface](https://jenkins.tails.boum.org/manage/pluginManager/installed).
* [x] Manually delete removed plugins from `/var/lib/jenkins/plugins`.
# Wrap up
* [x] Restart builds that were interrupted by Jenkins restart.
* [x] Schedule next update.Zen FuZen Fuhttps://gitlab.tails.boum.org/tails/sysadmin/-/issues/18037jenkins-jobs fails because of postbuildscript version string2023-09-29T11:19:49ZZen Fujenkins-jobs fails because of postbuildscript version stringWe had manually worked around this problem before (https://gitlab.tails.boum.org/tails/tails/-/issues/19051#note_191808) but it hit us again after the upgrade of the Jenkins VM to Bookworm (sysadmin#17982).
For now, it's worked around b...We had manually worked around this problem before (https://gitlab.tails.boum.org/tails/tails/-/issues/19051#note_191808) but it hit us again after the upgrade of the Jenkins VM to Bookworm (sysadmin#17982).
For now, it's worked around by manually patching the `/usr/lib/python3/dist-packages/jenkins_jobs/modules/publishers.py` file (see below).
Upstream bug report: https://storyboard.openstack.org/#!/story/2009943
<details>
<summary>Failure log</summary>
```shell
zen@jenkins ~ [2]> sudo -u jenkins /usr/local/sbin/deploy_jenkins_jobs update
HEAD is now at 80901b2c Automatic Tails_ISOS jobs update.
INFO:jenkins_jobs.cli.subcommand.update:Updating jobs in ['/etc/jenkins_jobs/jobs', '/etc/jenkins_jobs/jobs/Tails_ISOS', '/etc/jenkins_jobs/jobs/macros'] ([])
INFO:root:Caching type properties of properties = jenkins_jobs.modules.properties:Properties
INFO:root:Caching type builders of builders = jenkins_jobs.modules.builders:Builders
INFO:root:Caching type wrappers of wrappers = jenkins_jobs.modules.wrappers:Wrappers
INFO:root:Caching type parameters of parameters = jenkins_jobs.modules.parameters:Parameters
INFO:root:Caching type scm of scm = jenkins_jobs.modules.scm:SCM
INFO:root:Caching type triggers of triggers = jenkins_jobs.modules.triggers:Triggers
INFO:root:Caching type publishers of publishers = jenkins_jobs.modules.publishers:Publishers
Traceback (most recent call last):
File "/usr/bin/jenkins-jobs", line 10, in <module>
sys.exit(main())
^^^^^^
File "/usr/lib/python3/dist-packages/jenkins_jobs/cli/entry.py", line 174, in main
jjb.execute()
File "/usr/lib/python3/dist-packages/jenkins_jobs/cli/entry.py", line 154, in execute
ext.obj.execute(self.options, self.jjb_config)
File "/usr/lib/python3/dist-packages/jenkins_jobs/cli/subcommand/update.py", line 134, in execute
builder, xml_jobs, xml_views = self._generate_xmljobs(options, jjb_config)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/jenkins_jobs/cli/subcommand/update.py", line 119, in _generate_xmljobs
xml_jobs = xml_job_generator.generateXML(job_data_list)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/jenkins_jobs/xml_config.py", line 79, in generateXML
xml_objs.append(self._getXMLForData(data))
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/jenkins_jobs/xml_config.py", line 92, in _getXMLForData
self._gen_xml(xml, data)
File "/usr/lib/python3/dist-packages/jenkins_jobs/xml_config.py", line 109, in _gen_xml
module.gen_xml(xml, data)
File "/usr/lib/python3/dist-packages/jenkins_jobs/modules/publishers.py", line 8347, in gen_xml
self.registry.dispatch("publisher", publishers, action)
File "/usr/lib/python3/dist-packages/jenkins_jobs/registry.py", line 309, in dispatch
func(self, xml_parent, component_data, **kwargs)
File "/usr/lib/python3/dist-packages/jenkins_jobs/modules/publishers.py", line 4497, in postbuildscript
version = pkg_resources.parse_version(info.get("version", str(sys.maxsize)))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/pkg_resources/_vendor/packaging/version.py", line 266, in __init__
raise InvalidVersion(f"Invalid version: '{version}'")
pkg_resources.extern.packaging.version.InvalidVersion: Invalid version: '3.2.0-550.v88192b_d3e922'
```
</details>
<details>
<summary>Patch for publishers.py</summary>
```diff
diff --git a/jenkins_jobs/modules/publishers.py b/jenkins_jobs/modules/publishers.py
index e2f1219c..8726b1f4 100755
--- a/jenkins_jobs/modules/publishers.py
+++ b/jenkins_jobs/modules/publishers.py
@@ -30,6 +30,7 @@ import pkg_resources
import sys
import xml.etree.ElementTree as XML
+import re
import six
from jenkins_jobs.errors import InvalidAttributeError
@@ -4497,7 +4498,8 @@ def postbuildscript(registry, xml_parent, data):
info = registry.get_plugin_info("postbuildscript")
# Note: Assume latest version of plugin is preferred config format
- version = pkg_resources.parse_version(info.get("version", str(sys.maxsize)))
+ _version = re.sub('-.*', '', info.get("version", str(sys.maxsize)))
+ version = pkg_resources.parse_version(_version)
if version >= pkg_resources.parse_version("2.0"):
pbs_xml = XML.SubElement(pbs_xml, "config")
```https://gitlab.tails.boum.org/tails/sysadmin/-/issues/18036transmission often not running on bittorrent.lizard2023-09-25T05:33:10Zanonymtransmission often not running on bittorrent.lizardWhile preparing Tails 5.17.1 on 2023-09-14 the `transmission` daemon wasn't running and I had to ask a sysadmin to start it. Today (2023-09-20) I was gonna finish the release process, including stopping seeding Tails 5.17, but the `trans...While preparing Tails 5.17.1 on 2023-09-14 the `transmission` daemon wasn't running and I had to ask a sysadmin to start it. Today (2023-09-20) I was gonna finish the release process, including stopping seeding Tails 5.17, but the `transmission` daemon was again not running.
I think this started happening since the upgrade to Debian Bookworm.https://gitlab.tails.boum.org/tails/sysadmin/-/issues/18035GitLab CI pipelines have been failing since September 112023-09-18T08:43:59ZintrigeriGitLab CI pipelines have been failing since September 11See job history of https://gitlab.tails.boum.org/tails/tails/-/pipelines.
It seems the jobs timeout.See job history of https://gitlab.tails.boum.org/tails/tails/-/pipelines.
It seems the jobs timeout.groentegroentehttps://gitlab.tails.boum.org/tails/sysadmin/-/issues/18034git annex mirror services fail silently2024-03-26T14:28:12Zgroentegit annex mirror services fail silentlywhen the systemd services defined in `tails::git_annex::mirror` fail (or at least when they encounter a permission denial), the service is not set to 'failed' and doesn't trigger any monitoring.when the systemd services defined in `tails::git_annex::mirror` fail (or at least when they encounter a permission denial), the service is not set to 'failed' and doesn't trigger any monitoring.Zen FuZen Fuhttps://gitlab.tails.boum.org/tails/sysadmin/-/issues/18033Update Puppet modules: 2024 edition2023-11-29T14:51:45ZgroenteUpdate Puppet modules: 2024 editionReminder to update our modules somewhere in 2024.Reminder to update our modules somewhere in 2024.2024-12-31