GitLab

This is an ugly workaround to get rid of noise in tails-translations
mailing list.

The Python version in Debian Stretch has a bug that leaks memory in some
situations and will spit out the following error:

------------------------------------------------------------------------------
Exception ignored in: <function WeakValueDictionary.__init__.<locals>.remove at 0x7fe488c03048>
Traceback (most recent call last):
  File "/usr/lib/python3.5/weakref.py", line 117, in remove
TypeError: 'NoneType' object is not callable
Exception ignored in: <function WeakValueDictionary.__init__.<locals>.remove at 0x7fe488c03048>
Traceback (most recent call last):
  File "/usr/lib/python3.5/weakref.py", line 117, in remove
TypeError: 'NoneType' object is not callable
Exception ignored in: <function WeakValueDictionary.__init__.<locals>.remove at 0x7f69c5515ea0>
Traceback (most recent call last):
  File "/usr/lib/python3.5/weakref.py", line 117, in remove
TypeError: 'NoneType' object is not callable
Exception ignored in: <function WeakValueDictionary.__init__.<locals>.remove at 0x7f69c5515ea0>
Traceback (most recent call last):
  File "/usr/lib/python3.5/weakref.py", line 117, in remove
TypeError: 'NoneType' object is not callable
------------------------------------------------------------------------------

As our scripts run frequently, we are getting many e-mails that are
actually false positives.

The differences from the original file are:

------------------------------------------------------------------------------
--- weakref.py  2020-07-28 22:54:59.837435324 +0000
+++ /usr/lib/python3.5/weakref.py       2020-07-09 13:00:10.000000000 +0000
@@ -106,7 +106,7 @@
         self, *args = args
         if len(args) > 1:
             raise TypeError('expected at most 1 arguments, got %d' % len(args))
-        def remove(wr, selfref=ref(self), _atomic_removal=_remove_dead_weakref):
+        def remove(wr, selfref=ref(self)):
             self = selfref()
             if self is not None:
                 if self._iterating:
@@ -114,7 +114,7 @@
                 else:
                     # Atomic removal is necessary since this function
                     # can be called asynchronously by the GC
-                    _atomic_removal(d, wr.key)
+                    _remove_dead_weakref(d, wr.key)
         self._remove = remove
         # A list of keys to be removed
         self._pending_removals = []
------------------------------------------------------------------------------

See the following links for more context:

    https://helpdesk.tetrane.com/hc/en-us/articles/360034629091-I-am-getting-exceptions-from-usr-lib-python3-5-weakref-py-is-it-an-issue-
    https://bugs.python.org/issue29519

This will have to be removed once the machine is upgraded to a newer
Debian version.

This reverts commit b7905976.

The `jenkins-tools` repository is cloned as system user `jenkins`, which
doesn't have keys configured to access GitLab (only user has). I'll
revert this change and flag about this in the issue.

Maintaining HTTPS for now, as it's not a regression in this case.

We can probably use the role-jenkins-isotester user for this once the
SSH key of GitLab has been deployed in isotesters. I'm not sure if yet
another key would have to be added that user's config in GitLab.

See: https://salsa.debian.org/tails-team/gitlab-migration/-/issues/97

See: https://salsa.debian.org/tails-team/gitlab-migration/-/issues/72

We're migrating check-mirrors.git to GitLab, and the repo in
git.tails.boum.org will go away.

Manual steps done previous to this:

  - Creation of user role-check-mirrors in GitLab.

  - Configuration of the SSH key of tails_check_mirrors@misc.lizard in
    the role-check-mirrors user account in GitLab.

Bug introduced in: 107adf3e

Pushes from weblate-gatekeeper should be done as the
"role-weblate-gatekeeper" GitLab user, and for that we need to use a
specific SSH key when pushing.

This commit adds a new "Host" in gitolite's SSH config and changes the
remote URL of the gatekeeper repo to use the that new host, thus making
use of the correct key to connect to GitLab.

See: https://salsa.debian.org/tails-team/gitlab-migration/-/issues/74

We'll use a new user with new SSH keys to configure pushing of changes
made by the Translation Platform into GitLab's tails.git. This commit
puts the newly generated keys in place.

See: https://salsa.debian.org/tails-team/gitlab-migration/-/issues/74

Due to a wrong understanding of how the Translation Platform integration
scripts work, we have previously configured force-pushes from the
integration repo into the new gatekeeper repository.

Actually, such forcing shouldn't be necessary because:

  - Gatekeeper should always be a subset of GitLab, and
  - Integration should fail early when fast-forwarding is not possible.

This commit only reverts the pushing from integration to gatekeeper to
be a normal push instead of a force-push.

We changed the Jenkins configuration to pull from GitLab while the mirroring to
Gitolite was not working:

  tails/jenkins-jobs@4db52b22

But we did not adjust this post-receive hook accordingly,
so the "pushing to Git triggers a Jenkins build" behavior
is broken.

Let's repair this, until we make up our mind (tails/sysadmin#17715)
wrt. whether we should keep pointing Jenkins to GitLab's tails/tails,
or use the local mirror we have on puppet-git.lizard.

This reverts commit 8f58ff06.

Without disabling CSRF (complicated, will probably become impossible at some
point), we did not manage to have jenkins-job-builder use the API
with current Jenkins LTS. Using the standard security setup
(https://wiki.jenkins.io/display/JENKINS/Standard+Security+Setup),
an actual user, and its API token, repairs communication with the API.

refs tails/sysadmin#16955

refs: tails/sysadmin#16955

Closes tails/tails#17755, tails/puppet-tails!7

Now that we're running check-mirrors.rb 4 times more often,
there's greater chances that 2 of them end up running at the same time.
Let's avoid having to reason about how check-mirrors.rb manages
shared resources such as its failures.json state file.

refs: tails/tails#17755

refs: tails/tails#17755

refs: tails/tails#17755

See: https://salsa.debian.org/tails-team/gitlab-migration/-/issues/55

I'll change manually in repo during deploy.

See: https://salsa.debian.org/tails-team/gitlab-migration/-/issues/74

I'll modify manually in the server.

For some reason, our script is not detecting some dependency constraints
correctly, so i'm pinning this one manually.

From: https://github.com/django-compressor/django-compressor/blob/2.4/setup.py#L148

It is now installed with pip. I'll remove the package manually from the
server after deploy.