Skip to content

GitLab

Last edited by intrigeri
Page history

GitLab

This is about migrating our data and workflow from Redmine to GitLab, which is tracked as tails#15878 (closed).

Access control

Objects

  • Canonical Git repo: the authoritative tails.git repository hosted on GitLab

  • Major branches: master, stable, testing, devel, and possibly feature/bullseye

  • Release tags: a signed Git tag that identifies the source code used to build a specific Tails release; currently all tags in the authoritative tails.git repository are release tags; the tag name is a version number, with '~' replaced by '-'.

  • Particularly sensitive data: confidential data that specific teams like Fundraising and Accounting need to handle, but that other contributors generally don't need direct access to. This definitely include issues; this might include Git repositories at some point.

    Note that as of 2019-11-20, it is undefined:

    • What subset of this data can go to a web-based issue tracker or not.
      This is already a problem with Redmine.
      Fixing this will require discussions between various stakeholders.

    • What subset of this data could live in a cleartext Git repository hosted here or there, as opposed to requiring end-to-end encryption between members of these teams. This is a hypothetical problem for now.

Subjects

  • An admin can do anything that other roles can, and:

    • can delete issues
    • can edit team membership
    • MUST comply with our "Level 3" security policy
    • can view issues that contain particularly sensitive data
    • can create new groups and projects that are not forks

  • A committer:

    • can push and force-push to any ref in the canonical Git repo, including major branches and release tags;
      incidentally, this ensures the following requirement is met:
    • their branches are picked up by Jenkins; it follows that they MUST comply with our "Infrastructure" security policy
    • can merge MRs into major branches
    • can modify issues metadata
    • MAY be allowed to view confidential issues in the main GitLab project; if it's the case, then particularly sensitive data MUST live somewhere else with stricter access control
    • can edit other users' comments
    • MAY be allowed to add new team members
    • MUST comply with our "Level 3" security policy

  • A regular, particularly trusted contributor:

    • can push and force-push to a subset of refs in the canonical Git repo; this subset MUST NOT include any major branch nor release tag;
      this is required to ensure the following requirement is met:
    • their branches are picked up by Jenkins; it follows that they MUST comply with our "Infrastructure" security policy
    • can modify issues metadata
    • MAY be allowed to view confidential issues in the main GitLab project; if it's the case, then particularly sensitive data MUST live somewhere else with stricter access control

  • A regular contributor:

    • can fork the Git repositories and push changes to their own fork
    • can modify issues metadata
    • MAY be allowed to view confidential issues in the main GitLab project; if it's the case, then particularly sensitive data MUST live somewhere else with stricter access control

  • Anybody with a GitLab account on the instance we use:

    • can submit issues
    • can submit MRs

Implementation

Relevant GitLab doc

Common settings

  • pipelines: disabled
  • wiki: disabled

The parent "tails" group

  • Visibility: Public
  • The root user has "Owner" access to this group.
  • Release managers have "Reporter" access to every project in this group (after releasing a new Tails, they need to postpone issues, including confidential ones, and manage group labels), until tails#17589 (closed) is done.
  • Nobody else is a member of this group.
  • Allow users to request access: disabled

Most public projects

The following applies to each of our public project that has no dedicated section below:

  • Protected branch flow: it's the most adequate for regular contributors, since it's the least disruptive in terms of workflow and habits and requires no work to adjust our Jenkins CI setup.
  • We mark our major branches as "Protected".
  • Committers get "Maintainer" access.
  • Maintainers are allowed to merge and to push to protected branches.
  • The Git workflow remains unchanged for regular developers who are not granted full commit access but have access to our CI: they get "Developer" access, can push a topic branch to the canonical Git repository and our CI will pick it up. The only difference is that they are not restricted to pushing to their own $nickname/* namespace, which makes things simpler and has other advantages, e.g. they can use the wip/ prefix (so our Jenkins CI ignores the branch) and collaborate with others on shared branches.
  • Other contributors get access strictly lower than "Developer". They push topic branches to their own fork of the repository and create merge requests.
  • Our current Jenkins CI jobs generation process remains unchanged. (Technically, we could adjust it so it generates CI jobs for any merge request (based on refs/merge-requests/*/head), but this would give arbitrary code execution on our CI to anyone. Our infrastructure is not currently equipped to cope with this.)
  • issues: disabled except for the tails/tails project
  • MRs: enabled
  • Allow users to request access: enabled

rm

  • visibility: private
  • issues: disabled
  • MRs: disabled (gcrypt)
  • repository: enabled
  • Allow users to request access: disabled
  • Nobody has access except RMs

Public sysadmin projects

Projects:

  • bitcoin
  • bitcoin_libunivalue → rename to libunivalue
  • public puppet-* (additional "Maintainer": role-lizard)
  • jenkins-jobs (additional "Maintainer": role-lizard)

Properties:

  • public
  • forking workflow
  • issues: disabled (tracked in tails/tails)
  • MRs: enabled
  • regular Tails contributors have "Reporter" access

tails/accounting, tails/fundraising, tails/private

  • visibility: private
  • issues: enabled
  • MRs: disabled
  • repository: disabled (could be enabled later if these teams want their git-remote-gcrypt repo migrated)
  • Allow users to request access: disabled
  • Maintainer access given to:
    • accounting: accounting team
    • fundraising: fundraising team
    • private: tails@ members

tails/sysadmin

  • visibility: public
  • issues: enabled
  • MRs: disabled
  • repository: disabled (contributors with "Guest" access would be allowed to clone it)
  • Allow users to request access: disabled
  • regular contributors have "Guest" access level (so they can create new issues but not see confidential ones)
  • Maintainer access given to: sysadmin team

test-suite-shared-secrets

  • visibility: private
  • issues: disabled
  • MRs: enabled
  • repository: enabled
  • Allow users to request access: disabled
  • Nobody has access except committers

mirror-pool

  • visibility: public
  • issues: disabled
  • MRs: enabled
  • repository: enabled
  • Allow users to request access: disabled
  • mirrors team members have "Maintainer" access
  • forking workflow
  • regular Tails contributors have "Reporter" access

Management

With Redmine + Gitolite, we centrally manage, in 1-2 single places, team membership and the corresponding access level. For example, when someone gets commit access or joins a team, we add them to the corresponding Gitolite group and Redmine role. That's very convenient.

If we don't do anything special, to implement the access control specified above, we would need to:

  • During the migration to GitLab: give dozens of users "Reporter", "Maintainer" or "Developer" access on dozens of project.
  • Day-to-day operations: occasionally, give someone "Reporter", "Maintainer" or "Developer" access on dozens of project.

I don't think we want to do any of this manually.

It seems we could maintain user groups via the GitLab web interface and share the relevant projects with these user groups. These groups would not contain any project themselves: their primary purpose is to manage access to projects in batch.

And in passing, such user groups would allow us to mention a team with one single @xyz-team or @xyz-group. For example, @sysadmins-team.

Another option would be to use subgroups, and have for example tails/accounting-team/accounting, but references to issues etc. become pretty long, e.g. "tails/accounting-team/accounting#1234", compared to "tails/accounting#1234" in the current implementation. Unless there's a way around this problem, that's a usability deal breaker and we should not use subgroups merely to simplify access level management.

Wiki

It's out of scope for the first iteration but at some point, we might want to migrate our blueprints to GitLab's wiki: tails#9174 (closed)).

Git repositories

Migration includes preserving access rights, which are documented for each repository below.

Must be migrated

From git.tails.boum.org aka. git-tails.immerda.ch:

  • bitcoin (public; read-write: sysadmins)
  • bitcoin_libunivalue (public; read-write: sysadmins)
  • chutney (public; read-write: committers)
  • htp (public; read-write: committers)
  • jenkins-tools (public; read-write: committers)
  • liveusb-creator → rename to installer (public; read-write: committers)
  • network-manager (public; read-write: committers)
  • onioncircuits (public; read-write: committers)
  • rm (private; read-write: RMs)
  • tails-workarounds → rename to workarounds (public; read-write: committers)
  • test-suite-shared-secrets (private; read-write: committers)
  • torbrowser-launcher (public; read-write: committers)
  • ux (public; read-write: committers)
  • verification-extension (public; read-write: committers)
  • whisperback (public; read-write: committers)

From puppet-git.lizard:

  • etcher-binary (public; read-write: committers)
  • mirror-pool (public; read-write: mirrors team)
  • mirror-pool-dispatcher (public; read-write: committers)
  • promotion-material (public; read-write: committers)

Read-only mirrors to be migrated from git.tails.boum.org aka. git-tails.immerda.ch (with a description that makes it clear to people with commit access that it's a read-only mirror):

  • public puppet-* (public; read-write: role-lizard)
  • jenkins-jobs (public; read-write: role-lizard)

Leave on immerda for now

  • download-and-verify-extension: obsolete
  • iceweasel: very big, unused since years
  • thunderbird: very big, hopefully obsolete soon
  • uui-binary: obsolete

Interfaces that will need adjustments

Git repository → various mirrors

Our Git repository keeps a number of mirrors up-to-date:

In particular:

  • repositories that must now be mirrored on our GitLab (in addition to, or instead of, the current immerda remote):
    • public puppet-*
    • jenkins-jobs

Git repositories → website

A refresh of our website is triggered by:

  • pushing to the master branch of our Git repository
  • pushing to a few "underlay" repositories

Current implementation:

Git repository → Jenkins

Pushing to our Git repository pings Jenkins so it can run jobs as needed: https://git.tails.boum.org/puppet-tails/tree/templates/gitolite/hooks/tails-post-receive.erb

Jenkins → Git

Jenkins jobs are generated on jenkins.lizard, from the list of branches in our main Git repository. For details and pointers to the corresponding code, see the corresponding blueprint.

Here are the kinds of jobs relevant in this discussion:

  • check_PO_master runs the check_po script on all PO files

    • this script comes from a Git submodule referenced by tails.git's master branch (tails::tester::check_po)
    • in a Jenkins isobuilder
    • as a sudoer user

  • build_website_master runs ./build-website

    • from tails.git's master branch
    • in a Jenkins isobuilder
    • as a sudoer user

  • build_Tails_ISO_* and reproducibly_build_Tails_ISO_* run rake build

    • from the corresponding tails.git branch
    • in a Jenkins isobuilder
    • as a sudoer user

  • test_Tails_ISO_* run ./run_test_suite

    • from the corresponding tails.git branch
    • in a Jenkins isotester
    • as root via sudo

Wrt. who can push to which branch in tails.git, see the "Access control" section.

Jenkins → Redmine

Tails images build reproducibility is tested if the corresponding ticket has "Needs Validation" status: https://git.tails.boum.org/puppet-tails/tree/files/jenkins/slaves/isobuilders/decide_if_reproduce

(Currently disabled:) Email notifications are sent on job status change if the ticket has "Needs Validation" status: https://git.tails.boum.org/puppet-tails/tree/files/jenkins/slaves/isobuilders/output_ISO_builds_and_tests_notifications

Ticket triaging → Redmine

Our Redmine email reminder sends email to users who have at least one ticket assigned to them, that satisfies at least one of these criteria:

  • "Stalled work-in-progress": status "In Progress", that were not updated since more than 6 months

  • "Reviews waiting for a long time": not been updated since 45 days or more

Current implementation:

With GitLab one can define policies (YAML) and have them applied automatically to issues and merge requests:

Potential future work: tails#17589 (closed).

Translation platform → Git

A hook ensures that Weblate only pushes changes to our main Git repository that meet certain conditions:

https://git.tails.boum.org/puppet-tails/tree/files/gitolite/hooks/tails-weblate-update.hook

GitLab hosting options

See https://salsa.debian.org/tails-team/gitlab-migration/wikis/hosting/comparison.

Resources