Skip to content

Jenkins: Document the use of the fixed port 42585 for serving the API

During one upgrade (#17730 (closed)), a change in how Jenkins controller makes URLs available to agents forced us to make several changes to our setup:

  • Before 2.248, Jenkins used to include the request URL as a fallback address for agent to controller connections.
  • Because our agents fetch the connection info from http://jenkins.lizard/..., they used to connect to the controller doing plain HTTP to jenkins.lizard.
  • In 2.248 such alternate URL was removed from the connection info sent from the controller to the agent (https://github.com/jenkinsci/jenkins/commit#/3580e595617b4f863986f474523b74be194d2c4f) and the only URL included started to be the one configured in the "Jenkins URL" field in https://jenkins.tails.boum.org/configure.
  • Our agents started to fail because they now tried to connect via HTTPS (port 443) to jenkins.tails.boum.org, which locally maps to 192.168.122.1.

What we did:

  • Configure lizard's firewall to NAT incoming traffic to ports 80/443 from agents to the www VM ports 1180/11443.
  • Configure unauthenticated non-public ports in www.lizard (1180 and 11443) to handle TLS and reverse-proxy to jenkins VM.
  • Use a fixed port 42585 for agents connecting to controller (instead of default Jenkins config of choosing a random port each time it starts): https://jenkins.tails.boum.org/configureSecurity/
  • Configure lizards firewall to NAT incoming traffic to port 42585 from agents to the jenkins VM.

Note: port 80 can probably be removed from the setup above, as agents are connecting directly to 443.

This is a complicated setup and needs to be properly documented.

Documentation

These are the steps a Jenkins agent does when connecting to the controller:

  1. Fetch connection info from http://jenkins.lizard:8080.
  2. Receive the connection URL https://jenkins.tails.boum.org ("Jenkins URL" manually configured in Configure System).
  3. Resolve jenkins.tails.boum.org to 192.168.122.1 (because of libvirt config).
  4. Connect using HTTPS to jenkins.tails.boum.org:443.
  5. Learn about port 42585 (fixed "TCP port for inbound agents" manually configured in Configure Global Security).
  6. Connect using HTTP to jenkins.tails.boum.org:42585.

For those steps to work, the following configuration exists outside of Jenkins VMs:

  • Firewall rules in Lizard to forward agents traffic to www.lizard:
    • 192.168.122.1:80www.lizard:1180
    • 192.168.122.1:443www.lizard:11443
    • 192.168.122.1:42585www.lizard:42585
  • Nginx config in www.lizard to reverse proxy traffic to jenkins.lizard:
    • www.lizard:1180jenkins.lizard:80
    • www.lizard:11443jenkins.lizard:443
    • www.lizard:42585jenkins.lizard:42585

What could be improved:

  • Ports 1180/80 are probably not needed in the configs above.
  • Port 42585 could be directly forwarded to jenkins.lizard.
Edited by Zen Fu
To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information