Jenkins: Document the use of the fixed port 42585 for serving the API
During one upgrade (#17730 (closed)), a change in how Jenkins controller makes URLs available to agents forced us to make several changes to our setup:
- Before 2.248, Jenkins used to include the request URL as a fallback address for agent to controller connections.
- Because our agents fetch the connection info from
http://jenkins.lizard/...
, they used to connect to the controller doing plain HTTP tojenkins.lizard
. - In 2.248 such alternate URL was removed from the connection info sent from the controller to the agent (https://github.com/jenkinsci/jenkins/commit#/3580e595617b4f863986f474523b74be194d2c4f) and the only URL included started to be the one configured in the "Jenkins URL" field in https://jenkins.tails.boum.org/configure.
- Our agents started to fail because they now tried to connect via HTTPS (port 443) to
jenkins.tails.boum.org
, which locally maps to192.168.122.1
.
What we did:
- Configure
lizard
's firewall to NAT incoming traffic to ports 80/443 from agents to thewww
VM ports 1180/11443. - Configure unauthenticated non-public ports in
www.lizard
(1180 and 11443) to handle TLS and reverse-proxy tojenkins
VM. - Use a fixed port
42585
for agents connecting to controller (instead of default Jenkins config of choosing a random port each time it starts): https://jenkins.tails.boum.org/configureSecurity/ - Configure
lizard
s firewall to NAT incoming traffic to port42585
from agents to thejenkins
VM.
Note: port 80 can probably be removed from the setup above, as agents are connecting directly to 443.
This is a complicated setup and needs to be properly documented.
Documentation
These are the steps a Jenkins agent does when connecting to the controller:
- Fetch connection info from
http://jenkins.lizard:8080
. - Receive the connection URL
https://jenkins.tails.boum.org
("Jenkins URL" manually configured in Configure System). - Resolve
jenkins.tails.boum.org
to192.168.122.1
(because of libvirt config). - Connect using HTTPS to
jenkins.tails.boum.org:443
. - Learn about port
42585
(fixed "TCP port for inbound agents" manually configured in Configure Global Security). - Connect using HTTP to
jenkins.tails.boum.org:42585
.
For those steps to work, the following configuration exists outside of Jenkins VMs:
- Firewall rules in Lizard to forward agents traffic to
www.lizard
:-
192.168.122.1:80
→www.lizard:1180
-
192.168.122.1:443
→www.lizard:11443
-
192.168.122.1:42585
→www.lizard:42585
-
- Nginx config in
www.lizard
to reverse proxy traffic tojenkins.lizard
:-
www.lizard:1180
→jenkins.lizard:80
-
www.lizard:11443
→jenkins.lizard:443
-
www.lizard:42585
→jenkins.lizard:42585
-
What could be improved:
- Ports 1180/80 are probably not needed in the configs above.
- Port 42585 could be directly forwarded to
jenkins.lizard
.
Edited by Zen Fu