Update Jenkins core and plugins (2024Q2 edition)

Last time: #18060 (closed)

Preparation

• Go through the changelog, paying attention to changes on how agents connect to controller, config changes that may need update, important changes in plugins, etc.
  • Current version: 2.440.1
  • Available version: 2.452.2

Deployment

• Use the Groovy scripts to get a list of queued and running jobs to manually restart in the end of this process
• Deploy Jenkins upgrade to latest version available
• Use the Groovy and Python3 scripts to generate a list of up-to-date plugins
• Deploy plugin upgrades
• Restart all agents from lizard:

  for i in isobuilder{1,2,3,4} isoworker{1,2,3,4,5,6,7,8}; ssh $i sudo systemctl reboot; end

• Manually run the Update jobs script (may be needed so XML is valid with current Jenkins) in jenkins.dragon:

  sudo -u jenkins /usr/local/sbin/deploy_jenkins_jobs update

• Go through warnings in Jenkins interface.
• Manually uninstall removed plugins via Jenkins interface.
• Manually delete removed plugins from /var/lib/jenkins/plugins.

Wrap up

• Restart builds that were interrupted by Jenkins restarts.
• Schedule next update.

Note: For S11, this fits in:

• B.2 - Keep our infrastructure up-to-date and secure: Having an up-to-date CI build/test system is fundamental not only for the security of the infra but for the final product itself.