Skip to content
GitLab
  • Menu
Projects Groups Snippets
  • /
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in / Register
  • S sysadmin
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Issues 94
    • Issues 94
    • List
    • Boards
    • Service Desk
    • Milestones
  • Monitor
    • Monitor
    • Incidents
  • Analytics
    • Analytics
    • Value stream
  • Wiki
    • Wiki
  • Activity
  • Create a new issue
  • Issue Boards
Collapse sidebar
  • tails
  • sysadmin
  • Issues
  • #17702
Closed
Open
Created May 10, 2020 by intrigeri@intrigeriMaintainer1 of 2 tasks completed1/2 tasks

Ensure check-mirrors.rb does not run forever

Originally created by @intrigeri on #17702 (Redmine)

Today on misc.lizard there were 3 instances of that cronjob running: 1 current + 2 that started respectively on April 22 and 23. The 2 oldest ones were all stuck waiting for a wget command to complete.

This might explain some of the leftover temporary directories we’ve seen recently (tails#17679 (closed)): one hypothesis is that these directories were not leftovers of “old instances of check-mirrors.rb was killed and did not clean up after itself”, but rather a consequence of “weeks-old check-mirrors.rb instances were still running”.

Either way, I think we need 2 things:

  • sysadmin: a lock on the cronjob (like we have for most similar cronjobs already) and possibly a timeout (see e.g. timeout(1) which seems to fit the bill), in the cronjob. In case the timeout expires or the lock can’t be taken by a new cronjob instance, I think the mirrors team should be made aware.
  • check-mirrors.rb itself: tails#17736

Related issues

  • Related to tails#17679 (closed)
Edited Apr 06, 2021 by Zen Fu
To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information
Assignee
Assign to
Time tracking