Improve the way we monitor mirrors speeds and handle slow mirrors
The current approach with check-mirrors.sh
doesn't give us a good historic overview of mirror speeds (only e-mail notifications), and we would like to improve that.
Initial proposal:
- Use https://exchange.nagios.org/directory/Plugins/Network-and-Systems-Management/check_speed-2Esh/details
- Set it up with the current latest image URL
- Iterate from there
- Sysadmins will be notified when a mirror is slow, then we can:
- Check history
- If it's always good, acknowledge for a couple of days
- Elif it's bumpy for a bit longer, adjust priority/weight and contact mirror admin
- Else remove it completely
Note: we could even move the canonical configuration of the mirror pool to hiera and have puppet generate the mirrors.json file so we don't have to keep mirrors.json and icinga config in sync manually.
Edited by Zen Fu