Centralize our servers' logs
Originally created by @intrigeri on #11880 (Redmine)
Right now we have volatile Journal + some persistent log files managed by rsyslog and individual applications. This sometimes makes it painful to debug problems since one has to cross-match info from various sources. It would be nicer if all our logs landed in a single place.
An initial idea to start brainstorming about it would be:
- on each of our systems, send all logs to journald
- configure these journalds to have volatile storage only (that’s the default and what we currently do) and to forward them to a single journald instance running in a central place
- in the central logging location, either have journald store logs in a persistent manner, or forward them to a fancy system like Graylog (that seems much easier to setup than an ELK stack)
To upload designs, you'll need to enable LFS. More information