Had an issue recently on a relatively old Ubuntu server that had had a distribution upgrade between LTS versions.
Months after the upgrade, I realised that at least some of the rsyslog log files were logging new lines to the .log.1 file, not to the .log file – for example, any new SSH logging was written to /var/log/auth.log.1, not /var/log/auth.log as I expected.
This broke a few things – fail2ban stopped blocking SSH attempts, because it was monitoring auth.log and not auth.log.1, and a Grafana visualisation showing SSH login attempts broken because it was parsing a file that never changed.
The problem appears to be in logrotate. The config file /etc/logrotate.d/rsyslog looks something like this:
...
/var/log/messages
{
rotate 4
monthly
missingok
notifempty
compress
delaycompress
sharedscripts
postrotate
invoke-rc.d rsyslog rotate > /dev/null
endscript
}
I finally noticed there was a rsyslog.dpkg-dist file in there which looks like this:
...
/var/log/messages
{
rotate 4
monthly
missingok
notifempty
compress
delaycompress
sharedscripts
postrotate
/usr/lib/rsyslog/rsyslog-rotate
endscript
}
Changing the postrotate line seems to have fixed the issue. Looking at the script it seems to check some systemd directory, so I guess some core config changed at some point between whatever ancient distribution this was running (Debian 8, I think) and the current one (Debian 10).