[arch-general] syslog-ng + systemd-journald = no logs for syslog-ng
## vanilla /etc/syslog-ng/syslog-ng.conf # grep -v '^#' /etc/systemd/journald.conf [Journal] Storage=none ForwardToSyslog=no ForwardToKMsg=no ForwardToConsole=no ForwardToWall=no # note: ForwardToSyslog set to no as per the first few paragraphs of # https://wiki.archlinux.org/index.php/syslog-ng # I tried ForwardToSyslog=yes as well, doesn't work either # # systemd-journald and syslog-ng have been (re)started # logger -p mail.info "mail argh" # logger "user notice argh" # logger -p local0.alert "local argh" # cd /var/log/ # grep -l argh * # # nothing to be seen here. In fact, the only thing that seem to log anything "live" is syslog itself. Note that this setup used to work just fine until prior to some 10 to 15 days ago, i.e., I expect an update broke it. This is with an up-to-date system. (syslog-ng 3.6.2-1 and systemd 218-1) Any ideas, anybody? I'd really like to NOT store anything via systemd-journald, in fact, I'd really like to systemctl disable systemd-journald and never be bothered by it again, so please don't tell me I should be just using systemd-journald ;) Regards, -Martin
On Sun, Feb 15, 2015 at 3:37 AM, <Ephaeton@gmx.net> wrote:
## vanilla /etc/syslog-ng/syslog-ng.conf # grep -v '^#' /etc/systemd/journald.conf
It sounds like syslog-ng doesn't understand where the source is - traditionally that's /dev/log, /dev/kmsg and /proc/kmsg - and it's selected by a source module. Check your systemd-journald symlink to ensure it's not broken, you should have a symlink from /dev/log over to a socket like so: $ ls -l /dev/log /run/systemd/journal/dev-log lrwxrwxrwx 1 root root 28 Feb 14 09:05 /dev/log -> /run/systemd/journal/dev-log srw-rw-rw- 1 root root 0 Feb 14 09:05 /run/systemd/journal/dev-log If that's fine, try generating a processed config like so: $ syslog-ng -f /etc/syslog-ng/syslog-ng.conf --syntax-only --preprocess-into=/tmp/test.cfg --no-caps Inside that /tmp/test.cfg you should see a source section and module to go with it: $ grep ^@module /tmp/test.cfg ... @module system-source ...and a definition of where that's coming from: $ grep -A9 "source src" /tmp/test.cfg source src { channel { source { systemd-journal(); }; # source }; # channel ; internal(); }; I suspect that something in your configuration has become disconnected from the upgrade, and syslog-ng is no longer able to read from the source that it used to 10-15 days ago pre-upgrade...here's the link to the guide, look on page 96, section 6.11: http://www.balabit.com/support/documentation/syslog-ng-ose-3.6-guides/en/sys... hth, -te
On 2015-02-15 08:34:23, Troy Engel wrote:
On Sun, Feb 15, 2015 at 3:37 AM, <Ephaeton@gmx.net> wrote:
## vanilla /etc/syslog-ng/syslog-ng.conf # grep -v '^#' /etc/systemd/journald.conf
It sounds like syslog-ng doesn't understand where the source is - (...)
In fact, you pointed me to the solution. Leonid was suspecting it, and your manual link actually confirmed it. The generated configuration file is "fine", the symlinks are there etc. Here's the relevant portion of the manual text: "If the host is running under systemd, syslog-ng OSE reads directly from the systemd journal file using the systemd-journal() source" (6.11, p96 :) Now, with journald.conf "Storage=None", this cannot work, obviously. systemd-induced design brainfuck spreading... why would one follow a file when you have a socket already in place for some 30 years. I'll have a more detailed look at Leonid's suggestion, as that seems to be the way to go. Thanks Troy.
TL:DR: Syslog-ng in [extra] is kind of broken, so you'll need to a few steps to get it to work in your environment. On Sun, Feb 15, 2015 at 10:37:22AM +0100, Ephaeton@gmx.net wrote:
## vanilla /etc/syslog-ng/syslog-ng.conf # grep -v '^#' /etc/systemd/journald.conf
[Journal] ForwardToSyslog=no
1. Set this to "yes". 2. Replace the stock syslog-ng.service with this: ------ $ cat /etc/systemd/system/syslog-ng.service [Unit] Description=System Logger Daemon Documentation=man:syslog-ng(8) Requires=syslog.socket [Service] ExecStart=/usr/bin/syslog-ng -F ExecReload=/usr/bin/kill -HUP $MAINPID StandardOutput=null [Install] WantedBy=multi-user.target Alias=syslog.service ------ 3. Fix the src{} in syslog-ng.conf: ------ source src { #system(); unix-dgram("/run/systemd/journal/syslog"); internal(); file("/proc/kmsg"); }; ------ 4. Reenable syslog-ng.service and restart syslog. Explanation: Syslog indeed imports data from journald, but _only_ if journald stores the logs (I guess that syslog-ng reads logs from the journald file on-disk). If you don't want journald storage, you'll have to use the "old" way by making systemd forward logs to syslog. The stock syslog-ng.service file is broken because it doesn't pull in syslog.socket. And, no, I haven't filed any bugreports, so go ahead if you feel like it. But I personally recommend simply repackaging syslog-ng without systemd support... HTH, -- Leonid Isaev GPG fingerprints: DA92 034D B4A8 EC51 7EA6 20DF 9291 EE8A 043C B8C4 C0DF 20D0 C075 C3F1 E1BE 775A A7AE F6CB 164B 5A6D
On 2015-02-15 15:50:49, Leonid Isaev wrote:
TL:DR: Syslog-ng in [extra] is kind of broken, so you'll need to a few steps to get it to work in your environment.
1. (...) 2. (...) 3. (...) 4. (Re)enable syslog-ng.service and restart syslog.
A cookie, a mug of hot steaming morning drink and a big thanks for Leonid Isaev. Indeed, that fixed the problem. Damn systemd induced quirks. Regards, -Martin
participants (4)
-
Ephaeton＠gmx.net
-
Leonid Isaev
-
Martin S. Weber
-
Troy Engel