Shipping your Linux system log files using rsyslog

Alex Berber

Most Unix systems these days come with pre-installed rsyslog, which is a great light weight service to consolidate logs.

You can configure rsyslog to monitor logs from vireos system facilities on your local system (kernel, user-level messages, system daemons, security/authorization messages, etc.) and ship them over to Logz.io.

Requirements

The setup assumes that you have a sudo access
Rsyslog version 5.8.0 and above
Allow outgoing TCP traffic to destination port 5000
A common linux distribution

Automatic configuration

Run the following in order to configure your rsyslog daemon

curl -sLO https://github.com/logzio/logzio-shipper/raw/master/dist/logzio-rsyslog.tar.gz && tar xzf logzio-rsyslog.tar.gz && sudo rsyslog/install.sh -t linux -a "ovKAKHOqicsjudKFfLTUkITZlXjHdOQK" -l "listener.logz.io"
 Copy

Manual Configuration

Step 1 Create a new configuration file for Log.io

sudo vi /etc/rsyslog.d/22-logzio-linux.conf
 Copy

Add the additional configuration

#  -------------------------------------------------------
#      Syslog Logging Directives for Logz.io
#  -------------------------------------------------------
# Define the template used for sending logs to Logz.io. Do not change this format.
$template logzioFormat,"[ovKAKHOqicsjudKFfLTUkITZlXjHdOQK] <%pri%>%protocol-version% %timestamp:::date-rfc3339% %HOSTNAME% %app-name% %procid% %msgid% [type=syslog] %msg%\n"
$WorkDirectory RSYSLOG_SPOOL_DIR # where to place spool files
$ActionQueueFileName fwdRule1 # unique name prefix for spool files
$ActionQueueMaxDiskSpace 1g   # 1gb space limit (use as much as possible)
$ActionQueueSaveOnShutdown on # save messages to disk on shutdown
$ActionQueueType LinkedList   # run asynchronously
$ActionResumeRetryCount -1    # infinite retries if host is down
# Send messages to Logz over TCP using the template.
*.*             @@listener.logz.io:5000;logzioFormat
 Copy

Step 2 Restart rsyslog

After editing and saving the file, execute the following command

sudo service rsyslog restart
 Copy

Step 3 Verify that logs are been sent

You can verify that logs are been sent by searching for syslog typed log on Kibana.

Troubleshooting

If you have any trouble with rsyslog you can check our rsyslog troubleshooting guide here.