Skip to content

Start typing to search

Network Monitoring and Alerts: What to Watch and Alert On

How to Set Up a Syslog Server on Ubuntu with rsyslog

Build a central syslog server on Ubuntu 24.04 with rsyslog: open UDP 514, split logs per sender, connect a switch, plan disk and rotation, and add TLS on 6514.

Written by

Setting up a central syslog server on Ubuntu comes down to writing one configuration file. The long part isn’t the setup; it’s everything after: deciding what each device sends, keeping what arrives readable, and making sure the disk doesn’t fill.

Advertisement

This article is part of the network monitoring and alerts guide. The previous article compared syslog to a court clerk who writes down what is said, as it is said, with the time. This one picks up where the clerk stops. The transcripts are written; where do they go? To the court archive. And an archive’s value isn’t in how many shelves it has. It’s in finding the one you’re looking for.

Below, on a real Ubuntu 24.04 server I named sercesyslog, I set up the collector, connect my lab switch to it, and take in the first real records. Along the way, the most-copied first step turns out to do nothing on this machine. Commands, outputs, and names come from the author’s lab; outputs are left exactly as captured, including the few that were written in Turkish.

Before You Start: Server, Disk, and Time

This guide assumes the Ubuntu Server that will run the collector is already installed. The machine here is modest: Ubuntu 24.04.4 LTS, 2 vCPUs, 1.9 GB of RAM. Collecting syslog doesn’t need CPU; it needs disk.

Half the Disk May Be Unallocated

On a log server, disk matters more than CPU or memory, so before you start, confirm how much space you really have. On an Ubuntu installed with LVM, it may not be what you think:

Bash
lsblk -o NAME,SIZE,FSTYPE,TYPE,MOUNTPOINT
sda                         20G             disk
├─sda1                       1M             part
├─sda2                     1.8G ext4        part /boot
└─sda3                    18.2G LVM2_member part
  └─ubuntu--vg-ubuntu--lv   10G ext4        lvm  /

Look at the last line. I gave the machine a 20 GB disk; the LVM volume group is 18.2 GB, but only 10 GB went to the root volume. Ubuntu’s guided install deliberately leaves the rest unallocated.

On a machine that accumulates logs, that’s a quiet trap: you think the disk is 20 GB, it’s really 10, and the day it fills, logging just stops. On this server I added the free 8.2 GB to root, bringing it to 18 GB. Know the cost before you do the same: if you give all free space to root, there’s none left for LVM snapshots, and ext4 can grow online but can’t shrink, so the change is effectively permanent. On a log server, taking all of it is usually right; if you rely on snapshots, leave a margin.

Time Comes Before the Collector

The most critical field in an archive is the date. If two devices’ clocks disagree, their records side by side show events in the wrong order, and that’s hard to diagnose because every record is individually correct. So the order is: time first, collector second. Point every device, and the collector, at the same time source.

rsyslog Is Already Installed: Why We Skip apt

The server is ready. Next is software, and the first step is exactly the one to skip. Most guides start with apt install rsyslog. I measured.

What Ships in the Box

On a freshly installed server, with nothing done:

Bash
dpkg -l rsyslog | tail -1
systemctl is-active rsyslog
systemctl is-enabled rsyslog
ii  rsyslog  8.2312.0-3ubuntu9.3  amd64  reliable system and kernel logging daemon
active
enabled

The package is installed, the service is running, and it starts at boot. apt install on this machine just reports rsyslog is already the newest version and exits.

Running Isn’t the Same as Listening

So is the job done? A second measurement on the same machine:

Bash
sudo ss -lunp | grep 514
sudo ss -ltnp | grep 514
grep -rE '^\s*module\(load="(imudp|imtcp)"' /etc/rsyslog.conf /etc/rsyslog.d/
UDP 514 DINLENMIYOR
TCP 514 DINLENMIYOR
HICBIRI YUKLU DEGIL

The check script printed its verdicts in Turkish: “UDP 514 not listening,” “TCP 514 not listening,” “none loaded.” rsyslog is up but listening on nothing. The default configuration collects only the machine’s own logs; the modules that accept syslog from the network (imudp or imtcp) aren’t even loaded. The archive building is open and the clerk is at the desk, but the door is shut. Paperwork from outside can’t get in, and nobody notices, because the building’s own work goes on fine.

The job isn’t installing a package; it’s opening the door.

Opening the Collector: the imudp Module and Port 514

Don’t edit the existing files; add your own under /etc/rsyslog.d/. Package updates overwrite their own files and leave yours alone.

One File, Four Jobs

Bash
sudo nano /etc/rsyslog.d/10-uzak-loglar.conf

The file and template names are Turkish (uzak means “remote”); they’re kept as they are because every output below refers to them.

INI
# Collect syslog from remote devices
module(load="imudp")
input(type="imudp" port="514")

# One directory per sender, one file per program
template(name="UzakLog" type="string"
         string="/var/log/uzak/%HOSTNAME%/%PROGRAMNAME%.log")

# Every non-local source goes to the remote directory, and processing STOPS
if $fromhost-ip != "127.0.0.1" then {
    action(type="omfile" dynaFile="UzakLog"
           createDirs="on" fileCreateMode="0640" dirCreateMode="0755")
    stop
}
  1. module(load="imudp") loads the module that accepts syslog from the network. Nothing below works without it.
  2. input(type="imudp" port="514") says which port to listen on.
  3. template decides which file each record goes to. %HOSTNAME% is the sending device, %PROGRAMNAME% the process that produced the record.
  4. stop ends processing there. Why that’s essential will show up in a measurement shortly.

Validate the syntax before applying:

Bash
sudo rsyslogd -N1
rsyslogd: version 8.2312.0, config validation run (level 1), master config /etc/rsyslog.conf
rsyslogd: End of config validation run. Bye.

No error lines means clean. Restart and confirm the door is open:

Bash
sudo systemctl restart rsyslog
sudo ss -lunp | grep 514
UNCONN 0 0    0.0.0.0:514    0.0.0.0:*    users:(("rsyslogd",pid=3193,fd=6))
UNCONN 0 0       [::]:514       [::]:*    users:(("rsyslogd",pid=3193,fd=7))

Now it’s listening.

Why the 10- Prefix Matters

rsyslog reads the files under /etc/rsyslog.d/ in alphabetical order and applies rules in that order. Ubuntu’s own rules live in 50-default.conf. The 10- prefix makes our rules run before them, which is what lets stop work: it prevents every rule after it from running. Name the file 60- and Ubuntu’s defaults would already have processed the record; stop would be too late. In archive terms: sort incoming paperwork at the door, not after it has been handed around inside.

Adding TCP: Two More Lines

A message lost over UDP leaves no trace. To see loss, open TCP as well; both can run together and devices send over whichever they support. Add two lines to the same 10-uzak-loglar.conf, right below the UDP lines:

# --- These two lines are ALREADY in the file, leave them alone ---
module(load="imudp")
input(type="imudp" port="514")

# --- ADD THESE TWO (right below the lines above) ---
module(load="imtcp")
input(type="imtcp" port="514")

# --- The rest of the file (template and if block) stays as it is ---
Bash
sudo rsyslogd -N1 && sudo systemctl restart rsyslog
sudo ss -ltnp | grep 514
LISTEN 0 25    0.0.0.0:514    0.0.0.0:*    users:(("rsyslogd",pid=4021,fd=8))

On a Linux sender the difference is one character: @ is UDP, @@ is TCP. TCP makes loss visible but doesn’t encrypt anything; that’s covered in the TLS section below.

Verify Before Touching the Switch

The door is open. Before connecting a device, prove the pipeline works on its own, so that when “logs aren’t arriving” later, you can tell whether the collector or the sender is at fault. Skip this and you’re debugging a fault with two unknowns.

Testing the Network Path with logger

logger generates a syslog message by hand. The key flag is -n: it sends the message over the network to the given address instead of the local socket. Using the server’s own IP tests the real network path:

Bash
logger -n 192.168.1.46 -P 514 -d -t lab-testi "toplayici ag yolu dogrulama"

The directory should appear on its own (createDirs="on" does that):

Bash
sudo find /var/log/uzak -type f
sudo cat /var/log/uzak/*/lab-testi.log
/var/log/uzak/sercesyslog/lab-testi.log
2026-08-18T22:30:17.601035+00:00 sercesyslog lab-testi toplayici ag yolu dogrulama

The test message (Turkish for “collector network path check”) arrived and was written to the sender’s own directory.

Leak Check: Did the Record Land Twice?

This is the most often skipped check. If stop isn’t working, remote records land both in their own directory and in the server’s own /var/log/syslog: double the disk, and your server’s own logs buried under other devices’ noise.

Bash
sudo grep -c "toplayici ag yolu dogrulama" /var/log/syslog
0

Zero. The record went only where it should.

When Logs Don’t Arrive

In order, cheapest check first:

  1. Is the collector listening? sudo ss -lunp | grep 514. If empty, the module didn’t load; read rsyslogd -N1.
  2. Do packets reach the server? sudo tcpdump -ni any udp port 514. If nothing appears, the problem is the network or the sender, not rsyslog.
  3. Firewall: sudo ufw status. If active, sudo ufw allow 514/udp is needed. It was inactive on this machine.
  4. Write permissions: if no directory appears under /var/log/uzak, check createDirs="on" and the user rsyslog runs as.

The Sender: Connecting a Switch

The collector works. Now a real device: the lab switch, a managed enterprise switch.

One Line of Configuration

On network devices this is usually one line. On this switch’s operating system:

configure
logging 192.168.1.46
exit

The command drops you into a sub-mode offering three settings: description, level (the minimum severity to send), and port (default 514). The defaults are right for this setup.

The syntax took me three attempts, and the traps are worth writing down, because two of them are vendor quirks you’d never guess:

  • It’s logging <ip>, not logging host <ip>, which is the form many other platforms use.
  • There’s no show logging hosts command. To see the configured server, use show running-config | include logging. The configuration line the device writes itself is always more reliable than help output.
  • ? doesn’t always mean help. This one isn’t in the Turkish original. On most network operating systems, typing a command followed by ? lists the possible next arguments and changes nothing. On this switch, logging 192.168.1.46 ? didn’t list anything: it executed the line, configuring the server and entering the sub-mode. In other words, “checking what the command accepts” changed the configuration. On an unfamiliar platform, check what ? does on a harmless command first, and never use it on a line you don’t want applied.

Save It, or Lose It in the Next Power Cut

On network devices, the running configuration and the configuration read at boot are two separate things. The command above changed only the running configuration; if the switch restarts, it stops sending logs and nobody tells you. Before and after:

show startup-config | include logging
copy running-config startup-config
show startup-config | include logging
(bos)

This operation may take few minutes.
Are you sure you want to save? (y/n) y
Configuration Saved!

logging 192.168.1.46

The first query returned nothing ((bos) is the capture script’s Turkish placeholder for “empty”); after saving, the line is there. Now it’s permanent.

Split per Sender: One File Hides Everything

The switch is connected and records are flowing. Here’s what the template does:

Bash
sudo find /var/log/uzak -type f -printf "%s bayt\t%p\n" | sort -rn
21386 bayt	/var/log/uzak/sercebilisimsw01-1/DOT1S.log
  147 bayt	/var/log/uzak/sercebilisimsw01-1/CLI_WEB.log
   83 bayt	/var/log/uzak/sercesyslog/lab-testi.log

(bayt is bytes; the format string was typed in Turkish.) The switch got its own directory, and inside it records were split by the process that produced them, automatically, from %PROGRAMNAME%. The value is in the sizes. CLI_WEB.log is 147 bytes and holds one line:

<190> Aug 19 07:01:59 sercebilisimsw01-1 CLI_WEB[emWeb]: %% [CLI:ilker:192.168.1.115] User has succesfully logged in

Who logged in to the switch as an administrator, when, and from where. It’s the first line you’d look for in a security question, and it’s visible only because it’s in its own file. In one combined file it would be lost in 21 KB of noise. An archive gives each court its own shelf and each case type its own folder; piling everything into one room is also “archiving,” but you’ll never find anything.

Disk and Rotation: How Much Does One Switch Write?

The Measurement: Sixty Seconds, Sixty Lines

The standard field answer is “a switch barely talks; a port drops now and then.” I measured: after connecting it, I did nothing (no logins, no config changes) for sixty seconds.

60 satir / 8.670 bayt

That’s “60 lines / 8,670 bytes,” recorded in Turkish. At that rate:

PeriodValue
Per day~11 MB
Per year~4.3 GB
Time to fill an 18 GB disk~1,548 days

One switch, with nothing happening. A firewall logging every connection multiplies that, and if a retention rule requires you to keep logs for a year or more, the disk plan follows directly from this table.

Cutting Noise at the Source

The content matters more than the size. Of 149 lines collected, 148 were the same two messages repeating: the switch discarding a protocol message it couldn’t make sense of every two seconds. 99 percent of the volume was one sentence copied; the one meaningful record was 147 bytes. Filtering happens in three stages, and the first is the cheapest:

  1. Threshold at the sender. Set the minimum severity each device sends; the level setting in the sub-mode is for that. Leaving debug on permanently is the biggest source of noise.
  2. Separation at the collector. Already done: one file per sender and process.
  3. Rotation and retention. logrotate compresses and archives on a schedule. Ubuntu’s built-in rule in /etc/logrotate.d/rsyslog covers only its own files like /var/log/syslog; it knows nothing about /var/log/uzak, so that directory never rotates and grows forever.

The remote directory needs its own rule:

Bash
sudo nano /etc/logrotate.d/uzak-loglar
INI
/var/log/uzak/*/*.log {
    daily
    rotate 365
    compress
    delaycompress
    missingok
    notifempty
    create 0640 syslog syslog
    sharedscripts
    postrotate
        /usr/lib/rsyslog/rsyslog-rotate
    endscript
}

Three lines carry the decisions:

  • daily + rotate 365 keeps one year. That number is your policy decision, not a technical default: 365 matches, for example, the 12 months of audit log history PCI DSS v4.0 asks for; with no obligation, a much shorter window is fine. Pick it deliberately and write down why.
  • create 0640 syslog syslog keeps ownership on the new file. On this machine remote logs are created as syslog:syslog; on another setup, confirm with ls -l /var/log/uzak/*/, because the wrong owner makes rsyslog stop writing.
  • The postrotate block tells rsyslog to reopen its files. Skip it and rsyslog keeps writing to the deleted file, so records go nowhere, without an error.

Test the rule without applying it:

Bash
sudo logrotate -d /etc/logrotate.d/uzak-loglar

-d changes nothing; it only prints what it would do. Skip all three stages and the ending is predictable: the disk fills, logging stops, and you find out on the day you need the logs. For the numbers side of monitoring, graphs of counters polled from the same devices, see What Is SNMP? One keeps the event, the other the curve.

Encrypted Collection with TLS

Disk and noise are handled. But so far everything crosses the network in the clear. Syslog is plain text; anyone capturing packets on the same network reads usernames, device names, and configuration changes. Measured: one message to port 514 while capturing.

Bash
sudo tcpdump -ni lo -A "udp port 514" -c 2
logger -n 127.0.0.1 -P 514 -d -t gizli-test "PAROLA=CokGizli123 duz metin gidiyor"
<13>1 2026-08-19T00:58:11+00:00 sercesyslog gizli-test - - PAROLA=CokGizli123 duz metin gidiyor

The whole sentence, password included (Turkish: “PASSWORD=VerySecret123 going as plain text”), sits in the open.

Step 1: Create Your Own Certificate Authority

You don’t need a public certificate; an internal authority is enough for an internal network.

Bash
sudo mkdir -p /etc/rsyslog.d/tls && cd /etc/rsyslog.d/tls

# 1) The authority itself: valid ten years, the signing side
sudo openssl req -x509 -newkey rsa:2048 -nodes -days 3650 \
  -keyout ca-key.pem -out ca-cert.pem \
  -subj "/O=Example Lab/CN=Example Syslog CA"

# 2) A request for the collector
sudo openssl req -newkey rsa:2048 -nodes \
  -keyout sunucu-key.pem -out sunucu.csr \
  -subj "/O=Example Lab/CN=sercesyslog"

# 3) Sign it. The SAN line is essential: clients verify the name from it
printf "subjectAltName=DNS:sercesyslog,IP:192.168.1.46\n" | sudo tee san.cnf
sudo openssl x509 -req -in sunucu.csr -CA ca-cert.pem -CAkey ca-key.pem \
  -CAcreateserial -out sunucu-cert.pem -days 825 -extfile san.cnf

(sunucu means “server”; the file names match the configuration below and the error messages later.)

rsyslog runs as syslog, not root, so set ownership:

Bash
sudo chown -R root:syslog /etc/rsyslog.d/tls
sudo chmod 750 /etc/rsyslog.d/tls
sudo find /etc/rsyslog.d/tls -name "*.pem" -exec chmod 640 {} +

Step 2: Add a TLS Listener to the Collector

The TLS driver doesn’t ship by default:

Bash
sudo apt install rsyslog-gnutls

Then add this block to 10-uzak-loglar.conf, below the UDP lines and above the template:

# --- The UDP block is ALREADY in the file, leave it alone ---
module(load="imudp")
input(type="imudp" port="514")

# --- ADD THIS BLOCK: register the certificates and open 6514 ---
global(
  DefaultNetstreamDriver="gtls"
  DefaultNetstreamDriverCAFile="/etc/rsyslog.d/tls/ca-cert.pem"
  DefaultNetstreamDriverCertFile="/etc/rsyslog.d/tls/sunucu-cert.pem"
  DefaultNetstreamDriverKeyFile="/etc/rsyslog.d/tls/sunucu-key.pem"
)
module(load="imtcp" StreamDriver.Name="gtls"
       StreamDriver.Mode="1" StreamDriver.AuthMode="anon")
input(type="imtcp" port="6514")

# --- The template and if block stay as they are ---

StreamDriver.Mode="1" makes TLS mandatory; AuthMode="anon" doesn’t require clients to present a certificate: the server proves its identity, the client only listens. For mutual authentication, issue client certificates and use AuthMode="x509/name".

Bash
sudo rsyslogd -N1 && sudo systemctl restart rsyslog
sudo ss -lntp | grep 6514
LISTEN 0 25    0.0.0.0:6514    0.0.0.0:*    users:(("rsyslogd",...))

Step 3: The Sender Side

A Linux sender needs a copy of ca-cert.pem to verify the server. The configuration is one block:

INI
global(DefaultNetstreamDriverCAFile="/etc/rsyslog.d/tls/ca-cert.pem")
action(type="omfwd"
       target="192.168.1.46" port="6514" protocol="tcp"
       StreamDriver="gtls" StreamDriverMode="1" StreamDriverAuthMode="anon")

Searching the Capture Again

The same password, this time over 6514, with the capture running:

Bash
sudo tcpdump -ni any -A "tcp port 6514" -c 12
logger -t tls-test "PAROLA=CokGizli123 sifreli gidiyor"

Searching the capture for PAROLA:

eslesme sayisi: 0

“Match count: 0.” All that’s visible is TCP headers:

IP 192.168.1.46.41900 > 192.168.1.46.6514: Flags [S], seq 766642195
IP 192.168.1.46.6514 > 192.168.1.46.41900: Flags [S.], seq 3200597873
IP 192.168.1.46.41900 > 192.168.1.46.6514: Flags [.], ack 1, win 512

And the record reached the collector intact:

2026-08-19T00:58:18+00:00 sercesyslog tls-test: PAROLA=CokGizli123 sifreli gidiyor

Same sentence, same network, two ports: readable on one, not a word on the other.

Stuck on “Permission denied”? It May Not Be Permissions

On the first attempt the certificates were under /etc/rsyslog-tls/, and the service said:

rsyslogd: error: defaultnetstreamdriverkeyfile '/etc/rsyslog-tls/sunucu-key.pem'
could not be accessed: Permission denied

The reflex is to check permissions. But ls -l showed correct ownership and sudo cat could read the file. The kernel log had the answer:

apparmor="DENIED" operation="open" profile="rsyslogd"
name="/etc/rsyslog-tls/sunucu-key.pem" requested_mask="r" fsuid=0

Note fsuid=0: the process is trying to read as root and still being refused, because the barrier is the AppArmor profile, not file permissions. The diagnostic command:

Bash
sudo journalctl -k --since "-5min" | grep -i apparmor

Two fixes: move the certificates under /etc/rsyslog.d/, which the profile already allows (as this guide does), or add your own rule under /etc/apparmor.d/rsyslog.d/. The first is sturdier, because package updates don’t affect it.

Security Boundaries: Who Can Reach Port 514?

Encryption keeps records unreadable in transit. It doesn’t tell you who sent them. Syslog doesn’t authenticate senders: the collector assigns a record to a device by the packet’s source address, and over UDP that can be forged. Any machine on the network can produce records that appear to come from your switch.

The protocol doesn’t solve that, so the fix lives around it: open 514 not to the whole network but only to known senders.

Bash
sudo ufw allow from 192.168.1.2 to any port 514 proto udp
sudo ufw enable

It doesn’t replace authentication, but it raises the bar noticeably. By the same logic, don’t expose the collector beyond the management network.

Is this enough for a retention requirement? No. This setup solves collection: records leave the devices and accumulate in one place, split by sender. Retention rules usually also demand a guaranteed retention period, protection against tampering, and trustworthy timestamps, none of which is rsyslog’s job. Collection is the layer those guarantees are built on; without it they can’t exist, but it isn’t them.

Advertisement

Conclusion: Setting Up Is Easy, Keeping the Archive Alive Is the Work

A central syslog server on Ubuntu fit into one configuration file and didn’t even need a package install: load a module, listen on a port, split by sender, say stop.

The hard part is what comes after. The measurement showed that one switch writes sixty lines a minute while nothing is happening, and 99 percent of them tell you nothing. Set it up and walk away, and a year later you have 4.3 GB in a directory nobody opens.

An archive isn’t valuable because its shelves are full. It’s valuable because you can find the transcript you’re looking for.

Questions about running a syslog server

No. Current Ubuntu releases, 24.04 included, ship with rsyslog installed, running, and enabled at boot. The apt install step most guides start with does nothing on these machines. What's missing is the network listener: the default configuration only collects the machine's own logs.
It depends on how chatty your devices are, not how many there are. In the lab, one switch with nothing happening produced about 11 MB a day, roughly 4.3 GB a year. A firewall logging connections multiplies that. Measure a day's output per device type and plan from there.
UDP 514 is the default and works fine on a lightly loaded internal network. The one real reason to switch to TCP is to notice loss, because a UDP message lost in transit leaves no trace. If records must not be readable in transit, use TLS on port 6514.
No. It solves collection: logs leave the devices and land in one place, split by sender. Retention rules usually add a guaranteed retention period, tamper protection, and reliable timestamps, which aren't rsyslog's job. Collection is the prerequisite for all of that, not a substitute.

This article is adapted from a guide the author first published in Turkish on sercebilisim.com: Syslog Sunucusu Kurulumu: Ubuntu'da Merkezi Log

Advertisement

Written by

İlker Pehlivan

Network and systems engineer, founder of Serçe Bilişim

I run the networks and servers that other people's work depends on. Before founding my own consultancy I administered the backbone network, firewalls and core systems of a large multi-site organisation with thousands of users. I write about the things that actually broke.