Network Monitoring and Alerts: What to Watch and Alert On

An alert is only useful if someone reads it, and nobody reads a stream that is mostly noise. Monitoring a network isn’t installing a tool; it’s deciding what you need to know, how soon you need to know it, and who has to act when it happens.

Think of a car’s dashboard. It has two kinds of instruments. Gauges (speed, fuel, temperature) show a value continuously; you glance at them when you want to, and their value is in the trend: the fuel needle creeping toward empty. Warning lights stay dark until something needs your attention now: oil pressure, brakes. A good dashboard has many gauges and few lights. And everyone knows what happens to a warning light that is always on: after a week, nobody sees it anymore. Network monitoring works the same way: graphs are the gauges, alerts are the warning lights, and most monitoring setups fail by turning gauges into lights.

This guide is the map. It covers what devices already tell you and how, what to put on graphs and what to alert on, where to start, and how to keep alerts meaningful. The two protocols devices use to report on themselves have their own articles: What Is Syslog? and What Is SNMP?

Why Monitor at All?

Monitoring exists because faults happen when nobody is looking. A link that fills up at 2 p.m. is only visible to someone watching at 2 p.m. A port that drops for a minute at 3 a.m. leaves no trace by morning. A person can’t watch continuously; a program can.

The deeper reason is language. “The internet is slow” is an opinion, not a measurement: which site, what time, how slow? Without those three answers, no decision can be built on it. Monitoring turns opinions into measurements, and a problem that can be measured becomes solvable, even before it’s solved.

In practice, capacity problems don’t arrive suddenly; they grow quietly. Branch links are sized to save money, then everyone joins an online meeting at the same time and the link saturates. Without a trend graph, you can only guess at that complaint.

When don’t you need it? On one switch and a handful of devices, a monitoring server, software to maintain, and a dashboard to read may cost more than they return. The signs you’re past the threshold: more than one site, a leased line you pay for by capacity, or learning about outages from users. The last is the clearest.

What Devices Already Tell You: Logs and Metrics

Before choosing a tool, know what’s already being said. Network devices report on themselves through two channels, and they answer different questions.

Metrics are numbers, polled on a schedule, mostly over SNMP. A monitoring server asks each device every few minutes for its counters: bytes through an interface, CPU load, memory, temperature. One reading means little; two readings a few minutes apart give a rate, and a series of rates gives a trend. Graphs are drawn from metrics. SNMP also works the other way: a device can send a trap the moment something happens, without waiting to be asked.

Logs are sentences, pushed by the device as events happen, usually over syslog. Which port went down, who logged in, which process failed, which configuration changed. An incident investigation is read from logs.

A number doesn’t tell the story of an event, and a sentence doesn’t show a trend. SNMP says the CPU is at 80 percent; syslog says which process is causing it. A proper setup collects both. The mechanics, including what a syslog line’s leading number means, why SNMP’s community string is readable to anyone on the network, and a Windows firewall rule that silently blocks trap receivers, are measured in the two protocol articles: syslog and SNMP. Collecting the logs in one place is its own job, covered step by step in setting up a syslog server on Ubuntu.

Gauges vs. Warning Lights: Trends and Alarms

Capacity belongs on graphs; availability and failures belong on alerts. That one sentence prevents most monitoring mistakes.

Capacity changes slowly and needs context. A link at 70 percent is fine at noon and worrying if it was at 40 percent last month. Nobody should be woken up for that; someone should see the curve in a weekly review and plan an upgrade. Graphing tools built on round-robin databases (Cacti is the classic example) are designed for exactly this: long-term curves, low effort, easy to read. Setting one up and checking its first graph against the raw counters is covered in installing Cacti on Ubuntu.

Availability and failures need action now. A core switch that stops answering, a link that goes down, a disk that fills up, a UPS switching to battery: these are warning lights. Tools built around triggers and notifications (Zabbix is a common example) turn conditions into alarms.

The line between the two is sharp enough to state as a rule of thumb: one tool graphs capacity, the other turns availability into alarms. A graphing tool isn’t designed for complex trigger rules or modern notification integrations; an alerting tool is a poor place to read a year of bandwidth history. Mature environments run both side by side, and some add a third piece for inventory, so each question has one obvious place to look.

Where to Start: The First Graphs

With dozens of devices and hundreds of interfaces, the question isn’t what could be graphed but what goes on the screen first. Field experience gives a clear order.

First: the port that connects to your ISP’s router. All of the organization’s internet traffic passes through that one interface, so its graph answers both capacity and “why is the internet slow” at once. It’s the first place to look during an outage, but its real value shows on ordinary days: seeing what share of a 200 Mbps line is in use at different hours is more illuminating than most people expect. Whether the line hits its ceiling every morning or empties at lunch only shows on that curve.

Read the maximum, not the average. A graph usually shows current, average, and maximum values. For capacity decisions, the maximum is the one that matters: if a 100 Mbit link’s peak sits at 95, it has started to saturate even if the average looks comfortable. And know which direction you’re reading: inbound and outbound are defined relative to the monitored interface, not to you, so “inbound” on an uplink port is traffic arriving from the ISP, and the same data appears with the opposite name on the neighboring port.

Then compare like with like. When the screen fills up, resist putting five properties of one device side by side. Put the same property of as many different devices as fit: the internet uplink of every site, for example. The value is in the comparison. Busier sites use more bandwidth, which is normal; the danger is looking at one site’s uplink and concluding “they’re all 200 Mbps, so they’re all fine,” while the site that isn’t on the screen is the one at its ceiling.

Put it on a wall. A spare monitor or TV at head office showing a two-column grid of the most critical graphs moves monitoring off desktops and into the room. When eight graphs are always in view, a site hitting its ceiling or a switch going quiet gets noticed without anyone opening a dashboard.

Beyond Traffic: Device Health

Bandwidth isn’t the only thing SNMP can graph. The physical and system health of devices matters just as much, and it’s where monitoring finds problems no one would have guessed.

You may have put the same model of switch in every branch, but their health doesn’t have to match. What decides it is usually not the device but its environment: how clean the site is, how hot the room gets, whether the equipment room has cooling, how well the cabinet is ventilated. Two identical switches, one in a cool, dust-free cabinet and one in a storeroom that gets afternoon sun, age on completely different curves.

Worth graphing besides traffic:

• Temperature, CPU, and memory of switches, routers, and firewalls.
• Disk usage on servers, to see weeks ahead when a disk will fill instead of finding out when it does.
• UPS status: battery charge, load percentage, and remaining runtime.
• Latency and packet loss to key destinations, which show a link degrading long before it fills up.
• Environment sensors in the equipment room, if they speak SNMP.

Listening for the Crackle: Catching Faults Early

Before a radio link fails completely, the static gets worse. Networks do the same. Rising interface error counters on a port, memory on a switch creeping up week after week, a device’s temperature climbing a few degrees every summer: each is a quiet warning that an outage is on its way. Treat those small signals seriously and put the device into maintenance, and the outage never happens.

Two practices make this work. Graph the counters that indicate trouble, not just the ones that indicate load: errors and discards per interface, not only bits per second. And review the slow curves on a schedule, weekly or monthly, because nobody notices a gradual climb in daily glances.

Syslog carries the same kind of early warning, with a twist: a message repeating every few seconds isn’t information, it’s a symptom. In the lab behind What Is Syslog?, a switch that had run for months without a single complaint was discarding a protocol message it couldn’t understand every two seconds, and 99 percent of everything it logged was that repetition. The repetition didn’t make it unimportant; it was the only sign that something upstream was wrong.

Keeping Alerts Meaningful

Alert fatigue is the monitoring failure everyone recognizes: so many alerts fire that people stop reading them, and the one that matters is lost in the crowd. A year after setup, the warning light is just part of the dashboard. Four rules keep alerts meaningful.

Alert only on what needs action. If the correct response to an alert is “note it and carry on,” it belongs on a graph or in a report, not in someone’s pocket. Every alert should answer: what does the person receiving this do next?

Give every alert an owner. A notification should land with a named person or an on-call rotation, not a shared mailbox everyone assumes someone else reads. An alert sent to everyone is an alert sent to no one.

Set thresholds from history, not from defaults. A graph of the last few months shows what normal looks like for each link and device. Thresholds based on that stop firing on ordinary peaks. Thresholds based on a vendor default fire constantly or never.

Treat an ignored alert as a bug. If an alert fires and nobody acts on it, either the alert is wrong or the process is. Fix one or the other; don’t let it keep firing. The same goes for alerts that fire and then clear on their own every day: they are teaching people to ignore alerts.

One alert class deserves a special mention because it’s so often left unused: SNMP authentication-failure traps. When someone queries a device with the wrong community string, the querying side gets no answer at all, but the device can tell the monitoring server about the failed attempt. It’s silent to the wrong side and visible only to whoever listens on port 162, which makes it one of the cheapest intrusion signals a network has.

Live View or Recorded History?

Monitoring tools usually offer two ways to look. The default is recorded history: the poller measures every few minutes, stores the values, and you go back to the graph when you need it. Many tools also have a real-time mode that polls one graph every few seconds and draws a short live window.

Which is right depends on whether someone is at the screen. With an operator actively watching a problem, real-time is invaluable: you see the effect of a change without waiting five minutes. But the field reality is usually a small team juggling several jobs and opening monitoring only when needed. For them, recorded history is far more practical. A live screen nobody watches is worth nothing; the value is in a faithful record you can go back to when something breaks.

Whichever you use, five-minute polling hides short peaks. A spike that lasts seconds disappears into a five-minute average, which is why a link can drop calls while its graph looks fine. Poll critical links more often if short saturation matters, and use traps for events that can’t wait.

Foundations: Time, Access, and Retention

Three things sit underneath every graph and alert, and each fails quietly.

Time. Graphs and logs are only as good as their clocks. A monitoring server with a drifting clock shifts every curve, and graphs from different devices stop lining up; a log from one device and a graph from another can appear to show events in the wrong order. Sync the monitoring server and every device to the same reliable time source before trusting any timeline. How that time source works, and what a stratum looks like on the wire, is in What Is NTP?

Access. Monitoring protocols are powerful and, in their older versions, unprotected: SNMPv2c sends its community string in plain text, and syslog over UDP sends log lines anyone on the path can read. Keep monitoring traffic on a management network, use SNMPv3 where devices support it, and restrict who can query devices. Keeping that management traffic in its own segment is covered in the network segmentation guide.

Retention. Collecting everything forever is expensive and pointless. Decide how long graphs and logs are kept, filter noise at the source with sensible severity thresholds, collapse repeated messages, and archive old data. If your logs fall under a retention obligation, that decision isn’t yours alone, and the retention period, integrity protection, and reliable timestamps are built around the collector, not provided by the protocol. Plan disk for it: in the syslog lab, one idle switch produced about 11 MB of logs a day.

Conclusion: Installing Is Easy, Watching Is the Work

Installing a monitoring tool, adding a device, and drawing the first graph takes half a day. The real work is making sure someone looks. The most common failure isn’t a missing tool; it’s a tool set up “so we have one” and left ownerless, a dashboard nobody opens six months later.

A short checklist to start from:

• Graph the ISP uplink first, then inter-switch uplinks, then core device health.
• Read maximums, not averages, for capacity decisions.
• Compare sites side by side on the same screen.
• Collect both channels: SNMP for numbers, syslog for events.
• Alert only on what needs action, and give every alert a named owner.
• Review slow curves on a schedule, including error counters and temperatures.
• Sync every clock to one time source before trusting timelines.

The protocol articles in this guide show what those channels actually carry, measured on a real switch: What Is Syslog? and What Is SNMP?