24/7 monitoring: what to watch, what to alert on, who to wake up

Effective 24/7 monitoring is not about measuring everything: it is about detecting, as early as possible, whatever degrades the service your users actually experience — and waking a human only when immediate action is required. Three questions shape the whole setup: what to watch, at what threshold to alert, and who takes the call at three in the morning.
Watch the service, not just the servers
Most monitoring projects start with resources: CPU, memory, disk, network. Those metrics are necessary for diagnosis, but they say little about what matters: is the service working for the people who use it? A processor at 90% can be perfectly healthy; an API answering in eight seconds never is.
The metrics that count are read from the user's side: request latency, error rates, queue saturation, abnormally low traffic — often the earliest sign of an incident. Add external probes that replay critical journeys (login, order, payment) from outside your network: it is the only way to see what a customer actually sees. And wherever possible, surface a business metric — orders per minute, cases processed — because it is the one that gives meaning to all the others.
The approach starts from critical user journeys and works down to the infrastructure, never the other way around. For every monitored system, you should be able to answer: which user-facing service degrades if this component fails?
Thresholds that actually mean something
An arbitrary static threshold — "alert if CPU exceeds 80%" — mostly produces noise. Three refinements change everything:
- Persistence: a thirty-second spike is rarely an incident; ten minutes above the threshold usually is. Alert on sustained conditions, not snapshots.
- Trend: a disk that will be full in six hours at the current rate deserves an alert before it fills up. Predictive thresholds warn; static thresholds merely record.
- Context: production thresholds are not staging thresholds, and Monday-morning load is not Sunday-evening load. One threshold for every environment guarantees false alarms.
Finally, tie thresholds to your commitments: if your SLA promises a given availability level, alerts must fire well before the downtime budget starts burning, not after.
Alerting is not waking: the three levels
Not all anomalies are equal. A healthy setup distinguishes three levels:
- The ticket: useful information with no urgency — a certificate expiring in thirty days, a disk at 70%. Handled during business hours, never pushed in real time.
- The business-hours alert: a real degradation that can be tolerated for a few hours — a redundant node down, a localised slowdown.
- The page: confirmed or imminent impact on a critical service. This is the only level that rings at night, and it must stay rare.
Who gets woken up? A named person, not a mailing list. On-call works with a clear rotation, mandatory acknowledgement, and automatic escalation if an alert goes unanswered after a few minutes. And every page links to a runbook: symptoms, first diagnostics, workarounds, escalation criteria. Waking someone without telling them what to do throws away the precious minutes detection just earned — minutes whose cost can be calculated.
Alert fatigue, the silent enemy
The worst-case scenario is not the absence of monitoring: it is monitoring nobody listens to anymore. When non-actionable alerts pile up, teams acknowledge without reading, filter notifications, and the important alert slips through with the noise. Alert fatigue is fought like technical debt:
- every alert must demand an action; if the usual response is "wait and see", it should be deleted or downgraded to a ticket;
- a periodic review examines the month's alerts: which triggered a real action, which woke someone for nothing;
- every incident feeds the loop: did the alert catch the problem before users did? If not, what should have been watched?
It is continuous work, and it is exactly what makes on-call hard to sustain in-house: three people cannot run a viable 24/7 rotation. Outsourcing monitoring and on-call under SLA — which SOVALYX operates from Mauritius — is often cheaper than an exhausted internal rotation, and above all more reliable. Discuss it with an engineer rather than a salesperson.
Checklist: your monitoring in six points
- List critical user journeys and monitor them from outside your network.
- Classify every existing alert: page, business hours, or plain ticket.
- Add persistence, trend and context to every static threshold.
- Attach an up-to-date runbook to every alert capable of waking someone.
- Formalise on-call: rotation, acknowledgement, escalation, compensation.
- Hold a monthly noise review and delete whatever triggered no action.
How SOVALYX can help
SOVALYX runs 24/7 monitoring under SLA from Mauritius: service-oriented probes, thresholds tuned continuously, and a human on-call rotation equipped with up-to-date runbooks. Every alert that wakes an engineer maps to a real incident, and every incident feeds the review that cuts the noise. You know contractually who is watching, who responds, and how fast.
Talk disaster recovery with an engineer🧰 The companion tool: What does one hour of downtime cost you? — free · 2 minutes.
Reviewed and optimised by AI.