Alert on Docker container OOM-kill
A container OOM-kills, the restart policy revives it — no external signal until users complain.
Recipe
bash
#!/usr/bin/env bash
# /etc/cron.d/docker-oom-watch
# */5 * * * * root /opt/docker-oom.sh
# Check for OOM events in the last 5 minutes (kernel ring buffer).
HITS=$(dmesg --since="5min ago" 2>/dev/null | grep -ci 'killed.*oom-killer\|out of memory')
if [ "$HITS" -gt 0 ]; then
# Heartbeat ping with status=lag tells the monitor "I am alive
# AND something is wrong" — different from missed pings (host down).
curl -fsS "${HEARTBEAT_URL}?status=oom&hits=${HITS}" -m 5 || true
fi
Same thing in Enterno.io
Enterno heartbeat monitors distinguish "no ping at all" from "ping with error status". The cron above hits the second case — alerts fire instantly.
Related recipes
Your cron silently stopped running. Need an alert when the script misses its window.
Logs or backup files eat /var; in 24 hours the server falls over. A basic df check every 10 minutes saves a 2 AM incident.
Ensure your site returns 2xx every minute, alert to Slack/Telegram on failure.