Alert on a MySQL slow-query spike
long_query_time = 1, slow_query_log enabled. You need to know when the slow-query rate per minute suddenly jumps (a deploy broke an index, ORM went N+1).
Recipe
#!/usr/bin/env bash
# Count slow queries logged in the last 60 seconds. Alert above THRESHOLD.
LOG="/var/log/mysql/mysql-slow.log"
THRESHOLD=${THRESHOLD:-20}
SINCE=$(date -u -d "1 minute ago" "+%Y-%m-%dT%H:%M")
COUNT=$(awk -v t="$SINCE" '
/^# Time:/ { ts = substr($3 " " $4, 1, 16) }
/^# Query_time/ && ts >= t { c++ }
END { print c+0 }
' "$LOG")
if [ "$COUNT" -gt "$THRESHOLD" ]; then
curl -X POST "$SLACK_WEBHOOK" \
--data "{\"text\":\"MySQL: $COUNT slow queries in the last minute (threshold $THRESHOLD)\"}"
fi
Same thing in Enterno.io
A page-speed regression usually follows slow SQL. The PageSpeed checker + a Navigator-tier monitor with history surface the same signal at the TTFB level without needing slow-log access.
Related recipes
Every week some analyst fires off a heavy SELECT without LIMIT and blocks other transactions. You want to know within a minute, not within an hour.
A replica-set secondary falls behind the primary; the app will read stale data within a minute. Want an HTTP endpoint that says "ok" or "lag".
Memcached fills up and starts evicting keys under load; the app cache-misses and hammers the DB. Want an evictions/min threshold.