Logging pipeline — alert when ingest rate drops · monitoring cookbook · Enterno.io

Enterno.io Editorial Team

Logging pipeline — alert when ingest rate drops

Filebeat / Logstash silently died on one edge node. Elasticsearch ingest rate fell 40 % but no one watches dashboards. Sentry without logs is blindness.

Stack: Elasticsearch · curl · cron Tags: logging, elk, graylog, observability

Recipe

bash

#!/usr/bin/env bash
# Compares 1-minute index document count delta against expected floor.
ES="${ES:-http://localhost:9200}"
INDEX_PREFIX="${INDEX_PREFIX:-logs-}"
MIN_DOCS_PER_MIN="${MIN_DOCS_PER_MIN:-100}"

NOW=$(date -u -d '1 min ago' +"%Y-%m-%dT%H:%M:%SZ")
COUNT=$(curl -sS -X POST "$ES/${INDEX_PREFIX}*/_count" \
  -H "Content-Type: application/json" \
  -d "{\"query\":{\"range\":{\"@timestamp\":{\"gte\":\"$NOW\"}}}}" \
  | python3 -c 'import json,sys; print(json.load(sys.stdin).get("count", 0))')

[ -z "$COUNT" ] && { echo "no-data"; exit 1; }
[ "$COUNT" -lt "$MIN_DOCS_PER_MIN" ] && echo "low $COUNT" || echo "ok $COUNT"

Same thing in Enterno.io

Endpoint + Enterno HTTP monitor with "ok" keyword every minute = paged when the pipeline drops. Pair with a heartbeat on Filebeat itself — second line of defense.

Set up HTTP monitor → ← All recipes