LLM security monitoring

• Signal 1: blocked_count — counter of prompts blocked by pre-filter; baseline ≈ 0.1% of inputs
• Signal 2: output_safety_score — % of responses that pass post-filter; baseline > 99.5%
• Signal 3: per_user_rate — top 10 users by RPS; baseline = your known median
• Signal 4: spend_per_hour — sliding-window cost; baseline = daily budget / 24
• All 4 are pushed to one heartbeat monitor with different tokens; alert is composite "2 of 4 in red"

# Composite security heartbeat (Python)
import requests, redis, time

r = redis.Redis()

def compute_signals():
    return {
        'blocked_pct':  float(r.get('llm:blocked:5min') or 0) / max(1, float(r.get('llm:total:5min') or 1)) * 100,
        'safety_pct':   float(r.get('llm:safe:5min') or 0)    / max(1, float(r.get('llm:total:5min') or 1)) * 100,
        'top_user_rps': float(r.zrangebyscore('llm:user_rps:5min', '-inf', '+inf', withscores=True)[-1][1] if r.exists('llm:user_rps:5min') else 0),
        'spend_hour':   float(r.get('llm:spend:1h') or 0),
    }

while True:
    s = compute_signals()
    status = 'critical' if (s['blocked_pct'] > 1.0 or s['safety_pct'] < 99 or s['top_user_rps'] > 60) else 'ok'
    requests.get(
        'https://enterno.io/api/heartbeat',
        params={'token': SEC_TOKEN, 'status': status,
                'msg': f"blocked={s['blocked_pct']:.2f}%, safety={s['safety_pct']:.2f}%"}
    )
    time.sleep(60)

• Detecting prompt injection
• What is prompt injection

To set up security monitoring for an LLM app, implement comprehensive logging, utilize intrusion detection systems (IDS), enforce secure coding practices, and regularly perform vulnerability assessments. Tools like OWASP ZAP or Snyk can help identify security flaws, while services such as Datadog or Splunk can enable real-time monitoring and alerting of suspicious activities.

Real-time security monitoring is essential for safeguarding your LLM application against potential threats. Here’s how to implement an effective monitoring setup:

1. Set Up Logging

Begin by ensuring that all application logs are correctly configured to capture relevant security events. This includes:

• User authentication attempts
• Data access logs
• API request and response logs

Use structured logging to ensure that logs are easily parseable. For example, in a Node.js application, you can use the following code snippet to set up logging with Winston:

const winston = require('winston');

const logger = winston.createLogger({
  level: 'info',
  format: winston.format.json(),
  transports: [
    new winston.transports.File({ filename: 'security.log' })
  ]
});

logger.info('User authenticated', { userId: '12345', timestamp: new Date() });

2. Deploy Intrusion Detection Systems (IDS)

Integrate an IDS to monitor network traffic and identify suspicious activities. Tools such as Snort or Suricata can be deployed to analyze packets in real time. For example, to install Snort on a Linux server, you can run:

sudo apt-get update
sudo apt-get install snort

After installation, configure Snort to monitor your network interface:

snort -A console -c /etc/snort/snort.conf -i eth0

3. Use Security Information and Event Management (SIEM) Tools

Employ a SIEM tool like Splunk or Datadog to aggregate logs from different sources, analyze them, and generate alerts based on predefined security rules. For instance, with Datadog, you can set up a security monitoring dashboard to visualize logs and detect anomalies:

datadog monitor create --type log --query "service:llm_app status:error" --name "LLM App Error Monitoring"

4. Conduct Regular Vulnerability Assessments

Regularly assess your application for vulnerabilities using tools like OWASP ZAP or Snyk. For instance, to run a scan using OWASP ZAP, you can use the following command:

zap.sh -cmd -quickurl http://your-llm-app.com -quickout report.html

This generates a report highlighting potential security issues that need to be addressed. Ensure to review and remediate these vulnerabilities in a timely manner.

5. Establish Alerting Mechanisms

Set up alerts for critical security events, such as multiple failed login attempts or unauthorized data access. You can configure alerts in your SIEM tool or directly through log management tools. For example, in Splunk, you could use:

index=security sourcetype=access_log action=failed_login | stats count by user | where count > 5

This query alerts you when any user has more than five failed login attempts, indicating a possible brute-force attack.

Conclusion

By implementing these steps, you can establish a robust security monitoring framework for your LLM application. Regularly review your monitoring policies and update them as necessary to adapt to evolving threats.