Alerts

Monitor your LLM operations with configurable alerts. Get notified when error rates spike, latency degrades, or costs exceed thresholds.

Overview

Demeterics alerts help you:

• Detect issues early before they impact users
• Control costs with spending thresholds
• Monitor performance with latency alerts
• Track reliability with error rate monitoring

Alert Types

Error Rate

Monitor the percentage of failed LLM requests.

Example configuration:

P99 Latency

Monitor the 99th percentile response time in milliseconds.

Example configuration:

Cost Ratio

Monitor the cost relative to a baseline or budget.

Example configuration:

Note: Low credit balance alerts are configured separately in Settings → Credits. See Credits & Pricing for auto-reload and balance alert configuration.

Notification Channels

Email

Alerts are sent to the email addresses configured for your account.

Configuration:

{
  "channel": "email",
  "recipients": ["ops@example.com", "oncall@example.com"]
}

Slack

Send alerts to a Slack channel via incoming webhook.

Setup:

1. Create a Slack app at https://api.slack.com/apps
2. Add an Incoming Webhook
3. Copy the webhook URL
4. Add to Demeterics: Settings → Integrations → Slack

Configuration:

{
  "channel": "slack",
  "webhook_url": "https://hooks.slack.com/services/T00/B00/xxx"
}

Webhook

Send alerts to any HTTP endpoint.

Configuration:

{
  "channel": "webhook",
  "url": "https://your-server.com/alerts",
  "headers": {
    "Authorization": "Bearer your-token"
  }
}

Webhook Payload:

{
  "alert_id": "alt_abc123",
  "alert_name": "High Error Rate",
  "severity": "critical",
  "metric": "error_rate",
  "current_value": 12.5,
  "threshold": 10,
  "triggered_at": "2025-01-29T12:00:00Z",
  "message": "Error rate is 12.5%, exceeding critical threshold of 10%"
}

Managing Alerts

Alerts are managed through the Demeterics dashboard. A REST API for programmatic alert management is planned for a future release.

Create Alert

1. Navigate to Alerts in the main menu (or Settings → Alerts)
2. Click Create Alert
3. Select metric type (error_rate, p99_latency_ms, or cost_ratio)
4. Set warning and critical thresholds
5. Configure the evaluation window
6. Choose notification channels (email, Slack, webhook)
7. Click Save

Edit Alert

1. Navigate to Alerts
2. Click the alert you want to modify
3. Update thresholds or notification channels
4. Click Save

Enable/Disable Alert

Toggle the Enabled switch on any alert to pause or resume monitoring without deleting the configuration.

Delete Alert

1. Navigate to Alerts
2. Click the alert you want to remove
3. Click Delete and confirm

Alert History

View triggered alerts and their resolution status in the dashboard.

Viewing History

Navigate to Alerts → History to see:

• All triggered alerts
• Severity level (warning or critical)
• Trigger time
• Resolution status
• Current value vs threshold

Acknowledging Alerts

When an alert triggers:

1. Click Acknowledge to indicate you're investigating
2. The alert remains visible but marked as acknowledged
3. Once the issue is resolved, click Resolve

Alert Lifecycle

1. Active - Condition is currently met, notifications sent
2. Acknowledged - Someone is investigating
3. Resolved - Condition no longer met or manually resolved

Best Practices

Start Conservative

Begin with higher thresholds and tighten them as you understand your baseline:

1. Set warning threshold at 2x your normal baseline
2. Set critical threshold at 3x your normal baseline
3. Monitor for 1-2 weeks
4. Adjust based on actual alert frequency

Avoid Alert Fatigue

Too many alerts lead to ignored alerts:

• Use warning for investigation, critical for action
• Set appropriate time windows (5-15 minutes)
• Group related alerts
• Review and tune thresholds monthly

Use Multiple Channels

For critical alerts, configure redundant notification channels. In the alert configuration, enable multiple channels (email + Slack + webhook) to ensure notifications are received even if one channel fails.

Test Your Alerts

Verify alerts work before you need them:

1. Create a test alert with a low threshold
2. Trigger the condition intentionally
3. Verify notification delivery
4. Reset to production thresholds

Troubleshooting

Alerts Not Triggering

1. Verify alert is enabled (Settings → Alerts)
2. Check threshold values are appropriate
3. Confirm time window covers recent activity
4. Verify there's traffic to monitor

Not Receiving Notifications

1. Check email spam folder
2. Verify Slack webhook URL is correct
3. Test webhook endpoint independently
4. Check notification channel configuration

Too Many Alerts

1. Increase thresholds
2. Extend time window
3. Use hysteresis (require sustained condition)
4. Review baseline metrics

What's Next?

• Credits & Pricing - Set up low balance alerts
• API Reference - Full API documentation
• Dashboard - View real-time metrics