LND Alertmanager Setup -- Fix Prometheus Alert Rules and Routing Errors

What Is LND Alertmanager Integration

LND exposes Prometheus metrics that can be scraped and used to trigger Alertmanager notifications. Common errors arise during setup, rule configuration, or alert routing.

Common Errors

1. AlertManager Connection Refused

Error: dial tcp 127.0.0.1:9093: connect: connection refused

Fix: Ensure Alertmanager is running. Check: systemctl status alertmanager. Verify lnd.conf has prometheus.listen set correctly.

2. No Alerts Firing

Symptom: Rules load but no alerts trigger even when thresholds are crossed.

Fix: Verify your rule expressions with promtool check rules alerts.yml. Check that LND metrics are actually being scraped: curl http://localhost:9090/api/v1/targets

3. Alert Rule Syntax Error

Error: yaml: unmarshal errors / unexpected field

Fix: Use promtool check rules to validate. Common issues: wrong indentation, missing groups key, incorrect expr syntax.

4. Missing LND Metrics in Prometheus

Symptom: lnd_* metrics missing from Prometheus.

Fix: Ensure prometheus.listen=0.0.0.0:9090 in lnd.conf. Restart LND. Verify with: curl http://localhost:8080/metrics | grep lnd_

5. Alertmanager Not Receiving from Prometheus

Fix: In prometheus.yml check alerting section: alertmanagers: - static_configs: - targets: ["localhost:9093"]

Key Alerts to Configure

Channel balance low: lnd_channels_local_balance_sat < 100000

Node offline: up{job="lnd"} == 0

Payment failure rate high: rate(lnd_payments_failed_total[5m]) > 0.1

Force close detected: increase(lnd_channels_force_closed_total[1h]) > 0

Debug Checklist

1. LND Prometheus endpoint responding: curl localhost:8080/metrics

2. Prometheus scraping LND: check Targets page

3. Alert rules loading: promtool check rules

4. Alertmanager reachable: curl localhost:9093/-/healthy

5. Test alert fires: curl -XPOST localhost:9093/api/v1/alerts -d '[{"labels":{"alertname":"TestAlert"}}]'