About VictorOps and Grafana
The VictorOps and Grafana integration allows you to forward Grafana alerts and dashboards into VictorOps to notify the correct on-call users. Create on-call schedules, rotations, and escalation policies in VictorOps, then route Grafana alerts and charts based on those parameters. Monitor and visualize logs, metrics and traces in Grafana and set thresholds to optimize alerts, create charts and surface actionable system insights.
When events meet predetermined monitoring criteria, Grafana sends an alert notification. Then, in the VictorOps timeline, users can route and escalate critical alert data to the correct people. With the VictorOps and Grafana integration, on-call responders can collaborate in real-time around system data to reduce MTTA/MTTR and resolve incidents faster.
Optimize Alerts and Incident Visibility throughout the Entire Organization
- Track system performance over time, visualize service health and alert on-call teams when applicable monitoring thresholds are met
- Surface alert context in the form of metrics, logs, graphs and links to runbooks
- Automatically route and escalate Grafana alert data through VictorOps on-call software and collaborate in a single-pane-of-glass to drive incident workflows forward
- Chat in-line with incident context to improve collaboration and quickly share critical infrastructure information and dashboards, reducing MTTA/MTTR
Grafana is a general purpose dashboard and graph composer. It’s focused on providing rich ways to visualize time series metrics from Graphite, but also works with several other popular metric databases like InfluxDB, OpenTSDB, and Prometheus. The Grafana and VictorOps integration allows you to understand your alerts and metrics so you get the most out of your data. The following guide will walk you through this easy to implement integration.
Enable Grafana in VictorOps
From the main timeline navigate to Settings >> Alert Behavior >> Integrations.
Select the Grafana integration and copy the Service API Endpoint to your clipboard.
Connect VictorOps in Grafana
From the main dashboard, select the options menu then Alerting then Notification Channels.
Add a New Channel.
Give the notification a name select the VictorOps type. In the VictorOps settings paste in your Service API Endpoint from your clipboard. Be sure to replace the “$routing_key” section with the actual routing key you intend to use. (For more information on setting up routing keys, see our Knowledge Base article on Routing Keys)
You can now test and save the integration.
The resulting test alert in VictorOps should look similar to this:
For those with full-stack VictorOps features, you can include the Grafana image of the alert into VictorOps. When configuring the notification channel in Grafana, select the box to Include image. This will automatically add the correct “image_url” field to the payload.
In VictorOps, add the following Rules Engine rule that will surface the image_url as an annotation to the incident. (Found by navigating to Settings >> Alert Behavior >> Rules Engine)
Your resulting VictorOps incident should now include the image as an annotation:
Once the test comes through successfully, save the notification in Grafana.