1. Home
  2. Prometheus Integration Guide – VictorOps

Prometheus Integration Guide – VictorOps

Prometheus is an open-source systems monitoring and alerting toolkit with an active ecosystem.  Alerting with Prometheus is separated into two parts.  Alerting rules in Prometheus servers send alerts to an Alertmanager.  The Alertmanager then manages those alerts, including silencing, inhibition, aggregation and sending out notifications via methods such as VictorOps.  The following guide will walk you through this integration.

In VictorOps

From the VictorOps web portal, select Settings, then Alert Behavior, then Integrations.

Select the Prometheus integration option.

Click Enable Integration.

Copy the Service API Key to the clipboard.

In Prometheus

Download and configure the Alertmanager for Prometheus.  Here is the code to use in the YAML configuration file for the Alertmanager.  Make sure to replace the “api_key” with the API key from the Service API Key from the “In VictorOps” section and change the “routing_key” to the actual routing key you intend to use.

  group_by: ['alertname', 'cluster', 'service']
  group_wait: 30s
  group_interval: 5m
  repeat_interval: 3h
  receiver: victorOps-receiver

- name: victorOps-receiver
    - api_key: 558e7ebc-XXXX-XXXX-XXXX-XXXXXXXXXXXX
      routing_key: Sample_route
      state_message: 'Alert: {{ .CommonLabels.alertname }}. Summary:{{ .CommonAnnotations.summary }}. RawData: {{ .CommonLabels }}'

Start Prometheus from the command line and configure it to talk to the Alertmanager.  In this example, “prometheus.yml” is the configuration file for Prometheus and “http://localhost:9093” is the instance of the Alertmanager that Prometheus is pointing to.

./prometheus -config.file=prometheus.yml -alertmanager.url=http://localhost:9093

Next, start the Alertmanager from the command line using the Alertmanager configuration file from earlier.  In this example, “alertmanager.yml” is the name of the configuration file.

./alertmanager -config.file=alertmanager.yml

Alerts from Prometheus should now appear in the Alertmanager as they are generated.




You can also send a test message to the Alertmanager if you don’t want to wait for an alert to be generated by Prometheus.  Below is an example using curl with an instance of Alertmanager at “http://localhost:9093”.

curl -H "Content-Type: application/json" -d '[{"labels":{"alertname":"TestAlert"}}]' localhost:9093/api/v1/alerts

You have completed setting up this integration.  If you have any questions, please contact VictorOps support.

Updated on May 22, 2017

Was this article helpful?