1. Home
  2. Integrations
  3. REST Endpoint Integration Guide – VictorOps

REST Endpoint Integration Guide – VictorOps

About VictorOps and the REST Endpoint

The REST Endpoint allows you to send customized alerts and incident details into the VictorOps timeline. Alerts get sent into the VictorOps incident workflow with fields such as message_typeentity_id, and/or state_message. As these alerts come into VictorOps, they are routed directly to the proper person or team with incident details that give the on-call responder more context.

VictorOps REST Endpoint Establishes Incident Lifecycle Workflows

  • Send alerts straight into VictorOps with detailed incident context
  • Collaborate around issues in the VictorOps timeline via incident prioritization, escalation, native chat, runbook links, and more
  • Remediate incidents more quickly with automated contextual alerting, communication, and incident workflow management in one centralized location

The VictorOps REST Endpoint accepts alerts from any source via an HTTPS POST request in JSON format. As long as you can configure the content of the request, you can trigger, acknowledge, or resolve incidents in VictorOps.

Enable the REST Endpoint Integration

In VictorOps, click on Settings >> Alert Behavior >> Integrations >>  REST – Generic.

 

If the REST endpoint integration has not been enabled, click the blue Enable button to generate your endpoint destination URL. Note that the REST endpoint is different than the general API and is the preferred method to create incidents.

REST Endpoint Integration Routing Key

 

The routing key that will be used for this integration should be included in the destination URL.  It is essential that you replace what you see here with the actual routing key you intend to use.  Everything after the final forward slash must be replaced with your key.  For example, assuming a routing_key value of “database”:

………36437/$routing_key    ==>   ……..36437/database

Routing keys in VictorOps can be set up and associated by clicking on Settings >> Alert Behavior >> Route Keys.

For more information on routing keys and best practices, click HERE.


Recommended REST Endpoint Integration Fields

Technically, only the message_type field is absolutely required.  You could use {“message_type”:”critical”} as the body and generate an incident.  However, in order to get the most out of the integration, we recommend including the fields listed below.  These fields help establish the workflow and lifecycle of an incident in VictorOps, but keep in mind that you can also include any number of additional, custom fields that you may find useful. For more information on different fields that VictorOps uses please see the glossary of fields table.

message_typeBehavior of the alert [String]

Valid values:
- CRITICAL: Triggers an incident
- WARNING: May trigger an incident, depending on your settings
- ACKNOWLEDGEMENT: Acks an incident
- INFO: Creates a timeline event but doesnot trigger an incident
- RECOVERY: Resolves an incident
entity_idID of the Incident [String]

This field is the identity of the incident and must remain the same throughout the life-cycle of an incident. Make sure you use the same entity_id when triggering, acking, or resolving an incident. Example: "/", i.e... "diskspace/db01.mycompany.com"

If no entity_id is included, a random string will be used.
entity_display_nameDisplay Name in the UI and Notifications [String]

This field allows you to give custom, human-friendly, summary of your incidents without affecting the life-cycle workflow.

If you are using an entity_id value of "dskspc/db465/<0.15volumeOnDisk/this.com", a good entity_display_name might be "Disk Space is below 15% on Database Server 465. You may want to fix that"
state_messageVerbose message field [String]

This field has a high character limit and is intended for a long, verbose, explanation of the problem. This field will be included in notifications (full content in email, truncated version in push/phone/SMS notifications) Any URL links included in this field will be rendered as clickable links in email notifications
state_start_timeTime this issue began [Number] (Linux/Unix time)

The time this entity entered its current state (seconds since epoch). Defaults to the time alert is received.

Expected Responses

Success messages will display the result and the entity_id of the incident.

1
2
3
4
{
"result":"success",
"entity_id":"Your entity_id here"
}

Failure messages will display the result as well as an error message explaining the reason for the failure.

1
2
3
4
{
"result":"failure",
"message":"Missing fields: message_type"
}

Example Alerts

Trigger a Critical Incident

This request will open a new incident.

1
2
3
4
5
6
{
	"message_type":"CRITICAL",
	"entity_id":"disk space/db01.mycompany.com",
	"entity_display_name":"Critically Low Disk Space on DB01",
	"state_message":"The disk is really really full.  Here is abunch of information about the problem"
}

Response:

1
2
3
4
{
  "result" : "success",
  "entity_id" : "disk space/db01.mycompany.com"
}

Incident created:

Resolve an Incident

This request will resolve the same incident opened in the example above (notice the entity_id is the same, allowing our system to identify to which incident this request should be applied)

1
2
3
4
5
6
{
	"message_type":"RECOVERY",
	"entity_id":"disk space/db01.mycompany.com",
	"entity_display_name":"Critically Low Disk Space on DB01",
	"state_message":"Memory was added to the disk.  All is well now"
}

Response:

1
2
3
4
{
  "result" : "success",
  "entity_id" : "disk space/db01.mycompany.com"
}

Incident Resolved:


cURL and Incident Response

You can also invoke this with a cURL command by imitating this syntax (You must insert your own REST endpoint key and routing_key value):

1
curl -X POST -d '{"entity_id":"ID of the incident","message_type":"critical","state_message":"hi, this is some state message."}' https://alert.victorops.com/integrations/generic/20131114/alert/[YOUR_REST_ENDPOINT_KEY]/[ROUTING_KEY_HERE]

Annotations

By adding an annotation field to the alert payload you can include annotations in an alert being sent into VictorOps via the REST Endpoint. INFO, WARNING, and CRITICAL alerts can be annotated by adding the annotation field to the JSON payload.

Note: When an incident is in an acknowledged state and a new alert with the same entity_id comes into the VictorOps Timeline the alert will be aggregated under the open incident. Any new annotations carried in the most recent alert will be added to the annotation tab of the incident. The annotations will also be reflected within each alert payload.

Annotation Payloads:

There are three types of annotations accepted by VictorOps through the syntax below. Note: You can customize the title of your annotation at the end of your annotation syntax. For example, in the above screenshot the title of the URL annotation will appear in VictorOps as “Runbook”.

Example Payloads:

URL: vo_annotate.u.url
1
2
3
4
5
6
7
8
{
 "monitoring_tool": "API",
 "message_type":"INFO",
 "entity_id":"disk.space/db01",
 "entity_display_name":"Approaching Low Disk Space on DB01",
 "state_message":"The disk is really really full. Here is a bunch of information about the problem",
 "vo_annotate.u.Runbook":"https://help.victorops.com/knowledge-base/rest-endpoint-integration-guide/"
}
Note: vo_annotate.s.note

Note Annotations have a 1124 character limit

1
2
3
4
5
6
7
8
{
 "monitoring_tool": "API",
 "message_type":"INFO",
 "entity_id":"disk.space/db01",
 "entity_display_name":"Approaching Low Disk Space on DB01",
 "state_message":"The disk is really really full. Here is a bunch of information about the problem",
 "vo_annotate.s.Note":"Once Disk Space is critically low there will be an incident!"
}
Image URL: vo_annotate.i.image
1
2
3
4
5
6
7
8
{
 "monitoring_tool": "API",
 "message_type":"INFO",
 "entity_id":"disk.space/db01",
 "entity_display_name":"Approaching Low Disk Space on DB01",
 "state_message":"The disk is really really full. Here is a bunch of information about the problem",
 "vo_annotate.i.Graph":"https://community.iotawatt.com/uploads/db6340/original/1X/266a3917cc86317830ae9cda3e91c7689a6c73a7.png"
}

 

Updated on October 2, 2019

Was this article helpful?

Related Articles