Notifications begin to be aggregated when large number of alerts are received in a short period of time. VictorOps will automatically start this process when an alert storm is detected, to limit the number of notifications paging out. In order to completely mute all or specific alerts, see Rules Engine Transformations.
Notification Aggregation is triggered only in the event that three unique incidents are opened within a 60 second window. Once Notification Aggregation is triggered notifications for new incidents will not immediately send upon opening but rather on a one minute interval. A single notification will then display the number of incidents the user is being paged for — rather than for each. The medium of the notification sent will align with whichever methods are specified within the user’s personal paging policy of which they are in. Furthermore, if no new incidents arrive and the triggered alerts remain unacknowledged the next page will follow the user’s paging policy.
When notification aggregation is not triggered, multiple incidents will page traditionally exactly has outline in the personal paging policy of the user.
In order to exit Notification Aggregation, all incidents associated with the user– regardless of when they were opened in the process — must be acknowledged. Another way of phrasing this is the user must have zero incidents associated with them in the triggered state.
When multiple alerts arrive with identical entity_id’s VictorOps will wrap all of the alerts into one single incident. When you acknowledge an incident in a triggered state, be aware that new critical alerts will automatically be grouped into the already acknowledged incident. In this way it is a best practice to either promptly resolve incidents as they are fixed, or to use the Configure Incidents settings. An incident sitting for a long period of time in an acknowledged sate can unknowingly inhibit necessary pages from reaching on-call users.