PagerDuty on-call announcements and scheduled incidents

Two things teams schedule with PagerDuty: telling everyone who's on call, and opening incidents from checks that run on a timer.

#Example: announce today's on-call in Slack

{
  "name": "On-call announcement",
  "cron": { "expression": "0 9 * * *", "time_zone": "America/New_York" },
  "nodes": [
    { "id": "oncall", "pagerduty_list_on_calls": { "credential_id": "YOUR_PAGERDUTY_API_KEY_ID", "escalation_policy": "PABC123", "limit": 1 } },
    { "id": "post", "slack_send_message": {
        "webhook_url": "https://hooks.slack.com/services/T000/B000/XXXX",
        "text": ":rotating_light: On call today: *{{ $.oncalls[0].user.summary }}* ({{ $.oncalls[0].schedule.summary }})" } }
  ]
}

#Example: open an incident from a nightly check

Your endpoint reports problems, and cronservices turns them into an incident:

{
  "name": "Nightly reconciliation alert",
  "cron": { "expression": "30 2 * * *", "time_zone": "UTC" },
  "nodes": [
    { "id": "check", "request": { "method": "GET", "endpoint": "https://api.acme.com/internal/reconcile/report", "credential_id": "YOUR_INTERNAL_TOKEN_ID" } },
    { "id": "page", "pagerduty_trigger_incident": {
        "credential_id": "YOUR_PAGERDUTY_ROUTING_KEY_ID",
        "summary": "Reconciliation found {{ $.mismatches }} mismatched payments",
        "source": "reconcile-job",
        "severity": "error",
        "dedup_key": "reconcile-{{ format(now(), \"YYYY-MM-DD\") }}" } }
  ]
}

This pages every night. To page only when something is wrong, have the endpoint answer with an error status when everything is fine (which stops the job before the incident step), or use Healthchecks.io for "alert when it stops working".

#Set up

• On-call list: create a REST API key (read-only is enough) under Integrations → API Access Keys, and save it as a pagerduty_api_key credential.
• Incidents: add an Events API v2 integration to the service and save its integration key as a pagerduty_routing_key credential.

#Reference

#Read who’s on call in PagerDuty: pagerduty_list_on_calls

Sends GET https://api.pagerduty.com/oncalls with header Authorization: Token token=…. Output for the next step: PagerDuty’s response: oncalls, each with user.summary, schedule.summary and escalation_level.

#Trigger a PagerDuty incident: pagerduty_trigger_incident

Sends POST https://events.pagerduty.com/v2/enqueue (Events API v2). Output for the next step: PagerDuty’s response, with the dedup_key.