# Webhooks

A **webhook** pushes HostTracker events to your own HTTPS endpoint as they happen - a monitor going down or up, an
incident opening or closing, a certificate or domain about to expire, a maintenance window ending, a monitor or
contact being changed - so you do not have to poll the [API](/integrations/rest-api/). Every delivery is signed, so
your endpoint can prove it came from HostTracker.

Webhooks are managed through the API and the tools built on it ([SDKs](/integrations/sdks/),
[ht-cli](/integrations/cli/), [MCP](/integrations/mcp/), [Terraform](/integrations/terraform/) as
`hosttracker_webhook`). There is no webhook page in the app. For a simple alert-only destination you can set up in
the app, use a [webhook alert contact](/alerts/channels/webhook/) instead (see [the difference](#webhooks-vs-webhook-contacts)).

## Settings reference

| Field | Type / allowed values | Default | Limits | What it does for you |
|---|---|---|---|---|
| `url` | `https://` URL, publicly reachable | - (required) | 2,048 characters; one webhook per URL on an account | Where deliveries are POSTed. |
| `events` | Array of event names (see [Events](#events)), at least one | - (required) | The 13 subscribable events | Which events are sent. No wildcard - list each one. |
| `scope` | Exactly one of `{"all": true}`, `{"monitorIds": [...]}`, `{"tags": [...]}` | - (required) | Up to 500 monitors | Which monitors' events are sent. A `tags` scope also picks up monitors tagged later. Contact events ignore scope. |
| `name` | Text or `null` | None | 100 characters | A label for you. |
| `headers` | Array of `{"header": "...", "value": "..."}` | None | 20 headers; name 100 chars, value 1,000 chars; names starting with `HT-` or `webhook-` are refused | Extra headers on every delivery, for example your own auth header. |
| `secret` | Your own string (16-128 characters), or omit to have one generated; `{"rotate": true}` on update | Generated (`whsec_...`) | - | The key deliveries are signed with. Shown in full only in the create answer and a rotate answer. |
| `enabled` | `true` / `false` (update only) | `true` | - | Pause deliveries, or re-enable after an automatic disable. |

Read-only fields on the webhook: `id`, `consecutiveFailures`, `disabledReason` (`deliveryFailure` or `gone`),
`lastDeliveryAt`, `secret` (`{"set": true, "updatedAt": ..., "previousValidUntil": ...}`), `created`, `updated`.

## Create a webhook

Scope `webhook:write` (and `webhook:read` to list and read them).

```bash
curl -X POST https://api2.host-tracker.com/webhook \
  -H "Authorization: Bearer $HT_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://hooks.example.com/hosttracker",
    "events": ["monitor.down", "monitor.up", "incident.opened", "incident.closed"],
    "scope": { "tags": ["prod"] },
    "name": "Production alerts to our bus"
  }'
```

The answer is `201 Created`:

```json
{
  "id": "0c3c7b07-cecb-43dd-9b76-8516d3b9c771",
  "url": "https://hooks.example.com/hosttracker",
  "events": ["monitor.down", "monitor.up", "incident.opened", "incident.closed"],
  "scope": { "tags": ["prod"], "monitorCount": 8 },
  "name": "Production alerts to our bus",
  "enabled": true,
  "consecutiveFailures": 0,
  "secret": { "set": true, "updatedAt": 1785712680, "value": "whsec_tPlGftcqGUhzFSMwQgQEMPPiBYChRnIP7ku03iytipE=" },
  "created": 1785712681,
  "updated": 1785712680
}
```

**Store `secret.value` now** - later reads show only `{"set": true, ...}`. If you lose it, rotate it.

With MCP, `create_webhook` does the same, but its arguments are flat: `events` is a comma-separated string, and
the scope is chosen with `monitorIds` **or** `tags` (comma-separated; send at most one, or neither for the whole
account) - there is no `scope` argument. The tool answer carries the secret once.

```
create_webhook(url="https://hooks.example.com/hosttracker", events="monitor.down,monitor.up",
               tags="prod", name="Prod down/up")
```

`update_webhook` can replace the events and set a `monitorIds` scope; to switch an existing webhook to a tag scope
or back to the whole account, use `api_request` with `PATCH /webhook/{id}` and a `scope` object.

Other operations:

| Task | Operation |
|---|---|
| List / read | `GET /webhook`, `GET /webhook/{id}` (`GET /webhook` also returns the live event catalogue in `summary.eventTypes`) |
| Change url, events, scope, name, headers, enabled | `PATCH /webhook/{id}` (only the members you send) |
| Rotate the secret | `PATCH /webhook/{id}` with `{"secret": {"rotate": true}}` |
| Send a test delivery | `POST /webhook/{id}/test` with optional `{"event": "monitor.down"}` |
| Delivery log | `GET /webhook/{id}/delivery` |
| Redeliver one delivery | `POST /webhook/{id}/delivery/{deliveryId}/redeliver` |
| Delete | `DELETE /webhook/{id}` - pending deliveries are dropped and the secret cannot be recovered |

## Events

| Event | Fires when | `data` carries |
|---|---|---|
| `monitor.down` | An alert-grade down transition, after recheck confirmation and any alert delay. | `monitor` (id, name, url, type), `state`, `occurredAt`, `failedAt`, `error` (`code`, `message`, `codename`), `recheck[]` (what each location saw) |
| `monitor.up` | The matching recovery. | as above, plus `firstFailedAt`, `lastFailedAt`, `failedChecks`, `downtimeSec` |
| `monitor.repeatedlyDown` | The "still down" reminder tier. | as `monitor.up` (state `down`) |
| `incident.opened` | A down episode opens - including one inside a maintenance window. | `incidentId`, `monitorId`, `start`, `cause`, `underMaintenance` |
| `incident.closed` | The episode resolves. | the above plus `end`, `durationSec`, `checkCount` |
| `monitor.created` | A monitor is created (one event per monitor, bulk included). | the full monitor, as `GET /monitor/{id}` returns it |
| `monitor.updated` | A monitor's configuration changes (one per monitor, bulk included). | the full monitor |
| `monitor.deleted` | A monitor is deleted (one per monitor, bulk included). | the delete receipt |
| `maintenance.ended` | A maintenance window ends on schedule, or is cancelled while active. There is no "started" event. | `maintenanceId`, `name`, `from`, `to`, `endedAt`, `endedEarly`, `monitorIds` |
| `certificate.expiring` | An SSL certificate check crosses a warning threshold. | `monitorId`, `notAfter`, `daysLeft`, `issuer`, `threshold`, `endpoint` |
| `domain.expiring` | A domain expiry check crosses a warning threshold. | `monitorId`, `expiresAt`, `daysLeft`, `registrar`, `sharedDomain`, `threshold` |
| `contact.confirmed` | A contact becomes confirmed. | `contactId`, `type`, `address`, `confirmedAt`, `via` (`code` or `inherit`) |
| `contact.updated` | A contact's configuration changes. | the full contact, as `GET /contact/{id}` returns it |

Two more events are **callback-only**: `job.completed` and `job.progress` are sent to the webhook a job-creating
request names in `"callback": {"webhookId": "..."}`. Putting them in `events` is refused with
`422 unknown_event_type`. Full payload fields: [Webhook events reference](/reference/webhook-events/).

## What a delivery looks like

Every delivery is a `POST` with a JSON envelope:

```json
{
  "id": "d_e944b34a57924935968d32ea94a07832",
  "event": "monitor.down",
  "occurredAt": 1785672266,
  "apiVersion": "v2",
  "data": { }
}
```

- `id` is the delivery id - the same on every retry and redelivery. **Deduplicate on it.**
- `occurredAt` is when the event happened, not when this attempt was sent.
- Members with no value are omitted; treat an absent member as null. New members may appear; ignore what you do not
  know.

Headers on every delivery:

| Header | Value |
|---|---|
| `HT-Event` | The event name, so you can route before parsing. |
| `HT-Delivery` | `d_<32 hex>` - same as the envelope `id`. |
| `HT-Webhook` | The webhook's id (what `GET /webhook/{id}` takes). |
| `HT-Attempt` | Attempt number, starting at 1. A redelivery continues the count. |
| `HT-Signature` | `t=<unix seconds>,v1=<hex>` - HostTracker's signature. |
| `webhook-id`, `webhook-timestamp`, `webhook-signature` | The same signature in [Standard Webhooks](https://www.standardwebhooks.com/) form. |

Your own configured headers are added too; they can never replace these.

## Verify a delivery

Both signature schemes are on every delivery; use whichever suits your stack. Always verify against the **raw body
bytes** as received - re-serialised JSON will not match.

**HT-Signature**

1. Split the header on commas; take `t` and every `v1` value (during a secret rotation there are two).
2. Reject the delivery if `t` is more than 300 seconds from your clock.
3. Compute HMAC-SHA256 over `t + "." + rawBody`, keyed with the **whole secret string as UTF-8, including the
   `whsec_` prefix**. Encode as lowercase hex.
4. Accept if it equals any `v1` value (use a constant-time comparison).

```python
import hmac, hashlib, time

def verify_ht(raw_body: bytes, header: str, secret: str, tolerance: int = 300) -> bool:
    parts = [p.strip().split("=", 1) for p in header.split(",") if "=" in p]
    t = next(v for k, v in parts if k == "t")
    sigs = [v for k, v in parts if k == "v1"]
    if abs(int(time.time()) - int(t)) > tolerance:
        return False
    expected = hmac.new(secret.encode(), t.encode() + b"." + raw_body, hashlib.sha256).hexdigest()
    return any(hmac.compare_digest(expected, s) for s in sigs)
```

**Standard Webhooks**

Pass the whole secret (with `whsec_`) to any Standard Webhooks library:

```python
from standardwebhooks import Webhook
payload = Webhook("whsec_...").verify(raw_body, request_headers)  # raises on a bad signature
```

By hand: the signed string is `webhook-id + "." + webhook-timestamp + "." + rawBody`; the key is the secret with
`whsec_` removed, then **base64-decoded**; the digest is HMAC-SHA256 in **base64**; the header holds space-separated
`v1,<base64>` entries.

:::caution[The two schemes use different keys]
HT-Signature keys on the secret string as-is. Standard Webhooks keys on the base64-decoded bytes after `whsec_`.
Mixing them up is the most common reason a signature never matches.
:::

`ht-cli webhooks verify --secret "$SECRET" --headers-file headers.txt < body.json` checks a captured delivery, and
the SDKs include a verifier. `POST /webhook/{id}/test` returns the `signatureSent`, which you can use as a test
vector.

## Retries and auto-disable

Answer with any `2xx` quickly - each attempt times out after **10 seconds** - and do the real work afterwards.

| Your response | What HostTracker does |
|---|---|
| `2xx` | Delivered. |
| `408`, `429`, any `5xx`, a TLS or network failure | Retryable (a `Retry-After` on your `429` is honoured when longer than the next delay). |
| Any other `4xx` | Permanent failure - not retried. |
| `410 Gone` | Disables the webhook at once. Use it when you decommission an endpoint. |

**Retry ladder** for events raised by the monitoring engine (`monitor.down`, `monitor.up`,
`monitor.repeatedlyDown`, `incident.*`, `certificate.expiring`, `domain.expiring`, and `maintenance.ended` when a
window ends on schedule): one attempt, then up to five retries after **10 s, 60 s, 5 min, 30 min and 2 h** (about
2.6 hours in total, each delay with up to 20% jitter). While a delivery waits for a retry, the log shows it as
`pending` with a `nextRetryAt`. The queue is durable, so HostTracker deploys do not lose retries.

**Not retried:** events caused by API calls (`monitor.created`, `monitor.updated`, `monitor.deleted`,
`contact.confirmed`, `contact.updated`, `maintenance.ended` from an early cancel), job callbacks, test sends and
redeliveries get one attempt. The state they announce is always readable from the API; use redelivery if you missed
one.

**Auto-disable.** A webhook is switched off (`enabled: false` with a `disabledReason`) after **20 consecutive
failed deliveries**, after **24 hours in which every delivery failed**, or at once on a `410`. The account's
confirmed email contacts get one email ("Your HostTracker webhook has been disabled") naming the URL, the reason and
the last error. Fix the endpoint, then re-enable with `PATCH /webhook/{id}` and `{"enabled": true}` - this also
resets the failure counter. Deliveries that failed while it was off are not replayed automatically.

`GET /webhook/{id}` (`enabled`, `disabledReason`, `consecutiveFailures`, `lastDeliveryAt`) is the reliable health
signal to watch.

## Delivery log and redelivery

`GET /webhook/{id}/delivery` lists recent deliveries, one row per delivery with all its attempts:

- Row fields: `id`, `event`, `occurredAt`, `outcome` (`pending`, `delivered`, `failed`, `dropped`), `nextRetryAt`,
  `attempts[]` (`at`, `statusCode`, `latencyMs`, `error`), `payloadDigest` (SHA-256 of the body), `resourceId`.
- Filters: `from`, `to`, `event`, `outcome` (lists are any-of, e.g. `outcome=failed,dropped`). Cursor-paged.
- Deliveries and their payloads are kept for **7 days**.

`POST /webhook/{id}/delivery/{deliveryId}/redeliver` resends a past delivery: same body, same `HT-Delivery` id (so a
deduplicating receiver sees a repeat), a fresh signature and timestamp, and the attempt count continues. It is
refused with `422` when the webhook is disabled (`webhook_disabled`) or the payload was not kept
(`payload_not_retained` - bodies over 32 KB are not stored).

`POST /webhook/{id}/test` sends a synthetic event (default `monitor.down`) through the real signed path and answers
with the outcome, latency, any error and the `signatureSent`. It works on a disabled webhook too, so you can check a
fix before re-enabling.

## Rotate the secret

`PATCH /webhook/{id}` with `{"secret": {"rotate": true}}` returns the new secret once. For the next **24 hours**
every delivery is signed with both the new and the previous secret (two `v1` entries); `secret.previousValidUntil`
says when the old one stops. Deploy the new secret any time inside that window - as long as your verifier accepts
any matching signature.

## Transport rules

- **HTTPS only** (`422 invalid_url`, reason `scheme_not_allowed` for `http://`), with a certificate that validates
  against the public trust chain. Self-signed certificates fail every delivery and eventually auto-disable the
  webhook.
- **Public destinations only.** Loopback, private (10/8, 172.16/12, 192.168/16), carrier-NAT, link-local, IPv6
  unique-local and cloud metadata addresses are refused (`422 invalid_url`, reason `destination_not_allowed`),
  and checked again on the resolved address at every delivery.
- **Redirects** are followed up to 3 hops, only to https; each hop gets the full signed request.
- To test against your own machine, use a public HTTPS tunnel (ngrok, Cloudflare Tunnel) rather than `localhost`.

## Limits and gotchas

- **20 webhooks per account** on every plan (`403 package_limit`, `feature: "webhooks"`). Webhooks do not count
  against your contact limit.
- One webhook per URL: a second one for the same URL answers `409`.
- `enabled: false` is refused on create (`422`, reason `not_on_create`).
- Your receiver should be idempotent (deduplicate on `HT-Delivery`) and quick (answer within 10 seconds).
- Do not build a workflow that depends on a lifecycle event arriving - those are sent once. Reconcile with the API.

## Webhooks vs webhook contacts

| | v2 webhook (this page) | Webhook alert contact |
|---|---|---|
| Set up in | API, SDKs, CLI, MCP, Terraform | The app (**Contacts**) or `POST /contact` with `type: "http"` |
| Events | 13 typed events incl. lifecycle and expiry | Monitor alerts (down, up, still down) through its subscriptions |
| Payload | Fixed, documented JSON envelope | Default JSON/form, or your own template with `[[token]]` placeholders |
| Signed | Yes (HMAC, two schemes) | No |
| Retries | Durable ladder for engine events | Per the alert pipeline |
| HTTPS | Required | Not required |

A webhook contact is set up per monitor subscription like any other alert channel - see
[Webhook alerts](/alerts/channels/webhook/). Deleting one never affects the other, and `GET /contact` does not list
v2 webhooks.

## Related

- [Webhook events reference](/reference/webhook-events/)
- [REST API v2](/integrations/rest-api/)
- [Webhook alert contacts](/alerts/channels/webhook/)
- [Connect an AI assistant with MCP](/integrations/mcp/)