webhook on a job or monitor to get a JSON POST when it reports, instead of polling. Verify the signature before you trust a call. Any 2xx answer acknowledges it. Everything on this page is free. Webhooks need an API key.
Events
Every source sends the same top-level fields:id is the job or monitor, event says what happened, eventId names the call, and your metadata comes last.
A job sends
started (a crawl gives total), then page as each page finishes, then one of completed, failed or cancelled. Use the last for final counts and cost.
failedandcancelledhave the same fields, andfailedaddsmessage.- A
pagecall holdsresultfor a page that was read, orerrorfor one that failed. When your address falls far behind, it holds onlyurlandresultOmitted: true. Read that page with the job’s status call. eventIdstays the same when a call is sent again. Skip an ID you already handled.creditsUsedis an older field for the same cost ascostUsd.
Set a webhook
Sendwebhook as a URL, or as an object. For example, "events": ["completed"] limits the calls to successful completions.
Get the signing secret
One account secret signs the calls of every job. A monitor can have its own. Keep it on your server.data.secret, and keep the whole string including the whsec_ prefix. Send { "rotate": true } to replace it. The old secret stops signing at once. For a planned rotation, accept both for a while. If the old one was exposed, accept only the new one.
Verify signatures
Every call has these headers.v1 is the HMAC-SHA256, in hex, of <t>.<body> with your secret as the key, where <body> is the exact request text. Read the raw body before you parse it. Compare in constant time. Reject a missing or malformed signature, or a t more than 5 minutes from your clock.
After verification, store the body by eventId. If the ID already exists, answer 2xx without processing again. Answer before slow work.
Retries
Answer2xx within 5 seconds. Redirects are not followed. A failed call is sent again after 0.5, 2 and 8 seconds, so 4 attempts in all. The calls of one job or monitor go in order, one at a time. After 3 failed deliveries in a row, the ones still waiting are skipped.
Delivery history and retry
Records are kept for 30 days.
A delivery has
id (whd_ and a uuid), sourceType, sourceId, event, url, status, attempts and the time and status code of the last one.
Limits and errors
Codes are on Errors.- The payload is kept 7 days, and never for zero data retention jobs or after you delete a batch’s results.
404 notFound: the delivery belongs to another account or is over 30 days old.400 invalidRequest: awebhook.urlthat is not public, awebhook.eventsname the source does not send, or a retry when the payload was not kept, is over 7 days old, or the monitor was deleted or has no webhook.