Skip to main content
Set webhook on a job or monitor to get a JSON POST when it reports, instead of polling. Verify the signature before you trust a call. Any 2xx answer acknowledges it. Everything on this page is free. Webhooks need an API key.

Events

Every source sends the same top-level fields: id is the job or monitor, event says what happened, eventId names the call, and your metadata comes last. A job sends started (a crawl gives total), then page as each page finishes, then one of completed, failed or cancelled. Use the last for final counts and cost.
  • failed and cancelled have the same fields, and failed adds message.
  • A page call holds result for a page that was read, or error for one that failed. When your address falls far behind, it holds only url and resultOmitted: true. Read that page with the job’s status call.
  • eventId stays the same when a call is sent again. Skip an ID you already handled.
  • creditsUsed is an older field for the same cost as costUsd.

Set a webhook

Send webhook as a URL, or as an object. For example, "events": ["completed"] limits the calls to successful completions.

Get the signing secret

One account secret signs the calls of every job. A monitor can have its own. Keep it on your server.
Read data.secret, and keep the whole string including the whsec_ prefix. Send { "rotate": true } to replace it. The old secret stops signing at once. For a planned rotation, accept both for a while. If the old one was exposed, accept only the new one.

Verify signatures

Every call has these headers. v1 is the HMAC-SHA256, in hex, of <t>.<body> with your secret as the key, where <body> is the exact request text. Read the raw body before you parse it. Compare in constant time. Reject a missing or malformed signature, or a t more than 5 minutes from your clock. After verification, store the body by eventId. If the ID already exists, answer 2xx without processing again. Answer before slow work.

Retries

Answer 2xx within 5 seconds. Redirects are not followed. A failed call is sent again after 0.5, 2 and 8 seconds, so 4 attempts in all. The calls of one job or monitor go in order, one at a time. After 3 failed deliveries in a row, the ones still waiting are skipped.

Delivery history and retry

Records are kept for 30 days. A delivery has id (whd_ and a uuid), sourceType, sourceId, event, url, status, attempts and the time and status code of the last one.

Limits and errors

Codes are on Errors.
  • The payload is kept 7 days, and never for zero data retention jobs or after you delete a batch’s results.
  • 404 notFound: the delivery belongs to another account or is over 30 days old.
  • 400 invalidRequest: a webhook.url that is not public, a webhook.events name the source does not send, or a retry when the payload was not kept, is over 7 days old, or the monitor was deleted or has no webhook.