Skip to main content
POST
Best for a list of addresses you already have.

Authorizations

Authorization
string
header
required

Bearer authentication header of the form Bearer <token>, where <token> is your auth token.

Headers

Idempotency-Key
string

A value you choose for one call. Send the same key with the same method, path and body within 24 hours and the first answer comes back with idempotent-replayed: true, with the same cost headers and no second charge. A call that sends zeroDataRetention and answers with page data is not kept, so a retry runs again. Needs an API key.

Query Parameters

maxTextLength
integer

Cut every text value in the response to this many characters. A value that was cut ends with …, and a page whose text was cut has isTruncated set to true. A screenshot is never cut. Leave it out to get the full text.

Required range: x >= 1

Body

application/json
urls
string[]
required

The pages or PDFs to read. An address sent twice is read once.

Minimum array length: 1
Maximum string length: 2048
webhook

Where to be told about the job: a plain address, or { url } which gets a JSON POST when the job starts, for each finished page, and when the job ends. headers are sent with every call, metadata comes back in every call, and events picks which of started, page, action, completed, failed and cancelled to send (all by default). An agent run sends action for each step it takes, and page too. Every call is signed: read Stophy-Signature (t=,v1=), where v1 is the HMAC-SHA256 of "." with your webhook secret from webhook.secret. A call that gets no 2xx is sent again, up to 3 times. Every call has an eventId that stays the same when the call is sent again, so you can skip one you already handled. If your address answers slowly, a page event can carry only the page url. Read that page with the status call.

maxConcurrency
integer

Most pages read at once for this batch. We read at most 4 at once, so a larger number gives the same speed.

Required range: 1 <= x <= 100
appendToId
string

The id of a batch of yours that is still running: add urls to it instead of starting a new batch. The batch keeps its own options and webhook. Answers jobFinished when the batch is over.

Pattern: ^[A-Za-z0-9_-]{1,100}$
ignoreInvalidURLs
boolean
default:true

Skip addresses that are not public website URLs and list them in invalidURLs. Send false to refuse the whole batch instead.

formats
enum<string> · object · object · object · object · object · object · object · object · object[]

What to return: markdown (clean page text), summary (a short summary of the page), html (the cleaned page), rawHtml (the page as sent), audio and video (for a YouTube link: the audio track as an M4A file, or the video as an MP4 file, as a link that works for 1 hour), rawBase64 (the bytes the site sent, Base64 encoded, from a live read; metadata.contentType says what they are), links (every link on the page), images (every image address), screenshot (a link to a JPEG of the visible page, kept for 7 days), changeTracking (what changed in the markdown since your key last read this page with the same options, kept 30 days; add modes ["git-diff","json"] with a schema or prompt to compare fields too), question (send { "type": "question", "question": "..." } to get answer, left out with a warning when the page does not state one), product (title, brand, category, url, description and variants with price, sale, availability and images, left out with a warning when there is none) and menu (merchant, currency and sections of items with prices, left out with a warning when there is none) network (the JSON the page loaded while it rendered, up to 30 responses) and branding (the page's colors, fonts, typography, spacing, button and input styles, logo and brand images, read from how the page looks, with a field left out when the page does not show it). Send { "type": "screenshot", "fullPage": true } for the whole page. Send { "type": "highlights", "query": "refund policy" } for the passages that best match a query. Send { "type": "json", "schema": { "type": "object", "properties": { "price": { "type": "number" } } } } or { "type": "json", "prompt": "the price and the stock" } for fields filled from the page, each checked against it. Send { "type": "json", "parser": "amazon-product" } to get a site's own data as json instead of reading the page; GET /v1/parsers lists the parsers. Send { "type": "json", "rules": { "title": "h1", "links": { "selector": "a", "type": "list", "output": "@href" } } } to read the page with CSS rules and no model, at the page's own price. Send { "type": "attributes", "selectors": [{ "selector": "a", "attribute": "href" }] } for attribute values, returned in attributes. Without a parser or rules, json needs a schema or a prompt and is filled from the page. json with a schema or a prompt, summary, highlights, question, product, menu and changeTracking in json mode add $3 per 1,000 pages each when they return something. JSON with CSS rules and attributes add nothing. A named parser uses its own price. Any page read or other paid format you ask for is charged separately. A screenshot, network or branding read costs $1 per 1,000 pages, the same as any page. A PDF comes back as markdown only, and adds $0.50 per 1,000 pages of text from its text layer and $2.50 per 1,000 pages of text read from its picture that comes back. A CSV or an Excel .xlsx file comes back as markdown tables.

Required array length: 1 - 20 elements
Available options:
markdown,
summary,
html,
rawHtml,
rawBase64,
links,
images,
network,
branding,
audio,
video,
screenshot,
changeTracking,
product,
menu
onlyMainContent
boolean
default:true

Keep only the page's main content, without menus, headers, footers and ads.

onlyCleanContent
boolean

Send true to have a model remove what onlyMainContent leaves behind in the markdown: cookie banners, ads, share widgets, breadcrumbs, sign-up prompts, comment prompts and related-article lists. It only removes whole paragraphs and lists; it never rewrites or adds text, and code blocks are always kept. Skipped with a warning when the page is too long to clean or the model does not answer. Not with zeroDataRetention. Summary, answers and highlights are made from the cleaned markdown.

includeTags
string[]

Keep only the parts of the page that match these CSS selectors, such as article or .price. When set, onlyMainContent is not applied.

Minimum array length: 1
Minimum string length: 1
excludeTags
string[]

Remove the parts of the page that match these CSS selectors, such as .comments.

Minimum array length: 1
Minimum string length: 1
maxAge
integer

Accept a stored answer up to this many milliseconds old, at most 30 days. Without it, 2 days, or less for a site whose pages change often, and a copy up to twice that age (at most 7 days) comes back at once as stale while a fresh read runs. 0 always reads the page live. metadata.cacheState says which one you got.

Required range: 0 <= x <= 9007199254740991
minAge
integer

Answer only from a stored copy that is at least this many milliseconds old, and never read the page live. 0 accepts any stored copy. A page with no such copy is notFound.

Required range: 0 <= x <= 9007199254740991
headers
object

Extra request headers to send to the site, such as { "x-plan": "pro" }. A read with headers is never answered from or saved to a stored copy.

waitFor
integer

Extra milliseconds to wait before reading, up to 60000, for pages that load their content late. To wait for an element, add a wait action with a selector. A read that waits costs $1 per 1,000 pages, the same as any page.

Required range: 0 <= x <= 60000
mobile
boolean

Read the page as a phone would: a phone-sized screen and the mobile version of the site. The page loads in a browser.

skipTlsVerification
boolean

Send true to read a page whose certificate is expired, self-signed or for another name. The read is not private against someone between us and the site.

timeout
integer
default:60000

How long to keep trying, in milliseconds, from 1 second to 5 minutes; 60 seconds by default. A page that needs a browser needs at least 5 seconds, and pages behind a bot check can need 10 to 30 seconds the first time. A read that runs out of time is an error and costs nothing.

Required range: 1000 <= x <= 300000
parsers
(string | object)[]

Only the pdf parser exists. Send "pdf" or { "type": "pdf", mode, maxPages, pages, blocks, pageMarkers } with the same meaning as in parse. Without mode a PDF is read as auto: from its text layer, and from the picture of any page that has no text. fast reads the text layer only, and ocr reads the picture of every page, up to 20 pages. Each page of text that comes back costs $0.50 per 1,000 pages from its text layer and $2.50 per 1,000 pages from its picture. Pictures are read only with an API key.

Minimum array length: 1
Allowed value: "pdf"
actions
object[]

Up to 50 steps to run on the page, in order, before reading it: { "type": "wait", "milliseconds": 500 } (up to 60 seconds), { "type": "wait", "selector": ".results" }, { "type": "click", "selector": "button.more" }, { "type": "write", "text": "tides" }, { "type": "press", "key": "Enter" }, { "type": "scroll", "direction": "down" } (also up, left or right; amount is pixels, "viewport" or "max"; maxScrolls 1 to 50; selector scrolls an inner container; "max" keeps scrolling until the page stops growing, for feeds that load as you go), { "type": "screenshot", "fullPage": true } (links come back in actions.screenshots), { "type": "scrape" } (the page at that point, in actions.scrapes), { "type": "executeJavascript", "script": "document.title" } (results in actions.javascriptReturns), { "type": "pdf", "format": "A4" } (links in actions.pdfs) or { "type": "perform", "action": "Click the Load more button" } (a step in plain words, up to 5 browser steps on this page, never leaving the site; at most 5 per call). A read with actions costs $1 per 1,000 pages, the same as any page; the actions themselves add nothing. If an action fails, you still get the page as it stood, with a warning that names the action and says why.

Required array length: 1 - 50 elements
location
object

Where to read the page from. The default is the United States.

removeBase64Images
boolean

Images inside the page that carry their data in the address (data:image) are dropped by default. Send false to keep them in markdown and html.

blockAds
boolean

Ads and trackers are not loaded when a page runs in a browser. Send false to load them.

fastMode
boolean

Send true to read the page without opening a browser: the answer comes sooner, but a page that needs a browser, or that refuses a plain request, comes back as an error instead of being retried. Ignored when the call needs a browser (actions, screenshot, waitFor, mobile, profile, network, branding).

proxy
enum<string>

auto: basic first, enhanced once if the site refuses. basic: one plain attempt. enhanced: the stronger read from the start. stealth is the same as enhanced. Same price.

Available options:
auto,
basic,
enhanced,
stealth
storeInCache
boolean
default:true

Send false to neither read nor keep a stored copy of this page.

lockdown
boolean

Send true to answer only from a stored copy of the page and never read it live, whatever its age. A page with no stored copy is notFound. The same as minAge 0.

checkPromptInjection
boolean

Send true to scan the page text for instructions aimed at an AI model before anything is built from it. A page that has them is refused with 403 promptInjectionDetected and costs nothing. The scan adds $2.50 per 1,000 pages when it runs; a page it cannot scan fully comes back with a warning and no charge.

redactPII

Send true, or an object, to hide personal data in the markdown and in everything made from it (summary, answers, highlights, json). Links, html and rawHtml are not changed. A page the model cannot scan is refused rather than returned unredacted.

profile
object

Read the page with the cookies and local storage that earlier calls saved under this name, kept for your account for 30 days after the last use. A call with a profile always loads the page in a browser, costs $1 per 1,000 pages, the same as any page, and is never answered from a stored copy. Only one call at a time can save to a profile.

zeroDataRetention
boolean
default:false

Send true to keep nothing from this call: no stored copy is read or made, and nothing from it is logged. Adds $2 per 1,000 scraped URLs, not per PDF page. Not with a screenshot.

How links appear in the markdown. inline keeps text [blocked]. numbered writes Pricing⟨3⟩ and lists each address once at the end. none keeps only the link text, which makes link-heavy pages 10 to 70% shorter; ask for the links format to get the addresses.

Available options:
inline,
numbered,
none
maxChars
integer

Cut the markdown at this many characters; isTruncated says when that happened and nextOffset where to read on. With query, how much of the page to keep, 8000 by default.

Required range: 500 <= x <= 1000000
query
string

What you are looking for on the page, such as refund window. A page longer than maxChars (8000 by default) comes back as the passages that match, in page order, each with the text around it and its table header; a shorter page comes back whole. Costs nothing extra, and a PDF is charged for the share of its pages that comes back. Not with offset.

Required string length: 1 - 10000

Response

The data, and what this call cost in dollars.

success
boolean
required
invalidURLs
string[]
required
id
string
url
string
status
string
Allowed value: "scraping"
queuePosition
integer
Required range: -9007199254740991 <= x <= 9007199254740991