SerpBase

API reference · Web scraping

Web Crawl

Start a bounded crawl from a public website URL. The catalog price is fixed per customer request, not multiplied by returned pages.

POSThttps://api.serpbase.dev/web/crawl
In development · Paused

This endpoint is being reworked and is paused for now. It cannot be called until it reopens.

Example

BASH
curl -X POST 'https://api.serpbase.dev/web/crawl' \
  -H "Content-Type: application/json" \
  -H 'X-API-Key: YOUR_API_KEY' \
  --max-time 60 \
  -d '{"url":"https://example.com"}'

No recorded response is published for this endpoint yet. Run the example in the Playground to see a live one.

Parameters

format
"json" | "markdown"
json (default) returns the parsed result.
More

markdown returns the same result as Markdown text (text/markdown) for language models: no envelope, no duplicate alias fields, no inline images. Same credits; failures keep the JSON envelope.

url
string
Example: "https://example.com"
country
string
Public alias for locale.country.
More

Use one spelling at a time; the API validates the parameters and values supported by this product.

cursor
string
Public alias for pagination.cursor.
More

Use one spelling at a time; the API validates the parameters and values supported by this product.

device
string
Public alias for options.device.
More

Use one spelling at a time; the API validates the parameters and values supported by this product.

gl
string
Public alias for locale.country.
More

Use one spelling at a time; the API validates the parameters and values supported by this product.

hl
string
Public alias for locale.language.
More

Use one spelling at a time; the API validates the parameters and values supported by this product.

html
boolean
Public alias for include_html.
More

Use one spelling at a time; the API validates the parameters and values supported by this product.

language
string
Public alias for locale.language.
More

Use one spelling at a time; the API validates the parameters and values supported by this product.

limit
integer
Public alias for pagination.limit.
More

Use one spelling at a time; the API validates the parameters and values supported by this product.

markdown
boolean
Public alias for include_markdown.
More

Use one spelling at a time; the API validates the parameters and values supported by this product.

mobile
boolean
Public alias for options.device.
More

Use one spelling at a time; the API validates the parameters and values supported by this product.

num
integer
Public alias for pagination.limit.
More

Use one spelling at a time; the API validates the parameters and values supported by this product.

page
integer
Public alias for pagination.page.
More

Use one spelling at a time; the API validates the parameters and values supported by this product.

q
string
Public alias for query.
More

Use one spelling at a time; the API validates the parameters and values supported by this product.

The sample shows example parameters, not the complete request contract. Known fields are validated by the API; additional parameters are allowed.

What comes back

The fields are those in the response above; every endpoint also carries status, credits_charged and request_id.

If it fails

A failed request has a non-zero status and an error, and is not charged. The ones you will see here:

1000A parameter is missing or wrongFix the request
1020No credits leftTop up
1029Over your requests per secondWait Retry-After seconds
1502 · 1504The source could not be read this timeRetry
All status codes →