API reference · Web scraping
Web Crawl
Start a bounded crawl from a public website URL. The catalog price is fixed per customer request, not multiplied by returned pages.
https://api.serpbase.dev/web/crawlThis endpoint is being reworked and is paused for now. It cannot be called until it reopens.
Example
curl -X POST 'https://api.serpbase.dev/web/crawl' \
-H "Content-Type: application/json" \
-H 'X-API-Key: YOUR_API_KEY' \
--max-time 60 \
-d '{"url":"https://example.com"}'No recorded response is published for this endpoint yet. Run the example in the Playground to see a live one.
Parameters
formatMore
markdown returns the same result as Markdown text (text/markdown) for language models: no envelope, no duplicate alias fields, no inline images. Same credits; failures keep the JSON envelope.
urlcountryMore
Use one spelling at a time; the API validates the parameters and values supported by this product.
cursorMore
Use one spelling at a time; the API validates the parameters and values supported by this product.
deviceMore
Use one spelling at a time; the API validates the parameters and values supported by this product.
glMore
Use one spelling at a time; the API validates the parameters and values supported by this product.
hlMore
Use one spelling at a time; the API validates the parameters and values supported by this product.
htmlMore
Use one spelling at a time; the API validates the parameters and values supported by this product.
languageMore
Use one spelling at a time; the API validates the parameters and values supported by this product.
limitMore
Use one spelling at a time; the API validates the parameters and values supported by this product.
markdownMore
Use one spelling at a time; the API validates the parameters and values supported by this product.
mobileMore
Use one spelling at a time; the API validates the parameters and values supported by this product.
numMore
Use one spelling at a time; the API validates the parameters and values supported by this product.
pageMore
Use one spelling at a time; the API validates the parameters and values supported by this product.
qMore
Use one spelling at a time; the API validates the parameters and values supported by this product.
The sample shows example parameters, not the complete request contract. Known fields are validated by the API; additional parameters are allowed.
What comes back
The fields are those in the response above; every endpoint also carries status, credits_charged and request_id.
If it fails
A failed request has a non-zero status and an error, and is not charged. The ones you will see here:
1000A parameter is missing or wrongFix the request1020No credits leftTop up1029Over your requests per secondWait Retry-After seconds1502 · 1504The source could not be read this timeRetry