> ## Documentation Index
> Fetch the complete documentation index at: https://docs.stophy.dev/llms.txt
> Use this file to discover all available pages before exploring further.

# Crawl

> Crawl a website in the background.

Best for reading a whole section of a site without holding a call open.

```bash cURL theme={null}
curl -X POST https://api.stophy.dev/v1/crawl \
  -H "Authorization: Bearer $STOPHY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url":"https://bun.com/docs","limit":2}'
```

```json 200 theme={null}
{
  "success": true,
  "id": "job_qqmWroHTkWk61kDjA32Q3g",
  "url": "https://api.stophy.dev/v1/crawl/job_qqmWroHTkWk61kDjA32Q3g"
}
```


## OpenAPI

````yaml api-reference/openapi.json POST /v1/crawl
openapi: 3.1.0
info:
  title: Stophy API
  version: '1'
  description: >-
    Each operation shows its method and its input: a JSON object body, path and
    query parameters, or a file upload.


    Responses leave out empty values: null, empty strings, empty objects and
    empty nested lists are not sent.


    Text comes back whole. To get shorter responses, add
    ?maxTextLength=<characters> to the URL. Every text value longer than that is
    cut and ends with …. A screenshot is never cut, and a page whose text was
    cut has isTruncated set to true.


    The x-cost-usd response header is what the request cost in dollars. An
    operation's description gives its pricing units and add-ons. The OpenAPI
    fields x-cost-usd and x-cost-usd-max are list-price estimates, not receipts
    or spending limits. When a call starts a job, check the job result for its
    total cost.


    When a response has a cursor, send it back unchanged with the same input to
    get the next page. When it has a page, send the next page number.
servers:
  - url: https://api.stophy.dev
security:
  - apiKey: []
paths:
  /v1/crawl:
    post:
      summary: Crawl a website in the background
      description: >-
        Costs $1 per 1,000 pages, however they are reached. checkPromptInjection
        adds $2.50 per 1,000 pages when the scan runs, json with a schema or a
        prompt, summary, highlights, question, product, menu and changeTracking
        in json mode add $3 per 1,000 pages each when they return something. CSS
        rules and attributes add nothing. A named parser uses its own price. Any
        page read or other paid format you ask for is charged separately. A PDF
        adds $0.50 per 1,000 pages of text from its text layer and $2.50 per
        1,000 pages of text read from its picture that come back, so a PDF read
        with maxChars, offset or query is charged for the share of its pages
        that it returns, at least 1 page. Browser actions add nothing. An answer
        from a stored copy within maxAge costs $1 per 1,000 pages like any page.
        An unreadable page costs nothing. The base page price is waived for a
        readable 404 or 410. Each page is charged when it finishes. Starting the
        job is free.
      operationId: crawl
      parameters:
        - $ref: '#/components/parameters/MaxTextLength'
        - $ref: '#/components/parameters/IdempotencyKey'
      requestBody:
        required: true
        content:
          application/json:
            schema:
              type: object
              properties:
                url:
                  description: The page to start from, such as https://bun.com/docs.
                  type: string
                prompt:
                  description: >-
                    What you are looking for, such as sqlite performance. Pages
                    about it are read first, and the crawl stops once new pages
                    stop mentioning it.
                  type: string
                  minLength: 1
                  maxLength: 1000
                includePaths:
                  default: []
                  description: >-
                    Read only pages whose path matches one of these regular
                    expressions, such as blog/.*.
                  maxItems: 100
                  type: array
                  items:
                    type: string
                    minLength: 1
                    maxLength: 200
                excludePaths:
                  default: []
                  description: >-
                    Skip pages whose path matches one of these regular
                    expressions, such as tag/.*.
                  maxItems: 100
                  type: array
                  items:
                    type: string
                    minLength: 1
                    maxLength: 200
                maxDiscoveryDepth:
                  description: >-
                    How many links away from the start page to go. Without it
                    there is no depth limit, and limit bounds the crawl.
                  type: integer
                  minimum: 0
                  maximum: 9007199254740991
                sitemap:
                  default: include
                  description: >-
                    include also uses the site's sitemaps. only reads just the
                    pages the sitemaps name. skip ignores them.
                  type: string
                  enum:
                    - skip
                    - include
                    - only
                ignoreQueryParameters:
                  default: false
                  description: >-
                    Treat addresses that differ only in their ?query part as one
                    page.
                  type: boolean
                regexOnFullURL:
                  default: false
                  description: >-
                    Match includePaths and excludePaths against the whole
                    address, not only the path.
                  type: boolean
                limit:
                  default: 10000
                  description: Stop after reading this many pages in total.
                  type: integer
                  minimum: 1
                  maximum: 10000
                crawlEntireDomain:
                  default: false
                  description: >-
                    Read the whole site, not only pages under the start page's
                    path.
                  type: boolean
                deduplicateSimilarURLs:
                  default: true
                  description: >-
                    Read one page when addresses differ only by www, a trailing
                    slash, a #fragment, tracking parameters such as utm_source,
                    or the order of the ?query. Send false to read each address.
                  type: boolean
                allowExternalLinks:
                  default: false
                  description: >-
                    Also follow links to other sites. They count toward limit
                    and maxDiscoveryDepth. Each other site's robots.txt is read
                    too, for up to 25 new sites per page. A page on a site whose
                    robots.txt cannot be read, or that comes after those 25, is
                    skipped.
                  type: boolean
                allowSubdomains:
                  default: false
                  description: >-
                    Also read pages on subdomains, such as docs.bun.com for
                    bun.com.
                  type: boolean
                delay:
                  description: >-
                    Seconds to wait between pages. With delay the crawl reads
                    one page at a time. At most 60.
                  type: number
                  minimum: 0
                  maximum: 60
                maxConcurrency:
                  description: >-
                    Most pages read at once. We read at most 4 at once, so a
                    larger number gives the same speed.
                  type: integer
                  minimum: 1
                  maximum: 100
                scrapeOptions:
                  description: >-
                    How to read each page, the same options and formats as
                    scrape. Without formats, each page comes back as markdown.
                  type: object
                  properties:
                    formats:
                      description: >-
                        What to return: markdown (clean page text), summary (a
                        short summary of the page), html (the cleaned page),
                        rawHtml (the page as sent), audio and video (for a
                        YouTube link: the audio track as an M4A file, or the
                        video as an MP4 file, as a link that works for 1 hour),
                        rawBase64 (the bytes the site sent, Base64 encoded, from
                        a live read; metadata.contentType says what they are),
                        links (every link on the page), images (every image
                        address), screenshot (a link to a JPEG of the visible
                        page, kept for 7 days), changeTracking (what changed in
                        the markdown since your key last read this page with the
                        same options, kept 30 days; add modes
                        ["git-diff","json"] with a schema or prompt to compare
                        fields too), question (send { "type": "question",
                        "question": "..." } to get answer, left out with a
                        warning when the page does not state one), product
                        (title, brand, category, url, description and variants
                        with price, sale, availability and images, left out with
                        a warning when there is none) and menu (merchant,
                        currency and sections of items with prices, left out
                        with a warning when there is none) network (the JSON the
                        page loaded while it rendered, up to 30 responses) and
                        branding (the page's colors, fonts, typography, spacing,
                        button and input styles, logo and brand images, read
                        from how the page looks, with a field left out when the
                        page does not show it). Send { "type": "screenshot",
                        "fullPage": true } for the whole page. Send { "type":
                        "highlights", "query": "refund policy" } for the
                        passages that best match a query. Send { "type": "json",
                        "schema": { "type": "object", "properties": { "price": {
                        "type": "number" } } } } or { "type": "json", "prompt":
                        "the price and the stock" } for fields filled from the
                        page, each checked against it. Send { "type": "json",
                        "parser": "amazon-product" } to get a site's own data as
                        json instead of reading the page; GET /v1/parsers lists
                        the parsers. Send { "type": "json", "rules": { "title":
                        "h1", "links": { "selector": "a", "type": "list",
                        "output": "@href" } } } to read the page with CSS rules
                        and no model, at the page's own price. Send { "type":
                        "attributes", "selectors": [{ "selector": "a",
                        "attribute": "href" }] } for attribute values, returned
                        in attributes. Without a parser or rules, json needs a
                        schema or a prompt and is filled from the page. json
                        with a schema or a prompt, summary, highlights,
                        question, product, menu and changeTracking in json mode
                        add $3 per 1,000 pages each when they return something.
                        JSON with CSS rules and attributes add nothing. A named
                        parser uses its own price. Any page read or other paid
                        format you ask for is charged separately. A screenshot,
                        network or branding read costs $1 per 1,000 pages, the
                        same as any page. A PDF comes back as markdown only, and
                        adds $0.50 per 1,000 pages of text from its text layer
                        and $2.50 per 1,000 pages of text read from its picture
                        that comes back. A CSV or an Excel .xlsx file comes back
                        as markdown tables.
                      default:
                        - markdown
                      minItems: 1
                      maxItems: 20
                      type: array
                      items:
                        anyOf:
                          - type: string
                            enum:
                              - markdown
                              - summary
                              - html
                              - rawHtml
                              - rawBase64
                              - links
                              - images
                              - network
                              - branding
                              - audio
                              - video
                              - screenshot
                              - changeTracking
                              - product
                              - menu
                          - type: object
                            properties:
                              type:
                                type: string
                                enum:
                                  - markdown
                                  - summary
                                  - html
                                  - rawHtml
                                  - rawBase64
                                  - links
                                  - images
                                  - network
                                  - branding
                                  - audio
                                  - video
                            required:
                              - type
                            additionalProperties: false
                          - type: object
                            properties:
                              type:
                                type: string
                                const: changeTracking
                              modes:
                                description: >-
                                  git-diff returns a unified diff of the
                                  markdown. json compares the fields in schema
                                  or prompt between this read and the last one.
                                  Without modes you get git-diff.
                                maxItems: 2
                                type: array
                                items:
                                  type: string
                                  enum:
                                    - git-diff
                                    - json
                              schema:
                                description: >-
                                  The fields to fill, as a JSON Schema (draft-07
                                  or 2020-12) object with properties. Pydantic's
                                  model_json_schema() and zod's toJSONSchema()
                                  output work as they are. enum, const, anyOf,
                                  oneOf, nullable types, $ref with $defs,
                                  required, descriptions and any key names are
                                  read. Up to 100 top-level fields and 32 KB. A
                                  field the page does not state comes back null.
                                type: object
                                propertyNames:
                                  type: string
                                additionalProperties:
                                  $ref: '#/components/schemas/crawl.input.schema0'
                              prompt:
                                description: >-
                                  With json mode: what to compare, in plain
                                  words, such as the price and stock.
                                type: string
                                minLength: 1
                                maxLength: 10000
                              tag:
                                type: 'null'
                            required:
                              - type
                            additionalProperties: false
                          - type: object
                            properties:
                              type:
                                type: string
                                const: question
                              question:
                                type: string
                                minLength: 1
                                maxLength: 10000
                                description: >-
                                  A question to answer from the page, such as
                                  what the refund window is.
                            required:
                              - type
                              - question
                            additionalProperties: false
                          - type: object
                            properties:
                              type:
                                type: string
                                const: product
                            required:
                              - type
                            additionalProperties: false
                          - type: object
                            properties:
                              type:
                                type: string
                                const: menu
                            required:
                              - type
                            additionalProperties: false
                          - type: object
                            properties:
                              type:
                                type: string
                                const: screenshot
                              fullPage:
                                default: false
                                description: >-
                                  Capture the whole page from top to bottom
                                  instead of the visible part.
                                type: boolean
                              quality:
                                description: JPEG quality from 1 to 100. The default is 70.
                                type: integer
                                minimum: 1
                                maximum: 100
                              viewport:
                                description: >-
                                  The window size in pixels, such as { width:
                                  390, height: 844 }.
                                type: object
                                properties:
                                  width:
                                    type: integer
                                    minimum: 200
                                    maximum: 3840
                                  height:
                                    type: integer
                                    minimum: 200
                                    maximum: 2160
                                required:
                                  - width
                                  - height
                                additionalProperties: false
                            required:
                              - type
                            additionalProperties: false
                          - type: object
                            properties:
                              type:
                                type: string
                                const: highlights
                              query:
                                type: string
                                minLength: 1
                                maxLength: 10000
                                description: >-
                                  What you are looking for on the page, such as
                                  refund policy.
                              count:
                                default: 5
                                description: How many passages to return, best match first.
                                type: integer
                                minimum: 1
                                maximum: 20
                            required:
                              - type
                              - query
                            additionalProperties: false
                          - type: object
                            properties:
                              type:
                                type: string
                                const: json
                              parser:
                                description: >-
                                  Read the page with a named parser, which gives
                                  the site's own data as json. The url must be
                                  the kind of page the parser reads. Parsers are
                                  the only way to read some sites. GET
                                  /v1/parsers lists them.
                                type: string
                              options:
                                description: >-
                                  Only with parser: extra inputs for it, such as
                                  { "page": 2 } for the next page of results or
                                  { "minRating": 4.5 } for a filter. GET
                                  /v1/parsers lists what each parser accepts.
                                type: object
                                propertyNames:
                                  type: string
                                additionalProperties:
                                  $ref: '#/components/schemas/crawl.input.schema1'
                              schema:
                                description: >-
                                  The fields to fill, as a JSON Schema (draft-07
                                  or 2020-12) object with properties. Pydantic's
                                  model_json_schema() and zod's toJSONSchema()
                                  output work as they are. enum, const, anyOf,
                                  oneOf, nullable types, $ref with $defs,
                                  required, descriptions and any key names are
                                  read. Up to 100 top-level fields and 32 KB. A
                                  field the page does not state comes back null.
                                type: object
                                propertyNames:
                                  type: string
                                additionalProperties:
                                  $ref: '#/components/schemas/crawl.input.schema0'
                              prompt:
                                description: >-
                                  What to pull from the page in plain words,
                                  such as the version and the release date.
                                type: string
                                minLength: 1
                                maxLength: 10000
                              rules:
                                description: >-
                                  Read the page with CSS rules instead of a
                                  model, no extra fee. Each field is a selector
                                  such as "h1" (the first match's text),
                                  "a@href" (an attribute) or { "selector": "li",
                                  "type": "list", "output": "text" | "html" |
                                  "@attribute" | { nested fields } }. A missing
                                  item is null and an unmatched list is [].
                                  Relative links become absolute. Up to 100
                                  fields, 5 levels deep. Not with schema, prompt
                                  or parser.
                                type: object
                                propertyNames:
                                  type: string
                                additionalProperties:
                                  $ref: '#/components/schemas/crawl.input.schema2'
                              checkPromptInjection:
                                description: >-
                                  Deprecated: send checkPromptInjection at the
                                  top level of the call.
                                type: boolean
                            required:
                              - type
                            additionalProperties: false
                          - type: object
                            properties:
                              type:
                                type: string
                                const: attributes
                              selectors:
                                minItems: 1
                                maxItems: 50
                                type: array
                                items:
                                  type: object
                                  properties:
                                    selector:
                                      type: string
                                      minLength: 1
                                    attribute:
                                      type: string
                                      minLength: 1
                                      maxLength: 100
                                  required:
                                    - selector
                                    - attribute
                                  additionalProperties: false
                                description: >-
                                  The elements and attributes to read, such as {
                                  selector: 'a.buy', attribute: 'href' }.
                            required:
                              - type
                              - selectors
                            additionalProperties: false
                    onlyMainContent:
                      default: true
                      description: >-
                        Keep only the page's main content, without menus,
                        headers, footers and ads.
                      type: boolean
                    onlyCleanContent:
                      description: >-
                        Send true to have a model remove what onlyMainContent
                        leaves behind in the markdown: cookie banners, ads,
                        share widgets, breadcrumbs, sign-up prompts, comment
                        prompts and related-article lists. It only removes whole
                        paragraphs and lists; it never rewrites or adds text,
                        and code blocks are always kept. Skipped with a warning
                        when the page is too long to clean or the model does not
                        answer. Not with zeroDataRetention. Summary, answers and
                        highlights are made from the cleaned markdown.
                      type: boolean
                    includeTags:
                      description: >-
                        Keep only the parts of the page that match these CSS
                        selectors, such as article or .price. When set,
                        onlyMainContent is not applied.
                      minItems: 1
                      type: array
                      items:
                        type: string
                        minLength: 1
                    excludeTags:
                      description: >-
                        Remove the parts of the page that match these CSS
                        selectors, such as .comments.
                      minItems: 1
                      type: array
                      items:
                        type: string
                        minLength: 1
                    maxAge:
                      description: >-
                        Accept a stored answer up to this many milliseconds old,
                        at most 30 days. Without it, 2 days, or less for a site
                        whose pages change often, and a copy up to twice that
                        age (at most 7 days) comes back at once as stale while a
                        fresh read runs. 0 always reads the page live.
                        metadata.cacheState says which one you got.
                      type: integer
                      minimum: 0
                      maximum: 9007199254740991
                    minAge:
                      description: >-
                        Answer only from a stored copy that is at least this
                        many milliseconds old, and never read the page live. 0
                        accepts any stored copy. A page with no such copy is
                        notFound.
                      type: integer
                      minimum: 0
                      maximum: 9007199254740991
                    headers:
                      description: >-
                        Extra request headers to send to the site, such as {
                        "x-plan": "pro" }. A read with headers is never answered
                        from or saved to a stored copy.
                      type: object
                      propertyNames:
                        type: string
                        pattern: ^[A-Za-z0-9-]{1,64}$
                      additionalProperties:
                        type: string
                        maxLength: 2000
                        pattern: ^[\t\x20-\x7e]*$
                    waitFor:
                      description: >-
                        Extra milliseconds to wait before reading, up to 60000,
                        for pages that load their content late. To wait for an
                        element, add a wait action with a selector. A read that
                        waits costs $1 per 1,000 pages, the same as any page.
                      type: integer
                      minimum: 0
                      maximum: 60000
                    mobile:
                      description: >-
                        Read the page as a phone would: a phone-sized screen and
                        the mobile version of the site. The page loads in a
                        browser.
                      type: boolean
                    skipTlsVerification:
                      description: >-
                        Send true to read a page whose certificate is expired,
                        self-signed or for another name. The read is not private
                        against someone between us and the site.
                      type: boolean
                    timeout:
                      description: >-
                        How long to keep trying each page, in milliseconds, from
                        1 second to 5 minutes; 20 seconds, plus waitFor, by
                        default. A page that runs out of time is a failed page
                        and costs nothing.
                      type: integer
                      minimum: 1000
                      maximum: 300000
                    parsers:
                      description: >-
                        Only the pdf parser exists. Send "pdf" or { "type":
                        "pdf", mode, maxPages, pages, blocks, pageMarkers } with
                        the same meaning as in parse. Without mode a PDF is read
                        as auto: from its text layer, and from the picture of
                        any page that has no text. fast reads the text layer
                        only, and ocr reads the picture of every page, up to 20
                        pages. Each page of text that comes back costs $0.50 per
                        1,000 pages from its text layer and $2.50 per 1,000
                        pages from its picture. Pictures are read only with an
                        API key.
                      minItems: 1
                      type: array
                      items:
                        anyOf:
                          - type: string
                            const: pdf
                          - type: object
                            properties:
                              type:
                                type: string
                                const: pdf
                              mode:
                                description: >-
                                  fast reads the text layer only. auto (the
                                  default) reads the text layer and reads the
                                  picture of any page that has none. ocr reads
                                  the picture of every page.
                                type: string
                                enum:
                                  - fast
                                  - auto
                                  - ocr
                              maxPages:
                                description: >-
                                  Read at most this many pages, up to 500, which
                                  is also the default.
                                type: integer
                                minimum: 1
                                maximum: 500
                              pages:
                                description: >-
                                  Send true to also get each page's markdown in
                                  pages, as { pageNumber, markdown }. No extra
                                  charge.
                                type: boolean
                              blocks:
                                description: >-
                                  Send true to also get the page's layout in
                                  blocks: each title, section header or run of
                                  text, with its page, reading order, position
                                  on the page (bbox as left, top, right, bottom
                                  from 0 to 1) and where its text sits in that
                                  page's markdown (span, as in pages). Tables,
                                  figures and formulas are not told apart. A
                                  page read from its picture is one block. No
                                  extra charge.
                                type: boolean
                              pageMarkers:
                                description: >-
                                  Send true to mark page breaks in the markdown:
                                  pages are joined with "---" and a <!-- page N
                                  --> line, where N is the page that follows. No
                                  extra charge.
                                type: boolean
                            required:
                              - type
                            additionalProperties: false
                    actions:
                      description: >-
                        Up to 50 steps to run on the page, in order, before
                        reading it: { "type": "wait", "milliseconds": 500 } (up
                        to 60 seconds), { "type": "wait", "selector": ".results"
                        }, { "type": "click", "selector": "button.more" }, {
                        "type": "write", "text": "tides" }, { "type": "press",
                        "key": "Enter" }, { "type": "scroll", "direction":
                        "down" } (also up, left or right; amount is pixels,
                        "viewport" or "max"; maxScrolls 1 to 50; selector
                        scrolls an inner container; "max" keeps scrolling until
                        the page stops growing, for feeds that load as you go),
                        { "type": "screenshot", "fullPage": true } (links come
                        back in actions.screenshots), { "type": "scrape" } (the
                        page at that point, in actions.scrapes), { "type":
                        "executeJavascript", "script": "document.title" }
                        (results in actions.javascriptReturns), { "type": "pdf",
                        "format": "A4" } (links in actions.pdfs) or { "type":
                        "perform", "action": "Click the Load more button" } (a
                        step in plain words, up to 5 browser steps on this page,
                        never leaving the site; at most 5 per call). A read with
                        actions costs $1 per 1,000 pages, the same as any page;
                        the actions themselves add nothing. If an action fails,
                        you still get the page as it stood, with a warning that
                        names the action and says why.
                      minItems: 1
                      maxItems: 50
                      type: array
                      items:
                        anyOf:
                          - type: object
                            properties:
                              type:
                                type: string
                                const: wait
                              milliseconds:
                                type: integer
                                minimum: 1
                                maximum: 60000
                            required:
                              - type
                              - milliseconds
                            additionalProperties: false
                          - type: object
                            properties:
                              type:
                                type: string
                                const: wait
                              selector:
                                type: string
                                minLength: 1
                            required:
                              - type
                              - selector
                            additionalProperties: false
                          - type: object
                            properties:
                              type:
                                type: string
                                const: click
                              selector:
                                type: string
                                minLength: 1
                              all:
                                type: boolean
                                const: false
                            required:
                              - type
                              - selector
                            additionalProperties: false
                          - type: object
                            properties:
                              type:
                                type: string
                                const: write
                              text:
                                type: string
                                minLength: 1
                            required:
                              - type
                              - text
                            additionalProperties: false
                          - type: object
                            properties:
                              type:
                                type: string
                                const: press
                              key:
                                type: string
                                minLength: 1
                                maxLength: 40
                            required:
                              - type
                              - key
                            additionalProperties: false
                          - type: object
                            properties:
                              type:
                                type: string
                                const: scroll
                              direction:
                                default: down
                                type: string
                                enum:
                                  - up
                                  - down
                                  - left
                                  - right
                              amount:
                                anyOf:
                                  - type: integer
                                    minimum: 1
                                    maximum: 100000
                                  - type: string
                                    enum:
                                      - viewport
                                      - max
                              maxScrolls:
                                type: integer
                                minimum: 1
                                maximum: 50
                              selector:
                                type: string
                                minLength: 1
                            required:
                              - type
                            additionalProperties: false
                          - type: object
                            properties:
                              type:
                                type: string
                                const: screenshot
                              fullPage:
                                type: boolean
                              quality:
                                type: integer
                                minimum: 1
                                maximum: 100
                              viewport:
                                type: object
                                properties:
                                  width:
                                    type: integer
                                    minimum: 200
                                    maximum: 3840
                                  height:
                                    type: integer
                                    minimum: 200
                                    maximum: 2160
                                required:
                                  - width
                                  - height
                                additionalProperties: false
                            required:
                              - type
                            additionalProperties: false
                          - type: object
                            properties:
                              type:
                                type: string
                                const: scrape
                            required:
                              - type
                            additionalProperties: false
                          - type: object
                            properties:
                              type:
                                type: string
                                const: executeJavascript
                              script:
                                type: string
                                minLength: 1
                                maxLength: 20000
                            required:
                              - type
                              - script
                            additionalProperties: false
                          - type: object
                            properties:
                              type:
                                type: string
                                const: pdf
                              format:
                                type: string
                                enum:
                                  - Letter
                                  - Legal
                                  - Tabloid
                                  - Ledger
                                  - A0
                                  - A1
                                  - A2
                                  - A3
                                  - A4
                                  - A5
                                  - A6
                              landscape:
                                type: boolean
                              scale:
                                type: number
                                minimum: 0.1
                                maximum: 2
                            required:
                              - type
                            additionalProperties: false
                          - type: object
                            properties:
                              type:
                                type: string
                                const: perform
                              action:
                                type: string
                                minLength: 1
                                maxLength: 500
                            required:
                              - type
                              - action
                            additionalProperties: false
                    location:
                      description: >-
                        Where to read the page from. The default is the United
                        States.
                      type: object
                      properties:
                        country:
                          description: >-
                            Two-letter country code to read the page from, such
                            as DE or JP.
                          type: string
                        languages:
                          description: >-
                            Languages to ask the site for, best first, such as
                            ["de", "en"]. Sent as Accept-Language. A read with
                            languages is never answered from or saved to a
                            stored copy.
                          minItems: 1
                          maxItems: 5
                          type: array
                          items:
                            type: string
                            pattern: ^[A-Za-z]{2,3}(?:-[A-Za-z0-9]{2,8})*$
                      additionalProperties: false
                    removeBase64Images:
                      description: >-
                        Images inside the page that carry their data in the
                        address (data:image) are dropped by default. Send false
                        to keep them in markdown and html.
                      type: boolean
                    blockAds:
                      description: >-
                        Ads and trackers are not loaded when a page runs in a
                        browser. Send false to load them.
                      type: boolean
                    fastMode:
                      description: >-
                        Send true to read the page without opening a browser:
                        the answer comes sooner, but a page that needs a
                        browser, or that refuses a plain request, comes back as
                        an error instead of being retried. Ignored when the call
                        needs a browser (actions, screenshot, waitFor, mobile,
                        profile, network, branding).
                      type: boolean
                    proxy:
                      description: >-
                        auto: basic first, enhanced once if the site refuses.
                        basic: one plain attempt. enhanced: the stronger read
                        from the start. stealth is the same as enhanced. Same
                        price.
                      type: string
                      enum:
                        - auto
                        - basic
                        - enhanced
                        - stealth
                    storeInCache:
                      default: true
                      description: >-
                        Send false to neither read nor keep a stored copy of
                        this page.
                      type: boolean
                    lockdown:
                      description: >-
                        Send true to answer only from a stored copy of the page
                        and never read it live, whatever its age. A page with no
                        stored copy is notFound. The same as minAge 0.
                      type: boolean
                    checkPromptInjection:
                      description: >-
                        Send true to scan the page text for instructions aimed
                        at an AI model before anything is built from it. A page
                        that has them is refused with 403
                        promptInjectionDetected and costs nothing. The scan adds
                        $2.50 per 1,000 pages when it runs; a page it cannot
                        scan fully comes back with a warning and no charge.
                      type: boolean
                    redactPII:
                      description: >-
                        Send true, or an object, to hide personal data in the
                        markdown and in everything made from it (summary,
                        answers, highlights, json). Links, html and rawHtml are
                        not changed. A page the model cannot scan is refused
                        rather than returned unredacted.
                      anyOf:
                        - type: boolean
                        - type: object
                          properties:
                            mode:
                              default: accurate
                              description: >-
                                accurate finds names and places with the model
                                and the rest by pattern. aggressive adds guesses
                                that catch more and may hide a little too much.
                                fast uses patterns only, so it finds no ordinary
                                names or places.
                              type: string
                              enum:
                                - accurate
                                - aggressive
                                - fast
                            entities:
                              default:
                                - PERSON
                                - EMAIL
                                - PHONE
                                - LOCATION
                                - FINANCIAL
                                - SECRET
                              description: >-
                                What to hide: PERSON, EMAIL, PHONE, LOCATION,
                                FINANCIAL, SECRET. All by default.
                              minItems: 1
                              type: array
                              items:
                                type: string
                                enum:
                                  - PERSON
                                  - EMAIL
                                  - PHONE
                                  - LOCATION
                                  - FINANCIAL
                                  - SECRET
                            replaceStyle:
                              default: tag
                              description: >-
                                tag writes <EMAIL>, mask writes * for each
                                character, remove deletes it.
                              type: string
                              enum:
                                - tag
                                - mask
                                - remove
                          additionalProperties: false
                          description: Options for redactPII.
                    profile:
                      description: >-
                        Read the page with the cookies and local storage that
                        earlier calls saved under this name, kept for your
                        account for 30 days after the last use. A call with a
                        profile always loads the page in a browser, costs $1 per
                        1,000 pages, the same as any page, and is never answered
                        from a stored copy. Only one call at a time can save to
                        a profile.
                      type: object
                      properties:
                        name:
                          type: string
                          minLength: 1
                          maxLength: 128
                          description: >-
                            A name you choose, such as shop-login. Calls with
                            the same name share state.
                        saveChanges:
                          default: true
                          description: >-
                            Keep the cookies and local storage this read leaves
                            behind for the next call with this name. Send false
                            to read with the profile without changing it.
                          type: boolean
                      required:
                        - name
                      additionalProperties: false
                    zeroDataRetention:
                      default: false
                      description: >-
                        Send true to keep nothing from this call: no stored copy
                        is read or made, and nothing from it is logged. Adds $2
                        per 1,000 scraped URLs, not per PDF page. Not with a
                        screenshot.
                      type: boolean
                zeroDataRetention:
                  default: false
                  description: >-
                    Send true to keep nothing from this crawl: no stored copy is
                    read or made, page results are held for one hour at most,
                    and nothing is logged. Adds $2 per 1,000 pages.
                  type: boolean
                webhook:
                  description: >-
                    Where to be told about the job: a plain address, or { url }
                    which gets a JSON POST when the job starts, for each
                    finished page, and when the job ends. headers are sent with
                    every call, metadata comes back in every call, and events
                    picks which of started, page, action, completed, failed and
                    cancelled to send (all by default). An agent run sends
                    action for each step it takes, and page too. Every call is
                    signed: read Stophy-Signature (t=<unix seconds>,v1=<hex>),
                    where v1 is the HMAC-SHA256 of "<t>.<body>" with your
                    webhook secret from webhook.secret. A call that gets no 2xx
                    is sent again, up to 3 times. Every call has an eventId that
                    stays the same when the call is sent again, so you can skip
                    one you already handled. If your address answers slowly, a
                    page event can carry only the page url. Read that page with
                    the status call.
                  anyOf:
                    - type: string
                    - type: object
                      properties:
                        url:
                          type: string
                        headers:
                          type: object
                          propertyNames:
                            type: string
                            pattern: ^[A-Za-z0-9-]{1,64}$
                          additionalProperties:
                            type: string
                            maxLength: 1000
                        metadata:
                          type: object
                          propertyNames:
                            type: string
                          additionalProperties:
                            $ref: '#/components/schemas/crawl.input.schema0'
                        events:
                          minItems: 1
                          type: array
                          items:
                            type: string
                            enum:
                              - started
                              - page
                              - action
                              - completed
                              - failed
                              - cancelled
                      required:
                        - url
                      additionalProperties: false
              required:
                - url
              additionalProperties: false
            example:
              url: https://bun.com/docs
              limit: 2
      responses:
        '200':
          description: The data, and what this call cost in dollars.
          headers:
            X-Request-ID:
              $ref: '#/components/headers/RequestId'
            x-credits-used:
              $ref: '#/components/headers/CreditsUsed'
            x-cost-usd:
              $ref: '#/components/headers/CostUsd'
            X-RateLimit-Limit:
              $ref: '#/components/headers/RateLimitLimit'
            X-RateLimit-Remaining:
              $ref: '#/components/headers/RateLimitRemaining'
            X-RateLimit-Reset:
              $ref: '#/components/headers/RateLimitReset'
            x-cache:
              $ref: '#/components/headers/Cache'
            x-cache-age:
              $ref: '#/components/headers/CacheAge'
            idempotent-replayed:
              $ref: '#/components/headers/IdempotentReplayed'
          content:
            application/json:
              schema:
                type: object
                additionalProperties: false
                properties:
                  success:
                    type: boolean
                    const: true
                  id:
                    type: string
                  url:
                    type: string
                  status:
                    type: string
                    const: scraping
                  queuePosition:
                    type: integer
                    minimum: -9007199254740991
                    maximum: 9007199254740991
                required:
                  - success
        4XX:
          $ref: '#/components/responses/Error'
        5XX:
          $ref: '#/components/responses/Error'
components:
  parameters:
    MaxTextLength:
      name: maxTextLength
      in: query
      required: false
      description: >-
        Cut every text value in the response to this many characters. A value
        that was cut ends with …, and a page whose text was cut has isTruncated
        set to true. A screenshot is never cut. Leave it out to get the full
        text.
      schema:
        type: integer
        minimum: 1
    IdempotencyKey:
      name: Idempotency-Key
      in: header
      required: false
      description: >-
        A value you choose for one call. Send the same key with the same method,
        path and body within 24 hours and the first answer comes back with
        idempotent-replayed: true, with the same cost headers and no second
        charge. A call that sends zeroDataRetention and answers with page data
        is not kept, so a retry runs again. Needs an API key.
      schema:
        type: string
  schemas:
    crawl.input.schema0:
      anyOf:
        - type: string
        - type: number
        - type: boolean
        - type: 'null'
        - type: array
          items:
            $ref: '#/components/schemas/crawl.input.schema0'
        - type: object
          propertyNames:
            type: string
          additionalProperties:
            $ref: '#/components/schemas/crawl.input.schema0'
    crawl.input.schema1:
      anyOf:
        - type: string
        - type: number
        - type: boolean
        - type: 'null'
        - type: array
          items:
            $ref: '#/components/schemas/crawl.input.schema1'
        - type: object
          propertyNames:
            type: string
          additionalProperties:
            $ref: '#/components/schemas/crawl.input.schema1'
    crawl.input.schema2:
      anyOf:
        - type: string
        - type: number
        - type: boolean
        - type: 'null'
        - type: array
          items:
            $ref: '#/components/schemas/crawl.input.schema2'
        - type: object
          propertyNames:
            type: string
          additionalProperties:
            $ref: '#/components/schemas/crawl.input.schema2'
    Error:
      type: object
      required:
        - success
        - error
        - code
      properties:
        success:
          const: false
        error:
          type: string
          description: What went wrong and what to do next.
        code:
          type: string
          enum:
            - apiKeyRequired
            - busy
            - checkRunning
            - cliSessionCompleted
            - cliSessionExists
            - cliSessionNotFound
            - forbidden
            - freeLimit
            - insufficientCredits
            - internalError
            - inviteClaimed
            - inviteDisabled
            - inviteExpired
            - inviteUsedUp
            - invalidCliSession
            - invalidCliVerifier
            - invalidRequest
            - jobFinished
            - methodNotAllowed
            - notFound
            - payloadTooLarge
            - profileBusy
            - profileTooLarge
            - prohibitedPermission
            - promptInjectionDetected
            - rateLimited
            - sourceChanged
            - sourceRefused
            - siteNotRead
            - sourceTimeout
            - sourceUnavailable
            - unauthorized
            - unsupportedMediaType
        retryable:
          type: boolean
          description: true when trying again can help.
        retryAfterSeconds:
          type: integer
        requestId:
          type: string
        requiredCredits:
          type: number
          description: >-
            Older field: the most the call can cost, in $0.002 steps. Sent when
            the balance is too low.
        availableCredits:
          type: number
          description: 'Older field: the balance in $0.002 steps. Sent with requiredCredits.'
        requiredUsd:
          type: number
          description: >-
            The most the call can cost, in dollars. Sent when the balance is too
            low.
        availableUsd:
          type: number
          description: What is left on the account, in dollars. Sent with requiredUsd.
        billingUrl:
          type: string
          description: Where to add money. Sent with requiredUsd.
        signupUrl:
          type: string
          const: https://stophy.dev/signup
          description: >-
            Where a person gets a key. Sent with apiKeyRequired, freeLimit and
            an invalid key.
        signup:
          type: string
          const: POST https://api.stophy.dev/v1/signup
          description: >-
            The call that gets an agent a key with no human. Sent next to
            signupUrl.
  headers:
    RequestId:
      description: The request id. Quote it when you report a problem.
      schema:
        type: string
    CreditsUsed:
      description: >-
        Older header: what this call cost, as whole steps of $0.002. Use
        x-cost-usd.
      schema:
        type: integer
    CostUsd:
      description: What this call cost, in dollars.
      schema:
        type: string
    RateLimitLimit:
      description: Requests allowed in the current window.
      schema:
        type: integer
    RateLimitRemaining:
      description: Requests left in the current window.
      schema:
        type: integer
    RateLimitReset:
      description: Unix time in seconds when the current window resets.
      schema:
        type: integer
    Cache:
      description: hit if this result came from the cache, miss if it was fetched live.
      schema:
        type: string
    CacheAge:
      description: Seconds since the returned result was written to the cache.
      schema:
        type: integer
    IdempotentReplayed:
      description: >-
        true when this call reuses the result of an earlier call with the same
        Idempotency-Key.
      schema:
        type: string
    RetryAfter:
      description: Seconds to wait before retrying.
      schema:
        type: integer
  responses:
    Error:
      description: The request failed. Retry-After says when trying again can help.
      headers:
        X-Request-ID:
          $ref: '#/components/headers/RequestId'
        Retry-After:
          $ref: '#/components/headers/RetryAfter'
      content:
        application/json:
          schema:
            $ref: '#/components/schemas/Error'
  securitySchemes:
    apiKey:
      type: http
      scheme: bearer

````

This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.