> ## Documentation Index
> Fetch the complete documentation index at: https://help.decodo.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Python SDK

> The official Python SDK for the Decodo Web Data API, with typed targets, IDE autocomplete, and sync, async, and batch scraping methods.

> [!NOTE]
>
> View the [Python SDK GitHub repository](https://github.com/Decodo/sdk-python) for the latest installation instructions, examples, and source code.

Build typed scraping workflows for search engines, eCommerce platforms, social media, AI tools, and more with the official Decodo Python SDK.

* Fully typed targets and parameters with IDE autocomplete
* Sync, async, and batch scraping methods
* Web search, site mapping, and crawling through the Web Data API
* Built on `httpx`; requires Python 3.12 or later
* Typed error hierarchy for safer integrations

## What is the Decodo Python SDK?

The Decodo Python SDK provides a typed interface for interacting with Decodo targets like Google, Amazon, TikTok, YouTube, ChatGPT, and more. Instead of manually constructing HTTP requests and validating payloads, you work with fully typed methods and target-specific parameter classes directly in your editor.

## Why use the SDK?

* **Typing and autocomplete** – target parameters are fully typed for better editor support and fewer mistakes.
* **Unified scraping interface** – work with search engines, eCommerce platforms, social media, and AI tools through one SDK.
* **Async and batch workflows** – create scraping tasks, poll task status, and process batches at scale.
* **Typed errors** – handle authentication, validation, timeout, and rate-limit failures safely.
* **Minimal setup** – single dependency on `httpx`, no additional HTTP client required.

## Requirements

* Python 3.12+

## Installation

```bash theme={null}
pip install decodo-sdk
```

## Quick start

Create a new project:

```bash theme={null}
mkdir scrape-with-decodo
cd scrape-with-decodo
pip install decodo-sdk
touch main.py
```

Copy your Web Data API key from your Web Data API subscription on the [**Decodo dashboard**](https://dashboard.decodo.com/web-data/playground). Keep it out of your source by storing it in an environment variable:

```bash theme={null}
export DECODO_API_KEY=your_api_key
```

> [!NOTE]
>
> Older plans only have a basic authentication token. Pass it as `token` instead of `api_key` – see [Configuration](#configuration).

Every target has a corresponding parameter class. Import the one you need, fill in its fields, and pass it to `scrape()`:

```python theme={null}
# main.py
import os

from decodo import (
    DecodoClient,
    DecodoConfig,
    GoogleSearchParams,
    WebScrapingApiConfig,
)

client = DecodoClient(
    DecodoConfig(
        web_scraping_api=WebScrapingApiConfig(
            api_key=os.environ["DECODO_API_KEY"],
        ),
    )
)

result = client.web_scraping_api.scrape(
    GoogleSearchParams(
        query="coffee shops",
        geo="United States",
        parse=True,
    )
)

print(result)
```

Run the script:

```bash theme={null}
python main.py
```

Parameter classes are bundled with the package, so no extra step is needed after `pip install decodo-sdk`.

Each class pins its own target and accepts only the fields that target supports - your IDE flags misspelled or unsupported fields, and runtime validation catches them before the request is sent.

### Alternative: dictionary payloads

Scraping methods also accept a plain Python dictionary, validated against the bundled schema before the request is sent. Nothing is checked while you write the code, so typos surface only at runtime:

```python theme={null}
result = client.web_scraping_api.scrape({
    "target": "google_search",
    "query": "coffee shops",
    "geo": "United States",
    "parse": True,
})
print(result)
```

Use the `Target` enum instead of a raw string to avoid mistyping the target name. Typed parameter classes are recommended for anything beyond a quick experiment.

### Updating types to a newer schema

The types bundled in the package reflect the schema at release time. To update them without waiting for a new release, run the type generator:

```bash theme={null}
python -m decodo.codegen.codegen --out-dir ./decodo_generated
```

Then import from that directory instead:

```python theme={null}
from decodo_generated.targets import GoogleSearchParams
```

The directory name passed to `--out-dir` becomes the import namespace, so `./decodo_generated` gives you `from decodo_generated.targets import ...`. Any name works, but keep it consistent across your project.

## Configuration

```python theme={null}
from decodo import (
    DecodoClient,
    DecodoConfig,
    WebScrapingApiConfig,
)

client = DecodoClient(
    DecodoConfig(
        web_scraping_api=WebScrapingApiConfig(
            api_key="<api_key>",
        ),
        timeout_ms=120_000,  # optional, request timeout in ms (default: 180000)
    )
)
```

Pass either `api_key` or `token`, never both:

```python theme={null}
# Basic authentication token (older plans)
client = DecodoClient(
    DecodoConfig(
        web_scraping_api=WebScrapingApiConfig(
            token="<basic_auth_token>",
        ),
    )
)
```

| Parameter | Description |
| - | - |
| `api_key` | Web Data API key from your subscription on the dashboard. **Recommended** |
| `token` | Basic authentication token for older plans - the base64-encoded `user:password` string from the dashboard. Use it instead of `api_key` |
| `timeout_ms` | Request timeout in milliseconds (default: 180000) |

> [!NOTE]
>
> Setting both raises `ValueError: web_scraping_api accepts either token or api_key, not both. Provide only one.` Setting neither raises `ValueError: web_scraping_api requires api_key or token in DecodoConfig.`
>
> Which credential you pass also decides which API the SDK calls. An `api_key` uses the Web Data API at `data.decodo.com`, which is the only way to reach [search, map, and crawl](#web-data-api-search-map-and-crawl). A `token` uses the older Web Scraping API at `scraper-api.decodo.com`, which supports scraping only.

## Scraping

Access the API via `client.web_scraping_api`.

### Sync scrape

Waits for the scraping result before returning:

```python theme={null}
from decodo import AmazonProductParams

result = client.web_scraping_api.scrape(
    AmazonProductParams(
        query="B09H74FXNW",
        parse=True,
    )
)
print(result)
```

### Async scrape

Creates a scraping task and returns immediately. You then poll for its status and results:

```python theme={null}
import time
from decodo import GoogleSearchParams

task = client.web_scraping_api.scrape_async(
    GoogleSearchParams(query="laptop reviews", parse=True)
)
task_id = task["id"]

while True:
    status = client.web_scraping_api.get_status(task_id)["status"]
    if status == "done":
        break
    if status == "faulted":
        raise RuntimeError(f"Scraping task {task_id} failed.")
    time.sleep(2)

result = client.web_scraping_api.get_results(task_id)
print(result)
```

`get_status` returns `pending`, `done`, or `faulted`.

### Batch scrape

Send multiple inputs in a single request using the batch variant of a target's parameter class. The primary input accepts a list:

```python theme={null}
import time
from decodo import GoogleSearchBatchParams

batch = client.web_scraping_api.scrape_batch(
    GoogleSearchBatchParams(query=["coffee", "tea", "juice"], parse=True)
)

for task in batch["queries"]:
    task_id = task["id"]
    while True:
        status = client.web_scraping_api.get_status(task_id)["status"]
        if status == "done":
            break
        if status == "faulted":
            raise RuntimeError(f"Scraping task {task_id} failed.")
        time.sleep(2)
    result = client.web_scraping_api.get_results(task_id)
    print(result)
```

## Web Data API: search, map, and crawl

These methods are served by the Web Data API at `data.decodo.com` and **require an `api_key`**. Calling them on a client configured with a basic authentication `token` raises `ValueError`. They live on the same `client.web_scraping_api` namespace.

Parameters are plain dictionaries. `SearchRequest`, `MapRequest`, and `CrawlRequest` are `TypedDict`s you can use for editor completion - they are always importable and need no code generation. Bodies are validated against the bundled schema before the request is sent, just like `scrape`.

### Search

Runs a web search and returns the parsed result document:

```python theme={null}
from decodo import SearchRequest

result = client.web_scraping_api.search(
    SearchRequest(query="coffee shops", geo="United States", page=1)
)

for item in result.get("organic", []):
    print(item["rank"], item["title"], item["link"])
```

> [!NOTE]
>
> A search that ran but could not be scraped still answers with HTTP 200 and `status: "failed"` plus a `message`. Check `result["status"]` before reading the other fields.

### Map

Discovers the links on a site, from its pages and its sitemap:

```python theme={null}
from decodo import MapRequest

result = client.web_scraping_api.map(
    MapRequest(url="https://www.example.com", limit=100, sitemap="include")
)

for link in result["results"][0]["content"]["links"]:
    print(link["url"])
```

### Crawl

`crawl()` starts a crawl, waits for it to finish, and returns every page with its content:

```python theme={null}
from decodo import (
    CrawlError,
    CrawlRequest,
    CrawlTimeoutError,
)

try:
    pages = client.web_scraping_api.crawl(
        CrawlRequest(url="https://www.example.com", limit=20, max_depth=2),
        poll_interval=5,
        timeout=600,
    )
except CrawlTimeoutError as err:
    print(f"Still running, fetch it later with get_crawl_results({err.crawl_id!r})")
except CrawlError as err:
    print(f"Crawl {err.crawl_id} ended {err.status}: {err.reason}")
else:
    for page in pages:
        print(page["url"], page["status"], page.get("content", "")[:80])
```

A crawl that stops early because of `max_duration` or a vendor limit ends with status `truncated`, and `crawl()` returns the pages it did collect. Each page carries its own `status` (`pending`, `done`, or `faulted`) and, when the fetch failed, a `content_error` instead of `content`.

The lower-level methods let you drive a crawl yourself - for example, start it in one process and collect it in another:

```python theme={null}
started = client.web_scraping_api.start_crawl(
    CrawlRequest(url="https://www.example.com")
)
crawl_id = started["crawl_id"]

status = client.web_scraping_api.get_crawl_status(crawl_id)
print(status["status"], status["completed"], status["in_flight"])

pages = client.web_scraping_api.get_crawl_results(crawl_id)

client.web_scraping_api.cancel_crawl(crawl_id)
```

`get_crawl_status` returns `pending`, `done`, `faulted`, `cancelled`, or `truncated`, with a `reason` when the crawl stopped early.

| Method | Description |
| - | - |
| `search(params)` | Run a web search and return parsed results |
| `map(params)` | List the links on a site without fetching their content |
| `crawl(params, *, poll_interval=5, timeout=None)` | Start a crawl, wait for it, and return every page |
| `start_crawl(params)` | Start a crawl and return its `crawl_id` immediately |
| `get_crawl_status(crawl_id)` | Report progress and the current status |
| `get_crawl_results(crawl_id, *, content=True)` | Fetch the crawled pages; pass `content=False` for metadata only |
| `cancel_crawl(crawl_id)` | Stop a running crawl; pages collected so far stay readable |

## Targets and parameter classes

Every target in the `Target` enum has a matching parameter class. The enum member name is followed by `Params` – for example, `Target.GoogleSearch` → `GoogleSearchParams` and `Target.Chatgpt` → `ChatgptParams`. For batch calls, insert `Batch` before `Params`, as in `GoogleSearchBatchParams`.

You don't need to pass a target explicitly because each parameter class already pins its own target. For example, `GoogleSearchParams(query="coffee shops")` is sufficient on its own. The `Target` enum remains useful for dictionary payloads and for reading the `target` field from a response.

> [!NOTE]
>
> One target breaks the naming rule: `Target.Target` maps to `TargetStoreParams` (not `TargetParams`), to avoid a name clash. Every other target follows the pattern.

Each target accepts one primary input parameter (`url`, `query`, `product_id`, or `prompt`) together with optional configuration.

### Search engines

| Target | Parameter class | Description | Example input |
| - | - | - | - |
| `Target.GoogleSearch` | `GoogleSearchParams` | Google Search results for a query | `query="coffee shops"` |
| `Target.GoogleLens` | `GoogleLensParams` | Google Lens reverse image search | `query="https://example.com/cat.jpg"` |
| `Target.GoogleTravelHotels` | `GoogleTravelHotelsParams` | Google Travel hotel listings | `query="hotels in paris"` |
| `Target.GoogleAds` | `GoogleAdsParams` | Google Ads results for a query | `query="laptop"` |
| `Target.BingSearch` | `BingSearchParams` | Bing Search results | `query="electric vehicles"` |
| `Target.Bing` | `BingParams` | Raw Bing URL scraping | `url="https://bing.com/search?q=laptop"` |

### eCommerce

| Target | Parameter class | Description | Example input |
| - | - | - | - |
| `Target.AmazonProduct` | `AmazonProductParams` | Amazon product detail page by ASIN | `query="B09H74FXNW"` |
| `Target.AmazonSearch` | `AmazonSearchParams` | Amazon search results | `query="laptop"` |
| `Target.AmazonPricing` | `AmazonPricingParams` | Amazon pricing and offers | `query="B09H74FXNW"` |
| `Target.AmazonSellers` | `AmazonSellersParams` | Amazon seller listings | `query="B09H74FXNW"` |
| `Target.AmazonBestsellers` | `AmazonBestsellersParams` | Amazon bestsellers by category | `query="electronics"` |
| `Target.WalmartProduct` | `WalmartProductParams` | Walmart product page by product ID | `product_id="15296401808"` |
| `Target.WalmartSearch` | `WalmartSearchParams` | Walmart search results | `query="laptop"` |
| `Target.Walmart` | `WalmartParams` | Raw Walmart URL scraping | `url="https://walmart.com/ip/15296401808"` |
| `Target.TargetProduct` | `TargetProductParams` | Target.com product page by product ID | `product_id="92186007"` |
| `Target.TargetSearch` | `TargetSearchParams` | Target.com search results | `query="laptop"` |
| `Target.Target` | `TargetStoreParams` | Raw Target.com URL scraping | `url="https://target.com/p/-/A-92186007"` |

### Social media

| Target | Parameter class | Description | Example input |
| - | - | - | - |
| `Target.YoutubeSearch` | `YoutubeSearchParams` | YouTube search results | `query="ambient music"` |
| `Target.YoutubeMetadata` | `YoutubeMetadataParams` | YouTube video metadata by ID | `query="dFu9aKJoqGg"` |
| `Target.YoutubeSubtitles` | `YoutubeSubtitlesParams` | YouTube video subtitles by ID | `query="dFu9aKJoqGg"` |
| `Target.YoutubeChannel` | `YoutubeChannelParams` | YouTube channel by handle or ID | `query="@decodo_official"` |
| `Target.TiktokPost` | `TiktokPostParams` | TikTok post by URL | `url="https://www.tiktok.com/@nba/video/..."` |
| `Target.TiktokShopSearch` | `TiktokShopSearchParams` | TikTok Shop search results | `query="wireless earbuds"` |
| `Target.TiktokShopProduct` | `TiktokShopProductParams` | TikTok Shop product page by product ID | `product_id="7100000000000000000"` |
| `Target.Tiktok` | `TiktokParams` | Raw TikTok URL scraping | `url="https://www.tiktok.com/@nba"` |

### AI tools

| Target | Parameter class | Description | Example input |
| - | - | - | - |
| `Target.Chatgpt` | `ChatgptParams` | ChatGPT response for a prompt | `prompt="What are the top three dog breeds?"` |
| `Target.Gemini` | `GeminiParams` | Gemini response for a prompt | `prompt="What are the top three dog breeds?"` |
| `Target.GoogleAiMode` | `GoogleAiModeParams` | Google AI Mode response | `query="What are the top three dog breeds?"` |

### Other

| Target | Parameter class | Description | Example input |
| - | - | - | - |
| `Target.Bbb` | `BbbParams` | Better Business Bureau listing by URL | `url="https://bbb.org/us/ny/new-york/..."` |
| `Target.Autotrader` | `AutotraderParams` | Autotrader listing by URL | `url="https://autotrader.com/cars-for-sale/..."` |
| `Target.Mobile` | `MobileParams` | Mobile.de listing by URL | `url="https://mobile.de/auto/..."` |
| `Target.Airbnb` | `AirbnbParams` | Airbnb listing by URL | `url="https://airbnb.com/rooms/12345"` |
| `Target.AppleAppStore` | `AppleAppStoreParams` | Apple App Store app by URL | `url="https://apps.apple.com/app/id12345"` |

### Universal scraping

| Target | Parameter class | Description | Example input |
| - | - | - | - |
| `Target.Universal` | `UniversalParams` | Any URL via the universal scraper | `url="https://example.com"` |
| `Target.Google` | `GoogleParams` | Raw Google URL scraping | `url="https://google.com/search?q=laptop"` |
| `Target.Amazon` | `AmazonParams` | Raw Amazon URL scraping | `url="https://amazon.com/dp/B09H74FXNW"` |

For the full target list and parameter details, see [**Target list**](https://help.decodo.com/docs/web-scraping-api-targets.md) and [**Parameters**](https://help.decodo.com/docs/web-scraping-api-parameters-overview.md).

## Error handling

The SDK raises typed errors that map to API error codes:

```python theme={null}
import time
from decodo import (
    AuthenticationError,
    DecodoError,
    GoogleSearchParams,
    RateLimitError,
    TimeoutError,
    ValidationError,
)

params = GoogleSearchParams(query="coffee shops", geo="United States", parse=True)

try:
    response = client.web_scraping_api.scrape(params)
except AuthenticationError:
    raise SystemExit("Invalid API key or token. Check your credentials in the dashboard.")
except RateLimitError:
    time.sleep(5)
    response = client.web_scraping_api.scrape(params)
except ValidationError as err:
    print("Payload rejected:", err.errors or err)
except TimeoutError:
    print("Timed out. Increase timeout_ms or switch to scrape_async for slow targets.")
except DecodoError as err:
    print(f"Request failed with {err.status_code}: {err}")
```

| Error | Raised when | Useful attributes |
| - | - | - |
| `AuthenticationError` | The API returns `401` or `403` | `status_code` |
| `RateLimitError` | The API returns `429` | `status_code` |
| `ValidationError` | The payload fails local schema validation, or the API returns `422` | `errors`, `status_code` |
| `TimeoutError` | The request exceeds `timeout_ms` | none |
| `CrawlTimeoutError` | `crawl()` exceeds its `timeout`. Subclass of `TimeoutError`; the crawl keeps running | `crawl_id` |
| `CrawlError` | `crawl()` ends `faulted` or `cancelled`. Subclass of `DecodoError` | `crawl_id`, `status`, `reason` |
| `CrawlInterruptedError` | A status or results request fails after the crawl was started. Subclass of `DecodoError` | `crawl_id` |
| `DecodoError` | Any other unsuccessful response - base class for the errors above | `status_code`, `api_status` |

> [!NOTE]
>
> `TimeoutError` sits outside the `DecodoError` hierarchy, so `except DecodoError` won't catch it. Handle it separately. Importing it from `decodo` also shadows the built-in `TimeoutError` in that module.
>
> Typed parameters can also fail before a request is sent. An unknown field or incorrect type raises `pydantic.ValidationError` when the parameter object is created.

***

## Related

* [**CLI**](https://help.decodo.com/docs/cli.md) – Run Decodo scraping workflows from the command line.
* [**TypeScript SDK**](https://help.decodo.com/docs/typescript-sdk.md) – Integrate the Decodo APIs into TypeScript and JavaScript projects.
* [**Go SDK**](https://help.decodo.com/docs/go-sdk.md) – Integrate the Decodo APIs into Go projects.
* [**MCP Server**](https://help.decodo.com/docs/mcp.md) – Connect AI agents and MCP-compatible tools to Decodo.
* [GitHub repository ](https://github.com/Decodo/sdk-python)– View the Python SDK source code, examples, and latest releases.

***

> [card] [Support](https://direct.lc.chat/12092754)
>
> Need help or just want to say hello? Our support is available 24/7. \
> You can also reach us anytime via email at [support@decodo.com](mailto:support@decodo.com).

> [card] [Feedback](mailto:feedback@decodo.com)
>
> Can't find what you're looking for? Request an article! \
> Have feedback? Share your thoughts on how we can improve.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.