> ## Documentation Index
> Fetch the complete documentation index at: https://docs.scrapegraphai.com/llms.txt
> Use this file to discover all available pages before exploring further.

# JavaScript SDK

> Official JavaScript/TypeScript SDK for ScrapeGraphAI v2

> [card] [NPM Package](https://www.npmjs.com/package/scrapegraph-js)
>
> [![npm version](https://badge.fury.io/js/scrapegraph-js.svg)](https://badge.fury.io/js/scrapegraph-js)

> [card] [Source on GitHub](https://github.com/ScrapeGraphAI/scrapegraph-js)
>
> Issues, PRs, and the changelog

> [card] [License](https://opensource.org/licenses/MIT)
>
> [![License](https://img.shields.io/badge/License-MIT-blue.svg)](https://opensource.org/licenses/MIT)

> [!NOTE]
>
> These docs cover **`scrapegraph-js` ≥ 2.1.0**. The v2 SDK is **ESM-only** and requires **Node ≥ 22**. Earlier `0.x`/`1.x` releases expose a different, deprecated API.

> [!WARNING]
>
> **Breaking in 2.1.0 (types only):** all exported TypeScript types and Zod schemas dropped the `Api` prefix and now match `scrapegraph-py` 1:1 (`ApiScrapeRequest` → `ScrapeRequest`, `ApiFetchConfig` → `FetchConfig`, `apiScrapeRequestSchema` → `scrapeRequestSchema`, etc.). Monitor input types are also renamed: `ApiMonitorCreateInput` → `MonitorCreateRequest`, `ApiMonitorUpdateInput` → `MonitorUpdateRequest`, `ApiMonitorActivityParams` → `MonitorActivityRequest`. `ApiResult<T>` is the only type that keeps the prefix. Runtime JS code is unchanged — only TypeScript consumers need to rename imports.

## Installation

```bash theme={null}
# npm
npm i scrapegraph-js@latest     # pins a version >= 2.1.0

# pnpm
pnpm add scrapegraph-js@latest

# yarn
yarn add scrapegraph-js@latest

# bun
bun add scrapegraph-js@latest
```

## What's new in v2

* **New entry point**: `import { ScrapeGraphAI } from "scrapegraph-js"` and instantiate once — no more passing the API key to every call.
* **Nested resources**: `sgai.crawl.*`, `sgai.monitor.*`, `sgai.history.*`.
* **`ApiResult<T>` wrapper**: no throws — every call returns `{ status, data, error, elapsedMs }`.
* **Auto-picks the API key** from `SGAI_API_KEY` (or pass `{ apiKey }` to the factory).
* **Removed**: `markdownify`, `agenticScraper`, `sitemap`, `feedback` — use `sgai.scrape()` with the right format entry instead.

> [!WARNING]
>
> v2 is a breaking change. See the [Migration Guide](https://docs.scrapegraphai.com/transition-from-v1-to-v2) if you're upgrading from v1.

## Quick Start

```javascript theme={null}
import { ScrapeGraphAI } from "scrapegraph-js";

// reads SGAI_API_KEY from env, or pass explicitly: ScrapeGraphAI({ apiKey: "..." })
const sgai = ScrapeGraphAI();

const res = await sgai.scrape({
  url: "https://example.com",
  formats: [{ type: "markdown" }],
});

if (res.status === "success") {
  console.log(res.data?.results.markdown?.data?.[0]);
} else {
  console.error(res.error);
}
```

> [!NOTE]
>
> Store your API keys securely in environment variables. Use `.env` files and libraries like `dotenv` to load them into your app.

## Return Type

Every method returns `ApiResult<T>`:

```typescript theme={null}
type ApiResult<T> = {
  status: "success" | "error";
  data: T | null;
  error?: string;
  elapsedMs: number;
};
```

Check `res.status` before accessing `res.data`.

## Services

### `sgai.scrape()`

Fetch a page in one or more formats (markdown, html, screenshot, json, links, images, summary, branding).

```javascript theme={null}
const res = await sgai.scrape({
  url: "https://example.com",
  formats: [
    { type: "markdown", mode: "reader" },
    { type: "screenshot", fullPage: true, width: 1440, height: 900 },
    { type: "json", prompt: "Extract product info" },
  ],
  contentType: "text/html",      // optional, auto-detected
  fetchConfig: {                 // optional
    mode: "js",
    stealth: true,
    timeout: 30000,
    wait: 2000,
    scrolls: 3,
  },
});
```

#### Parameters

| Parameter | Type | Required | Description |
| - | - | - | - |
| `url` | `string` | Yes | URL to scrape |
| `formats` | `FormatConfig[]` | No | Defaults to `[{ type: "markdown" }]` |
| `contentType` | `string` | No | Override detected content type (e.g. `"application/pdf"`) |
| `fetchConfig` | `FetchConfig` | No | Fetch configuration |

**Formats:**

* `markdown` — Clean markdown (modes: `normal`, `reader`, `prune`)
* `html` — Raw HTML (modes: `normal`, `reader`, `prune`)
* `links` — All links on the page
* `images` — All image URLs
* `summary` — AI-generated summary
* `json` — Structured extraction with prompt/schema
* `branding` — Brand colors, typography, logos
* `screenshot` — Page screenshot (`fullPage`, `width`, `height`, `quality`)

> [accordion] Multi-format example
>
> ```javascript theme={null}
> const res = await sgai.scrape({
>   url: "https://example.com",
>   formats: [
>     { type: "markdown", mode: "reader" },
>     { type: "links" },
>     { type: "images" },
>     { type: "screenshot", fullPage: false, width: 1440, height: 900, quality: 90 },
>   ],
> });
>
> if (res.status === "success") {
>   const r = res.data?.results;
>   console.log("Markdown:", r?.markdown?.data?.[0]?.slice(0, 200));
>   console.log("Links:", r?.links?.metadata?.count);
>   console.log("Screenshot URL:", r?.screenshot?.data.url);
> }
> ```

### `sgai.extract()`

Extract structured data from a URL, HTML, or markdown.

```javascript theme={null}
const res = await sgai.extract({
  url: "https://example.com",
  prompt: "Extract the main heading and description",
});

if (res.status === "success") {
  console.log(res.data?.json);
  console.log("Tokens:", res.data?.usage);
}
```

#### Parameters

| Parameter | Type | Required | Description |
| - | - | - | - |
| `url` | `string` | Yes\* | URL of the page |
| `html` | `string` | Yes\* | Raw HTML (alternative to `url`) |
| `markdown` | `string` | Yes\* | Raw markdown (alternative to `url`) |
| `prompt` | `string` | Yes | What to extract |
| `schema` | `object` | No | JSON schema for structured output |
| `mode` | `string` | No | HTML processing mode: `"normal"`, `"reader"`, `"prune"` |
| `contentType` | `string` | No | Override the detected content type |
| `fetchConfig` | `FetchConfig` | No | Fetch configuration |

> [!NOTE]
>
> \*One of `url`, `html`, or `markdown` is required.

> [accordion] With a JSON schema
>
> ```javascript theme={null}
> const res = await sgai.extract({
>   url: "https://example.com/article",
>   prompt: "Extract the article information",
>   schema: {
>     type: "object",
>     properties: {
>       title: { type: "string" },
>       author: { type: "string" },
>       publishDate: { type: "string" },
>       content: { type: "string" },
>     },
>     required: ["title"],
>   },
> });
>
> if (res.status === "success") {
>   console.log(res.data?.json);
> }
> ```

### `sgai.search()`

Web search with optional AI extraction.

```javascript theme={null}
const res = await sgai.search({
  query: "best programming languages 2024",
  numResults: 5,
});

if (res.status === "success") {
  for (const r of res.data?.results ?? []) {
    console.log(`${r.title} - ${r.url}`);
  }
}
```

#### Parameters

| Parameter | Type | Required | Description |
| - | - | - | - |
| `query` | `string` | Yes | Search query (1–500 chars) |
| `numResults` | `number` | No | Number of results (1–20). Default: `3` |
| `prompt` | `string` | No | Prompt for AI extraction from the fetched results |
| `schema` | `object` | No | JSON schema (requires `prompt`) |
| `format` | `string` | No | `"markdown"` (default) or `"html"` |
| `timeRange` | `string` | No | `"past_hour"`, `"past_24_hours"`, `"past_week"`, `"past_month"`, `"past_year"` |
| `locationGeoCode` | `string` | No | Two-letter country code (e.g. `"us"`) |
| `allowedTypes` | `string[]` | No | Non-empty MIME allowlist. Omit it for all supported types; `"all"` and `"*"` are not accepted |
| `processors` | `object[]` | No | Omit for the 25-page PDF cap. `{ type: "pdf" }` also defaults to 25; set `maxPages` only to override it, or use `-1` for unlimited. PDF processing costs 1 credit per page actually processed. [Configuration examples](https://docs.scrapegraphai.com/services/scrape.md#configure-pdf-page-limits) |
| `fetchConfig` | `FetchConfig` | No | Fetch configuration |

> [accordion] Search + extraction
>
> ```javascript theme={null}
> const res = await sgai.search({
>   query: "typescript best practices",
>   numResults: 5,
>   prompt: "Extract the main tips and recommendations",
>   schema: {
>     type: "object",
>     properties: {
>       tips: { type: "array", items: { type: "string" } },
>     },
>   },
> });
>
> if (res.status === "success") {
>   console.log("Results:", res.data?.results.length);
>   console.log("Extracted:", res.data?.json);
> }
> ```

### `sgai.crawl.*`

Crawl a site. Access the resource via `sgai.crawl`.

```javascript theme={null}
const start = await sgai.crawl.start({
  url: "https://example.com",
  formats: [{ type: "markdown" }],
  maxPages: 50,
  maxDepth: 2,
  maxLinksPerPage: 10,
  includePatterns: ["/blog/*"],
  excludePatterns: ["/admin/*"],
});

const crawlId = start.data?.id;

// Status
await sgai.crawl.get(crawlId);

// Control
await sgai.crawl.stop(crawlId);
await sgai.crawl.resume(crawlId);
await sgai.crawl.delete(crawlId);
```

#### `crawl.start()` parameters

| Parameter | Type | Required | Description |
| - | - | - | - |
| `url` | `string` | Yes | Starting URL |
| `formats` | `FormatConfig[]` | No | Defaults to `[{ type: "markdown" }]` |
| `maxDepth` | `number` | No | Maximum crawl depth. Default: `2` |
| `maxPages` | `number` | No | Maximum pages (1–1000). Default: `50` |
| `maxLinksPerPage` | `number` | No | Links followed per page. Default: `10` |
| `allowExternal` | `boolean` | No | Allow crossing domains. Default: `false` |
| `includePatterns` | `string[]` | No | URL patterns to include |
| `excludePatterns` | `string[]` | No | URL patterns to exclude |
| `allowedTypes` | `string[]` | No | Non-empty MIME allowlist; omit it to allow every supported type |
| `processors` | `object[]` | No | Omit for the 25-page PDF cap. `{ type: "pdf" }` also defaults to 25; set `maxPages` only to override it, or use `-1` for unlimited. PDF processing costs 1 credit per page actually processed. [Configuration examples](https://docs.scrapegraphai.com/services/scrape.md#configure-pdf-page-limits) |
| `fetchConfig` | `FetchConfig` | No | Fetch configuration |

### `sgai.monitor.*`

Scheduled monitoring jobs.

```javascript theme={null}
// Create
const res = await sgai.monitor.create({
  url: "https://example.com",
  name: "Price Monitor",
  interval: "0 * * * *",       // cron expression
  formats: [{ type: "markdown" }],
  webhookUrl: "https://...",   // optional
});

const cronId = res.data?.cronId;

// Manage
await sgai.monitor.list();
await sgai.monitor.get(cronId);
await sgai.monitor.update(cronId, { interval: "0 */6 * * *" });
await sgai.monitor.pause(cronId);
await sgai.monitor.resume(cronId);
await sgai.monitor.delete(cronId);
```

#### `monitor.activity()` — poll tick history

Paginate through per-run ticks.

```javascript theme={null}
const activity = await sgai.monitor.activity(cronId, { limit: 20 });

if (activity.status === "success") {
  for (const tick of activity.data?.ticks ?? []) {
    const changed = tick.changed ? "CHANGED" : "no change";
    console.log(`[${tick.createdAt}] ${tick.status} - ${changed} (${tick.elapsedMs}ms)`);
  }

  if (activity.data?.nextCursor) {
    const next = await sgai.monitor.activity(cronId, {
      limit: 20,
      cursor: activity.data.nextCursor,
    });
  }
}
```

Params: `limit` (1–100, default `20`) and `cursor` for pagination. Each tick exposes `id`, `createdAt`, `status`, `changed`, `elapsedMs`, and `diffs`.

### `sgai.history.*`

```javascript theme={null}
const list = await sgai.history.list({
  service: "scrape",   // optional filter
  page: 1,
  limit: 20,
});

const entry = await sgai.history.get("request-id");
```

### `sgai.credits()` / `sgai.healthy()`

```javascript theme={null}
const credits = await sgai.credits();
// { remaining: 1000, used: 500, plan: "pro", jobs: { crawl: {...}, monitor: {...} } }

const health = await sgai.healthy();
// { status: "ok", uptime: 12345 }
```

## Configuration Objects

### FetchConfig

Controls how pages are fetched. See the [proxy configuration guide](https://docs.scrapegraphai.com/services/additional-parameters/proxy) for details.

```javascript theme={null}
{
  mode: "js",          // "auto" (default) | "fast" | "js"
  stealth: true,        // Residential proxies / anti-bot headers
  timeout: 15000,       // ms (1000–60000)
  wait: 2000,           // ms after page load (0–30000)
  scrolls: 3,           // 0–100
  country: "us",        // ISO 3166-1 alpha-2
  headers: { "X-Custom": "header" },
  cookies: { key: "value" },
  mock: false,          // Enable mock mode for testing
}
```

## Error Handling

```javascript theme={null}
const res = await sgai.extract({
  url: "https://example.com",
  prompt: "Extract the title",
});

if (res.status === "success") {
  console.log(res.data);
} else {
  console.error(`Request failed: ${res.error}`);
}
```

## Environment Variables

| Variable | Description | Default |
| - | - | - |
| `SGAI_API_KEY` | Your ScrapeGraphAI API key | — |
| `SGAI_API_URL` | Override API base URL | `https://v2-api.scrapegraphai.com/api` |
| `SGAI_DEBUG` | Enable debug logging (`"1"`) | off |
| `SGAI_TIMEOUT` | Request timeout in seconds | `120` |

## Support

> [card] [GitHub](https://github.com/ScrapeGraphAI/scrapegraph-js)
>
> Report issues and contribute to the SDK

> [card] [Email Support](mailto:support@scrapegraphai.com)
>
> Get help from our development team


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.