# BabelWrap

> The web, as an API, for your agents.

BabelWrap is a REST API and MCP server that lets AI agents interact with any website using natural language. Instead of CSS selectors or XPath, agents describe what they want in plain English ("click the Login button", "fill the email field", "extract all product prices"). BabelWrap translates these into real browser interactions and returns structured, LLM-readable JSON snapshots of every page.

## Who It's For

- AI agent builders (Claude, GPT, LangChain, LlamaIndex, custom agents)
- Backend developers who need web automation without Selenium/Playwright boilerplate
- Data teams extracting structured data from websites

## Core Concepts

### Sessions
A session is an isolated browser context with its own cookies, storage, and page state. Create one with POST /v1/sessions, interact with it, then close it when done. Sessions expire after 1 hour of inactivity.

### Actions
Actions are browser interactions: navigate, click, fill, submit, extract, scroll, etc. Every action is described in natural language and returns a structured snapshot of the page after the action completes.

### Snapshots
The core innovation. Every action returns a JSON snapshot with these fields:
- url: current page URL
- title: document title
- content: main readable text (up to 15,000 characters)
- inputs: all form fields with label, type, value
- actions: all clickable elements (buttons, links)
- forms: logical groupings of fields with submit buttons
- navigation: site navigation links
- alerts: error messages, success banners, warnings
- tables: structured table data with headers and rows
- lists: ordered and unordered list items

### Element Resolution
When you say "click the Login button", BabelWrap uses an LLM (Claude Haiku) to match your description to the correct element on the page. This means your automations survive DOM changes and site redesigns.

## Base URL

https://api.babelwrap.com/v1

## Authentication

All API requests require a Bearer token:
Authorization: Bearer bw_your_api_key_here

Get your API key at https://babelwrap.com/dashboard/signup

## REST API Endpoints

### Session Management
- POST /v1/sessions — Create a browser session (body: {metadata?: object})
- GET /v1/sessions/{id} — Get session info
- DELETE /v1/sessions/{id} — Close and clean up a session

### Browser Actions (all return a snapshot)
- POST /v1/sessions/{id}/navigate — Load a URL (body: {url: string})
- POST /v1/sessions/{id}/snapshot — Read current page state (body: {})
- POST /v1/sessions/{id}/click — Click element (body: {target: string})
- POST /v1/sessions/{id}/fill — Fill form field (body: {target: string, value: string})
- POST /v1/sessions/{id}/submit — Submit form (body: {target?: string})
- POST /v1/sessions/{id}/extract �� Extract data via NL query (body: {query: string})
- POST /v1/sessions/{id}/screenshot — Take base64 PNG screenshot (body: {})
- POST /v1/sessions/{id}/press — Press keyboard key (body: {key: string})
- POST /v1/sessions/{id}/upload — Upload file (body: {target: string, file_base64: string, filename: string})
- POST /v1/sessions/{id}/hover — Hover over element (body: {target: string})
- POST /v1/sessions/{id}/back — Navigate back in history (body: {})
- POST /v1/sessions/{id}/forward — Navigate forward (body: {})
- POST /v1/sessions/{id}/scroll — Scroll page (body: {direction: "up"|"down", amount: "page"|"half"|"pixels"})
- POST /v1/sessions/{id}/wait_for — Wait for condition (body: {text?: string, selector?: string, url_contains?: string, timeout_ms?: int})
- POST /v1/sessions/{id}/batch — Execute multiple actions (body: {actions: [...], continue_on_error: bool})

### Session Utilities
- GET /v1/sessions/{id}/pages — List open tabs/popups
- POST /v1/sessions/{id}/switch_page — Switch to tab (body: {index: int})
- POST /v1/sessions/{id}/cookies — Add cookies (body: {cookies: [...]})
- GET /v1/sessions/{id}/history — Get action history

### Account
- GET /v1/usage — Current month action count, plan limits, estimated cost
- GET /v1/health — Health check (no auth required)

## Site Mapping

BabelWrap can automatically map any website into typed tools. An AI agent (Claude Sonnet 4) explores the target site, discovers its structure (entities, page types, action flows), and generates replayable "recipes" — multi-step action sequences with parameter interpolation.

Recipes are converted into typed MCP tools with domain-prefixed names. For example, mapping linkedin.com/jobs creates tools like `linkedin_search_jobs(query: str, location: str = "")`. Your agent calls one function instead of manually navigating page by page.

### Pipeline
1. **Explore** — AI agent browses ~10-20 pages, discovers entities, pages, and recipes
2. **Validate** — Structural checks + live test execution for safe recipes (search, get, list)
3. **Generate Tools** — Recipes become typed tool definitions with domain prefix
4. **Register & Persist** — Tools registered on MCP server, model saved to PostgreSQL, reloaded on startup

### Recipes
A recipe is a parameterized sequence of browser actions:
- Steps: navigate, fill, click, submit, extract, scroll, conditional
- Parameters: typed (str, int, bool) with defaults; interpolated via ${param_name}
- Self-healing: when a step fails, snapshot → LLM correction → retry → persist fix

### Authentication (4 layers)
- Layer 1: Cookie injection into browser sessions
- Layer 2: Stored credentials for automatic login
- Layer 3: Login redirect detection → auto-run login recipe
- Layer 4: Expired cookie detection → re-authenticate → save fresh cookies

### Site Mapping Endpoints
- POST /v1/sites/map — Map a website (body: {start_url: string, auth_cookies?: array})
  - Returns existing site from catalog (status: "ready") or Stripe checkout URL (status: "pending_payment", amount_cents: 1000)
  - Errors: 402 (upgrade_required), 409 (modeling_in_progress)
- POST /v1/sites/{site_id}/refresh — Re-model existing site ($10 via Stripe checkout)
- GET /v1/sites — List all mapped sites (returns: [{site_id, domain, status, entities, tools_count}])
- GET /v1/sites/orders — List site model orders (returns: {orders: [{order_id, domain, order_type, status, amount_cents, created_at, completed_at}]})
- GET /v1/sites/{site_id} — Get full site details (entities, pages, recipes, tools)
- GET /v1/sites/{site_id}/tools — List generated tools with params and return types
- POST /v1/sites/{site_id}/tools/{tool_name} — Execute a generated tool (body: {params: {key: value}})
  - Returns: {success: bool, data: any, duration_ms: int, steps_executed: int}
- DELETE /v1/sites/{site_id} — Delete a mapped site and its tools

### Site Catalog (public, no auth required)
Every mapped site is added to a public catalog. If a site is already in the catalog, calling POST /v1/sites/map returns it immediately for free.
- GET /v1/catalog — List public site models (query: page, per_page; returns: {total, page, per_page, sites: [{site_id, domain, start_url, recipe_count, version, last_validated_at, created_at}]})
- GET /v1/catalog/search?domain=... — Search by domain (returns: {results: [{site_id, domain, start_url, recipe_count, version}]})
- GET /v1/catalog/{site_id} — Get catalog entry with tools and recipes

### Generated MCP Tools
Mapped sites generate additional MCP tools beyond the 16 built-in tools. These have typed parameters (e.g., `linkedin_search_jobs(query: str, location: str = "")`) and appear in MCP clients (Claude Desktop, Cursor, etc.) alongside the built-in tools. Example tools: linkedin_search_jobs, linkedin_view_profile, github_search_repos, amazon_search_products.

## MCP Server

BabelWrap includes a Model Context Protocol (MCP) server with 16 tools. Works with Claude Desktop, Cursor, Claude Code, and any MCP client.

### Setup (Claude Desktop)
Add to your claude_desktop_config.json:
{
  "mcpServers": {
    "babelwrap": {
      "command": "uvx",
      "args": ["babelwrap-mcp"],
      "env": {
        "BABELWRAP_API_KEY": "bw_your_api_key_here"
      }
    }
  }
}

### MCP Tools
- babelwrap_new_session(metadata?: dict) — Create browser session
- babelwrap_close_session(session_id: str) — Close session
- babelwrap_navigate(session_id: str, url: str) — Navigate to URL
- babelwrap_snapshot(session_id: str) — Read current page state
- babelwrap_click(session_id: str, target: str) — Click element by description
- babelwrap_fill(session_id: str, target: str, value: str) — Fill form field
- babelwrap_submit(session_id: str, target?: str) — Submit form
- babelwrap_extract(session_id: str, query: str) — Extract structured data
- babelwrap_screenshot(session_id: str) — Take screenshot
- babelwrap_press(session_id: str, key: str) — Press keyboard key
- babelwrap_upload(session_id: str, target: str, file_base64: str, filename: str) — Upload file
- babelwrap_hover(session_id: str, target: str) — Hover over element
- babelwrap_back(session_id: str) — Navigate back
- babelwrap_forward(session_id: str) — Navigate forward
- babelwrap_scroll(session_id: str, direction: str, amount: str) — Scroll page
- babelwrap_wait_for(session_id: str, text?: str, selector?: str, url_contains?: str, timeout_ms?: int) — Wait for condition

## Python SDK

Install:
pip install babelwrap

Usage:
```python
from babelwrap import BabelWrap

with BabelWrap(api_key="bw_...") as bw:
    with bw.create_session() as session:
        snap = session.navigate("https://news.ycombinator.com")
        print(snap.title)
        data = session.extract("all story titles and point counts")
        # [{title: "...", points: 312}, ...]
```

Async usage:
```python
from babelwrap import AsyncBabelWrap

async with AsyncBabelWrap(api_key="bw_...") as bw:
    async with await bw.create_session() as session:
        snap = await session.navigate("https://example.com")
        data = await session.extract("all links on the page")
```

## Quick Example (Raw API)

```python
import httpx, os

API_KEY = os.environ["BABELWRAP_API_KEY"]
BASE = "https://api.babelwrap.com/v1"
HEADERS = {"Authorization": f"Bearer {API_KEY}"}

async with httpx.AsyncClient(timeout=30.0) as client:
    # Create session
    sid = (await client.post(f"{BASE}/sessions", headers=HEADERS, json={})).json()["session_id"]

    # Navigate and extract
    await client.post(f"{BASE}/sessions/{sid}/navigate", headers=HEADERS, json={"url": "https://news.ycombinator.com"})
    resp = await client.post(f"{BASE}/sessions/{sid}/extract", headers=HEADERS, json={"query": "all story titles and point counts"})
    stories = resp.json()["data"]  # [{title: "...", points: 312}, ...]

    # Clean up
    await client.delete(f"{BASE}/sessions/{sid}", headers=HEADERS)
```

## Pricing

- Free: $0/month, 500 actions/month, 2 concurrent sessions, all 16 tools, MCP access
- Usage-based: $0.01/action ($10 per 1,000 actions), unlimited actions, 50 concurrent sessions
- Site Mapping: $10 per site (one-time). Pre-mapped sites from the catalog are free to use.

No monthly fee. Pay only for what you use. Usage resets on the 1st of each month.

## Documentation

- Quickstart: https://babelwrap.com/docs/quickstart
- API Reference: https://babelwrap.com/docs/api
- MCP Server Guide: https://babelwrap.com/docs/mcp
- Site Mapping: https://babelwrap.com/docs/site-mapping
- Sites & Catalog API: https://babelwrap.com/docs/sites-api
- Using Mapped Sites: https://babelwrap.com/docs/guides/mapped-sites
- Snapshot Format: https://babelwrap.com/docs/snapshot
- Guides: https://babelwrap.com/docs/guides
- Code Examples: https://babelwrap.com/examples
- Pricing: https://babelwrap.com/pricing
- Dashboard: https://babelwrap.com/dashboard