> ## Documentation Index
> Fetch the complete documentation index at: https://help.decodo.com/llms.txt
> Use this file to discover all available pages before exploring further.

# LangChain (Python)

> The langchain-decodo package: Decodo scrape and search tools for LangChain agents, plus a document loader for RAG pipelines.

`langchain-decodo` is the Python LangChain package for the Decodo Web Data API. It gives LangChain agents two tools, one that scrapes any URL and one that searches Google or Amazon, and adds a document loader that turns web pages into LangChain `Document` objects for RAG pipelines.

> [!NOTE]
>
> View the [**langchain repository**](https://github.com/Decodo/langchain/tree/master/libs/partners/decodo) for the source code and latest releases. Building in Node.js? See [**LangChain (TypeScript)**](https://help.decodo.com/docs/langchain.md).

## Features

* **Web scraping** – scrape any URL and get the page back as Markdown
* **Search** – Google Search and Amazon Search with parsed, structured results
* **Document loader** – load one or more URLs as LangChain `Document` objects for RAG
* **LangChain integration** – standard `BaseTool` and `BaseLoader` classes that plug into any agent or chain

## Requirements

* Python 3.10+
* A Decodo [**Web Data API plan**](https://dashboard.decodo.com/web-data/pricing). Free trials are available on the [**dashboard**](https://dashboard.decodo.com/).

## Installation

```bash theme={null}
pip install langchain-decodo
```

## Authentication

Copy your Web Data API key from the [**Web Data API playground**](https://dashboard.decodo.com/web-data/playground) and store it in the `DECODO_API_TOKEN` environment variable. Every class reads it from there:

```bash theme={null}
export DECODO_API_TOKEN=your_api_key
```

Then pass `auth_mode="token"` to each class you create:

```python theme={null}
from langchain_decodo import DecodoWebScrapeTool

tool = DecodoWebScrapeTool(auth_mode="token")
```

> [!WARNING]
>
> `auth_mode` defaults to `"basic"`, not `"token"`. If you leave it out, the API key is sent as a basic authentication token and every request fails with `Decodo API error: Username invalid.`

Older plans only have a basic authentication token (a base64-encoded `username:password`). Put that in `DECODO_API_TOKEN` and leave `auth_mode` out, since `"basic"` is the default.

You can also pass the credential directly instead of using the environment variable: `decodo_api_token=` on the two tools, `api_token=` on the loader.

## Quick start

A LangChain agent that searches Google, picks a page and scrapes it. This example uses OpenAI through `langchain-openai`, but any chat model LangChain supports works:

```bash theme={null}
pip install langchain-decodo langchain langchain-openai
export DECODO_API_TOKEN=your_api_key
export OPENAI_API_KEY=your_openai_key
```

```python theme={null}
# main.py
from langchain.agents import create_agent
from langchain_decodo import DecodoSearchTool, DecodoWebScrapeTool

agent = create_agent(
    model="openai:gpt-5-mini",
    tools=[
        DecodoSearchTool(auth_mode="token"),
        DecodoWebScrapeTool(auth_mode="token"),
    ],
)

result = agent.invoke(
    {
        "messages": [
            {
                "role": "user",
                "content": "Search Google for the 2025 NBA Finals, scrape the Wikipedia page about it and tell me who won.",
            }
        ]
    }
)

print(result["messages"][-1].content)
```

## Available tools

| Class | Tool name exposed to the agent | Purpose |
| - | - | - |
| `DecodoWebScrapeTool` | `decodo_scrape_url` | Scrape any URL and return Markdown |
| `DecodoSearchTool` | `decodo_search` | Google Search or Amazon Search with parsed results |

You can also call a tool directly, without an agent:

```python theme={null}
from langchain_decodo import DecodoSearchTool, DecodoWebScrapeTool

scrape = DecodoWebScrapeTool(auth_mode="token")
markdown = scrape.invoke("https://quotes.toscrape.com")

search = DecodoSearchTool(auth_mode="token")
results = search.invoke({"query": "wireless keyboard", "engine": "amazon"})
```

### `DecodoWebScrapeTool`

Takes a single `url`, which must include `http://` or `https://`. It calls the `universal` target with `markdown: true` and returns the page as a Markdown string.

### `DecodoSearchTool`

Takes these inputs:

| Input | Default | Description |
| - | - | - |
| `query` | – | Search query. Required |
| `engine` | `google` | `google` calls the `google_search` target, `amazon` calls `amazon_search` |
| `num_results` | `10` | Sent to the API as `limit`. Accepts 1 to 100 |

It always sends `parse: true` and returns a JSON string: a list with one entry per result, each holding `content`, `status_code` and `url`. The parsed results are in `content`, for example the Google organic results at `content.results.results.organic`.

If the search fails, the tool raises a `RuntimeError` with the status code. A failed search can come back from the API as HTTP 200, so raising lets the agent retry instead of treating it as "no results".

> [!NOTE]
>
> Pass `engine` with each call, as in the example above. Setting it when you create the tool (`DecodoSearchTool(engine="amazon")`) has no effect, and the search still goes to Google.

> [!NOTE]
>
> `num_results` doesn't cap the results returned. A Google search with `num_results` set to 3 still returns the full first page of organic results.

## Document loader

`DecodoLoader` scrapes one or more URLs and returns each page as a LangChain `Document`, ready to split, embed and store in a RAG pipeline:

```python theme={null}
from langchain_decodo import DecodoLoader

loader = DecodoLoader(
    urls=["https://quotes.toscrape.com", "https://books.toscrape.com"],
    auth_mode="token",
)

for doc in loader.lazy_load():
    print(doc.metadata["url"], doc.metadata["status_code"], len(doc.page_content))
```

Each `Document` holds the page as Markdown in `page_content`. Its `metadata` has `source`, `url` and `status_code`, plus `task_id`, `created_at` and `updated_at` when the API returns them. URLs are scraped one at a time, in order. Use `load()` for a list of every document, or `lazy_load()` to process them as they arrive.

| Argument | Default | Description |
| - | - | - |
| `urls` | – | A single URL or a list of URLs. Required |
| `api_token` | `DECODO_API_TOKEN` | API key, or basic authentication token on older plans |
| `auth_mode` | `"basic"` | Set to `"token"` for an API key |
| `timeout` | `180` | Request timeout in seconds, per URL |
| `continue_on_error` | `True` | Keep loading the other URLs when one fails |

With `continue_on_error` on, a failed URL still produces a `Document`, with empty `page_content` and the error message in `metadata["error"]`. Filter those out before embedding. Set `continue_on_error=False` to raise on the first failure instead.

For the full list of API parameters, see [**Parameters**](https://help.decodo.com/docs/web-scraping-api-parameters-overview.md).

***

## Related

* [**LangChain (TypeScript)**](https://help.decodo.com/docs/langchain.md) – The Node.js LangChain tools.
* [**LlamaIndex**](https://help.decodo.com/docs/llamaindex.md) – Decodo readers and agent tools for LlamaIndex.
* [**Python SDK**](https://help.decodo.com/docs/python-sdk.md) – Typed access to every Decodo target from Python.
* [**MCP Server**](https://help.decodo.com/docs/mcp.md) – Connect AI agents and MCP-compatible tools to Decodo.
* [GitHub repository](https://github.com/Decodo/langchain/tree/master/libs/partners/decodo) – View the source code and latest releases.

***

> **[Support](https://direct.lc.chat/12092754)**
>
> Need help or just want to say hello? Our support is available 24/7. \
> You can also reach us anytime via email at [support@decodo.com](mailto:support@decodo.com).

> **[Feedback](mailto:feedback@decodo.com)**
>
> Can't find what you're looking for? Request an article! \
> Have feedback? Share your thoughts on how we can improve.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.