Langchain

Web Scraping API node for Lanchain applications

A Decodo Node.js LangChain plugin that enables developers to use Decodo's Scraper API alongside their LangChain applications.

Decodo Lanchain Tool integration is available with Web API Advanced plans.

📘

More information can be found in GitHub here.

Features

  • Web Scraping: Scrape any URL
  • Google Search: Search Google and retrieve structured results
  • Amazon Search: Search Amazon and retrieve structured product data
  • Reddit Scraping: Scrape Reddit posts and subreddits
  • Full TypeScript Support: Complete type definitions for all parameters
  • LangChain Integration: Seamless integration with LangChain's Tool system

Quick start

Installation:

npm install @decodo/langchain-ts

Once you have a Advanced plan activated, take a note of your generated username and password:

A simple agentic example:

import dotenv from 'dotenv';
import { ChatOpenAI } from '@langchain/openai';
import { createReactAgent } from '@langchain/langgraph/prebuilt';
import { DecodoUniversalTool } from '@decodo/langchain-ts';

dotenv.config();

const main = async () => {
  const username = process.env.SCRAPER_API_USERNAME!;
  const password = process.env.SCRAPER_API_PASSWORD!;

  const decodoUniversalTool = new DecodoUniversalTool({ username, password });

  const model = new ChatOpenAI({
    model: 'gpt-4o-mini',
  });

  const agent = createReactAgent({
    llm: model,
    tools: [decodoUniversalTool],
  });

  const result = await agent.invoke({
    messages: [
      {
        role: 'user',
        content: 'scrape the wikipedia NBA 2025 season page and tell me who won in 2025?',
      },
    ],
  });

  console.log(result.messages[result.messages.length - 1].content);
};

if (require.main === module) {
  main();
}

All tools accept a DecodoConfig object:

type DecodoConfig = {
  username: string; // Your Web Advanced product username
  password: string; // Your Web Advanced product password
};

Parameters

**Web Scraping: **

ParameterRequiredDescription
urlWebsite URL
geoThe geographical location on which the result depends. Read more.
JsRenderWhen set to true, enables javascript rendering
markdownWhen set to true, returns markdown response

**Google Search: **

ParameterRequiredDescription
querySearch query
parseAutomatically parses results when set to true
geoThe geographical location on which the result depends. Read more.
JsRenderWhen set to true, enables javascript rendering.

Amazon Search:

ParameterRequiredDescription
querySearch query
parseAutomatically parses results when set to true
geoThe geographical location on which the result depends. Read more.
JsRenderWhen set to true, enables javascript rendering.

Reddit Scraping:

ParameterRequiredDescription
urlReddit subreddit URL
geoThe geographical location on which the result depends. Read more
parseAutomatically parses results when set to true

Support

Still can't find an answer? Want to say hi? We take pride in our 24/7 customer support. Alternatively, you can reach us via our support email at [email protected].

Feedback

Something's missing? Request an article!
Got feedback? Let us know how we're doing and what can be improved.