Migrating to Web Data API

Web Data API replaces two legacy products:

Web Scraping API is replaced by /scrape.
Fast Search API is replaced by /search.

Existing setups keep working, so you can migrate at your own pace. The request parameters stay the same. You only change the API key, the base URL and the authentication header.

What changes

	Legacy	Web Data API
Base URL	`https://scraper-api.decodo.com` (Web Scraping API) `https://fastsearch.decodo.com` (Fast Search API)	`https://data.decodo.com/v1`
Authentication	`Authorization: Basic TOKEN VALUE`	`Authorization: Bearer TOKEN VALUE`
Credentials	A separate token for each product	One API key for every endpoint

Endpoint mapping

Legacy endpoint	Web Data API endpoint
`POST https://scraper-api.decodo.com/v2/scrape`	`POST https://data.decodo.com/v1/scrape`
`POST https://scraper-api.decodo.com/v3/task`	`POST https://data.decodo.com/v1/task`
`POST https://scraper-api.decodo.com/v3/task/batch`	`POST https://data.decodo.com/v1/task/batch`
`GET https://scraper-api.decodo.com/v3/task/{task_id}`	`GET https://data.decodo.com/v1/task/{task_id}`
`GET https://scraper-api.decodo.com/v3/task/{task_id}/results`	`GET https://data.decodo.com/v1/task/{task_id}/results`
`POST https://fastsearch.decodo.com/v0/search`	`POST https://data.decodo.com/v1/search`

Migration steps

1. Get a Web Data API key

Your legacy token doesn’t work with Web Data API, and the new API key doesn’t work with the legacy endpoints. Copy your Web Data API key from the dashboard.

2. Update the URL and the authentication header

Replace the legacy URL with its Web Data API endpoint from the table above. Then change Basic to Bearer in the Authorization header and use your new API key.

curl --request 'POST' \
    --url 'https://scraper-api.decodo.com/v2/scrape' \
    --header 'Accept: application/json' \
    --header 'Authorization: Basic TOKEN VALUE' \
    --header 'Content-Type: application/json' \
    --data '
    {
      "target": "google_search",
      "query": "laptop",
      "parse": true
    }
'

curl --request 'POST' \
    --url 'https://data.decodo.com/v1/scrape' \
    --header 'Accept: application/json' \
    --header 'Authorization: Bearer TOKEN VALUE' \
    --header 'Content-Type: application/json' \
    --data '
    {
      "target": "google_search",
      "query": "laptop",
      "parse": true
    }
'

curl --request 'POST' \
    --url 'https://fastsearch.decodo.com/v0/search' \
    --header 'Accept: application/json' \
    --header 'Authorization: Basic TOKEN VALUE' \
    --header 'Content-Type: application/json' \
    --data '
    {
      "query": "laptop"
    }
'

curl --request 'POST' \
    --url 'https://data.decodo.com/v1/search' \
    --header 'Accept: application/json' \
    --header 'Authorization: Bearer TOKEN VALUE' \
    --header 'Content-Type: application/json' \
    --data '
    {
      "query": "laptop"
    }
'

3. Keep your request body

You don’t need to change any parameters:

/scrape accepts the same parameters as Web Scraping API, including target, headless, proxy_pool and parse. The response format is the same too.
/search accepts the same parameters as Fast Search API. The response includes organic, top_stories and status, and can include extra fields such as answer_box, people_also_ask and related_searches.

4. Test, then switch

Send a few requests with the new setup and compare the results with your legacy integration. Keep the legacy integration running until you’ve confirmed that the new one works.

What’s new

Web Data API also adds two endpoints that the legacy products didn’t have:

/crawl collects up to 10,000 pages from a site, starting from the URL you submit.
/map lists the URLs found on a page and in the site’s top-level sitemap, without scraping page content.

Support

Need help or just want to say hello? Our support is available 24/7.
You can also reach us anytime via email at support@decodo.com.

Feedback

Can’t find what you’re looking for? Request an article!
Have feedback? Share your thoughts on how we can improve.

ABOUT

RESIDENTIAL PROXIES

STATIC RESIDENTIAL (ISP) PROXIES

DATACENTER PROXIES

FREE PROXIES

SITE UNBLOCKER

WEB DATA API

DECODO TOOLS

AI

INTEGRATIONS

ACCOUNT & DASHBOARD

PAYMENTS

TROUBLESHOOTING

TIPS & TRICKS

USEFUL LINKS

Migrating to Web Data API

What changes

Endpoint mapping

Migration steps

1. Get a Web Data API key

2. Update the URL and the authentication header

3. Keep your request body

4. Test, then switch

What’s new

Support

Feedback

​What changes

​Endpoint mapping

​Migration steps

​1. Get a Web Data API key

​2. Update the URL and the authentication header

​3. Keep your request body

​4. Test, then switch

​What’s new

Support

Feedback

What changes

Endpoint mapping

Migration steps

1. Get a Web Data API key

2. Update the URL and the authentication header

3. Keep your request body

4. Test, then switch

What’s new