Fetch and XHR Request Capturing
Web Scraping API supports retrieving list of XHR requests.
Often, extracting the necessary data directly from one or more Fetch or XHR requests made by the browser during page load is more efficient than parsing the HTML content. Therefore, Web Scraping API supports retrieving list of XHR requests.
How to get XHR request list
To obtain a list of XHR requests in the response, you will need to add the xhr with value true, for example:
curl --request 'POST' \
--url 'https://scraper-api.decodo.com/v2/scrape' \
--header 'Accept: application/json' \
--header 'Authorization: Basic TOKEN VALUE' \
--header 'Content-Type: application/json' \
--data '
{
"url": "https://dummyjson.com/",
"headless": "html",
"xhr": true
}
'You will receive a response containing a list of JSON objects, where each object corresponds to a single XHR request.
To receive both the raw HTML and XHR responses in a single request, add
?type=raw,xhrto the request URL:For synchronous requests:
https://scraper-api.decodo.com/v2/scrape?type=raw,xhrFor retrieving asynchronous requests results:
https://scraper-api.decodo.com/v2/task/{task_id}/results?type=raw,xhrMore information can be found here.
Output
| Key | Description |
|---|---|
url | URL of the XHR request |
status_code | Status code returned by the HTTP response |
method | Request’s HTTP method (e.g., GET, POST) |
request_headers | A set of key-value pairs for the request headers, with cookies included |
request_payload | Payload content; this key appears only for POST, PUT, and PATCH requests |
response_headers | A set of key-value pairs for the response headers, with cookies included |
response_body | Content returned in the body of the HTTP response |
Response example
{
"results": [
{
"content": [
{
"url": "https://dummyjson.com/cdn-cgi/rum?",
"method": "POST",
"status_code": 204,
"response_body": "",
"request_headers": {
"accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7",
"origin": "https://dummyjson.com",
"referer": "https://www.yahoo.com/",
"sec-gpc": "1",
"sec-ch-ua": "\"Chromium\";v=\"119\", \"Not)A;Brand\";v=\"24\", \"Opera\";v=\"105\"",
"user-agent": "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/119.0.6045.193 Safari/537.36 OPR/105.0.4970.29",
"content-type": "application/json",
"sec-fetch-dest": "document",
"sec-fetch-mode": "navigate",
"sec-fetch-site": "same-origin",
"sec-fetch-user": "?1",
"accept-encoding": "gzip, deflate, br",
"accept-language": "en-US,en;q=0.8",
"sec-ch-ua-mobile": "?0",
"sec-ch-ua-platform": "\"Linux\"",
"upgrade-insecure-requests": "1"
},
"request_payload": "{\"memory\":{\"totalJSHeapSize\":3750592,\"usedJSHeapSize\":2468952,\"jsHeapSizeLimit\":4294705152},\"resources\":[],\"referrer\":\"\",\"eventType\":1,\"firstPaint\":0,\"firstContentfulPaint\":0,\"startTime\":1755774141001.2,\"versions\":{\"fl\":\"2024.11.0\",\"js\":\"2024.6.1\",\"timings\":2},\"pageloadId\":\"968fbcee-484b-4e53-b4ff-97c0058ba179\",\"location\":\"https://dummyjson.com/\",\"nt\":\"navigate\",\"timingsV2\":{\"unloadEventStart\":0,\"unloadEventEnd\":0,\"domInteractive\":1427.800000011921,\"domContentLoadedEventStart\":2259.9000000059605,\"domContentLoadedEventEnd\":2259.9000000059605,\"domComplete\":2269.5,\"loadEventStart\":2269.5,\"loadEventEnd\":2269.5,\"type\":\"navigate\",\"redirectCount\":0,\"criticalCHRestart\":0,\"activationStart\":0,\"initiatorType\":\"navigation\",\"nextHopProtocol\":\"h2\",\"deliveryType\":\"\",\"workerStart\":0,\"redirectStart\":0,\"redirectEnd\":0,\"fetchStart\":1,\"domainLookupStart\":1,\"domainLookupEnd\":1,\"connectStart\":56.20000001788139,\"connectEnd\":595.3000000119209,\"secureConnectionStart\":457.30000001192093,\"requestStart\":595.4000000059605,\"responseStart\":738,\"responseEnd\":738.9000000059605,\"transferSize\":5175,\"encodedBodySize\":4875,\"decodedBodySize\":24758,\"responseStatus\":200,\"firstInterimResponseStart\":0,\"renderBlockingStatus\":\"non-blocking\",\"finalResponseHeadersStart\":738,\"name\":\"https://dummyjson.com/\",\"entryType\":\"navigation\",\"startTime\":0,\"duration\":2269.5},\"dt\":\"\",\"siteToken\":\"a67792548ccc48faac5fa0d91120d5e7\",\"st\":2}",
"response_headers": {
"nel": "{\"report_to\":\"cf-nel\",\"success_fraction\":0.0,\"max_age\":604800}",
"date": "Thu, 21 Aug 2025 11:02:23 GMT",
"vary": "Origin, accept-encoding",
"cf-ray": "9729a34c9e38f032-DFW",
"server": "cloudflare",
"alt-svc": "h3=\":443\"; ma=86400",
"report-to": "{\"group\":\"cf-nel\",\"max_age\":604800,\"endpoints\":[{\"url\":\"https://a.nel.cloudflare.com/report/v4?s=1sDYaS2C%2F52rYP10Hp1fu0zokI2eEK0oBZnbM7fLRdM4rxEiqYBMM3vq6yK9wWJkNPj4lmML6rsK9f7eCD%2FA5BPIvQxnJJBjyWbdavc%3D\"}]}",
"content-type": "text/plain",
"server-timing": "cfEdge;dur=3,cfOrigin;dur=0",
"access-control-max-age": "86400",
"access-control-allow-origin": "https://dummyjson.com",
"access-control-allow-methods": "POST,OPTIONS",
"access-control-allow-credentials": "true"
}
}
],
"headers": {
"age": "40350",
"nel": "{\"report_to\":\"cf-nel\",\"success_fraction\":0.0,\"max_age\":604800}",
"date": "Thu, 21 Aug 2025 11:02:21 GMT",
"vary": "Accept-Encoding",
"cf-ray": "9729a3416da3f032-DFW",
"server": "cloudflare",
"alt-svc": "h3=\":443\"; ma=86400",
"report-to": "{\"group\":\"cf-nel\",\"max_age\":604800,\"endpoints\":[{\"url\":\"https://a.nel.cloudflare.com/report/v4?s=nyhZyWx2mEa9rX30X9iPrHhcsiowOcG89XIWtH7in5vmJKmf3H3LGlmjowKxk2qRcaaiDO2ODdqj6x2zRPJXHCJ4hUyhki1NAxXf7aA%3D\"}]}",
"content-type": "text/html; charset=utf-8",
"x-powered-by": "Cats on Keyboards",
"cache-control": "no-store",
"server-timing": "cfCacheStatus;desc=\"HIT\", cfEdge;dur=11,cfOrigin;dur=0",
"x-railway-edge": "railway/us-east4-eqdc4a",
"cf-cache-status": "HIT",
"x-frame-options": "SAMEORIGIN",
"content-encoding": "br",
"x-xss-protection": "1; mode=block",
"x-ratelimit-limit": "100",
"x-ratelimit-reset": "1755612661",
"x-download-options": "noopen",
"x-railway-request-id": "HnoVBL2CSRWSyS9Vg4a9AQ",
"x-ratelimit-remaining": "99",
"x-content-type-options": "nosniff",
"x-dns-prefetch-control": "off",
"strict-transport-security": "max-age=15552000; includeSubDomains",
"access-control-allow-origin": "*"
},
"cookies": [],
"status_code": 200,
"task_id": "7364250500075630593",
"created_at": "2025-08-21 11:02:20",
"updated_at": "2025-08-21 11:02:35"
}
]
}XHR support for Target Templates
xhr parameter is supported on these Target Templates for Advanced plan holders:
- Universal
- Google Search
- Google Ads
- Bing
- Bing Search
- Amazon
- Amazon Product
- Amazon Pricing
- Amazon Search
- Amazon Sellers
- Amazon Bestsellers
- Target
- Target Product
- Target Search
- Walmart
- Walmart product
- Tiktok Post
Support
Still can't find an answer? Want to say hi? We take pride in our 24/7 customer support. Alternatively, you can reach us via our support email at [email protected].
Updated 1 day ago