Skip to main content
Often, extracting the necessary data directly from one or more Fetch or XHR requests made by the browser during page load is more efficient than parsing the HTML content. Therefore, the Web Scraping API supports retrieving a list of XHR requests.

​
How to get the XHR request list

To obtain a list of XHR requests in the response, you will need to add the xhr with value true, for example: 
# update 'TOKEN VALUE' with your authorization token
curl --request 'POST' \
        --url 'https://scraper-api.decodo.com/v2/scrape' \
        --header 'Accept: application/json' \
        --header 'Authorization: Basic TOKEN VALUE' \
        --header 'Content-Type: application/json' \
        --data '
    {
      "url": "https://dummyjson.com/",
      "headless": "html",
      "xhr": true
    }
'
You will receive a response containing a list of JSON objects, where each object corresponds to a single XHR request. It’s possible to receive a multi-type response; more information can be found here.

​
Output

KeyDescription
urlURL of the XHR request
status_codeStatus code returned by the HTTP response
methodRequest’s HTTP method (e.g., GET, POST)
request_headersA set of key-value pairs for the request headers, with cookies included
request_payloadPayload content: this key appears only for POST, PUT, and PATCH requests
response_headersA set of key-value pairs for the response headers, with cookies included
response_bodyContent returned in the body of the HTTP response

​
Response Example

{
  "results": [
      {
          "content": [
              {
                  "url": "https://dummyjson.com/cdn-cgi/rum?",
                  "method": "POST",
                  "status_code": 204,
                  "response_body": "",
                  "request_headers": {
                      "accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7",
                      "origin": "https://dummyjson.com",
                      "referer": "https://www.yahoo.com/",
                      "sec-gpc": "1",
                      "sec-ch-ua": "\"Chromium\";v=\"119\", \"Not)A;Brand\";v=\"24\", \"Opera\";v=\"105\"",
                      "user-agent": "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/119.0.6045.193 Safari/537.36 OPR/105.0.4970.29",
                      "content-type": "application/json",
                      "sec-fetch-dest": "document",
                      "sec-fetch-mode": "navigate",
                      "sec-fetch-site": "same-origin",
                      "sec-fetch-user": "?1",
                      "accept-encoding": "gzip, deflate, br",
                      "accept-language": "en-US,en;q=0.8",
                      "sec-ch-ua-mobile": "?0",
                      "sec-ch-ua-platform": "\"Linux\"",
                      "upgrade-insecure-requests": "1"
                  },
                  "request_payload": "{\"memory\":{\"totalJSHeapSize\":3750592,\"usedJSHeapSize\":2468952,\"jsHeapSizeLimit\":4294705152},\"resources\":[],\"referrer\":\"\",\"eventType\":1,\"firstPaint\":0,\"firstContentfulPaint\":0,\"startTime\":1755774141001.2,\"versions\":{\"fl\":\"2024.11.0\",\"js\":\"2024.6.1\",\"timings\":2},\"pageloadId\":\"968fbcee-484b-4e53-b4ff-97c0058ba179\",\"location\":\"https://dummyjson.com/\",\"nt\":\"navigate\",\"timingsV2\":{\"unloadEventStart\":0,\"unloadEventEnd\":0,\"domInteractive\":1427.800000011921,\"domContentLoadedEventStart\":2259.9000000059605,\"domContentLoadedEventEnd\":2259.9000000059605,\"domComplete\":2269.5,\"loadEventStart\":2269.5,\"loadEventEnd\":2269.5,\"type\":\"navigate\",\"redirectCount\":0,\"criticalCHRestart\":0,\"activationStart\":0,\"initiatorType\":\"navigation\",\"nextHopProtocol\":\"h2\",\"deliveryType\":\"\",\"workerStart\":0,\"redirectStart\":0,\"redirectEnd\":0,\"fetchStart\":1,\"domainLookupStart\":1,\"domainLookupEnd\":1,\"connectStart\":56.20000001788139,\"connectEnd\":595.3000000119209,\"secureConnectionStart\":457.30000001192093,\"requestStart\":595.4000000059605,\"responseStart\":738,\"responseEnd\":738.9000000059605,\"transferSize\":5175,\"encodedBodySize\":4875,\"decodedBodySize\":24758,\"responseStatus\":200,\"firstInterimResponseStart\":0,\"renderBlockingStatus\":\"non-blocking\",\"finalResponseHeadersStart\":738,\"name\":\"https://dummyjson.com/\",\"entryType\":\"navigation\",\"startTime\":0,\"duration\":2269.5},\"dt\":\"\",\"siteToken\":\"a67792548ccc48faac5fa0d91120d5e7\",\"st\":2}",
                  "response_headers": {
                      "nel": "{\"report_to\":\"cf-nel\",\"success_fraction\":0.0,\"max_age\":604800}",
                      "date": "Thu, 21 Aug 2025 11:02:23 GMT",
                      "vary": "Origin, accept-encoding",
                      "cf-ray": "9729a34c9e38f032-DFW",
                      "server": "cloudflare",
                      "alt-svc": "h3=\":443\"; ma=86400",
                      "report-to": "{\"group\":\"cf-nel\",\"max_age\":604800,\"endpoints\":[{\"url\":\"https://a.nel.cloudflare.com/report/v4?s=1sDYaS2C%2F52rYP10Hp1fu0zokI2eEK0oBZnbM7fLRdM4rxEiqYBMM3vq6yK9wWJkNPj4lmML6rsK9f7eCD%2FA5BPIvQxnJJBjyWbdavc%3D\"}]}",
                      "content-type": "text/plain",
                      "server-timing": "cfEdge;dur=3,cfOrigin;dur=0",
                      "access-control-max-age": "86400",
                      "access-control-allow-origin": "https://dummyjson.com",
                      "access-control-allow-methods": "POST,OPTIONS",
                      "access-control-allow-credentials": "true"
                  }
              }
          ],
          "headers": {
              "age": "40350",
              "nel": "{\"report_to\":\"cf-nel\",\"success_fraction\":0.0,\"max_age\":604800}",
              "date": "Thu, 21 Aug 2025 11:02:21 GMT",
              "vary": "Accept-Encoding",
              "cf-ray": "9729a3416da3f032-DFW",
              "server": "cloudflare",
              "alt-svc": "h3=\":443\"; ma=86400",
              "report-to": "{\"group\":\"cf-nel\",\"max_age\":604800,\"endpoints\":[{\"url\":\"https://a.nel.cloudflare.com/report/v4?s=nyhZyWx2mEa9rX30X9iPrHhcsiowOcG89XIWtH7in5vmJKmf3H3LGlmjowKxk2qRcaaiDO2ODdqj6x2zRPJXHCJ4hUyhki1NAxXf7aA%3D\"}]}",
              "content-type": "text/html; charset=utf-8",
              "x-powered-by": "Cats on Keyboards",
              "cache-control": "no-store",
              "server-timing": "cfCacheStatus;desc=\"HIT\", cfEdge;dur=11,cfOrigin;dur=0",
              "x-railway-edge": "railway/us-east4-eqdc4a",
              "cf-cache-status": "HIT",
              "x-frame-options": "SAMEORIGIN",
              "content-encoding": "br",
              "x-xss-protection": "1; mode=block",
              "x-ratelimit-limit": "100",
              "x-ratelimit-reset": "1755612661",
              "x-download-options": "noopen",
              "x-railway-request-id": "HnoVBL2CSRWSyS9Vg4a9AQ",
              "x-ratelimit-remaining": "99",
              "x-content-type-options": "nosniff",
              "x-dns-prefetch-control": "off",
              "strict-transport-security": "max-age=15552000; includeSubDomains",
              "access-control-allow-origin": "*"
          },
          "cookies": [],
          "status_code": 200,
          "task_id": "7364250500075630593",
          "created_at": "2025-08-21 11:02:20",
          "updated_at": "2025-08-21 11:02:35"
      }
  ]
}
xhr output is not supported by all Target Templates. Check the individual template documentation for compatibility.

Support

Need help or just want to say hello? Our support is available 24/7.
You can also reach us anytime via email at support@decodo.com.

Feedback

Can’t find what you’re looking for? Request an article!
Have feedback? Share your thoughts on how we can improve.