> ## Documentation Index
> Fetch the complete documentation index at: https://help.decodo.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Status, Results & Cancellation

Track a crawl's progress, fetch scraped pages, receive webhooks, and stop a crawl early

Crawls run asynchronously. When you start a crawl, the response includes a `crawl_id` and ready-to-use links for every endpoint below.

<table>
  <colgroup>
    <col width="295" />

    <col width="124" />

    <col width="435" />
  </colgroup>

  <thead>
    <tr>
      <th>Endpoint</th>
      <th>Method</th>
      <th>Purpose</th>
    </tr>
  </thead>

  <tbody>
    <tr>
      <td>`https://data.decodo.com/v1/crawl`</td>
      <td>POST</td>
      <td>Starts a crawl.</td>
    </tr>

    <tr>
      <td>`https://data.decodo.com/v1/crawl/{crawl_id}/status`</td>
      <td>GET</td>
      <td>Crawl status.</td>
    </tr>

    <tr>
      <td>`https://data.decodo.com/v1/crawl/{crawl_id}`</td>
      <td>GET</td>
      <td>List of crawled URLs, their status and Task IDs for each URL. You can then fetch each URL content individually using Task ID.</td>
    </tr>

    <tr>
      <td>`https://data.decodo.com/v1/crawl/{crawl_id}?type=content`</td>
      <td>GET</td>
      <td>Scraped page content inline.</td>
    </tr>

    <tr>
      <td>`https://data.decodo.com/v1/crawl/{crawl_id}/cancel`</td>
      <td>POST</td>
      <td>Stop the crawl and keep collected results.</td>
    </tr>
  </tbody>
</table>

## Check crawl status

##### **Endpoint:** `https://data.decodo.com/v1/crawl/{crawl_id}/status`

This is the `crawl_status` link from the crawl response.

<CodeGroup>
  ```shellscript cURL theme={null}
  curl --request 'GET' \
    --url 'https://data.decodo.com/v1/crawl/{crawl_id}/status' \
    --header 'Accept: application/json' \
    --header 'Authorization: Bearer YOUR_API_KEY'
  ```
</CodeGroup>

## Get results

<Note>
  Results are stored for 24 hours after the crawl finishes.
</Note>

Crawl duration depends on the site's size and how fast its pages respond, so there's no fixed completion time.

You can get crawl results in two ways.

1. List of crawled URLs

##### **Endpoint:** `https://data.decodo.com/v1/crawl/{crawl_id}`

This is the `results_tasks` link. It returns every URL in the crawl with its own status, so you can see which pages are finished, still running, or failed.

<ResponseExample>
  ```text Response example theme={null}
  {
              "task_id": "{task_id}",
              "status": "done",
              "created_at": "2026-10-01 04:48:26",
              "updated_at": "2026-10-01 04:48:29",
              "url": "https://decodo.com/",
              "depth": 0,
              "crawl_status": "https://data.decodo.com/v1/crawl/{crawl_id}/status",
              "results_tasks": "https://data.decodo.com/v1/crawl/{crawl_id}",
              "results_content": "https://data.decodo.com/v1/crawl/{crawl_id}?type=content"
          },
  ```
</ResponseExample>

You can then fetch each task content individually through `https://data.decodo.com/v1/task/{task_id}/results `endpoint

2. Full content inline in the reponse

##### **Endpoint:** `https://data.decodo.com/v1/crawl/{crawl_id}?type=content`

This is the `results_content` link. It returns the scraped content of every page, in the format you chose with `output`. Results are split into pages of up to 10 MB - use the `cur` parameter to get the next one.

<CodeGroup>
  ```shellscript cURL theme={null}
  curl --request 'GET' --url 'https://data.decodo.com/v1/crawl/{crawl_id}?type=content' \
  --header 'Accept: application/json' \
  --header 'Authorization: Bearer TOKEN VALUE'
  ```

  ```text file name theme={null}
  ```
</CodeGroup>

## Webhooks

Add `callback_url` to your crawl request and we'll send a `POST` request to it once all pages are processed, so you don't need to poll the status endpoint. To confirm a webhook really came from Decodo, add a `passthrough` value to your request. It's returned unchanged in the webhook payload.

<CodeGroup>
  ```shellscript cURL theme={null}
  curl --request 'POST' \
          --url 'https://data.decodo.com/v1/crawl' \
          --header 'Accept: application/json' \
          --header 'Authorization: Bearer TOKEN VALUE' \
          --header 'Content-Type: application/json' \
          --data '
      {
        "url": "https://decodo.com",
        "callback_url": "https://your.url",
        "passthrough": "your_note"
      }
  '
  ```
</CodeGroup>


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.