Skip to main content
Web Data API supports performing a number of browser actions before retrieving a desired result.
Browser actions run in a headless browser, so every request using browser_actions must also set headless to "html". Without it the request fails with Browser actions require "headless" parameter to be set to "html".

Actions

click

Example:

input

Example:

scroll

NameArgumentsDescription
scrollx: integer y: integerScrolls the content of a page a specified number of pixels.
Example:

scroll_to_bottom

NameArgumentsDescription
scroll_to_bottomtimeout_s: integerScrolls down the page for a set amount of seconds.
Example:

wait

NameArgumentsDescription
waitwait_time_s: integerPauses for a specified number of seconds.
Example:

wait_for_element

NameArgumentsDescription
wait_for_elementSelectors: *type: “xpath”/“css”/“text” *value: string timeout_s: integerWaits for a specified duration for element to load.
Example:

fetch_resource

If used with other instructions, fetch_resource should be the last action.
NameArgumentsDescription
fetch_resourcefilter: regex on_error:“error”/“skip”Retrieves the first Fetch or XHR resource that matches the specified pattern
Example:

General Arguments

Arguments available for all actions above

type

NameDescription
typeType of browser action used

timeout_s

NameDescription
timeout_sHow much time in seconds to wait at max until the execution of the action is terminated.

wait_time_s

NameDescription
wait_time_sHow much time in seconds to use explicitly to execute the action.

on_error

NameDescription
on_errorIndicates what to do with actions in case they fail: “error”: Stops the execution of browser actions. “skip”: Continues with the next action.

Fetching a Network Request

If a website populates content by fetching a JSON object, you can scrape just the network request and thus avoid having to deal with HTML altogether. To do this, you can use the fetch_resource browser action, as shown below.
fetch_resource can be combined with other browser actions, but it must be the last action in the list.
For example, when loading yahoo.com and opening the Network tab, we can see an exp.json file being loaded:
If you wish to scrape just the contents of this request, you can use the fetch_resource browser action. Note that filter is a regular expression that matches the filename:
Results:

Support

Need help or just want to say hello? Our support is available 24/7.
You can also reach us anytime via email at support@decodo.com.

Feedback

Can’t find what you’re looking for? Request an article!
Have feedback? Share your thoughts on how we can improve.