Debug

overview page of web interface
Debug tab in log inspection

Debugging is an essential function that helps you investigate unexpected scrape content, allowing you to replay it multiple times for analysis or integrate it into your test suite for better handling.

With our API, the complete response is saved along with the scraped content, and a link to the results is provided for easy access from external sources, as shown in the screen above.

If debug is activated while javascript rendering is enabled, a screenshot is taken

The debug content will be deleted when the log retention policy expires

Usage

Sign in 
import requests

url = "https://api.scrapfly.io/scrape?key=__API_KEY__&url=http%3A%2F%2Fhttpbin.dev%2Fanything&debug=true"

response = requests.request("GET", url)

print(response.text)

# import json
# print(json.loads(response.text)['result']['content'])
# print(json.loads(response.text)['result']['status_code'])

"https://api.scrapfly.io/scrape?key=&url=http%3A%2F%2Fhttpbin.dev%2Fanything&debug=true"

"api.scrapfly.io"
"/scrape"

key    = "" 
url    = "http://httpbin.dev/anything" 
debug  = "true"

Example of API Response

You need to pass your API key when requesting the URL given in "response_url" like https://api.scrapfly.io/scrape/debug/ee8484c6-ee5f-4775-a665-0a2b57631c1c?key={{ YOUR_API_KEY }} to be authorized. 
...
context {
    debug: {
        // screenshot is available only if rendering javascript is activated
        screenshot_url: "https://api.scrapfly.io/11cd6abe-5061-4dce-8d37-5d50e667a071/scrape/screenshot/ee8484c6-ee5f-4775-a665-0a2b57631c1c/debug",
        response_url: "https://api.scrapfly.io/scrape/debug/ee8484c6-ee5f-4775-a665-0a2b57631c1c"
    }
}
...

Pricing

No additional fee.

Integration