Screenshot

overview page of web interface
Screenshot tab of log inspection
This feature require Javascript Rendering enabled

You can take multiple screenshots of a webpage. You have a choice between full page or specific zone via CSS selector.

The screenshot feature works beside the scraper, which means you can still scrape as usual and use the screenshot features. The upstream response is not altered with a screenshot.

Screenshots can help you to debug, track visual changes, and generate thumbnails of a website.

You can take a maximum of 10 different screenshots per scrape.

API

In this example, we take three screenshots. One full-page and two others via CSS selector

Sign in 
curl -G \
--request "GET" \
--url "https://api.scrapfly.io/scrape" \
--data-urlencode "key=__API_KEY__" \
--data-urlencode "url=https://www.amazon.fr/MikroTik-CRS305-1G-4S-Interrupteur-Ethernet-routerOS/dp/B07LFKGP1L" \
--data-urlencode "render_js=true" \
--data-urlencode "screenshots[full-page]=fullpage" \
--data-urlencode "screenshots[similarities]=.similarities-widget" \
--data-urlencode "screenshots[price]=#price"
"https://api.scrapfly.io/scrape?key=&url=https%3A%2F%2Fwww.amazon.fr%2FMikroTik-CRS305-1G-4S-Interrupteur-Ethernet-routerOS%2Fdp%2FB07LFKGP1L&render_js=true&screenshots[full-page]=fullpage&screenshots[similarities]=.similarities-widget&screenshots[price]=%23price"

"api.scrapfly.io"
"/scrape"

key                        = "" 
url                        = "https://www.amazon.fr/MikroTik-CRS305-1G-4S-Interrupteur-Ethernet-routerOS/dp/B07LFKGP1L" 
render_js                  = "true" 
screenshots[full-page]     = "fullpage" 
screenshots[similarities]  = ".similarities-widget" 
screenshots[price]         = "#price"

Example Of Response

{
    ...
    "result": {
        ...
        "screenshots": {
            "main": {
                "extension": "png",
                "format": "fullpage",
                "css_selector": null,
                "size": 2867974,
                "url": "https://api.scrapfly.io/4d1b8e8f-3803-4aa6-88fa-39d5aa81b6b3/scrape/screenshot/db475202-a7c0-4d7b-9179-98089901fce3/main"
            },
            "similarities": {
                "extension": "png",
                "format": "element",
                "css_selector": ".similarities-widget",
                "size": 109817,
                "url": "https://api.scrapfly.io/4d1b8e8f-3803-4aa6-88fa-39d5aa81b6b3/scrape/screenshot/db475202-a7c0-4d7b-9179-98089901fce3/similarities"
            },
            "price": {
                "extension": "png",
                "format": "element",
                "css_selector": "#price",
                "size": 10364,
                "url": "https://api.scrapfly.io/4d1b8e8f-3803-4aa6-88fa-39d5aa81b6b3/scrape/screenshot/db475202-a7c0-4d7b-9179-98089901fce3/price"
            }
        }
        ...
    }
    ...
}

Download Programmatically

The url to download the screenshot is located in the response response['result']['screenshots']['${name_of_your_screenshot}']['url']. The only thing you have to add in order to download it, it's the api key param to authenticate your call (Because the URL even if it's private - it's protected) 

curl "https://api.scrapfly.io/4d1b8e8f-3803-4aa6-88fa-39d5aa81b6b3/scrape/screenshot/db475202-a7c0-4d7b-9179-98089901fce3/main?key=" > screenshot.jpg

All related errors are listed below. You can see full description and example of error response on Errors section

Limitations

This feature require javascript rendering to be enabled in order to work. In some situations, the screenshot system won't be able to take it. For many reasons:

  • GET Request: Only GET request are eligible to screenshot
  • Huge page: Require a large amount of memory - to keep our service stable, we prefer to shut down the rendering
  • Broken script: For unknown reason at the time, when web pages contain bugged / no well-scripted javascript, the rendering system might crash
  • Cache: If the cache feature is used and HIT then screenshots will be ignored

Frequently Asked Questions

  • Question: How much time screenshot are available to download?
  • Answer: Screenshot availability is the same as logs retention, more info about log retention
  • Question: The screenshot does not look like my browser page
  • Answer: Our goal is not to provide a screenshot service with 1:1 rendering - a lot of optimisation are done behind the scene to provide best performance for web scraping, which have tradeoff

Integration