DNS

DNS feature allows collecting DNS information and records for any web scraped target website. It's useful for monitoring and tracking DNS changes.

Note that enabling DNS may slow down scraping speed.

DNS details can also be found in the monitoring dashboard logs under the DNS tab:

overview page of web interface for Scrapfly DNS scraping — dns details on the monitoring logs page

Usage

To use DNS feature the DNS parameter can be enabled on any Scrapfly scrape request. This will include DNS details in the result.dns field:

require "uri"
require "net/http"

url = URI("https://api.scrapfly.io/scrape?dns=true&key=__API_KEY__&url=https%3A%2F%2Fweb-scraping.dev%2F")

https = Net::HTTP.new(url.host, url.port);
https.use_ssl = true

request = Net::HTTP::Get.new(url)

response = https.request(request)
puts response.read_body

"https://api.scrapfly.io/scrape?dns=true&key=&url=https%3A%2F%2Fweb-scraping.dev%2F"

"api.scrapfly.io"
"/scrape"

dns  = "true" 
key  = "" 
url  = "https://web-scraping.dev/"

Example Of Response

...
"result": {
    ...
    "dns": {
        "A": [
            {
                "host": "34.127.70.125",
                "ttl": 300
            }
        ],
        "CAA": [],
        "NS": [
            {
                "host": "ns-cloud-c4.googledomains.com",
                "ttl": -1
            },
            {
                "host": "ns-cloud-c2.googledomains.com",
                "ttl": -1
            },
            {
                "host": "ns-cloud-c1.googledomains.com",
                "ttl": -1
            },
            {
                "host": "ns-cloud-c3.googledomains.com",
                "ttl": -1
            }
        ],
        "SOA": [
            {
                "expires": 259200,
                "hostmaster": "cloud-dns-hostmaster.google.com",
                "minttl": 300,
                "nsname": "ns-cloud-c1.googledomains.com",
                "refresh": 21600,
                "serial": 1,
                "ttl": -1
            }
        ],
        "TXT": [
            {
                "text": "google-site-verification=WSRMzA3PyGeg1LYiNn979SCXSjVwfoqv9QuAXk-C_Dg",
                "ttl": -1
            }
        ]
    },
    ...
...

Pricing

No additional fee is applied on usage.

DNS

Usage

Example Of Response

Pricing

Summary