DNS

View as markdown
Copy for LLM Open in ChatGPT Open in Claude Open in Perplexity

DNS feature allows collecting DNS information and records for any web scraped target website. It's useful for monitoring and tracking DNS changes.

Note that enabling DNS may slow down scraping speed.

DNS details can also be found in the monitoring dashboard logs under the DNS tab:

dns details on the monitoring logs page

Usage

To use DNS feature the DNS parameter can be enabled on any Scrapfly scrape request. This will include DNS details in the result.dns field:

Ruby HTTP 
# gem install httparty

require 'httparty'
require 'json'

# Build query parameters
params = {
  'dns' => true,
  'key' => "__API_KEY__",
  'url' => "https://web-scraping.dev/",
}

url = "https://api.scrapfly.io/scrape"

options = {
  query: params,
  timeout: 160,
  open_timeout: 10,
}

begin
  response = HTTParty.get(url, options)

  # Check for HTTP errors
  unless response.success?
    error_data = response.parsed_response
    error_msg = error_data['message'] || error_data['description'] || 'Request failed'
    raise "HTTP error #{response.code}: #{error_msg}"
  end

  data = response.parsed_response
  puts JSON.pretty_generate(data)

  # Access the scrape result
  puts data['result'] if data['result']

rescue HTTParty::Error => e
  STDERR.puts "Request failed: #{e.message}"
  raise
rescue StandardError => e
  STDERR.puts "Error: #{e.message}"
  raise
end

https://api.scrapfly.io/scrape?dns=true&key=&url=https%3A%2F%2Fweb-scraping.dev%2F

Example Of Response

Pricing

No additional fee is applied on usage.

Summary