# General crawler rules
User-agent: *
Allow: /

Sitemap: https://scrapfly.io/blog/sitemap.xml

# learn robot skills at https://scrapfly.io/academy <3