# Scrapfly Blog Tutorials, guides, and insights on web scraping, data extraction, and automation 308 articles Search articles [ ](https://scrapfly.io/blog/feed.xml "Subscribe via RSS") [All](https://scrapfly.io/blog) [ai](https://scrapfly.io/blog/tag/ai) [anti-bot](https://scrapfly.io/blog/tag/anti-bot) [api](https://scrapfly.io/blog/tag/api) [beautifulsoup](https://scrapfly.io/blog/tag/beautifulsoup) [blocking](https://scrapfly.io/blog/tag/blocking) [cloud-browser](https://scrapfly.io/blog/tag/cloud-browser) [crawling](https://scrapfly.io/blog/tag/crawling) [css-selectors](https://scrapfly.io/blog/tag/css-selectors) [curl](https://scrapfly.io/blog/tag/curl) [data-parsing](https://scrapfly.io/blog/tag/data-parsing) [ecommerce](https://scrapfly.io/blog/tag/ecommerce) [fashion](https://scrapfly.io/blog/tag/fashion) [frameworks](https://scrapfly.io/blog/tag/frameworks) [golang](https://scrapfly.io/blog/tag/golang) [graphql](https://scrapfly.io/blog/tag/graphql) [headless-browser](https://scrapfly.io/blog/tag/headless-browser) [hidden-api](https://scrapfly.io/blog/tag/hidden-api) [http](https://scrapfly.io/blog/tag/http) [httpx](https://scrapfly.io/blog/tag/httpx) [java](https://scrapfly.io/blog/tag/java) [javascript](https://scrapfly.io/blog/tag/javascript) [jupyter](https://scrapfly.io/blog/tag/jupyter) [nodejs](https://scrapfly.io/blog/tag/nodejs) [parsel](https://scrapfly.io/blog/tag/parsel) [php](https://scrapfly.io/blog/tag/php) [playwright](https://scrapfly.io/blog/tag/playwright) [project](https://scrapfly.io/blog/tag/project) [proxies](https://scrapfly.io/blog/tag/proxies) [puppeteer](https://scrapfly.io/blog/tag/puppeteer) [python](https://scrapfly.io/blog/tag/python) [r](https://scrapfly.io/blog/tag/r) [real-estate](https://scrapfly.io/blog/tag/real-estate) [requests](https://scrapfly.io/blog/tag/requests) [ruby](https://scrapfly.io/blog/tag/ruby) [scaling](https://scrapfly.io/blog/tag/scaling) [scrapeguide](https://scrapfly.io/blog/tag/scrapeguide) [scrapy](https://scrapfly.io/blog/tag/scrapy) [screenshots](https://scrapfly.io/blog/tag/screenshots) [selenium](https://scrapfly.io/blog/tag/selenium) [seo](https://scrapfly.io/blog/tag/seo) [tools](https://scrapfly.io/blog/tag/tools) [typescript](https://scrapfly.io/blog/tag/typescript) [web-scraping](https://scrapfly.io/blog/tag/web-scraping) [xpath](https://scrapfly.io/blog/tag/xpath) ## // Articles 12 posts [ Latest python blocking headless-browser ### Best Stealth Browsers for Web Scraping in 2026 A neutral, benchmark-literate ranking of the best open-source stealth browsers for web scraping in 2026, with the success, speed, and maintenance tradeoffs. Jul 15, 2026 16 min read ](https://scrapfly.io/blog/posts/best-stealth-browsers) [ http blocking proxies ### Akamai Bot Manager: Understanding \_abck Cookies and Sensor Data Deep technical guide to Akamai Bot Manager detection layers, \_abck and ak\_bmsc cookies, sensor data mechanics, JavaScr... Jul 13, 2026 14 min read ](https://scrapfly.io/blog/posts/akamai-bot-manager-understanding-abck-cookies-and-sensor-data) [ ### AI Browser Agents - How to Use Them with Cloud Browsers Learn to integrate AI browser agents like Browser Use and Stagehand with cloud browsers. This guide shows you how to run... Jul 13, 2026 12 min read ](https://scrapfly.io/blog/posts/ai-browser-agents-how-to-use-them-with-cloud-browsers) [ ### JA3/JA4 TLS Fingerprinting: Guide to Detection and Evasion Learn how JA3 and JA4 TLS fingerprinting works, how anti-bot systems use it to detect scrapers, and how to bypass detect... Jul 13, 2026 20 min read ](https://scrapfly.io/blog/posts/ja3-ja4-tls-fingerprinting-guide-to-detection-and-evasion) [ ### TLS Cipher Suites in Web Scraping Learn how TLS cipher suites affect both transport security and scraper detection, with practical JA3/JA4 guidance, refer... Jul 07, 2026 14 min read ](https://scrapfly.io/blog/posts/tls-cipher-suites-in-web-scraping) [ ### HTTP Referer Header: Complete Guide for Web Scraping A deep dive into the HTTP Referer header for web scraping. Learn how websites use it for detection and how to implement ... Jul 07, 2026 15 min read ](https://scrapfly.io/blog/posts/http-referer-header-complete-guide-for-web-scraping) [ ### Web Scraping with Grok: Build an Autonomous Scraping Agent Learn web scraping with Grok in 2026 using Grok Build, grok-cli, the xAI API, and Scrapfly CLI for JavaScript rendering,... Jul 07, 2026 15 min read ](https://scrapfly.io/blog/posts/web-scraping-with-grok) [ blocking ### How Anti-Bot Detection Works Discover how anti-bot systems work by assigning a 'trust score' to every request. This guide breaks down the 5 key detec... Jul 07, 2026 17 min read ](https://scrapfly.io/blog/posts/how-anti-bot-detection-works) [ python ai hidden-api ### How to Scrape Perplexity AI in 2026 Scrape Perplexity AI with Python: render JS, bypass Cloudflare, capture answers + cited domains, and build a citation tr... Jul 07, 2026 19 min read ](https://scrapfly.io/blog/posts/how-to-scrape-perplexity) [ blocking api ai ### Web Scraping with Hermes Agent Hermes Agent ships with Firecrawl and Scrapling, but they break on protected sites. Add Scrapfly as a cloud fetch layer ... Jul 07, 2026 14 min read ](https://scrapfly.io/blog/posts/web-scraping-with-hermes-agent) [ blocking anti-bot ### What Is an Anti-Detect Browser? How They Work and When to Use One Anti-detect browsers spoof browser fingerprints and isolate profiles so sites see many separate users. Learn how they wo... Jul 07, 2026 11 min read ](https://scrapfly.io/blog/posts/what-is-an-anti-detect-browser) [ api headless-browser playwright ### Best Browser Automation Tools in 2026 Explore the best browser automation tools of 2026. This complete guide compares Cloud APIs, Open-Source frameworks, AI A... Jun 30, 2026 16 min read ](https://scrapfly.io/blog/posts/best-browser-automation-tools) 1 [2](https://scrapfly.io/blog?page=2) [3](https://scrapfly.io/blog?page=3) … [26](https://scrapfly.io/blog?page=26) [ ](https://scrapfly.io/blog?page=2) Page 1 of 26 · 308 articles ## ? Quick Answers ### nodejs [View all ](https://scrapfly.io/blog/answers) - [ Q How to take screenshots in NodeJS? ](https://scrapfly.io/blog/answers/how-to-take-screenshots-nodejs) ### css-selectors [View all ](https://scrapfly.io/blog/answers) - [ Q How to use CSS Selectors in Nim ? ](https://scrapfly.io/blog/answers/how-to-use-css-selectors-in-nim) - [ Q Is it possible to select preceding siblings using CSS selectors? ](https://scrapfly.io/blog/answers/how-to-select-preceding-sibling-element-css-selectors) - [ Q How to select following siblings using CSS selectors? ](https://scrapfly.io/blog/answers/how-to-select-following-sibling-element-css-selectors) - [ Q How to select elements by ID using CSS selectors? ](https://scrapfly.io/blog/answers/how-to-select-elements-by-id-css-selectors) - [ Q How to select elements by class using CSS selectors? ](https://scrapfly.io/blog/answers/how-to-select-elements-by-class-css-selectors) ### proxies [View all ](https://scrapfly.io/blog/answers) - [ Q What is The cURL (28) Error, Couldn't connect to server? ](https://scrapfly.io/blog/answers/what-is-the-curl-28-error) - [ Q How To Use Proxy With cURL? ](https://scrapfly.io/blog/answers/how-to-use-proxy-with-curl) - [ Q How to Solve the cURL (60) Error When Using Proxy? ](https://scrapfly.io/blog/answers/how-to-solve-the-curl-60-error-when-proxy) - [ Q What are SOCKS5 proxies and how they compare to HTTP proxies? ](https://scrapfly.io/blog/answers/what-are-socks5-proxies-in-web-scraping) - [ Q What are private proxies and how are they used in scraping? ](https://scrapfly.io/blog/answers/what-are-private-proxies-compared-to-shared) ### curl [View all ](https://scrapfly.io/blog/answers) - [ Q How to Set User Agent With cURL? ](https://scrapfly.io/blog/answers/how-to-set-curl-user-agent) - [ Q How to Use cURL Config Files? ](https://scrapfly.io/blog/answers/how-to-set-curl-config-file) - [ Q How to Set cURL Authentication - Full Examples Guide ](https://scrapfly.io/blog/answers/how-to-set-authorization-with-curl-full-examples-guide) - [ Q How To Send Multiple cURL Requests in Parallel? ](https://scrapfly.io/blog/answers/how-to-send-multiple-curl-requests-in-parallel) - [ Q How to Send a HEAD Request With cURL? ](https://scrapfly.io/blog/answers/how-to-send-curl-head-requests) ### http [View all ](https://scrapfly.io/blog/answers) - [ Q How to Copy as cURL With Safari? ](https://scrapfly.io/blog/answers/how-to-copy-as-curl-with-safari) - [ Q How to Copy as cURL With Firefox? ](https://scrapfly.io/blog/answers/how-to-copy-as-curl-with-firefox) - [ Q How to Copy as cURL With Edge? ](https://scrapfly.io/blog/answers/how-to-copy-as-curl-with-edge) - [ Q How To Copy as cURL With Google Chrome? ](https://scrapfly.io/blog/answers/how-to-copy-as-curl-with-chrome) - [ Q How to Copy as cURL With Brave? ](https://scrapfly.io/blog/answers/how-to-copy-as-curl-with-brave) ### python [View all ](https://scrapfly.io/blog/answers) - [ Q How to scrape HTML table to Excel Spreadsheet (.xlsx)? ](https://scrapfly.io/blog/answers/html-table-to-xlsx-python-beautifulsoup) - [ Q How to handle popup dialogs in Playwright? ](https://scrapfly.io/blog/answers/how-to-click-on-alert-dialog-in-playwright) - [ Q How to scrape images from a website? ](https://scrapfly.io/blog/answers/how-to-scrape-images-from-website) - [ Q What are some ways to parse JSON datasets in Python? ](https://scrapfly.io/blog/answers/what-are-some-ways-to-parse-json-datasets-in-python) - [ Q How to select dictionary key recursively in Python? ](https://scrapfly.io/blog/answers/how-to-select-dictionary-key-recursively-in-python) ### tools [View all ](https://scrapfly.io/blog/answers) - [ Q How to edit Local Storage data using browser Devtools ](https://scrapfly.io/blog/answers/how-to-edit-local-storage-using-devtools) - [ Q How to edit cookies in Chrome devtools? ](https://scrapfly.io/blog/answers/how-to-edit-cookies-using-chrome-devtools) ### selenium [View all ](https://scrapfly.io/blog/answers) - [ Q How to click on cookie popups and modal alerts in Selenium? ](https://scrapfly.io/blog/answers/how-to-click-on-modal-alerts-like-cookie-pop-up-in-selenium) - [ Q How to handle popup dialogs in Selenium? ](https://scrapfly.io/blog/answers/how-to-click-on-alert-dialog-in-selenium) - [ Q How to scroll to the bottom of the page with Selenium? ](https://scrapfly.io/blog/answers/how-to-scroll-to-the-bottom-with-selenium) - [ Q How to capture background requests and responses in Selenium? ](https://scrapfly.io/blog/answers/how-to-capture-xhr-requests-selenium) - [ Q How to block resources in Selenium and Python? ](https://scrapfly.io/blog/answers/how-to-block-resources-in-selenium) ### puppeteer [View all ](https://scrapfly.io/blog/answers) - [ Q How to click on cookie popups and modal alerts in Puppeteer? ](https://scrapfly.io/blog/answers/how-to-click-on-modal-alerts-like-cookie-pop-up-in-puppeteer) - [ Q How to handle popup dialogs in Puppeteer? ](https://scrapfly.io/blog/answers/how-to-click-on-alert-dialog-in-puppeteer) - [ Q How to scroll to the bottom of the page with Puppeteer? ](https://scrapfly.io/blog/answers/how-to-scroll-to-the-bottom-with-puppeteer) - [ Q Getting started with Puppeteer Stealth ](https://scrapfly.io/blog/answers/how-to-use-puppeteer-stealth-what-does-it-do) ### playwright [View all ](https://scrapfly.io/blog/answers) - [ Q How to click on cookie popups and modal alerts in Playwright? ](https://scrapfly.io/blog/answers/how-to-click-on-modal-alerts-like-cookie-pop-up-in-playwright) - [ Q How to scroll to the bottom of the page with Playwright? ](https://scrapfly.io/blog/answers/how-to-scroll-to-the-bottom-with-playwright) - [ Q How to run Playwright in Jupyter notebooks? ](https://scrapfly.io/blog/answers/playwright-in-ipython) ### scrapy [View all ](https://scrapfly.io/blog/answers) - [ Q What are scrapy middlewares and how to use them? ](https://scrapfly.io/blog/answers/what-are-scrapy-middlewares-and-how-to-use-them) - [ Q What are scrapy pipelines and how to use them? ](https://scrapfly.io/blog/answers/what-are-scrapy-pipelines-and-how-to-use-them) - [ Q How to pass custom parameters to scrapy spiders? ](https://scrapfly.io/blog/answers/how-to-pass-parameters-to-scrapy-spiders-cli) - [ Q What are scrapy Item and ItemLoader objects and how to use them? ](https://scrapfly.io/blog/answers/what-are-scrapy-items-and-itemloaders) - [ Q How to pass data from start\_requests to parse callbacks in scrapy? ](https://scrapfly.io/blog/answers/how-to-pass-data-from-start-request-to-callbacks-scrapy) ### xpath [View all ](https://scrapfly.io/blog/answers) - [ Q How to select elements by attribute value in XPath? ](https://scrapfly.io/blog/answers/how-to-select-elements-by-attribute-value) - [ Q How to select sibling elements in XPath? ](https://scrapfly.io/blog/answers/how-to-select-sibling-elements-using-xpath) - [ Q How to select elements of a specific position in XPath? ](https://scrapfly.io/blog/answers/how-to-select-elements-of-specific-position-in-xpath) - [ Q How to select any element using wildcard in XPath? ](https://scrapfly.io/blog/answers/how-to-select-elements-of-any-name-using-wildcards-in-xpath) - [ Q How to select elements by ID in XPath? ](https://scrapfly.io/blog/answers/how-to-select-elements-by-id-in-xpath) ### headless-browser [View all ](https://scrapfly.io/blog/answers) - [ Q How to use headless browsers with scrapy? ](https://scrapfly.io/blog/answers/how-to-use-headless-browsers-with-scrapy) ### data-parsing [View all ](https://scrapfly.io/blog/answers) - [ Q How to select last element in XPath? ](https://scrapfly.io/blog/answers/how-to-select-last-element-in-xpath) - [ Q How to select all elements between two elements in XPath? ](https://scrapfly.io/blog/answers/how-to-select-all-elements-between-two-known-elements-in-xpath) ### requests [View all ](https://scrapfly.io/blog/answers) - [ Q How to configure Python requests to use a proxy? ](https://scrapfly.io/blog/answers/python-requests-proxy-intro) ### blocking [View all ](https://scrapfly.io/blog/answers) - [ Q Web scraping - what is HTTP 520 status code? ](https://scrapfly.io/blog/answers/520-status-code) - [ Q Web scraping - what is HTTP 503 status code? ](https://scrapfly.io/blog/answers/503-status-code) - [ Q Web scraping - what is HTTP 499 status code? ](https://scrapfly.io/blog/answers/499-status-code) - [ Q What is 444 status code and how to avoid it? ](https://scrapfly.io/blog/answers/444-status-code) - [ Q Web scraping - what is HTTP 429 status code? ](https://scrapfly.io/blog/answers/429-status-code) ## Ready to scale your web scraping? Anti-bot bypass, browser rendering, and rotating proxies, all in one API. [ Try Scrapfly for FREE ](https://scrapfly.io/register) [ View Documentation ](https://scrapfly.io/docs/onboarding)