# Scrapfly Blog Tutorials, guides, and insights on web scraping, data extraction, and automation 326 articles Search articles [ ](https://scrapfly.io/blog/feed.xml "Subscribe via RSS") [All](https://scrapfly.io/blog) [ai](https://scrapfly.io/blog/tag/ai) [anti-bot](https://scrapfly.io/blog/tag/anti-bot) [api](https://scrapfly.io/blog/tag/api) [beautifulsoup](https://scrapfly.io/blog/tag/beautifulsoup) [blocking](https://scrapfly.io/blog/tag/blocking) [cloud-browser](https://scrapfly.io/blog/tag/cloud-browser) [crawling](https://scrapfly.io/blog/tag/crawling) [css-selectors](https://scrapfly.io/blog/tag/css-selectors) [curl](https://scrapfly.io/blog/tag/curl) [data-parsing](https://scrapfly.io/blog/tag/data-parsing) [ecommerce](https://scrapfly.io/blog/tag/ecommerce) [fashion](https://scrapfly.io/blog/tag/fashion) [frameworks](https://scrapfly.io/blog/tag/frameworks) [golang](https://scrapfly.io/blog/tag/golang) [graphql](https://scrapfly.io/blog/tag/graphql) [headless-browser](https://scrapfly.io/blog/tag/headless-browser) [hidden-api](https://scrapfly.io/blog/tag/hidden-api) [http](https://scrapfly.io/blog/tag/http) [httpx](https://scrapfly.io/blog/tag/httpx) [java](https://scrapfly.io/blog/tag/java) [javascript](https://scrapfly.io/blog/tag/javascript) [jupyter](https://scrapfly.io/blog/tag/jupyter) [nodejs](https://scrapfly.io/blog/tag/nodejs) [parsel](https://scrapfly.io/blog/tag/parsel) [php](https://scrapfly.io/blog/tag/php) [playwright](https://scrapfly.io/blog/tag/playwright) [project](https://scrapfly.io/blog/tag/project) [proxies](https://scrapfly.io/blog/tag/proxies) [puppeteer](https://scrapfly.io/blog/tag/puppeteer) [python](https://scrapfly.io/blog/tag/python) [r](https://scrapfly.io/blog/tag/r) [real-estate](https://scrapfly.io/blog/tag/real-estate) [requests](https://scrapfly.io/blog/tag/requests) [ruby](https://scrapfly.io/blog/tag/ruby) [scaling](https://scrapfly.io/blog/tag/scaling) [scrapeguide](https://scrapfly.io/blog/tag/scrapeguide) [scrapy](https://scrapfly.io/blog/tag/scrapy) [screenshots](https://scrapfly.io/blog/tag/screenshots) [selenium](https://scrapfly.io/blog/tag/selenium) [seo](https://scrapfly.io/blog/tag/seo) [tools](https://scrapfly.io/blog/tag/tools) [typescript](https://scrapfly.io/blog/tag/typescript) [web-scraping](https://scrapfly.io/blog/tag/web-scraping) [xpath](https://scrapfly.io/blog/tag/xpath) ## // Articles 12 posts [ Latest ai data-parsing ### 13 Best Web Data Sources for AI and LLM Training in 2026 A tiered index of open and public web data sources for AI and LLM training, from raw crawls to structured dumps to building your own. Aug 05, 2026 21 min read ](https://scrapfly.io/blog/posts/best-web-data-sources-for-ai-training) [ python scrapeguide ### How to Scrape Google Jobs in 2026 Learn how to scrape Google Jobs with Python and Scrapfly. Build a job-search URL, render and scroll the results, and ext... Aug 05, 2026 15 min read ](https://scrapfly.io/blog/posts/how-to-scrape-google-jobs) [ python hidden-api scrapeguide ### How to Scrape Google Play App Reviews and Data Scrape Google Play app metadata, ratings, and the full review set with Python, past the few-hundred-review ceiling the f... Aug 03, 2026 15 min read ](https://scrapfly.io/blog/posts/how-to-scrape-google-play-app-reviews-and-data) [ proxies tools ### 4 Best Open-Source Proxy Scrapers and Checkers in 2026 A maintenance-filtered ranking of the four open-source proxy scrapers and checkers still worth running in 2026, the stal... Aug 03, 2026 15 min read ](https://scrapfly.io/blog/posts/best-proxy-scrapers) [ api headless-browser ai ### 7 Best AI Browser Agents for Automation and Scraping in 2026 A ranked look at the 7 best AI browser agents in 2026, judged on whether they stay unblocked and stable in production, n... Aug 03, 2026 21 min read ](https://scrapfly.io/blog/posts/best-ai-browser-agents) [ python scrapeguide ### How to Scrape Marriott Hotel Prices and Availability Learn how to scrape Marriott hotel prices and room availability with Python in 2026, including search inputs, the JSON b... Aug 03, 2026 20 min read ](https://scrapfly.io/blog/posts/how-to-scrape-marriott) [ python scrapeguide ### How to Scrape Kayak Flights in 2026 A step-by-step guide to scraping Kayak flight search results with Python and the Scrapfly SDK. Covers URL construction, ... Jul 27, 2026 18 min read ](https://scrapfly.io/blog/posts/how-to-scrape-kayak) [ python scrapeguide ### How to Scrape RS-Online (rs-online.com) in 2026 How to scrape RS-Online's North American listings and product pages for pricing, stock, specifications, and datasheet li... Jul 24, 2026 19 min read ](https://scrapfly.io/blog/posts/how-to-scrape-rs-online) [ python playwright ai ### Browser Use vs Playwright: Which Should You Use? Compare Browser Use and Playwright for web scraping. Learn the architectural relationship, speed and cost tradeoffs, sil... Jul 24, 2026 15 min read ](https://scrapfly.io/blog/posts/browser-use-vs-playwright) [ blocking ### How to Bypass AWS WAF when Web Scraping in 2026 Learn how AWS WAF Bot Control detects web scrapers across five layers, how to identify an AWS WAF block from its respons... Jul 24, 2026 14 min read ](https://scrapfly.io/blog/posts/how-to-bypass-aws-waf-when-web-scraping) [ python scrapeguide ### 5 Best Open-Source Facebook Marketplace Scrapers on GitHub in 2026 A dated, freshness-first roundup of the open-source Facebook Marketplace scrapers on GitHub worth cloning in 2026, and t... Jul 24, 2026 19 min read ](https://scrapfly.io/blog/posts/best-facebook-marketplace-scrapers-github) [ python web-scraping scrapeguide ### Web Scraping Best Practices: 11 Rules for Reliable and Respectful Scraping in 2026 The 11 practices that separate a scraper from a data pipeline, sequenced in the order a real scraping project meets them... Jul 21, 2026 22 min read ](https://scrapfly.io/blog/posts/web-scraping-best-practices) 1 [2](https://scrapfly.io/blog?page=2) [3](https://scrapfly.io/blog?page=3) … [28](https://scrapfly.io/blog?page=28) [ ](https://scrapfly.io/blog?page=2) Page 1 of 28 · 326 articles ## ? Quick Answers ### nodejs [View all ](https://scrapfly.io/blog/answers) - [ Q How to take screenshots in NodeJS? ](https://scrapfly.io/blog/answers/how-to-take-screenshots-nodejs) ### css-selectors [View all ](https://scrapfly.io/blog/answers) - [ Q How to use CSS Selectors in Nim ? ](https://scrapfly.io/blog/answers/how-to-use-css-selectors-in-nim) - [ Q How to select elements by attribute using CSS selectors? ](https://scrapfly.io/blog/answers/how-to-select-elements-by-attribute-containing-value-css-selectors) - [ Q How to select elements by class using CSS selectors? ](https://scrapfly.io/blog/answers/how-to-select-elements-by-class-css-selectors) - [ Q How to select elements by ID using CSS selectors? ](https://scrapfly.io/blog/answers/how-to-select-elements-by-id-css-selectors) - [ Q How to select following siblings using CSS selectors? ](https://scrapfly.io/blog/answers/how-to-select-following-sibling-element-css-selectors) ### proxies [View all ](https://scrapfly.io/blog/answers) - [ Q How to Solve the cURL (60) Error When Using Proxy? ](https://scrapfly.io/blog/answers/how-to-solve-the-curl-60-error-when-proxy) - [ Q How To Use Proxy With cURL? ](https://scrapfly.io/blog/answers/how-to-use-proxy-with-curl) - [ Q What is The cURL (28) Error, Couldn't connect to server? ](https://scrapfly.io/blog/answers/what-is-the-curl-28-error) - [ Q Mobile vs Residential Proxies - which to choose for scraping? ](https://scrapfly.io/blog/answers/mobile-vs-residential-proxies-whats-the-difference) - [ Q What are private proxies and how are they used in scraping? ](https://scrapfly.io/blog/answers/what-are-private-proxies-compared-to-shared) ### curl [View all ](https://scrapfly.io/blog/answers) - [ Q How To Download a File With cURL? ](https://scrapfly.io/blog/answers/how-to-download-file-curl) - [ Q How to Follow Redirects In cURL? ](https://scrapfly.io/blog/answers/how-to-follow-redirects-in-curl) - [ Q How To Send cURL POST Requests? ](https://scrapfly.io/blog/answers/how-to-send-a-post-request-using-curl) - [ Q How to Send a HEAD Request With cURL? ](https://scrapfly.io/blog/answers/how-to-send-curl-head-requests) - [ Q How To Send Multiple cURL Requests in Parallel? ](https://scrapfly.io/blog/answers/how-to-send-multiple-curl-requests-in-parallel) ### http [View all ](https://scrapfly.io/blog/answers) - [ Q How to Copy as cURL With Brave? ](https://scrapfly.io/blog/answers/how-to-copy-as-curl-with-brave) - [ Q How To Copy as cURL With Google Chrome? ](https://scrapfly.io/blog/answers/how-to-copy-as-curl-with-chrome) - [ Q How to Copy as cURL With Edge? ](https://scrapfly.io/blog/answers/how-to-copy-as-curl-with-edge) - [ Q How to Copy as cURL With Firefox? ](https://scrapfly.io/blog/answers/how-to-copy-as-curl-with-firefox) - [ Q How to Copy as cURL With Safari? ](https://scrapfly.io/blog/answers/how-to-copy-as-curl-with-safari) ### tools [View all ](https://scrapfly.io/blog/answers) - [ Q How to edit Local Storage data using browser Devtools ](https://scrapfly.io/blog/answers/how-to-edit-local-storage-using-devtools) - [ Q How to edit cookies in Chrome devtools? ](https://scrapfly.io/blog/answers/how-to-edit-cookies-using-chrome-devtools) ### python [View all ](https://scrapfly.io/blog/answers) - [ Q How to scrape HTML table to Excel Spreadsheet (.xlsx)? ](https://scrapfly.io/blog/answers/html-table-to-xlsx-python-beautifulsoup) - [ Q How to handle popup dialogs in Playwright? ](https://scrapfly.io/blog/answers/how-to-click-on-alert-dialog-in-playwright) - [ Q How to scrape images from a website? ](https://scrapfly.io/blog/answers/how-to-scrape-images-from-website) - [ Q How to check if element exists in Playwright? ](https://scrapfly.io/blog/answers/how-to-check-for-element-in-playwright) - [ Q How to select dictionary key recursively in Python? ](https://scrapfly.io/blog/answers/how-to-select-dictionary-key-recursively-in-python) ### puppeteer [View all ](https://scrapfly.io/blog/answers) - [ Q How to handle popup dialogs in Puppeteer? ](https://scrapfly.io/blog/answers/how-to-click-on-alert-dialog-in-puppeteer) - [ Q How to click on cookie popups and modal alerts in Puppeteer? ](https://scrapfly.io/blog/answers/how-to-click-on-modal-alerts-like-cookie-pop-up-in-puppeteer) - [ Q How to scroll to the bottom of the page with Puppeteer? ](https://scrapfly.io/blog/answers/how-to-scroll-to-the-bottom-with-puppeteer) - [ Q Getting started with Puppeteer Stealth ](https://scrapfly.io/blog/answers/how-to-use-puppeteer-stealth-what-does-it-do) ### selenium [View all ](https://scrapfly.io/blog/answers) - [ Q How to handle popup dialogs in Selenium? ](https://scrapfly.io/blog/answers/how-to-click-on-alert-dialog-in-selenium) - [ Q How to click on cookie popups and modal alerts in Selenium? ](https://scrapfly.io/blog/answers/how-to-click-on-modal-alerts-like-cookie-pop-up-in-selenium) - [ Q How to block resources in Selenium and Python? ](https://scrapfly.io/blog/answers/how-to-block-resources-in-selenium) - [ Q How to capture background requests and responses in Selenium? ](https://scrapfly.io/blog/answers/how-to-capture-xhr-requests-selenium) - [ Q How to scroll to the bottom of the page with Selenium? ](https://scrapfly.io/blog/answers/how-to-scroll-to-the-bottom-with-selenium) ### playwright [View all ](https://scrapfly.io/blog/answers) - [ Q How to click on cookie popups and modal alerts in Playwright? ](https://scrapfly.io/blog/answers/how-to-click-on-modal-alerts-like-cookie-pop-up-in-playwright) - [ Q How to scroll to the bottom of the page with Playwright? ](https://scrapfly.io/blog/answers/how-to-scroll-to-the-bottom-with-playwright) - [ Q How to run Playwright in Jupyter notebooks? ](https://scrapfly.io/blog/answers/playwright-in-ipython) ### xpath [View all ](https://scrapfly.io/blog/answers) - [ Q How to select elements by attribute value in XPath? ](https://scrapfly.io/blog/answers/how-to-select-elements-by-attribute-value) - [ Q How to count selections in XPath and why? ](https://scrapfly.io/blog/answers/how-to-count-selectors-in-xpath-and-why) - [ Q How to get the name of an HTML element in XPath? ](https://scrapfly.io/blog/answers/how-to-get-name-of-selected-element-in-xpath) - [ Q How to join values using XPath concat? ](https://scrapfly.io/blog/answers/how-to-join-values-in-xpath) - [ Q How to reverse expressions in XPath? ](https://scrapfly.io/blog/answers/how-to-reverse-expression-in-xpath) ### scrapy [View all ](https://scrapfly.io/blog/answers) - [ Q What are scrapy middlewares and how to use them? ](https://scrapfly.io/blog/answers/what-are-scrapy-middlewares-and-how-to-use-them) - [ Q What are scrapy pipelines and how to use them? ](https://scrapfly.io/blog/answers/what-are-scrapy-pipelines-and-how-to-use-them) - [ Q How to pass custom parameters to scrapy spiders? ](https://scrapfly.io/blog/answers/how-to-pass-parameters-to-scrapy-spiders-cli) - [ Q How to pass data between scrapy callbacks in Scrapy? ](https://scrapfly.io/blog/answers/how-to-pass-data-between-scrapy-callbacks) - [ Q How to pass data from start\_requests to parse callbacks in scrapy? ](https://scrapfly.io/blog/answers/how-to-pass-data-from-start-request-to-callbacks-scrapy) ### headless-browser [View all ](https://scrapfly.io/blog/answers) - [ Q How to use headless browsers with scrapy? ](https://scrapfly.io/blog/answers/how-to-use-headless-browsers-with-scrapy) ### data-parsing [View all ](https://scrapfly.io/blog/answers) - [ Q How to select last element in XPath? ](https://scrapfly.io/blog/answers/how-to-select-last-element-in-xpath) - [ Q How to select all elements between two elements in XPath? ](https://scrapfly.io/blog/answers/how-to-select-all-elements-between-two-known-elements-in-xpath) ### requests [View all ](https://scrapfly.io/blog/answers) - [ Q How to configure Python requests to use a proxy? ](https://scrapfly.io/blog/answers/python-requests-proxy-intro) ### blocking [View all ](https://scrapfly.io/blog/answers) - [ Q Web scraping - what is HTTP 403 status code? ](https://scrapfly.io/blog/answers/403-status-code) - [ Q Web scraping - what is HTTP 429 status code? ](https://scrapfly.io/blog/answers/429-status-code) - [ Q What is 444 status code and how to avoid it? ](https://scrapfly.io/blog/answers/444-status-code) - [ Q Web scraping - what is HTTP 499 status code? ](https://scrapfly.io/blog/answers/499-status-code) - [ Q Web scraping - what is HTTP 503 status code? ](https://scrapfly.io/blog/answers/503-status-code) ## Ready to scale your web scraping? Anti-bot bypass, browser rendering, and rotating proxies, all in one API. [ Try Scrapfly for FREE ](https://scrapfly.io/register) [ View Documentation ](https://scrapfly.io/docs/onboarding)