# Scrapfly Blog Tutorials, guides, and insights on web scraping, data extraction, and automation 391 articles Search articles [ ](https://scrapfly.io/blog/feed.xml "Subscribe via RSS") [All](https://scrapfly.io/blog) [ai](https://scrapfly.io/blog/tag/ai) [anti-bot](https://scrapfly.io/blog/tag/anti-bot) [api](https://scrapfly.io/blog/tag/api) [beautifulsoup](https://scrapfly.io/blog/tag/beautifulsoup) [blocking](https://scrapfly.io/blog/tag/blocking) [cloud-browser](https://scrapfly.io/blog/tag/cloud-browser) [crawling](https://scrapfly.io/blog/tag/crawling) [css-selectors](https://scrapfly.io/blog/tag/css-selectors) [curl](https://scrapfly.io/blog/tag/curl) [data](https://scrapfly.io/blog/tag/data) [data-parsing](https://scrapfly.io/blog/tag/data-parsing) [debugging](https://scrapfly.io/blog/tag/debugging) [ecommerce](https://scrapfly.io/blog/tag/ecommerce) [elixir](https://scrapfly.io/blog/tag/elixir) [fashion](https://scrapfly.io/blog/tag/fashion) [frameworks](https://scrapfly.io/blog/tag/frameworks) [golang](https://scrapfly.io/blog/tag/golang) [graphql](https://scrapfly.io/blog/tag/graphql) [headless-browser](https://scrapfly.io/blog/tag/headless-browser) [hidden-api](https://scrapfly.io/blog/tag/hidden-api) [http](https://scrapfly.io/blog/tag/http) [httpx](https://scrapfly.io/blog/tag/httpx) [java](https://scrapfly.io/blog/tag/java) [javascript](https://scrapfly.io/blog/tag/javascript) [nodejs](https://scrapfly.io/blog/tag/nodejs) [parsel](https://scrapfly.io/blog/tag/parsel) [php](https://scrapfly.io/blog/tag/php) [playwright](https://scrapfly.io/blog/tag/playwright) [project](https://scrapfly.io/blog/tag/project) [proxies](https://scrapfly.io/blog/tag/proxies) [puppeteer](https://scrapfly.io/blog/tag/puppeteer) [python](https://scrapfly.io/blog/tag/python) [r](https://scrapfly.io/blog/tag/r) [real-estate](https://scrapfly.io/blog/tag/real-estate) [requests](https://scrapfly.io/blog/tag/requests) [ruby](https://scrapfly.io/blog/tag/ruby) [rust](https://scrapfly.io/blog/tag/rust) [scaling](https://scrapfly.io/blog/tag/scaling) [scrapeguide](https://scrapfly.io/blog/tag/scrapeguide) [scrapy](https://scrapfly.io/blog/tag/scrapy) [screenshots](https://scrapfly.io/blog/tag/screenshots) [selenium](https://scrapfly.io/blog/tag/selenium) [seo](https://scrapfly.io/blog/tag/seo) [tools](https://scrapfly.io/blog/tag/tools) [typescript](https://scrapfly.io/blog/tag/typescript) [web-scraping](https://scrapfly.io/blog/tag/web-scraping) [wget](https://scrapfly.io/blog/tag/wget) [xpath](https://scrapfly.io/blog/tag/xpath) ## // Articles 12 posts [ Latest nodejs proxies puppeteer ### How to Use Proxies with Puppeteer: Authentication, Rotation, and Per-Page Proxies A tested walkthrough of Puppeteer proxies covering page.authenticate, per-context proxies, rotation, SOCKS5 with proxy-chain, Chrome error codes and geo-matched browsers. Nov 12, 2026 29 min read ](https://scrapfly.io/blog/posts/how-to-use-proxies-with-puppeteer) [ headless-browser selenium puppeteer ### Puppeteer vs Selenium for Web Scraping: Tested on the Same Chrome A tested comparison of Puppeteer and Selenium for scraping: architecture, measured speed, language and browser support, ... Nov 12, 2026 26 min read ](https://scrapfly.io/blog/posts/puppeteer-vs-selenium) [ ai cloud-browser ### AI Browser Agent Security Risks: A Developer's Guide to Running Agents Safely Understand browser-agent risks, from indirect prompt injection to inherited sessions, with controlled browser demos and ... Nov 11, 2026 24 min read ](https://scrapfly.io/blog/posts/browser-agent-security-risks) [ python playwright proxies ### How to Use Proxies with Playwright: Authentication, Rotation, and Per-Context Proxies A tested walkthrough of Playwright proxies covering launch and context options, authentication, rotation, SOCKS5 limits,... Nov 10, 2026 27 min read ](https://scrapfly.io/blog/posts/how-to-use-proxies-with-playwright) [ python hidden-api scrapeguide ### How to Scrape Apple App Store Reviews and App Data Scrape Apple App Store app metadata and customer reviews with Python using iTunes Lookup, the customer-reviews JSON endp... Nov 10, 2026 24 min read ](https://scrapfly.io/blog/posts/how-to-scrape-app-store) [ python api ecommerce ### Google Shopping API: What Google Offers and How to Get Shopping Results Data An explainer on Google Shopping data access in 2026, separating the merchant-only Merchant API from collecting public Sh... Nov 10, 2026 19 min read ](https://scrapfly.io/blog/posts/google-shopping-api-and-alternatives) [ ai data-parsing ### 13 Best Web Data Sources for AI and LLM Training in 2026 A tiered index of open and public web data sources for AI and LLM training, from raw crawls to structured dumps to build... Nov 09, 2026 21 min read ](https://scrapfly.io/blog/posts/best-web-data-sources-for-ai-training) [ http python headless-browser ### 5 Best Scrapy Alternatives for Web Scraping in 2026 Five Scrapy alternatives by job: Crawlee for JS/TS crawlers, Playwright for browsers, Beautiful Soup with HTTPX for smal... Nov 09, 2026 22 min read ](https://scrapfly.io/blog/posts/best-scrapy-alternatives-for-web-scraping) [ python headless-browser selenium ### Selenium Alternatives for Web Scraping: What to Use in 2026 Choose a Selenium replacement for interactions, stealth, parsing or crawling. Includes tested Python examples and the li... Nov 09, 2026 23 min read ](https://scrapfly.io/blog/posts/best-selenium-alternatives-for-web-scraping) [ headless-browser proxies cloud-browser ### Camoufox for Web Scraping: Setup, Proxies and Servers A tested guide to running Camoufox for web scraping: Python setup, proxy and GeoIP configuration, remote servers, known ... Nov 09, 2026 35 min read ](https://scrapfly.io/blog/posts/camoufox-for-web-scraping) [ http tools data-parsing ### 7 Best Java Web Scraping Libraries in 2026 Seven Java web scraping libraries for fetching, static HTML, browser automation, and large crawl workloads, plus the fix... Nov 05, 2026 15 min read ](https://scrapfly.io/blog/posts/best-java-web-scraping-libraries) [ tools web-scraping elixir ### 4 Top Elixir Web Scraping Libraries in 2026 Four current Elixir libraries for HTTP retrieval, HTML parsing, crawl orchestration, and browser control, plus managed p... Nov 03, 2026 8 min read ](https://scrapfly.io/blog/posts/best-elixir-web-scraping-libraries) 1 [2](https://scrapfly.io/blog?page=2) [3](https://scrapfly.io/blog?page=3) … [33](https://scrapfly.io/blog?page=33) [ ](https://scrapfly.io/blog?page=2) Page 1 of 33 · 391 articles ## ? Quick Answers ### http [View all ](https://scrapfly.io/blog/answers) - [ Q How to Persist Sessions Across Scrapy Requests ](https://scrapfly.io/blog/answers/persist-sessions-across-scrapy-requests) - [ Q How to Copy as cURL With Brave? ](https://scrapfly.io/blog/answers/how-to-copy-as-curl-with-brave) - [ Q How To Copy as cURL With Google Chrome? ](https://scrapfly.io/blog/answers/how-to-copy-as-curl-with-chrome) - [ Q How to Copy as cURL With Edge? ](https://scrapfly.io/blog/answers/how-to-copy-as-curl-with-edge) - [ Q How to Copy as cURL With Firefox? ](https://scrapfly.io/blog/answers/how-to-copy-as-curl-with-firefox) ### nodejs [View all ](https://scrapfly.io/blog/answers) - [ Q How to take screenshots in NodeJS? ](https://scrapfly.io/blog/answers/how-to-take-screenshots-nodejs) ### css-selectors [View all ](https://scrapfly.io/blog/answers) - [ Q How to use CSS Selectors in Nim ? ](https://scrapfly.io/blog/answers/how-to-use-css-selectors-in-nim) - [ Q How to select elements by attribute using CSS selectors? ](https://scrapfly.io/blog/answers/how-to-select-elements-by-attribute-containing-value-css-selectors) - [ Q How to select elements by class using CSS selectors? ](https://scrapfly.io/blog/answers/how-to-select-elements-by-class-css-selectors) - [ Q How to select elements by ID using CSS selectors? ](https://scrapfly.io/blog/answers/how-to-select-elements-by-id-css-selectors) - [ Q How to select following siblings using CSS selectors? ](https://scrapfly.io/blog/answers/how-to-select-following-sibling-element-css-selectors) ### proxies [View all ](https://scrapfly.io/blog/answers) - [ Q How to Solve the cURL (60) Error When Using Proxy? ](https://scrapfly.io/blog/answers/how-to-solve-the-curl-60-error-when-proxy) - [ Q How To Use Proxy With cURL? ](https://scrapfly.io/blog/answers/how-to-use-proxy-with-curl) - [ Q What is The cURL (28) Error, Couldn't connect to server? ](https://scrapfly.io/blog/answers/what-is-the-curl-28-error) - [ Q Mobile vs Residential Proxies - which to choose for scraping? ](https://scrapfly.io/blog/answers/mobile-vs-residential-proxies-whats-the-difference) - [ Q What are private proxies and how are they used in scraping? ](https://scrapfly.io/blog/answers/what-are-private-proxies-compared-to-shared) ### curl [View all ](https://scrapfly.io/blog/answers) - [ Q How To Download a File With cURL? ](https://scrapfly.io/blog/answers/how-to-download-file-curl) - [ Q How to Follow Redirects In cURL? ](https://scrapfly.io/blog/answers/how-to-follow-redirects-in-curl) - [ Q How To Send cURL POST Requests? ](https://scrapfly.io/blog/answers/how-to-send-a-post-request-using-curl) - [ Q How to Send a HEAD Request With cURL? ](https://scrapfly.io/blog/answers/how-to-send-curl-head-requests) - [ Q How To Send Multiple cURL Requests in Parallel? ](https://scrapfly.io/blog/answers/how-to-send-multiple-curl-requests-in-parallel) ### tools [View all ](https://scrapfly.io/blog/answers) - [ Q How to edit Local Storage data using browser Devtools ](https://scrapfly.io/blog/answers/how-to-edit-local-storage-using-devtools) - [ Q How to edit cookies in Chrome devtools? ](https://scrapfly.io/blog/answers/how-to-edit-cookies-using-chrome-devtools) ### python [View all ](https://scrapfly.io/blog/answers) - [ Q How to scrape HTML table to Excel Spreadsheet (.xlsx)? ](https://scrapfly.io/blog/answers/html-table-to-xlsx-python-beautifulsoup) - [ Q How to handle popup dialogs in Playwright? ](https://scrapfly.io/blog/answers/how-to-click-on-alert-dialog-in-playwright) - [ Q How to scrape images from a website? ](https://scrapfly.io/blog/answers/how-to-scrape-images-from-website) - [ Q How to check if element exists in Playwright? ](https://scrapfly.io/blog/answers/how-to-check-for-element-in-playwright) - [ Q How to select dictionary key recursively in Python? ](https://scrapfly.io/blog/answers/how-to-select-dictionary-key-recursively-in-python) ### puppeteer [View all ](https://scrapfly.io/blog/answers) - [ Q How to handle popup dialogs in Puppeteer? ](https://scrapfly.io/blog/answers/how-to-click-on-alert-dialog-in-puppeteer) - [ Q How to click on cookie popups and modal alerts in Puppeteer? ](https://scrapfly.io/blog/answers/how-to-click-on-modal-alerts-like-cookie-pop-up-in-puppeteer) - [ Q How to scroll to the bottom of the page with Puppeteer? ](https://scrapfly.io/blog/answers/how-to-scroll-to-the-bottom-with-puppeteer) - [ Q Getting started with Puppeteer Stealth ](https://scrapfly.io/blog/answers/how-to-use-puppeteer-stealth-what-does-it-do) ### selenium [View all ](https://scrapfly.io/blog/answers) - [ Q How to handle popup dialogs in Selenium? ](https://scrapfly.io/blog/answers/how-to-click-on-alert-dialog-in-selenium) - [ Q How to click on cookie popups and modal alerts in Selenium? ](https://scrapfly.io/blog/answers/how-to-click-on-modal-alerts-like-cookie-pop-up-in-selenium) - [ Q How to block resources in Selenium and Python? ](https://scrapfly.io/blog/answers/how-to-block-resources-in-selenium) - [ Q How to capture background requests and responses in Selenium? ](https://scrapfly.io/blog/answers/how-to-capture-xhr-requests-selenium) - [ Q How to scroll to the bottom of the page with Selenium? ](https://scrapfly.io/blog/answers/how-to-scroll-to-the-bottom-with-selenium) ### playwright [View all ](https://scrapfly.io/blog/answers) - [ Q How to click on cookie popups and modal alerts in Playwright? ](https://scrapfly.io/blog/answers/how-to-click-on-modal-alerts-like-cookie-pop-up-in-playwright) - [ Q How to scroll to the bottom of the page with Playwright? ](https://scrapfly.io/blog/answers/how-to-scroll-to-the-bottom-with-playwright) - [ Q How to run Playwright in Jupyter notebooks? ](https://scrapfly.io/blog/answers/playwright-in-ipython) ### xpath [View all ](https://scrapfly.io/blog/answers) - [ Q How to select elements by attribute value in XPath? ](https://scrapfly.io/blog/answers/how-to-select-elements-by-attribute-value) - [ Q How to count selections in XPath and why? ](https://scrapfly.io/blog/answers/how-to-count-selectors-in-xpath-and-why) - [ Q How to get the name of an HTML element in XPath? ](https://scrapfly.io/blog/answers/how-to-get-name-of-selected-element-in-xpath) - [ Q How to join values using XPath concat? ](https://scrapfly.io/blog/answers/how-to-join-values-in-xpath) - [ Q How to reverse expressions in XPath? ](https://scrapfly.io/blog/answers/how-to-reverse-expression-in-xpath) ### scrapy [View all ](https://scrapfly.io/blog/answers) - [ Q What are scrapy middlewares and how to use them? ](https://scrapfly.io/blog/answers/what-are-scrapy-middlewares-and-how-to-use-them) - [ Q What are scrapy pipelines and how to use them? ](https://scrapfly.io/blog/answers/what-are-scrapy-pipelines-and-how-to-use-them) - [ Q How to pass custom parameters to scrapy spiders? ](https://scrapfly.io/blog/answers/how-to-pass-parameters-to-scrapy-spiders-cli) - [ Q How to pass data between scrapy callbacks in Scrapy? ](https://scrapfly.io/blog/answers/how-to-pass-data-between-scrapy-callbacks) - [ Q How to pass data from start\_requests to parse callbacks in scrapy? ](https://scrapfly.io/blog/answers/how-to-pass-data-from-start-request-to-callbacks-scrapy) ### headless-browser [View all ](https://scrapfly.io/blog/answers) - [ Q How to use headless browsers with scrapy? ](https://scrapfly.io/blog/answers/how-to-use-headless-browsers-with-scrapy) ### data-parsing [View all ](https://scrapfly.io/blog/answers) - [ Q How to select last element in XPath? ](https://scrapfly.io/blog/answers/how-to-select-last-element-in-xpath) - [ Q How to select all elements between two elements in XPath? ](https://scrapfly.io/blog/answers/how-to-select-all-elements-between-two-known-elements-in-xpath) ### requests [View all ](https://scrapfly.io/blog/answers) - [ Q How to configure Python requests to use a proxy? ](https://scrapfly.io/blog/answers/python-requests-proxy-intro) ### blocking [View all ](https://scrapfly.io/blog/answers) - [ Q Web scraping - what is HTTP 403 status code? ](https://scrapfly.io/blog/answers/403-status-code) - [ Q Web scraping - what is HTTP 429 status code? ](https://scrapfly.io/blog/answers/429-status-code) - [ Q What is 444 status code and how to avoid it? ](https://scrapfly.io/blog/answers/444-status-code) - [ Q Web scraping - what is HTTP 499 status code? ](https://scrapfly.io/blog/answers/499-status-code) - [ Q Web scraping - what is HTTP 503 status code? ](https://scrapfly.io/blog/answers/503-status-code) ## Ready to scale your web scraping? Anti-bot bypass, browser rendering, and rotating proxies, all in one API. [ Try Scrapfly for FREE ](https://scrapfly.io/register) [ View Documentation ](https://scrapfly.io/docs/onboarding)