# Scrapfly Blog Tutorials, guides, and insights on web scraping, data extraction, and automation 358 articles Search articles [ ](https://scrapfly.io/blog/feed.xml "Subscribe via RSS") [All](https://scrapfly.io/blog) [ai](https://scrapfly.io/blog/tag/ai) [anti-bot](https://scrapfly.io/blog/tag/anti-bot) [api](https://scrapfly.io/blog/tag/api) [beautifulsoup](https://scrapfly.io/blog/tag/beautifulsoup) [blocking](https://scrapfly.io/blog/tag/blocking) [cloud-browser](https://scrapfly.io/blog/tag/cloud-browser) [crawling](https://scrapfly.io/blog/tag/crawling) [css-selectors](https://scrapfly.io/blog/tag/css-selectors) [curl](https://scrapfly.io/blog/tag/curl) [data-parsing](https://scrapfly.io/blog/tag/data-parsing) [debugging](https://scrapfly.io/blog/tag/debugging) [ecommerce](https://scrapfly.io/blog/tag/ecommerce) [fashion](https://scrapfly.io/blog/tag/fashion) [frameworks](https://scrapfly.io/blog/tag/frameworks) [golang](https://scrapfly.io/blog/tag/golang) [graphql](https://scrapfly.io/blog/tag/graphql) [headless-browser](https://scrapfly.io/blog/tag/headless-browser) [hidden-api](https://scrapfly.io/blog/tag/hidden-api) [http](https://scrapfly.io/blog/tag/http) [httpx](https://scrapfly.io/blog/tag/httpx) [java](https://scrapfly.io/blog/tag/java) [javascript](https://scrapfly.io/blog/tag/javascript) [jupyter](https://scrapfly.io/blog/tag/jupyter) [nodejs](https://scrapfly.io/blog/tag/nodejs) [parsel](https://scrapfly.io/blog/tag/parsel) [php](https://scrapfly.io/blog/tag/php) [playwright](https://scrapfly.io/blog/tag/playwright) [project](https://scrapfly.io/blog/tag/project) [proxies](https://scrapfly.io/blog/tag/proxies) [puppeteer](https://scrapfly.io/blog/tag/puppeteer) [python](https://scrapfly.io/blog/tag/python) [r](https://scrapfly.io/blog/tag/r) [real-estate](https://scrapfly.io/blog/tag/real-estate) [requests](https://scrapfly.io/blog/tag/requests) [ruby](https://scrapfly.io/blog/tag/ruby) [scaling](https://scrapfly.io/blog/tag/scaling) [scrapeguide](https://scrapfly.io/blog/tag/scrapeguide) [scrapy](https://scrapfly.io/blog/tag/scrapy) [screenshots](https://scrapfly.io/blog/tag/screenshots) [selenium](https://scrapfly.io/blog/tag/selenium) [seo](https://scrapfly.io/blog/tag/seo) [tools](https://scrapfly.io/blog/tag/tools) [typescript](https://scrapfly.io/blog/tag/typescript) [web-scraping](https://scrapfly.io/blog/tag/web-scraping) [wget](https://scrapfly.io/blog/tag/wget) [xpath](https://scrapfly.io/blog/tag/xpath) ## // Articles 12 posts [ Latest tools data-parsing css-selectors ### HtmlAgilityPack vs AngleSharp: Which C# HTML Parser Should You Use? A fixture based comparison of Html Agility Pack and AngleSharp for C# HTML parsing, covering XPath versus CSS selectors, malformed markup, DOM behavior, and when each parser fits a project. Sep 11, 2026 16 min read ](https://scrapfly.io/blog/posts/htmlagilitypack-vs-anglesharp) [ python hidden-api scrapeguide ### How to Scrape DHL Tracking Status and Shipment Events A practical guide to scraping DHL parcel tracking status and shipment events in Python, covering Track & Trace page insp... Sep 11, 2026 19 min read ](https://scrapfly.io/blog/posts/how-to-scrape-dhl) [ python scrapeguide ### How to Scrape Goibibo Hotel and Flight Prices Learn how to scrape Goibibo hotel and flight prices with Python in 2026. Use Scrapfly ASP, India geo-targeting, and brow... Sep 11, 2026 18 min read ](https://scrapfly.io/blog/posts/how-to-scrape-goibibo-hotel-and-flight-prices) [ python data-parsing ### 5 Best Open-Source Amazon Scrapers in 2026 Five Amazon scraper repos on GitHub compared with rechecked stars, commit dates, licenses, code paths, and live failure ... Sep 11, 2026 17 min read ](https://scrapfly.io/blog/posts/best-amazon-scrapers-github) [ blocking playwright selenium ### Is Chrome CDP Detectable? CDP vs WebDriver for Browser Automation Chrome CDP is not stealth by itself. Learn how CDP, WebDriver, Playwright, Puppeteer, nodriver, Camoufox, and cloud brow... Sep 11, 2026 18 min read ](https://scrapfly.io/blog/posts/chrome-cdp-stealth-browser-automation-detection-explained) [ python api hidden-api ### Google Trends API Guide and Alternatives for Developers Google shipped an official Trends API in alpha. Here's what it covers, its limits, and when pytrends or a managed scrape... Sep 11, 2026 12 min read ](https://scrapfly.io/blog/posts/google-trends-api-alternatives) [ python api real-estate ### 6 Best Real Estate Data APIs for Developers in 2026 Six real estate data APIs compared by property-data job, from listings and property facts to valuations, parcels, zoning... Sep 11, 2026 12 min read ](https://scrapfly.io/blog/posts/best-real-estate-data-apis) [ python api ecommerce ### 7 Best E-commerce Product Data APIs for Developers in 2026 Seven e-commerce product data APIs compared by access model, from storefront catalogs and marketplace listings to seller... Sep 11, 2026 17 min read ](https://scrapfly.io/blog/posts/best-ecommerce-product-data-apis) [ python scrapy debugging ### Scrapy Error Messages: What They Mean and How to Fix Them A lookup catalog of Scrapy error messages. Each entry pairs the exact string with the layer that emitted it, one diagnos... Sep 11, 2026 17 min read ](https://scrapfly.io/blog/posts/scrapy-error-messages) [ tools data-parsing ### 7 Best XPath and CSS Selector Tools for Web Scraping in 2026 Compare seven XPath and CSS selector tools for testing selectors, reading live DOMs, generating locators, and recording ... Sep 11, 2026 19 min read ](https://scrapfly.io/blog/posts/best-xpath-css-selector-tools) [ python scrapeguide ### How to Scrape Capterra Reviews and Software Data Learn how to scrape Capterra software reviews and listings using Python and Scrapfly, bypassing Cloudflare Bot Managemen... Sep 11, 2026 19 min read ](https://scrapfly.io/blog/posts/how-to-scrape-capterra) [ nodejs tools javascript ### Best JavaScript Web Scraping Libraries in 2026 (Node.js Stack Guide) A layer-by-layer ranking of the 8 JavaScript and Node.js libraries worth composing into a scraper in 2026, from HTTP cli... Sep 07, 2026 19 min read ](https://scrapfly.io/blog/posts/best-javascript-web-scraping-libraries) 1 [2](https://scrapfly.io/blog?page=2) [3](https://scrapfly.io/blog?page=3) … [30](https://scrapfly.io/blog?page=30) [ ](https://scrapfly.io/blog?page=2) Page 1 of 30 · 358 articles ## ? Quick Answers ### nodejs [View all ](https://scrapfly.io/blog/answers) - [ Q How to take screenshots in NodeJS? ](https://scrapfly.io/blog/answers/how-to-take-screenshots-nodejs) ### css-selectors [View all ](https://scrapfly.io/blog/answers) - [ Q How to use CSS Selectors in Nim ? ](https://scrapfly.io/blog/answers/how-to-use-css-selectors-in-nim) - [ Q How to select elements by attribute using CSS selectors? ](https://scrapfly.io/blog/answers/how-to-select-elements-by-attribute-containing-value-css-selectors) - [ Q How to select elements by class using CSS selectors? ](https://scrapfly.io/blog/answers/how-to-select-elements-by-class-css-selectors) - [ Q How to select elements by ID using CSS selectors? ](https://scrapfly.io/blog/answers/how-to-select-elements-by-id-css-selectors) - [ Q How to select following siblings using CSS selectors? ](https://scrapfly.io/blog/answers/how-to-select-following-sibling-element-css-selectors) ### proxies [View all ](https://scrapfly.io/blog/answers) - [ Q How to Solve the cURL (60) Error When Using Proxy? ](https://scrapfly.io/blog/answers/how-to-solve-the-curl-60-error-when-proxy) - [ Q How To Use Proxy With cURL? ](https://scrapfly.io/blog/answers/how-to-use-proxy-with-curl) - [ Q What is The cURL (28) Error, Couldn't connect to server? ](https://scrapfly.io/blog/answers/what-is-the-curl-28-error) - [ Q Mobile vs Residential Proxies - which to choose for scraping? ](https://scrapfly.io/blog/answers/mobile-vs-residential-proxies-whats-the-difference) - [ Q What are private proxies and how are they used in scraping? ](https://scrapfly.io/blog/answers/what-are-private-proxies-compared-to-shared) ### curl [View all ](https://scrapfly.io/blog/answers) - [ Q How To Download a File With cURL? ](https://scrapfly.io/blog/answers/how-to-download-file-curl) - [ Q How to Follow Redirects In cURL? ](https://scrapfly.io/blog/answers/how-to-follow-redirects-in-curl) - [ Q How To Send cURL POST Requests? ](https://scrapfly.io/blog/answers/how-to-send-a-post-request-using-curl) - [ Q How to Send a HEAD Request With cURL? ](https://scrapfly.io/blog/answers/how-to-send-curl-head-requests) - [ Q How To Send Multiple cURL Requests in Parallel? ](https://scrapfly.io/blog/answers/how-to-send-multiple-curl-requests-in-parallel) ### http [View all ](https://scrapfly.io/blog/answers) - [ Q How to Copy as cURL With Brave? ](https://scrapfly.io/blog/answers/how-to-copy-as-curl-with-brave) - [ Q How To Copy as cURL With Google Chrome? ](https://scrapfly.io/blog/answers/how-to-copy-as-curl-with-chrome) - [ Q How to Copy as cURL With Edge? ](https://scrapfly.io/blog/answers/how-to-copy-as-curl-with-edge) - [ Q How to Copy as cURL With Firefox? ](https://scrapfly.io/blog/answers/how-to-copy-as-curl-with-firefox) - [ Q How to Copy as cURL With Safari? ](https://scrapfly.io/blog/answers/how-to-copy-as-curl-with-safari) ### tools [View all ](https://scrapfly.io/blog/answers) - [ Q How to edit Local Storage data using browser Devtools ](https://scrapfly.io/blog/answers/how-to-edit-local-storage-using-devtools) - [ Q How to edit cookies in Chrome devtools? ](https://scrapfly.io/blog/answers/how-to-edit-cookies-using-chrome-devtools) ### python [View all ](https://scrapfly.io/blog/answers) - [ Q How to scrape HTML table to Excel Spreadsheet (.xlsx)? ](https://scrapfly.io/blog/answers/html-table-to-xlsx-python-beautifulsoup) - [ Q How to handle popup dialogs in Playwright? ](https://scrapfly.io/blog/answers/how-to-click-on-alert-dialog-in-playwright) - [ Q How to scrape images from a website? ](https://scrapfly.io/blog/answers/how-to-scrape-images-from-website) - [ Q How to check if element exists in Playwright? ](https://scrapfly.io/blog/answers/how-to-check-for-element-in-playwright) - [ Q How to select dictionary key recursively in Python? ](https://scrapfly.io/blog/answers/how-to-select-dictionary-key-recursively-in-python) ### puppeteer [View all ](https://scrapfly.io/blog/answers) - [ Q How to handle popup dialogs in Puppeteer? ](https://scrapfly.io/blog/answers/how-to-click-on-alert-dialog-in-puppeteer) - [ Q How to click on cookie popups and modal alerts in Puppeteer? ](https://scrapfly.io/blog/answers/how-to-click-on-modal-alerts-like-cookie-pop-up-in-puppeteer) - [ Q How to scroll to the bottom of the page with Puppeteer? ](https://scrapfly.io/blog/answers/how-to-scroll-to-the-bottom-with-puppeteer) - [ Q Getting started with Puppeteer Stealth ](https://scrapfly.io/blog/answers/how-to-use-puppeteer-stealth-what-does-it-do) ### selenium [View all ](https://scrapfly.io/blog/answers) - [ Q How to handle popup dialogs in Selenium? ](https://scrapfly.io/blog/answers/how-to-click-on-alert-dialog-in-selenium) - [ Q How to click on cookie popups and modal alerts in Selenium? ](https://scrapfly.io/blog/answers/how-to-click-on-modal-alerts-like-cookie-pop-up-in-selenium) - [ Q How to block resources in Selenium and Python? ](https://scrapfly.io/blog/answers/how-to-block-resources-in-selenium) - [ Q How to capture background requests and responses in Selenium? ](https://scrapfly.io/blog/answers/how-to-capture-xhr-requests-selenium) - [ Q How to scroll to the bottom of the page with Selenium? ](https://scrapfly.io/blog/answers/how-to-scroll-to-the-bottom-with-selenium) ### playwright [View all ](https://scrapfly.io/blog/answers) - [ Q How to click on cookie popups and modal alerts in Playwright? ](https://scrapfly.io/blog/answers/how-to-click-on-modal-alerts-like-cookie-pop-up-in-playwright) - [ Q How to scroll to the bottom of the page with Playwright? ](https://scrapfly.io/blog/answers/how-to-scroll-to-the-bottom-with-playwright) - [ Q How to run Playwright in Jupyter notebooks? ](https://scrapfly.io/blog/answers/playwright-in-ipython) ### xpath [View all ](https://scrapfly.io/blog/answers) - [ Q How to select elements by attribute value in XPath? ](https://scrapfly.io/blog/answers/how-to-select-elements-by-attribute-value) - [ Q How to count selections in XPath and why? ](https://scrapfly.io/blog/answers/how-to-count-selectors-in-xpath-and-why) - [ Q How to get the name of an HTML element in XPath? ](https://scrapfly.io/blog/answers/how-to-get-name-of-selected-element-in-xpath) - [ Q How to join values using XPath concat? ](https://scrapfly.io/blog/answers/how-to-join-values-in-xpath) - [ Q How to reverse expressions in XPath? ](https://scrapfly.io/blog/answers/how-to-reverse-expression-in-xpath) ### scrapy [View all ](https://scrapfly.io/blog/answers) - [ Q What are scrapy middlewares and how to use them? ](https://scrapfly.io/blog/answers/what-are-scrapy-middlewares-and-how-to-use-them) - [ Q What are scrapy pipelines and how to use them? ](https://scrapfly.io/blog/answers/what-are-scrapy-pipelines-and-how-to-use-them) - [ Q How to pass custom parameters to scrapy spiders? ](https://scrapfly.io/blog/answers/how-to-pass-parameters-to-scrapy-spiders-cli) - [ Q How to pass data between scrapy callbacks in Scrapy? ](https://scrapfly.io/blog/answers/how-to-pass-data-between-scrapy-callbacks) - [ Q How to pass data from start\_requests to parse callbacks in scrapy? ](https://scrapfly.io/blog/answers/how-to-pass-data-from-start-request-to-callbacks-scrapy) ### headless-browser [View all ](https://scrapfly.io/blog/answers) - [ Q How to use headless browsers with scrapy? ](https://scrapfly.io/blog/answers/how-to-use-headless-browsers-with-scrapy) ### data-parsing [View all ](https://scrapfly.io/blog/answers) - [ Q How to select last element in XPath? ](https://scrapfly.io/blog/answers/how-to-select-last-element-in-xpath) - [ Q How to select all elements between two elements in XPath? ](https://scrapfly.io/blog/answers/how-to-select-all-elements-between-two-known-elements-in-xpath) ### requests [View all ](https://scrapfly.io/blog/answers) - [ Q How to configure Python requests to use a proxy? ](https://scrapfly.io/blog/answers/python-requests-proxy-intro) ### blocking [View all ](https://scrapfly.io/blog/answers) - [ Q Web scraping - what is HTTP 403 status code? ](https://scrapfly.io/blog/answers/403-status-code) - [ Q Web scraping - what is HTTP 429 status code? ](https://scrapfly.io/blog/answers/429-status-code) - [ Q What is 444 status code and how to avoid it? ](https://scrapfly.io/blog/answers/444-status-code) - [ Q Web scraping - what is HTTP 499 status code? ](https://scrapfly.io/blog/answers/499-status-code) - [ Q Web scraping - what is HTTP 503 status code? ](https://scrapfly.io/blog/answers/503-status-code) ## Ready to scale your web scraping? Anti-bot bypass, browser rendering, and rotating proxies, all in one API. [ Try Scrapfly for FREE ](https://scrapfly.io/register) [ View Documentation ](https://scrapfly.io/docs/onboarding)