https://scrapfly.io/blog 2026-10-30T22:50:20+00:00 Scrapfly Web scraping tutorials, guides, and articles by Scrapfly https://scrapfly.io/img/scrapfly-logo.png https://scrapfly.io/blog/posts/expedia-scraper 2026-10-30T15:02:35+00:00 2026-10-30T15:02:37+00:00

A repo-backed Python guide to scraping Expedia hotel and flight search results by capturing its GraphQL responses, with pagination.

https://scrapfly.io/blog/posts/how-to-scrape-loopnet 2026-10-27T14:45:22+00:00 2026-10-27T14:45:23+00:00

Scrape LoopNet search and listing pages with Python and Scrapfly's Unblocker: placards for search results, JSON-LD for property details.

https://scrapfly.io/blog/posts/best-python-html-parsers 2026-10-27T14:44:31+00:00 2026-10-27T14:44:32+00:00

Seven Python HTML parsers compared by selector model, malformed-markup behavior, dependency cost, and measured parse time on one real page.

https://scrapfly.io/blog/posts/nodriver-tutorial 2026-10-27T14:44:05+00:00 2026-10-28T13:51:55+00:00

A tested Nodriver tutorial covering install, selectors, background requests, authenticated proxies, headless mode, common errors and when to move on.

https://scrapfly.io/blog/posts/best-travel-flight-data-apis 2026-10-26T16:09:14+00:00 2026-10-30T15:07:22+00:00

Six flight and travel data APIs compared by what they actually return, from live status and aircraft tracking to schedules and fares.

https://scrapfly.io/blog/posts/how-to-use-proxies-with-selenium 2026-10-19T17:59:16+00:00 2026-10-19T17:59:17+00:00

A tested walkthrough of Python Selenium proxies: why Chrome drops credentials, the auth methods that still work, rotation, residential gateways and geo checks.

https://scrapfly.io/blog/posts/how-to-scrape-shopify-stores 2026-10-19T17:57:47+00:00 2026-10-19T17:57:49+00:00

Build a resilient Shopify storefront scraper around products.json, JSON-LD, response classification, and measured fallbacks.

https://scrapfly.io/blog/posts/csharp-httpclient-proxy 2026-10-16T20:02:23+00:00 2026-10-16T20:09:09+00:00

How to configure authenticated and rotating proxies with C# HttpClient and SocketsHttpHandler, including the connection-pool traps behind 407 errors.

https://scrapfly.io/blog/posts/how-to-test-web-scraper-before-production 2026-10-16T20:02:20+00:00 2026-10-16T20:02:21+00:00

Build parser tests, HTML fixtures, live checks, and deployment gates that catch silent web scraper failures before they reach production.

https://scrapfly.io/blog/posts/how-to-scrape-idealo 2026-10-16T20:02:17+00:00 2026-10-16T20:02:18+00:00

Scrape idealo product pages for the full merchant offer list, prices, and price history with Python and the Scrapfly Web Scraping API.

https://scrapfly.io/blog/posts/best-rust-web-scraping-libraries 2026-10-16T13:49:36+00:00 2026-10-28T15:53:24+00:00

Five current Rust libraries for static HTML, article extraction, and browser control, plus managed page access for blocked pages.

https://scrapfly.io/blog/posts/best-go-web-scraping-libraries 2026-10-14T15:53:35+00:00 2026-10-14T15:53:36+00:00

Nine Go web scraping libraries and APIs compared by the job each does best, current release, and license.

https://scrapfly.io/blog/posts/best-puppeteer-alternatives-for-web-scraping 2026-10-14T15:52:50+00:00 2026-10-28T15:53:21+00:00

A job-by-job guide to replacing Puppeteer for web scraping, covering library rewrites, crawl frameworks, no-browser routes and remote stealth browsers.

https://scrapfly.io/blog/posts/how-to-scrape-pinterest 2026-10-08T15:54:08+00:00 2026-10-08T15:54:11+00:00

Learn to scrape Pinterest pins, boards, profiles, and images with Python by starting a session, calling the hidden resource API, and following its bookmark pagination.

https://scrapfly.io/blog/posts/how-to-scrape-goodreads 2026-09-29T17:28:55+00:00 2026-10-09T15:58:17+00:00

A practical guide to scraping Goodreads book pages, ratings, and review samples in Python, reading JSON-LD and the embedded Next.js Apollo state before falling back to CSS selectors, plus how to discover books from list pages.

https://scrapfly.io/blog/posts/how-to-scrape-news-articles 2026-09-29T17:28:50+00:00 2026-10-05T21:03:27+00:00

Learn how to scrape news articles from any website in Python, extracting headline, author, date, and body with Newspaper4k, discovering article URLs at scale with RSS and sitemaps, and getting through JavaScript rendering and anti-bot systems on major outlets.

https://scrapfly.io/blog/answers/persist-sessions-across-scrapy-requests 2026-09-29T17:09:54+00:00 2026-10-01T11:38:49+00:00

How Scrapy keeps cookies within a crawl, isolates cookie jars per session, resumes JOBDIR state, and handles Playwright authentication across restarts.

https://scrapfly.io/blog/posts/best-twitter-scrapers-github 2026-09-17T14:12:01+00:00 2026-10-08T15:12:18+00:00

Four surviving X scraper repositories compared by coverage, account requirements, maintenance, and infrastructure burden, plus why older projects died.

https://scrapfly.io/blog/posts/best-financial-data-apis 2026-09-17T13:44:40+00:00 2026-10-05T21:31:57+00:00

Eight financial data APIs compared by workload, delivery method, history depth, entitlement, and what the licence actually lets you redistribute.

https://scrapfly.io/blog/posts/how-to-bypass-f5-when-web-scraping 2026-09-17T13:33:02+00:00 2026-10-05T21:31:47+00:00

Tell BIG-IP Bot Defense from F5 Distributed Cloud Bot Defense, inspect its browser telemetry, and configure a browser-backed fetch.

https://scrapfly.io/blog/posts/best-local-business-data-apis 2026-09-16T17:14:26+00:00 2026-10-23T16:21:35+00:00

Six local business data APIs compared by coverage and licensing, from platform place search and reviews to open map data and geographic context.

https://scrapfly.io/blog/posts/best-social-media-data-apis 2026-09-16T17:08:01+00:00 2026-10-23T16:22:15+00:00

Compare eight official social media APIs across posts, video, communities, research, professional data, and federated networks.

https://scrapfly.io/blog/posts/best-csharp-web-scraping-libraries 2026-09-16T17:06:46+00:00 2026-10-01T11:38:42+00:00

A workflow based comparison of seven C# web scraping libraries across three layers, covering HTML parsing with Html Agility Pack and AngleSharp, browser automation with Playwright for .NET, Selenium, and PuppeteerSharp, and crawl orchestration with Abot and DotnetSpider.

https://scrapfly.io/blog/posts/htmlagilitypack-vs-anglesharp 2026-09-11T20:35:21+00:00 2026-10-01T11:38:47+00:00

A fixture based comparison of Html Agility Pack and AngleSharp for C# HTML parsing, covering XPath versus CSS selectors, malformed markup, DOM behavior, and when each parser fits a project.

https://scrapfly.io/blog/posts/how-to-scrape-dhl 2026-09-11T20:32:31+00:00 2026-10-05T21:31:46+00:00

A practical guide to scraping DHL parcel tracking status and shipment events in Python, covering Track & Trace page inspection, JSON/XHR discovery, response classification, event normalization, and the official tracking API.

https://scrapfly.io/blog/posts/how-to-scrape-goibibo-hotel-and-flight-prices 2026-09-11T20:26:46+00:00 2026-10-23T16:21:54+00:00

Learn how to scrape Goibibo hotel and flight prices with Python in 2026. Use Scrapfly Unblocker (previously ASP), India geo-targeting, and browser sessions to capture and parse structured hotel and flight responses.

https://scrapfly.io/blog/posts/best-amazon-scrapers-github 2026-09-11T20:17:58+00:00 2026-10-28T15:53:14+00:00

Four Amazon scraper repos on GitHub compared with rechecked stars, commit dates, licenses, code paths, and live failure modes.

https://scrapfly.io/blog/posts/chrome-cdp-stealth-browser-automation-detection-explained 2026-09-11T20:05:00+00:00 2026-10-01T11:38:47+00:00

Chrome CDP is not stealth by itself. Learn how CDP, WebDriver, Playwright, Puppeteer, nodriver, Camoufox, and cloud browsers affect detection.

https://scrapfly.io/blog/posts/google-trends-api-alternatives 2026-09-11T19:48:15+00:00 2026-10-28T16:48:18+00:00

Google shipped an official Trends API in alpha. Here's what it covers, its limits, and when pytrends or a managed scraper still fits better.

https://scrapfly.io/blog/posts/best-real-estate-data-apis 2026-09-11T19:42:45+00:00 2026-10-23T16:21:15+00:00

Six real estate data APIs compared by property-data job, from listings and property facts to valuations, parcels, zoning, and neighborhood scores, plus guidance on when scraping a public portal is the better route.