Scrapfly Blog

Latest

python nodejs headless-browser

How to Scrape With Headless Firefox

Discover how to use headless Firefox with Selenium, Playwright, and Puppeteer for web scraping, including practical examples for each library.

Apr 17, 2024 10 min read

proxies tools

How to Use Tor For Web Scraping

In this article, we'll explain web scraping using Tor. For this, we'll use Tor as a proxy server to change the IP addres...

Apr 12, 2024 9 min read

blocking tools

How to Know What Anti-Bot Service a Website is Using?

In this article we'll take a look at two popular tools: WhatWaf and Wafw00f which can identify what WAF service is used.

Apr 09, 2024 10 min read

python scrapeguide

How to Scrape LinkedIn in 2026

LinkedIn aggressively blocks scrapers. This guide shows how to scrape profiles, companies, and jobs anyway using ScrapFl...

Apr 05, 2024 29 min read

python headless-browser selenium

Selenium Wire Tutorial: Intercept Background Requests

In this guide, we'll explore web scraping with Selenium Wire. We'll define what it is, how to install it, and how to use...

Apr 01, 2024 18 min read

python seo scrapeguide

How to Scrape SimilarWeb Website Traffic Analytics

In this guide, we'll explain how to scrape SimilarWeb through a step-by-step guide. We'll scrape comprehensive website t...

Mar 25, 2024 25 min read

python ecommerce hidden-api

How to Scrape BestBuy Product, Offer and Review Data

Learn how to scrape BestBuy, one of the most popular retail stores for electronic stores in the United States. We'll scr...

Mar 20, 2024 14 min read

http tools curl

Sending HTTP Requests With Curlie: A better cURL

In this guide, we'll explore Curlie, a better cURL version. We'll start by defining what Curlie is and how it compares t...

Mar 15, 2024 8 min read

http tools curl

How to Use cURL For Web Scraping

In this article, we'll go over a step-by-step guide on sending and configuring HTTP requests with cURL. We'll also explo...

Mar 12, 2024 11 min read

python tools hidden-api

How To Scrape TikTok in 2026

Complete guide to scraping TikTok in 2026. Learn TikTok's new anti-bot defenses, hidden JSON APIs, and production-ready ...

Mar 08, 2024 41 min read

python headless-browser playwright

Web Scraping Dynamic Websites With Scrapy Playwright

Learn about Selenium Playwright. A Scrapy integration that allows web scraping dynamic web pages with Scrapy. We'll expl...

Mar 06, 2024 16 min read

python headless-browser selenium

Web Scraping Dynamic Web Pages With Scrapy Selenium

Learn how to scrape dynamic web pages with Scrapy Selenium. You will also learn how to use Scrapy Selenium for common sc...

Mar 04, 2024 22 min read