Mazen is a lead contributor and scraper engineer at Scrapfly since 2024 and has been contributing articles and tutorials on web scraping, data extraction and automation. He has a background in software engineering with a focus on Python and web technologies.
How to Scrape YouTube in 2025
Learn how to scrape YouTube, channel, video, and comment data using Python directly in JSON.
SCRAPEGUIDE
PYTHON
HIDDEN-API
A Comprehensive Guide to TikTok API
Explore the various TikTok APIs, their features, use cases, and limitations.
API
How to Use cURL to Download Files
Master file downloads with curl and discover advanced use cases.
CURL
TOOLS
How to Use Yelp API to Extract Business and Review Data
Take an extensive look into Yelp API, its key features, pricing, and limitations. Additionally, explore potential alternatives.
API
How To Take Screenshots In Python?
Learn how to take Python screenshots through Selenium and Playwright, including common browser tips and tricks for customizing web page captures.
SCREENSHOTS
PYTHON
HEADLESS-BROWSER
What is the best Screenshot API in 2025?
Learn everything about the best screenshot API, from the features to consider to a list of the best services available and how to benchmark them.
SCREENSHOTS
Web Scraping with Go
Learn web scraping with Golang, from native HTTP requests and HTML parsing to a step-by-step guide to using Colly, the Go web crawling package.
GOLANG
HTTP
DATA-PARSING
FRAMEWORK
How to Power-Up LLMs with Web Scraping and RAG
In depth look at how to use LLM and web scraping for RAG applications using either LlamaIndex or LangChain.
AI
PYTHON
PROJECT
Web Scraping With Cloud Browsers
Introduction cloud browsers and their benefits and a step-by-step setup with self-hosted Selenium-grid cloud browsers.
HEADLESS-BROWSER
SCALING
How to Scrape Forms
Learn how to scrape forms through a step-by-step guide using HTTP clients and headless browsers.
HEADLESS-BROWSER
PYTHON
HTTPX
INTRO
NODEJS
How to Build Minimum Advertised Price (MAP) Monitoring Tool
Learn what minimum advertised price monitoring is and how to apply its concept using Python web scraping.
PROJECT
PYTHON
How to Scrape Reddit Posts, Subreddits and Profiles
In this article, we'll explore how to scrape Reddit. We'll extract various social data types from subreddits, posts, and user pages. All of which through plain HTTP requests without headless browser usage.
PYTHON
SCRAPEGUIDE
How to Scrape With Headless Firefox
Discover how to use headless Firefox with Selenium, Playwright, and Puppeteer for web scraping, including practical examples for each library.
HEADLESS-BROWSER
PUPPETEER
SELENIUM
NODEJS
PLAYWRIGHT
PYTHON
How to Use Tor For Web Scraping
In this article, we'll explain web scraping using Tor. For this, we'll use Tor as a proxy server to change the IP address randomly in either HTTP or SOCKS, as well as using it as a rotating proxy server.
TOOLS
PROXIES
How to Know What Anti-Bot Service a Website is Using?
In this article we'll take a look at two popular tools: WhatWaf and Wafw00f which can identify what WAF service is used.
BLOCKING
TOOLS
How to Scrape LinkedIn in 2025
In this scrape guide we'll be taking a look at one of the most popular web scraping targets - LinkedIn.com. We'll be scraping people profiles, company profiles as well as job listings and search.
PYTHON
SCRAPEGUIDE
Selenium Wire Tutorial: Intercept Background Requests
In this guide, we'll explore web scraping with Selenium Wire. We'll define what it is, how to install it, and how to use it to inspect and manipulate background requests.
PYTHON
HEADLESS-BROWSER
SELENIUM
TOOLS
How to Scrape SimilarWeb Website Traffic Analytics
In this guide, we'll explain how to scrape SimilarWeb through a step-by-step guide. We'll scrape comprehensive website traffic insights, websites comparing data, sitemaps, and trending industry domains.
SCRAPEGUIDE
SEO
SEARCH-ENGINE
PYTHON
How to Scrape BestBuy Product, Offer and Review Data
Learn how to scrape BestBuy, one of the most popular retail stores for electronic stores in the United States. We'll scrape different data types from product, search, review, and sitemap pages using different web scraping techniques.
SCRAPEGUIDE
HIDDEN-API
ECOMMERCE
PYTHON
Sending HTTP Requests With Curlie: A better cURL
In this guide, we'll explore Curlie, a better cURL version. We'll start by defining what Curlie is and how it compares to cURL. We'll also go over a step-by-step guide on using and configuring Curlie to send HTTP requests.
CURL
HTTP
TOOLS
How to Use cURL For Web Scraping
In this article, we'll go over a step-by-step guide on sending and configuring HTTP requests with cURL. We'll also explore advanced usages of cURL for web scraping, such as scraping dynamic pages and avoiding getting blocked.
HTTP
TOOLS
CURL
How To Scrape TikTok in 2025
In this tutorial, we'll explain how to scrape TikTok. We'll extract data from various TikTok sources, such as posts, comments, profiles and search pages. Moreover, we'll scrape these data through hidden TikTok APIs or hidden JSON datasets.
PYTHON
HIDDEN-API
SCRAPEGUIDE
Web Scraping Dynamic Websites With Scrapy Playwright
Learn about Selenium Playwright. A Scrapy integration that allows web scraping dynamic web pages with Scrapy. We'll explain web scraping with Scrapy Playwright through an example project and how to use it for common scraping use cases, such as clicking elements, scrolling and waiting for elements.
PYTHON
PLAYWRIGHT
SCRAPY
HEADLESS-BROWSER
Web Scraping Dynamic Web Pages With Scrapy Selenium
Learn how to scrape dynamic web pages with Scrapy Selenium. You will also learn how to use Scrapy Selenium for common scraping use cases, such as waiting for elements, clicking buttons and scrolling.
PYTHON
SCRAPY
HEADLESS-BROWSER
SELENIUM
Scrapy Splash Guide: Scrape Dynamic Websites With Scrapy
Learn about web scraping with Scrapy Splash, which lets Scrapy scrape dynamic web pages. We'll define Splash, cover installation and navigation, and provide a step-by-step guide for using Scrapy Splash.
PYTHON
HEADLESS-BROWSER
FRAMEWORK
SCRAPY
How to Track Competitor Prices Using Web Scraping
In this web scraping guide, we'll explain how to create a tool for tracking competitor prices using Python. It will scrape specific products from different providers, compare their prices and generate insights.
ECOMMERCE
PYTHON
PROJECT
Intro to Using Web Scraping For Sentiment Analysis
In this article, we'll explore using web scraping for sentiment analysis. We'll start by defining sentiment analysis and then walk through a practical example of performing sentiment analysis on web-scraped data with community Python libraries.
PROJECT
AI
PYTHON
Using API Clients For Web Scraping: Postman
In this article, we'll explore the use of API clients for web scraping. We'll start by explaining how to locate hidden API requests on websites. Then, we'll explore importing, manipulating, and exporting them using Postman to develop efficient API-based web scrapers.
HIDDEN-API
TOOLS
Intro to Parsing HTML and XML with Python and lxml
In this tutorial, we'll take a deep dive into lxml, a powerful Python library that allows for parsing HTML and XML effectively. We'll start by explaining what lxml is, how to install it and using lxml for parsing HTML and XML files. Finally, we'll go over a practical web scraping with lxml.
PYTHON
TOOLS
DATA-PARSING
Use Curl Impersonate to scrape as Chrome or Firefox
Learn how to prevent TLS fingerprinting by impersonating normal web browser configurations. We'll start by explaining what the Curl Impersonate is, how it works, how to install and use it. Finally, we'll explore using it with Python to avoid web scraping blocking.
TOOLS
BLOCKING
CURL
HTTP
FlareSolverr Guide: Bypass Cloudflare While Scraping
In this article, we'll explore the FlareSolverr tool and how to use it to get around Cloudflare while scraping. We'll start by explaining what FlareSolverr is, how it works, how to install and use it. Let's get started!
PYTHON
TOOLS
BLOCKING
HTTP
Web Scraping with CloudProxy
One of the most common challenges encountered while web scraping is IP throttling and blocking. Learn about the CloudProxy tool, how to install it and how to use it for cloud-based web scraping.
TOOLS
PROXIES
How to use Headless Chrome Extensions for Web Scraping
In this article, we'll explore different useful Chrome extensions for web scraping. We'll also explain how to install Chrome extensions with various headless browser libraries, such as Selenium, Playwright and Puppeteer.
PYTHON
NODEJS
TOOLS
PLAYWRIGHT
PUPPETEER
SELENIUM
How to Use Cache In Web Scraping for Major Performance Boost
Introduction to web scraping caches. How caching can significantly reduce scraping costs and drastically improve performance.
PYTHON
TOOLS
How to Parse XML
In this article, we'll explain about XML parsing. We'll start by defining XML files, their format and how to navigate them for data extraction.
PYTHON
CSS-SELECTORS
XPATH
DATA-PARSING
How to Build a Price Tracker Using Python
Extracting price data from websites is a popular web scraping use-case for e-commerce businesses. Learn how to create a price scraper using Python. It will crawl over pages, extract product data and record historical price changes.
PROJECT
PYTHON
How to Scrape Bing Search with Python
In this scrape guide we'll be taking a look at scraping Bing search results. It's the second biggest search engine in the world and it contains a lot of data - all retrievable with a bit a of Python.
PYTHON
SCRAPEGUIDE
SEO
How to Bypass CAPTCHA While Web Scraping in 2025
Captchas can ruin web scrapers but we don't have to teach our robots how to solve them - we can just get around it all!
BLOCKING
How to Bypass Kasada Anti-Bot When Web Scraping in 2025
In this article, we'll take a look at the popular anti-bot service Kasada. How does it detect web scrapers and bots and what can we do to prevent our scrapers from being detected?
BLOCKING
How to Scrape G2 Company Data and Reviews
In this scrapeguide we're taking a look at G2.com - one of the biggest digital product metawebsites out there. We'll be scraping product data, reviews and company profiles.
SCRAPEGUIDE
PYTHON
What are Honeypots and How to Avoid Them in Web Scraping
Introduction to web honeypots, their types and functions and how they are used to identify and block web scrapers and bots and how to avoid them.
BLOCKING
How to Scrape Etsy.com Product, Shop and Search Data
In this scrapeguide we're taking a look at Etsy.com - a popular e-commerce market for hand crafted and vintage items. We'll be using Python and HTML parsing to scrape search and product data.
SCRAPEGUIDE
ECOMMERCE
PROJECT
PYTHON
How to Hide Your IP Address
In this article we'll be taking a look at several ways to hide IP addresses: proxies, tor networks, vpns and other techniques.
BLOCKING
TOOLS
PROXIES
How to Scrape Trustpilot.com Reviews and Company Data
In today's scrapeguide we'll be taking a look at Trustpilot - one of the biggest sources of company reviews and how to scrape it using Python.
PYTHON
SCRAPEGUIDE
HIDDEN-API
Web Scraping to Google Sheets
Google sheets is an easy to store scraped data. In this tutorial we'll take a look at how to use this free online database for storing scraped data!
PYTHON
PROJECT
DATA-PARSING
How to Scrape Domain.com.au Real Estate Property Data
We'll be taking a look at another real estate target in Australia - domain.com.au. To scrape real estate data we'll be using Python and hidden web data scraping approach.
REAL-ESTATE
SCRAPEGUIDE
PYTHON
How to Scrape Realestate.com.au Property Listing Data
We're taking yet another look at real estate websites. This time we're going down under! Realtestate.com.au is the biggest real estate portal in Australia and let's take a look at how to scrape it.
PYTHON
REAL-ESTATE
SCRAPEGUIDE
How to Scrape Immowelt.de Real Estate Data
Immowelt.de is a major real estate website in Germany and it's suprisingly easy to scrape. In this tutorial, we'll be using Python and hidden web data scraping technique to scrape real estate property data.
SCRAPEGUIDE
PYTHON
REAL-ESTATE
How to Scrape Homegate.ch Real Estate Property Data
For this scrape guide we'll be taking a look at another real estate website in Switzerland - Homegate. For this we'll be using hidden web data scraping and JSON parsing.
PYTHON
SCRAPEGUIDE
REAL-ESTATE
How to Scrape Immobilienscout24.de Real Estate Data
In this scrape guide we'll be taking a look at another real estate giant from Germany - Immobilienscout24.de.
PYTHON
SCRAPEGUIDE
How to Scrape Immoscout24.ch Real Estate Property Data
In this scrape guide tutorial we'll be taking a look at the biggest real estate marketplace in Switzerland - ImmoScout24.ch. We'll be using hidden web data scraping technique and explore private APIs.
SCRAPEGUIDE
PYTHON
REAL-ESTATE
How to Handle Cookies in Web Scraping
Introduction to cookies in web scraping. What are they and how to take advantage of cookie process to authenticate or set website preferences.
HTTP
How to Scrape Seloger.com - Real Estate Listing Data
Learn about seloger.com web scraping and how to avoid its blocking. You will also learn how to scrape real estate data from seloger.com.
SCRAPEGUIDE
PYTHON
PROJECT
How to Web Scrape Leboncoin.fr using Python
Introduction to scraping leboncoin.fr without getting blocked. In this tutorial, we'll cover Leboncoin search and ad listing scraping using Python and Scrapfly.
SCRAPEGUIDE
PYTHON
Intro to Web Scraping Using Selenium Grid
In this guide, you will learn about installing and configuring Selenium Grid with Docker and how to use it for web scraping at scale.
SELENIUM
PYTHON
Web Scraping Without Blocking With Undetected ChromeDriver
In this tutorial we'll be taking a look at a new popular web scraping tool Undetected ChromeDriver which is a Selenium extension that allows to bypass many scraper blocking techniques.
BLOCKING
PYTHON
TOOLS
Web Scraping Emails using Python
In this tutorial we'll take a look at email scraping. How to crawl pages and extract email addresses using Python and what are some popular challenges.
PYTHON
DATA-PARSING
PROJECT
Web Scraping Phone Numbers with Python
In this article we'll dive into phone number scraping. We'll explore an example object and cover common phone number scraping challenges like obfuscation.
PYTHON
DATA-PARSING
PROJECT
How to Scrape Google Trends using Python
In this article we'll be taking a look at scraping Google Trends - what it is and how to scrape it? For this example, we'll dive into reverse engineering and scrape the secret Google Trends API.
PYTHON
SEO
How to Avoid Scraper Blocking when Scraping Images
Introduction to scraper blocking when it comes to image scraping. What are some popular scraper blocking techniques and how to avoid them.
BLOCKING
Intro to Web Scraping Images with Python
In this guide, we’ll explore how to scrape images from websites using different methods. We'll also cover the most common image scraping challenges and how to overcome them. By the end of this article, you will be an image scraping master!
INTRO
CRAWLING
DATA-PARSING
PYTHON
How to Scrape Google SEO Keyword Data and Rankings
In this article, we’ll take a look at SEO web scraping, what it is and how to use it for better SEO keyword optimization. We’ll also create an SEO keyword scraper that scrapes Google search rankings and suggested keywords.
PYTHON
PROJECT
SEO
How to Effectively Use User Agents for Web Scraping
In this article, we’ll take a look at the User-Agent header, what it is and how to use it in web scraping. We'll also generate and rotate user agents to avoid web scraping blocking.
HTTP
PYTHON
How to Observe E-Commerce Trends using Web Scraping
In this example web scraping project we'll be taking a look at monitoring E-Commerce trends using Python, web scraping and data visualization tools.
PYTHON
ECOMMERCE
PROJECT
How to Scrape in Another Language, Currency or Location
Localization allows for adapting websites content by changing language and currency. So, how do we scrape it? We'll take a look at the most common methods for changing language, currency and other locality details in web scraping.
PYTHON
HEADLESS-BROWSER
HTTP
JSON Parsing Made Easy with ChatGPT in Web Scraping
ChatGPT web scraping techniques allow for faster web scraping development. Here's how you can save a lot of time parsing JSON data with the help of chatGPT!
AI
DATA-PARSING
PYTHON
Finding Hidden Web Data with ChatGPT Web Scraping
In this article we take a look at how to get assistance from LLMs for hidden web data scraping.
AI
DATA-PARSING
Find Web Elements with ChatGPT and XPath or CSS selectors
ChatGPT is becoming a popular assistant in web scraper development. In this article, we'll take a look at how to use it in HTML using it to generate XPath and CSS selectors.
AI
PYTHON
