Scraping Overview

Web scraping is an automated process to collect web data. This data can be publicly available like product details on e-commerce or belong to you like your purchase receipts.

This process has many names like data harvesting, scraping and crawling — all of which define the same subject though in slightly different scenarios.

It takes very little imagination of what can be done with scraped data, but popular use cases like market analytics, lead generation, trends and price tracking, AI training and more.

Web scraping vs crawling — what's the difference?
Web Scraping Use Cases

For more on web scraping use case examples see our in-depth use case directory.

Yes. While laws can vary by country generally web scraping process is perfectly legal as long as these general rules are followed:

  • The scraped content is public or belongs to you
  • The scraping process doesn't damage the website
Web scraping is still a very young niche and there's a lot of nuance though it's safe to say web scraping is generally considered to be ethical and legal.

The Web Scraping Process

Web scraping is a unique programming field as it requires a diverse knowledge of many technology subjects. Most notably: web development — both front-end and back-end — networking, databases, security and more.

Given proper training web scraping can be relatively simple and accessible to anyone. That's the goal of Scrapfly Academy!

We'll start with a skim overview of the web scraping field and then start exploring everything through themed examples. Use the arrows below to navigate to the next page

< >

Summary