What's the difference between Web Scraping and Crawling?

Web crawling is a form of web scraping with exploration capabilities.
Web scraping tends to focus on specific targets like "a list of product URLs of X e-commerce website" while crawling extends this by adding exploration logic. Crawlers can find all product links in addition to scraping them by exploring the website.

Generally, most web scrapers implement some crawling functionality. For example, if we want to scrape products from an e-commerce website first we'd scrape the page where product URLs are located (like product category) and then scrape each product - this would be a mild form of crawling.

More extreme forms of crawling, like "broad crawling", can crawl and scrape many different website domains and many different web structures. For example, search engines like Google or Bing broad crawl the web and index the contents for search capabilities.

How to Crawl the Web with Python

For more on crawling, see our full introduction tutorial for web crawling with Python