JSON and XML are two major data formats encountered in web development — here's how they differ and which is one better for your use case.
Data Parsing Knowledgebase
How to scrape HTML table to Excel Spreadsheet (.xlsx)?
To scrape tables to Excel spreadsheet we can use bs4, requets and xlsxwriter packages for Python. Here's how.
How to parse dynamic CSS classes when web scraping?
Dynamic CSS can make be very difficult to scrape. There are a few tricks and common idioms to approach this though.
How to turn HTML to text in Python?
To turn HTML data to text in Python we can use BeautifulSoup's get_text() method which strips away HTML data and leaves text as is. Here's how.
Scraper doesn't see the data I see in the browser - why?
This means that scraper is not rendereding javascript that is changing the page contents. To verify this disable javascript in your browser.
Related
Related Blog Posts
JSON vs XML: Key Differences and Modern Uses
What is Parsing? From Raw Data to Insights
Learn about the fundamentals of parsing data, across formats like JSON, XML, HTML, and PDFs. Learn how to use Python parsers and AI models for efficient data extraction.
Web Scraping With Go
Learn web scraping with Golang, from native HTTP requests and HTML parsing to a step-by-step guide to using Colly, the Go web crawling package.
Intro to Parsing HTML and XML with Python and lxml
In this tutorial, we'll take a deep dive into lxml, a powerful Python library that allows for parsing HTML and XML effectively. We'll start by explaining what lxml is, how to install it and using lxml for parsing HTML and XML files. Finally, we'll go over a practical web scraping with lxml.
How to Parse XML
In this article, we'll explain about XML parsing. We'll start by defining XML files, their format and how to navigate them for data extraction.
Web Scraping to Google Sheets
Google sheets is an easy to store scraped data. In this tutorial we'll take a look at how to use this free online database for storing scraped data!
Web Scraping Emails using Python
In this tutorial we'll take a look at email scraping. How to crawl pages and extract email addresses using Python and what are some popular challenges.
Web Scraping Phone Numbers with Python
In this article we'll dive into phone number scraping. We'll explore an example object and cover common phone number scraping challenges like obfuscation.
Intro to Web Scraping Images with Python
In this guide, we’ll explore how to scrape images from websites using different methods. We'll also cover the most common image scraping challenges and how to overcome them. By the end of this article, you will be an image scraping master!
Ultimate XPath Cheatsheet for HTML Parsing in Web Scraping
Ultimate companion for HTML parsing using XPath selectors. This cheatsheet contains all syntax explanations with interactive examples.
Ultimate CSS Selector Cheatsheet for HTML Parsing
Ultimate companion for HTML parsing using CSS selectors. This cheatsheet contains all syntax explanations with interactive examples.
JSON Parsing Made Easy with ChatGPT in Web Scraping
ChatGPT web scraping techniques allow for faster web scraping development. Here's how you can save a lot of time parsing JSON data with the help of chatGPT!
Finding Hidden Web Data with ChatGPT Web Scraping
ChatGPT can help with different tasks including hidden data scraping. In this article, we’ll know about hidden data and how to use ChatGPT to find hidden web data. We will also scrape hidden data on a real website.
How to Parse Datetime Strings with Python and Dateparser
Dateparser is a popular Python package for parsing datetime strings. Here's how it can be used in web scraping and how to avoid common problems.
How to Scrape Sitemaps to Discover Scraping Targets
Usually to find scrape targets we look at site search or category pages but there's a better way - sitemaps! In this tutorial, we'll be taking a look at how to find and scrape sitemaps for target locations.