Ultimate Guide to JSON Parsing in Python
Learn JSON parsing in Python with this ultimate guide. Explore basic and advanced techniques using json, and tools like ijson and nested-lookup
How to turn HTML to text in Python?
When web scraping, we might need to represent scrape HTML data as plain text. For this we can use BeautifulSoup's get_text() method which extracts all visible HTML text and most importantly ignores invisible details such as <script> elements:
from bs4 import BeautifulSoup
soup = BeautifulSoup("""
<body>
<article>
<h1>Article title</h1>
<p>first paragraph and a <a>link</a></p>
<script>var invisible="javascript variable";</script>
</article>
</body>
""")
# if possible it's best to restrict html to a specific element
element = soup.find('article')
text = element.get_text()
print(text)
"""
Article title
first paragraph and a link
"""
Provided by Scrapfly
This knowledgebase is provided by Scrapfly data APIs, check us out! 👇
- Web Scraping API - scrape without blocking, control cloud browsers, and more.
- Extraction API - AI and LLM for parsing data.
- Screenshot API - capture pages or elements with no blocks.
Guide to Parsel - the Best HTML Parsing in Python
Learn to extract data from websites with Parsel, a Python library for HTML parsing using CSS selectors and XPath.
JSONL vs JSON
Learn the differences between JSON and JSONLines, their use cases, and efficiency. Why JSONLines excels in web scraping and real-time processing
Web Scraping and HTML Parsing with Jsoup and Java
Learn how to harness the power of jsoup, a lightweight and efficient Java library for web scraping and HTML parsing.