Web Scraping with Python and BeautifulSoup
Beautifulsoup is one the most popular libraries in web scraping. In this tutorial, we'll take a hand-on overview of how to use it, what is it good for and explore a real -life web scraping example.
When web scraping, sometimes it's easier to select a value by finding its sibling first. Using Python and Beautifulsoup, to find element siblings we can use
find_all() methods or CSS selectors and the
import bs4 soup = bs4.BeautifulSoup(""" <span class="price">Price including shipping:</span> <span>83.13</span> """) # using find() method: soup.find('span', class_="price").find_next_sibling('span').text "83.13" # using CSS selectors and `~` notation: soup.select('.price ~ span').text # or soup.select_one('.price ~ span').text "83.13"