When web scraping, sometimes it's easier to select a value by finding its sibling first. Using Python and Beautifulsoup, to find element siblings we can use find()
and find_all()
methods or CSS selectors and the select()
method:
import bs4
soup = bs4.BeautifulSoup("""
<span class="price">Price including shipping:</span>
<span>83.13</span>
""")
# using find() method:
soup.find('span', class_="price").find_next_sibling('span').text
"83.13"
# using CSS selectors and `~` notation:
soup.select('.price ~ span')[0].text
# or
soup.select_one('.price ~ span').text
"83.13"