How to Scrape Google Maps
We'll take a look at to find businesses through Google Maps search system and how to scrape their details using either Selenium, Playwright or ScrapFly's javascript rendering feature - all of that in Python.
To scroll to an element in Selenium we can find the HTML element using CSS or XPath selectors and execute javascript scrollIntoView()
function:
from selenium import webdriver
from selenium.webdriver.common.by import By
driver = webdriver.Chrome()
driver.get("https://some-url.com")
# find element to scroll to. In this example we select last element with product class:
element = driver.find_elements(By.CSS_SELECTOR, '.products .product')[-1]
# execute scrollIntoView script with our element as the argument:
driver.execute_script(
"arguments[0].scrollIntoView(scrollIntoView({ behavior: 'smooth', block: 'end', inline: 'end' });",
element
)
driver.close()
In the example above we're using scrollIntoView with smooth type of scrolling which performs more like a real user. We also specify block and inline arguments to end
value to scroll to the bottom of the horizontal right of the element - this ensures highest visibility of the element. For more see browser's scrollIntoView documentation