CSS selectors are one of the most popular ways to parse HTML pages when web scraping. In NodeJS and Puppeteer, CSS selectors can be used through the page.$ and page.$$ methods:
javascript
const puppeteer = require('puppeteer');
async function run() {
const browser = await puppeteer.launch();
const page = await browser.newPage();
await page.goto("https://httpbin.dev/html");
// to get the first matching element:
await page.$("p");
// to get ALL matching elements:
await page.$$("p");
// we can also modify the captured elements immediatly:
// get the text value:
await page.$eval("p", element => element.innerText);
// get attributes attribute:
await page.$eval("a", element => element.href);
// same with multiple elements, like count total appearances:
await page.$$eval("p", elements => elements.length)
browser.close();
}
run();âš It's possible that these commands will try to find elements before the page has fully loaded if it's a dynamic javascript page. For more see how-to-wait-for-page-to-load-in-puppeteer
