WorkerConf 2021 - Browser automation with Puppeteer

Transcript

1. @JecelynYeen Developer advocate, Google @ChromeDevTools
3. My road bike
4. Mountain bike My road bike
5. Mountain bike Hiking, hot and humid My road bike
6. Mountain bike Hiking, hot and humid Hiking, Switzerland My road bike
8. Browser automation with Puppeteer
9. PM YANG Professional developer PAUL
10. PM YANG I want a daily email on our server’s internet speed. Professional developer PAUL
11. PM YANG I want a daily email on our server’s internet speed. Professional developer PAUL Why does he need that?
12. PM YANG It’s easy. Go to fast.com on the server and send me the screenshot.
13. PM YANG It’s easy. Go to fast.com on the server and send me the screenshot. Professional developer PAUL I never said it’s hard...
16. Professional developer PAUL There are 2 ways to do this: Manually 1. Login to server 2. Open a browser 3. Go to fast.com 4. Take a screenshot 5. Send email
17. Professional developer PAUL There are 2 ways to do this: Manually 1. Login to server 2. Open a browser 3. Go to fast.com 4. Take a screenshot 5. Send email Automate it 1. Schedule a daily server task 2. Automate screenshot 3. Automate email sending
18. Professional developer PAUL Researching...
19. Professional developer PAUL Researching... Puppeteer
21. Browser automation library for Node.js
22. Control a browser with modern JavaScript API.
23. const puppeteer = require('puppeteer'); (async () => { })();
24. const puppeteer = require('puppeteer'); (async () => { const browser = await puppeteer.launch(); })();
25. const puppeteer = require('puppeteer'); (async () => { const browser = await puppeteer.launch(); const page = await browser.newPage(); })();
26. const puppeteer = require('puppeteer'); (async () => { const browser = await puppeteer.launch(); const page = await browser.newPage(); await page.goto('https://fast.com/'); })();
27. const puppeteer = require('puppeteer'); (async () => { const browser = await puppeteer.launch(); const page = await browser.newPage(); await page.goto('https://fast.com/'); await page.screenshot({ path: 'img/screenshot-yyyyMMdd.png' }); })();
28. const puppeteer = require('puppeteer'); (async () => { const browser = await puppeteer.launch(); const page = await browser.newPage(); await page.goto('https://fast.com/'); await page.screenshot({ path: 'img/screenshot-yyyyMMdd.png' }); await browser.close(); })();
29. Not working
31. const puppeteer = require('puppeteer'); (async () => { const browser = await puppeteer.launch(); const page = await browser.newPage(); await page.goto('https://fast.com/'); await page.waitForSelector('.speed-results-container.succeeded'); await page.screenshot({ path: 'img/screenshot-yyyyMMdd.png' }); await browser.close(); })();
32. Run it again
33. Professional developer PAUL
34. PM YANG Cool, it will be great if to have date in the screenshot. Possible?
35. const puppeteer = require('puppeteer'); (async () => { const browser = await puppeteer.launch(); const page = await browser.newPage(); await page.goto('https://fast.com/'); await page.waitForSelector('.speed-results-container.succeeded'); await page.$eval('#your-speed-message', (el, dt) => { el.innerText = `Internet speed as of ${dt} is`; }, 'yyyy-MM-dd')); await page.screenshot({ path: `img/screenshot-yyyyMMdd.png` }); await browser.close(); })();
36. PM YANG I want...
37. // Save as pdf, only work in headless mode await page.pdf({ path: `doc/speed-yyyyMMdd.pdf` }); // Click to show more details await page.click('#show-more-details-link'); await page.waitForSelector('#upload-label.succeeded', { timeout: 0 }); // Emulate mobile view and save screenshot // src/common/DeviceDescriptors.ts const tablet = puppeteer.devices['iPad Mini']; await page.emulate(tablet); await page.screenshot({ path: `img/screenshot-yyyyMMdd.png` }); // Reload page await page.reload();
38. // Save as pdf, only work in headless mode await page.pdf({ path: `doc/speed-yyyyMMdd.pdf` }); // Click to show more details await page.click('#show-more-details-link'); await page.waitForSelector('#upload-label.succeeded', { timeout: 0 }); // Emulate mobile view and save screenshot // src/common/DeviceDescriptors.ts const tablet = puppeteer.devices['iPad Mini']; await page.emulate(tablet); await page.screenshot({ path: `img/screenshot-yyyyMMdd.png` }); // Reload page await page.reload();
39. // Save as pdf, only work in headless mode await page.pdf({ path: `doc/speed-yyyyMMdd.pdf` }); // Click to show more details await page.click('#show-more-details-link'); await page.waitForSelector('#upload-label.succeeded', { timeout: 0 }); // Emulate mobile view and save screenshot // src/common/DeviceDescriptors.ts const tablet = puppeteer.devices['iPad Mini']; await page.emulate(tablet); await page.screenshot({ path: `img/screenshot-yyyyMMdd.png` }); // Reload page await page.reload();
40. // Save as pdf, only work in headless mode await page.pdf({ path: `doc/speed-yyyyMMdd.pdf` }); // Click to show more details await page.click('#show-more-details-link'); await page.waitForSelector('#upload-label.succeeded', { timeout: 0 }); // Emulate mobile view and save screenshot // src/common/DeviceDescriptors.ts const tablet = puppeteer.devices['iPad Mini']; await page.emulate(tablet); await page.screenshot({ path: `img/screenshot-yyyyMMdd.png` }); // Reload page await page.reload();
41. github.com/puppeteer
42. ❤
43. const browser = await puppeteer.launch(); const page = await browser.newPage(); await page.goto('https://v8.dev/blog/10-years'); // Create screenshots of your web app in Light Mode and Dark Mode. // Tip: check out `npx dark-mode-screenshot` for more options! await page.emulateMediaFeatures([{ name: 'prefers-color-scheme', value: 'light' }]); await page.screenshot({ path: 'light.png', fullPage: true }); await page.emulateMediaFeatures([{ name: 'prefers-color-scheme', value: 'dark' }]); await page.screenshot({ path: 'dark.png', fullPage: true }); await browser.close();
44. const browser = await puppeteer.launch(); const page = await browser.newPage(); await page.goto('https://mathiasbynens.be/demo/timezone'); page.evaluate(() => { globalThis.date = new Date(1479579154987); }); await page.emulateTimezone('America/Buenos_Aires'); console.log(await page.evaluate(() => date.toString())); // → 'Sat Nov 19 2016 15:12:34 GMT-0300 (Argentina Standard Time)' await page.emulateTimezone('Asia/Kuala_Lumpur'); console.log(await page.evaluate(() => date.toString())); // → 'Sun Nov 20 2016 02:12:34 GMT+0800 (Malaysia Time)' await browser.close();
45. const browser = await puppeteer.launch(); const page = await browser.newPage(); await page.goto('https://developer.chrome.com/tags/devtools/'); await page.emulateVisionDeficiency('blurredVision'); await page.screenshot({ path: 'blurred-vision.png', fullPage: true }); await page.emulateVisionDeficiency('achromatopsia'); await page.screenshot({ path: 'achromatopsia.png', fullPage: true }); await page.emulateVisionDeficiency('deuteranopia'); await page.screenshot({ path: 'deuteranopia.png', fullPage: true }); await browser.close();
46. addyosmani.com/blog/ puppeteer-recipes/
47. const browser = await puppeteer.launch();
48. const browser = await puppeteer.launch(); // Launch options const browser = await puppeteer.launch({ product: 'firefox' });
49. const browser = await puppeteer.launch(); // Launch options const browser = await puppeteer.launch({ product: 'firefox' }); const browser = await puppeteer.launch({ headless: false, slowMo: 3000 });
50. const browser = await puppeteer.launch(); // Launch options const browser = await puppeteer.launch({ product: 'firefox' }); const browser = await puppeteer.launch({ headless: false, slowMo: 3000 }); const browser = await puppeteer.launch({ devtools: true });
51. goo.gle/devtools-blog
52. @JecelynYeen Developer advocate, Google @ChromeDevTools