Puppeteerを導入してみた話

Puppeteerを実際に使ってみて良かった点やハマった点について書いてあります。

1. ©2018 Wantedly, Inc. PuppeteerΛಋೖͯ͠Έͨ࿩ We Are JavaScripters! #28 25.Jan.2019 - Naoki Kobayashi
2. ©2018 Wantedly, Inc. Naoki Kobayashi GitHub: @kobayang Twitter: @kbys_02 I’m an Engineer @Wantedly React / Rails React Native࢝Ί·ͨ͠
3. ©2018 Wantedly, Inc. Agenda - Puppeteer ͱ͸ - ͲΜͳͱ͜ΖͰ࢖ΘΕΔ͔ - ྫ - Puppeteer ͷಋೖ - ಋೖͨ͠৔ॴ - Α͔ͬͨͱ͜ΖɾϋϚͬͨͱ͜Ζ - ࡶײ
4. ©2018 Wantedly, Inc. Puppeteer ͱ͸
5. ©2018 Wantedly, Inc. Puppeteer ͱ͸
6. ©2018 Wantedly, Inc. ͲΜͳͱ͜ΖͰ࢖ΘΕΔʁ JS HTML Ϧιʔε ϒϥ΢β͕ղऍ ղऍ݁Ռ ΫϥΠΞϯτଆ
7. ©2018 Wantedly, Inc. ͲΜͳͱ͜ΖͰ࢖ΘΕΔʁ JS HTML Ϧιʔε ϒϥ΢β͕ղऍ ղऍ݁Ռ αʔόʔଆͰ΋ղऍ݁Ռ͕औΕΔͱศརͳ৔߹͕͋Δ
8. ©2018 Wantedly, Inc. ͲΜͳͱ͜ΖͰ࢖ΘΕΔʁ • Generate screenshots and PDFs of pages. • Crawl a SPA (Single-Page Application) and 
 generate pre-rendered content (i.e. "SSR"). • Automate form submission, UI testing, keyboard input, etc. • Create an up-to-date, automated testing environment. • Capture a timeline trace of your site • Test Chrome Extensions.
9. ©2018 Wantedly, Inc. ͲΜͳͱ͜ΖͰ࢖ΘΕΔʁ • Generate screenshots and PDFs of pages. • Crawl a SPA (Single-Page Application) and 
 generate pre-rendered content (i.e. "SSR"). • Automate form submission, UI testing, keyboard input, etc. • Create an up-to-date, automated testing environment. • Capture a timeline trace of your site • Test Chrome Extensions.
10. ©2018 Wantedly, Inc. SSR Example https://developers.google.com/web/tools/puppeteer/articles/ssr import puppeteer from “puppeteer"; async function ssr(url) { const browser = await puppeteer.launch({ headless: true }); const page = await browser.newPage(); await page.goto(url, { waitUntil: "networkidle0" }); const html = await page.content(); // serialized HTML of page DOM. await browser.close(); return html; }
11. ©2018 Wantedly, Inc. Puppeteer ͷಋೖ
12. ©2018 Wantedly, Inc. Ͳ͜Ͱ࢖͔ͬͨʁ - OGը૾ͷΧελϚΠθʔγϣϯ - ࢦఆͨ͠ը૾ʹ೔෇ͷόονΛ
 ৐ͤͨը૾Λฦ͢ - WantedlyͰ͸࠷ۙϛʔτΞοϓػ ೳΛϦϦʔεͨ͠ - γΣΞ͞Εͨը૾͕ϛʔτΞοϓ Ͱ͋Δ͜ͱΛ෼͔Γ΍͍ͨ͘͢͠
13. ©2018 Wantedly, Inc. Ͳ͜Ͱ࢖͔ͬͨʁ
14. ©2018 Wantedly, Inc. Puppeteer v.s. node-canvas - node-canvas ͳΒը૾ΛՃ޻ͯ͠ฦ͢͜ͱ͕Ͱ͖ͦ͏ - ଞͷࢪࡦͰ͢Ͱʹ࢖༻͞Ε͍ͯͨ (pros) - node-canvas༻ͷAPIΛΘ͟Θ֮͑ͯ͟σβΠϯΛ౰ͯΔͷ͸໘౗ͦ͏ (cons) - Puppeteer ͩͱ screenshot API Λ࢖͑͹ྑͦ͞͏ - গ͠ͷPuppeteerͷAPI͚֮ͩ͑Ε͹ྑͦ͞͏ (pros) - ελΠϧ͸CSS͕࢖͑ΔͷͰσβΠϯͷมߋʹ΋͙͢ରԠͰ͖ͦ͏ (pros) - ଎౓໘ɾ҆ఆੑͷෆ҆ (cons) - DockerͰಈ͔͘ෆ҆ (cons)
15. ©2018 Wantedly, Inc. Puppeteer v.s. node-canvas - OGը૾͸֤SNSʹΩϟογϡ͞ΕΔ - ଎౓΍ΞΫηεͷෛՙʹ͍ͭͯ͸͋·Γߟ͑ͳͯ͘ྑ͍ - ೔෇෦෼ͷσβΠϯ͕ޙʑมΘΔՄೳੑ͕ߴ͍ - node-canvasͰॻ͍ͨ΋ͷΛॻ͖׵͑Δͷ͸ίετ ࠓճͷέʔε Puppeteer Λબ୒
16. ©2018 Wantedly, Inc. ྑ͔ͬͨ͜ͱ - htmlΛฦ͢endpointͱɺPuppeteer͔ΒhtmlΛݺΜͰscreenshotΛऔΔendpointΛ࣮૷ - ೔෇ΛೖΕͨOGը૾Λฦ͢෦෼͸؆୯ʹ࣮૷Ͱ͖ͨ - ελΠϧͷ֬ೝ࣌ʹ͸ html Λฦ͢෦෼͚ͩݟΕ͹͍͍ͷͰ֬ೝ͕༰қ ϋϚͬͨ͜ͱ - DockerͰಈ͔͢ͷ͕গ͠໘౗ - webfont͕දࣔ͞Εͳ͍৔߹͕͋ͬͨ - Puppeteerͷىಈ͕஗͍
17. ©2018 Wantedly, Inc. DockerͰಈ͔͢ͷ͕গ͠໘౗ - `npm install puppeteer` Ͱ͸ಈ͔ͳ͍ - Alpine linux Ͱ͸ಈ͔ͤͳ͔ͬͨ - https://github.com/GoogleChrome/puppeteer/blob/master/docs/ troubleshooting.md#running-puppeteer-in-docker 
 ʹৄ͘͠ॻ͍ͯ͋ΔͷͰɺ͜ΕΛࢀߟʹ࣮૷
18. ©2018 Wantedly, Inc. webfont͕දࣔ͞Εͳ͍৔߹͕͋ͬͨ - “networkidle0”Λࢦఆ͢Δ͜ͱͰɺωοτϫʔΫ͕খ߁ঢ়ଶʹͳΔ·Ͱ଴ͬͯ͘ΕΔ - “networkidle0”Λࢦఆ͢Δͱɺࠓޙ͸ɺӬԕʹ଴ͪଓ͚Δ৔߹͕ى͖ͯ͠·ͬͨ - issue΋͋Δ https://github.com/GoogleChrome/puppeteer/issues/1908 - ݁ہwebfont͸ϩʔΧϧʹೖΕͯ࢖༻͢Δ͜ͱʹͨ͠
19. ©2018 Wantedly, Inc. Puppeteerͷىಈ͕஗͍ - ৭ΜͳExampleͰ͸ΤϯυϙΠϯτͷ࠷ॳʹ`puppeteer.launch`Λ͍ͯ͠Δ͕ɺ
 `puppeteer.launch` ͰͷPuppeteerͷ্ཱ͕ͪ͛ඇৗʹ஗͍ - 1ඵҎ্͔͔ͬͯ͠·ͬͨ - ͞ΒʹPuppeteerΛ্ཱͪ͛ͨͱ͖ʹɺchromeͷϓϩηε͕Ұ্ཱ͕ͭͪΔͷͰɺ
 ΞΫηε͕૿͑ͨͱ͖ʹෆ҆ - αʔόʔͷىಈ࣌ʹ `puppeteer.launch` Λ͓͍ͯͯ͠ɺsingleton Ͱ
 puppeteerͷঢ়ଶΛ؅ཧͨ͠ - Ϩεϙϯε࣌ؒ͸େମ300-1000ms͘Β͍ʹͳͬͨ
20. ©2018 Wantedly, Inc. [NOTE] Ϩεϙϯε࣌ؒ - தԝ஋Ͱ550ms΄Ͳ - ࣮ࡍʹ͸ΤϯυϙΠϯτͷ಺෦ͰAPIΛୟ͖ʹߦ͍ͬͯΔͷͰɺ
 Puppeteerࣗମͷॲཧ࣌ؒ͸΋͏গ͠୹͍
21. ©2018 Wantedly, Inc. ࡶײ - ௐࠪΛ࢝Ί͔ͯΒ3೔ఔ౓Ͱ࣮૷·ͰͰ͖ͨ - Puppeteerͷֶशίετ͸௿͍ʢجຊతʹ Example ͱҰॹʣ - ϋϚΓϙΠϯτ͸͍͔ͭ͋ͬͨ͘ɻಛʹαʔόʔͰಈ͔͢ͱ͖ɻ - ଎౓͸ग़ͳ͍ɻ଎౓͕ඞཁʹͳΔ৔߹͸ΩϟογϡͳͲ޻෉͢Δඞཁ͕͋Δ - ࣮૷ͨ͠ޙʹେ͖ͳ໰୊͸ى͖ͳ͔ͬͨ Α͍PuppeteerϥΠϑΛʂ