Greetings to headless Chrome

What headless browsers are, why we need them and what we can do with Chrome browser in headless mode.

Transcript

1. GreETinGs To tHE heADleSs ChRomE Marina Fadeeva
2. AboUT me
3. AboUT me • Mobile Web QA at Badoo • Worked as QA Lead in Yandex.Music • LinkedIn: www.linkedin.com/in/ marina-fadeeva
4. badoo.com
5. Available on multiple platforms: baDOo In NumBErS 350m Messages sent per day Registered users worldwide Monthly active users 349m 60m </> </> Mobile Web Desktop Web Android iOS Windows Phone
6. AGenDA •Badoo MW team QA process •Headless browsers •Existing solutions •Headless Chrome •Examples •Conclusions
7. TesTInG in BAdoO
8. wHat My TeAm DoEs •In Mobile Web team in Badoo we build and then test web applications •We test in different ways: •manually •with functional automated tests •with visual regression tests •Testing of web applications requires
 usage of browsers (LOL)
9. teStINg FloW In BadOo Visual Regression tests Functional automated tests Testing in branch Manual QA Manual QA Testing in build Visual Regression tests Functional automated tests Prod DEV Development
10. HEadLEsS bROwSer
11. WhaT Is a hEaDlESs BroWsER? •What is the solution? •Ordinary browsers need GUI, but… •Servers for automation tests don't have GUI. •We need a browser without GUI •Headless browser is a web browser without GUI
12. USe cASes FOr a hEaDlESs BroWsER •Test automation •Scraping web sites for data •Taking screenshots of web pages •Generating PDF •☠
13. BroWsERs uSAge PAtTerNs Human Browser Website Website Robot Headless Browser Browsers for humans Browsers for robots I/O devices API
14. BroWsERs uSAge PAtTerNs Human Browser Website Client Server Website Robot Headless Browser Browsers for humans Browsers for robots I/O devices API
15. How TO usE BroWsER on GUi-lESs MacHIne? Browser UI Rendering Engine API Browser Engine
16. How TO usE BroWsER on GUi-lESs MacHIne? Browser UI Rendering Engine API Browser Engine User I/O
17. How TO usE BroWsER on GUi-lESs MacHIne? Browser UI Rendering Engine API Browser Engine User I/O X-test xvfb
18. How TO usE BroWsER on GUi-lESs MacHIne? Browser UI Rendering Engine API Browser Engine User I/O Headless test X-test xvfb
19. How TO usE BroWsER on GUi-lESs MacHIne? Browser UI Rendering Engine API Browser Engine User I/O Headless test X-test xvfb
20. SomE HeAdLesS bRowSErS •PhantomsJS
 phantomjs.org •Nightmare
 www.nightmarejs.org •Google Chrome with Xvfb •Headless Chromium
 www.google.com/chrome/browser/canary.html •And many others…
 github.com/dhamaniasad/HeadlessBrowsers
21. PhaNtOM jS
22. PhaNtOMjS... •Scriptable with a JavaScript API •Based on QtWebKit, fork of WebKit •Supports various web standards: DOM handling, CSS selector, JSON, Canvas, and SVG •There is a lot of testing frameworks integrated with it: •Jasmine, QUnit, Mocha, Capybara etc
23. exAMpLe: lET’s TakE a ScReEnShoT Open a 
 web page Take a screenshot
 of the page Store the screenshot
 to the disk
24. ScReEnShoT - wiTh PhaNtOMjS var webPage = require('webpage'); 
 var page = webPage.create(); 
 
 page.viewportSize = { width: 1080, height: 1920 }; 
 
 page.open("http://m.badoo.com", function start(status) { 
 page.render('mbadoo.jpeg', { 
 format: 'jpeg',
 quality: '100'
 });
 phantom.exit(); 
 }); 

25. laUNcH it… Run script app.js $ phantomjs app.js Bingo!
26. …ScReEnShoT!
27. ProS & coNs OF pHanTOmJs Advantages •Popular •Big community •Open source Disadvantages •Not a real browser •Doesn’t support xPath, video&audio, WebGL etc. •Bugs everywhere •Maintainer doesn’t support it anymore
28. The EnD of AN erA " https://groups.google.com/forum/#!topic/phantomjs/9aI5d-LDuNE
29. X viRtUaL fRamEBufFEr
30. WhaT Is XvFb ? •X Virtual Framebuffer •In-memory display server for UNIX-like operating systems •Acts like a real X display server •Works with any real browser you choose
31. How TO inTEraCt WitH cHroME + XvFb? User I/O Headless test Browser UI Rendering Engine API Browser Engine
32. How TO inTEraCt WitH cHroME + XvFb? User I/O Headless test Browser UI Rendering Engine API Browser Engine X-test xvfb
33. usINg ChRomE + xVfB in BAdoO Advantages •Acts exactly like the the real display •Allows to use any browser you need •And all this is done on a server Disadvantages •More complex environment = less stable environment
34. AutOMatIoN EnVirONmeNt IN baDOo Functional Tests Selenium Grid Nodes Selenium ChromeDriver Chrome xvfb
35. HEadLEsS cHrOMe
36. GenERal OVerVIew •Headless Chrome allows to run Chrome in a headless/server environment •Headless mode is now available with Chrome 59 on Mac and Linux •Get it here:
 www.google.com/chrome/browser/desktop/ index.html •Windows support is coming soon!
37. DifFEreNcE FroM rEGulAR cHroME What do we get? •Blink with the latest web platform awesomeness •Profiling and introspection tools 
 (DevTools UI, tracing, etc.) •Everything below the User Interface What is missing? •No Chrome-level features (e.g., profiles, sync ...) •No UI - offscreen software rendering •No extensions (for now)
38. How TO inTEraCt WitH heADleSs ChRomE ? Browser UI Rendering Engine DevTools Protocol Browser Engine Client User I/O X-test xvfb
39. diFfERenT waYs TO coNtROl iT Headless Chrome can be driven in several ways: •via command line flags •using Chrome as a client •programmatically All of them use the DevTools Protocol.
40. ComMAnD liNE fLagS 1
41. StaRt ChRomE --heADleSs $ chrome --headless --disable-gpu https://badoo.com Temporarily needed (for now) Runs Chrome in a headless mode URL to be opened by Chrome
42. FlaGs = NO coDInG rEQuIreD! Window Size $ chrome --window-size=800,500 --headless
 --disable-gpu https://google.com Take a screenshot $ chrome --screenshot --headless
 --disable-gpu https://badoo.com
43. FlaGs = NO coDInG rEQuIreD! Window Size $ chrome --window-size=800,500 --headless
 --disable-gpu https://google.com Take a screenshot $ chrome --screenshot --headless
 --disable-gpu https://badoo.com Create a PDF $ chrome --print-to-pdf --headless
 --disable-gpu https://en.wikipedia.org/wiki/ Unicorn
44. FlaGs = NO coDInG rEQuIreD! The full list of commands: https://cs.chromium.org/chromium/src/headless/ app/headless_shell_switches.cc
45. reAL bRowSEr 2
46. UsiNg A reAL bRowSEr •Run Chrome with --remote-debugging-port=9222 •It starts an instance with the DevTools protocol enabled •Navigate to http://localhost:9222 in another browser •You'll see a list of inspectable pages •You can click through and see what Headless is rendering
47. deVEloPEr ToOlS, we KnOW tHem •You can use the familiar DevTools features •A powerful debugging tool
48. leT’s SeE a dEMo
49. leT’s SeE a dEMo
50. pRogRAmMatICalLy 3
51. DevTOolS pRotOCol •Allows to instrument, inspect, debug and profile Chromium, Chrome and other Blink-based browsers •Chrome DevTools use this protocol •The messaging is based on serialised JSON objects •Documentation here:
 chromedevtools.github.io/devtools-protocol/
52. DevTOolS pRotOCol •Allows to instrument, inspect, debug and profile Chromium, Chrome and other Blink-based browsers •Chrome DevTools use this protocol •The messaging is based on serialised JSON objects •Documentation here:
 chromedevtools.github.io/devtools-protocol/ Chrome DevTools
 Protocol Chrome Debugging Protocol = Chrome Remote Protocol = The
 SamE!
53. VerSIonS Of The PrOTocOL •The latest “tip-of-tree” (tot) •Changes frequently and can break at any time •Captures the full capabilities of the protocol •v8-inspector (v8) •Available in node 6.3+ •Enables debugging & profiling of Node.js apps •Stable 1.2 (1-2) •Stable release of the protocol (Chrome 54) •Smaller subset of the complete protocol
54. ThiRd-ParTy LibRAriES foR tHe pROtoCOl •Allow to write testing application using your favourite programming language: •Python •NodeJS •Go •etc. •Not all of them are in a good shape :) •More informations here:
 developer.chrome.com/devtools/docs/debugging- clients
55. AgaIN: leT’s TakE a ScReEnShoT Open a 
 web page Take a screenshot
 of the page Store the screenshot
 to the disk
56. PyThoN + ChRomE DevTOolS pRotOCol •We’re implementing a screenshot maker •We use the Python package Chromote •It’s a basic wrapper for DevTools Protocol
57. cLiEnT coDE (apP.pY) import base64 import json from chromote import Chromote chrome = Chromote() tab = chrome.tabs[0] tab.set_url('https://m.badoo.com') tab._send({"method": "Emulation.setVisibleSize", "params": {"width": 1080, "height": 1920}}) res = tab._send({"method": "Page.captureScreenshot", "params": {"fromSurface": True}}) res = json.loads(res) with open("imageToSave.png", "wb") as fh: fh.write(base64.decodebytes(str.encode(res['result']['data'])))
58. • cLiEnT coDE (apP.pY) import base64 import json from chromote import Chromote chrome = Chromote() tab = chrome.tabs[0] tab.set_url('https://m.badoo.com') tab._send({"method": "Emulation.setVisibleSize", "params": {"width": 1080, "height": 1920}}) res = tab._send({"method": "Page.captureScreenshot", "params": {"fromSurface": True}}) res = json.loads(res) with open("imageToSave.png", "wb") as fh: fh.write(base64.decodebytes(str.encode(res['result']['data'])))
59. Sends JSON Receives JSON CliENt + SerVEr iNtERacTIon {“method": “Emulation.setVisibleSize", “params": {"width": 500,"height": 500}} {"some":"data"} Client Code Server Headless Browser
60. laUNcH it… Launch Chrome in headless mode $ chrome --headless --remote-debugging-port=9222 --disable-gpu Run script app.py $ python app.py Bingo!
61. …sCreENsHot!
62. OtHer EXamPlES •Running Selenium With The New Headless Chrome
 intoli.com/blog/running-selenium-with-headless-chrome/ •Headless Chrome Protocol Exploration (plain Python)
 gist.github.com/llimllib/ 7f6143a1a6955d243161b2fec23b8f4d •How to finally use headless Chrome to power your automated tests (Headless Chrome + capybara)
 blog.faraday.io/headless-chromium/ •Using headless Chrome as an automated screenshot tool (Node.js)
 medium.com/@dschnr/using-headless-chrome-as-an- automated-screenshot-tool-4b07dffba79a
63. SOme USefUL liNkS •Getting Started with Headless Chrome: developers.google.com/web/updates/2017/04/ headless-chrome •Chrome DevTools Protocol Viewer:
 chromedevtools.github.io/devtools-protocol/ •Headless Chrome bugs:
 bugs.chromium.org/p/chromium/issues/list •DevTools Protocol:
 github.com/ChromeDevTools/devtools-protocol
64. The FUtuRE
65. ThE BriGhT fUTurE •Headless Chrome for Windows •Saving and restoring browser sessions •Working with pop-up windows •File downloads to local folder •The Headless Chrome team offered PhantomJS to move from WebKit fork to Chromium • #$
66. “HeAdLesS moDE” acROsS bRowSErS •Status in Chromium •Chrome for desktop: release 59 •Opera: release 46 •Consensus & Standardization •Firefox: In development •Edge: No public signals •Safari: No public signals •Web Devs and QAs: Positive and Enthusiastic
67. HeAdLesS cHroME & BadOo We wILl HavE it. or NOt.
68. ConClUSiOnS
69. WhY heADleSs ChRomE is COol? •Brings all modern web platform features to the headless browser world •Powerful debugging tool - you see what your testing application sees •Can be run on server •Uses the latest version of Blink •Supported by Google (Chromium Community)
70. WhY it IS noT sO CoOl (Yet) ? •The DevTools Protocol currently does not support multiple clients connected to the protocol simultaneously •There are only low-level APIs. High-level APIs are currently supported only by third-party clients. •Bleeding edge of development
71. • it’S coMInG!
72. Marina Fadeeva tHanK
 yoU!