by Kohei Ueno

Slide 1

Slide 1 text

Universal DevTools for all environments ্໺߁ฏKohei Ueno 2021/11/27

Slide 2

Slide 2 text

্໺ ߁ฏ / Kohei Ueno Software Engineer@LINE https://twitter.com/cola119_ https://github.com/cola119 👶

Slide 3

Slide 3 text

What I want to talk about today Chrome (PC) Safari (PC) Chrome DevTools Web Inspector

Slide 4

Slide 4 text

What I want to talk about today Chrome (PC) Chrome (Android) Safari (iOS) Safari (PC) Chrome DevTools Web Inspector

Slide 5

Slide 5 text

What I want to talk about today Chrome (PC) Chrome (Android) Safari (iOS) Safari (PC) Chrome DevTools Web Inspector WebView WebView WebView WebView

Slide 6

Slide 6 text

What I want to talk about today Chrome (PC) Chrome (Android) Safari (iOS) Safari (PC) Chrome DevTools Web Inspector WebView WebView WebView WebView Cannot debug WebView 😢

Slide 7

Slide 7 text

WebViews are widely used in native apps to render web content Chrome (PC) Chrome (Android) Safari (iOS) Safari (PC) WebView WebView WebView WebView Making WebViews Work for the Web Dominique Hazael-Massieux, TPAC2021 https://www.w3.org/2021/Talks/dhm-tpac-webviews/#5

Slide 8

Slide 8 text

Universal DevTools WebView WebView MINI App Super App LIFF All environments DevTools Debug

Slide 9

Slide 9 text

Contents 1. Anatomy of DevTools 2. Debug methods for mobile devices 3. Problem 4. Universal DevTools 5. Summary

Slide 10

Slide 10 text

1. Anatomy of DevTools 2. Debug methods for mobile devices 3. Problem 4. Universal DevTools 5. Summary

Slide 11

Slide 11 text

DevTools Chrome DevTools Web Inspector Firefox Developer Tools DevTools

Slide 12

Slide 12 text

Client-server model Chrome DevTools

Slide 13

Slide 13 text

Client-server model Chrome DevTools Server

Slide 14

Slide 14 text

Client-server model Server Client ChromeDevTools/devtools-frontend Chrome DevTools

Slide 15

Slide 15 text

Client-server model Chrome DevTools Server Client Command Event Tell me the DOM tree of your site! Sure, here you are.

Slide 16

Slide 16 text

Communication with DevTools Chrome DevTools blink V8 index.js index.js

Slide 17

Slide 17 text

Chrome DevTools blink V8 console API is called! index.js index.js Communication with DevTools JS engines notify API calls to DevTools (Event) OK, I’ll display them! Event

Slide 18

Slide 18 text

Chrome DevTools blink V8 console API is called! fetch API is called! index.js index.js Communication with DevTools JS engines notify API calls to DevTools (Event) DOM tree is changed! OK, I’ll display them!

Slide 19

Slide 19 text

Chrome DevTools blink index.js V8 Give me the DOM tree! index.js Communication with DevTools DevTools send commands to the browser

Slide 20

Slide 20 text

Chrome DevTools blink index.js V8 DOM tree Give me the DOM tree! index.js Communication with DevTools JS engines send the response to the command Thank you!

Slide 21

Slide 21 text

Chrome DevTools Protocol

Slide 22

Slide 22 text

Chrome DevTools Protocol Hi, could you change

text from “JSConf JP” to “Hello World”?

Slide 23

Slide 23 text

Chrome DevTools Protocol Hi, could you change

text from “JSConf JP” to “Hello World”?

Slide 24

Slide 24 text

Chrome DevTools Protocol Hi, could you change

text from “JSConf JP” to “Hello World”? OK, I changed.

Slide 25

Slide 25 text

Protocol monitor ⚙ Settings → Experiments → ☑ Protocol Monitor

Slide 26

Slide 26 text

1. Anatomy of DevTools 2. Debug methods for mobile devices 3. Problem 4. Universal DevTools 5. Summary

Slide 27

Slide 27 text

Simulator $ open /Applications/Xcode.app/Contents/Developer/Applications/Simulator.app $ ~/Library/Android/sdk/emulator -avd ${AVD_NAME}

Slide 28

Slide 28 text

Remote debug USB WiFi

Slide 29

Slide 29 text

Remote debug USB WiFi Server Client WebKit Inspector Protocol https://github.com/WebKit/webkit/tree/main/Source/JavaScriptCore/inspector/protocol

Slide 30

Slide 30 text

1. Anatomy of DevTools 2. Debug methods for mobile devices 3. Problem 4. Universal DevTools 5. Summary

Slide 31

Slide 31 text

WebView Your website is broken when I view it in Twitter’s in-app browser! <

Slide 32

Slide 32 text

WebView Your website is broken when I view it in Twitter’s in-app browser! < > Oops, let me check. > Wait, how can I debug in-app browser? I can’t inspect it with DevTools...

Slide 33

Slide 33 text

In-app browser, LIFF, Mini app, Super app How do you debug WebView-based applications? WebView Your website is broken when I view it in Twitter’s in-app browser! < > Oops, let me check. > Wait, how can I debug in-app browser? I can’t inspect it with DevTools...

Slide 34

Slide 34 text

Slide 35

Slide 35 text

Simulator WKWebView WebView If you can run an app using WebView on your simulator, you can debug without any problem. Good! But you can’t run Facebook, LINE, Twitter on your simulator. (unless you are working there) Simulator

Slide 36

Slide 36 text

Remote debug WKWebView WebView Your iPhone

Slide 37

Slide 37 text

Remote debug WKWebView WebView debugging must be enabled! Android: setWebContentsDebuggingEnabled(true) iOS: Debug build (beta version) WebView Your iPhone

Slide 38

Slide 38 text

Remote debug WKWebView WebView debugging must be enabled! Android: setWebContentsDebuggingEnabled(true) iOS: Debug build (beta version) Android: setWebContentsDebuggingEnabled(false) iOS: Production build (downloaded from App Store) WebView Your iPhone

Slide 39

Slide 39 text

Example: LIFF LINE Front-end Framework

Slide 40

Slide 40 text

LIFF Browser LIFF Browser WebView extended native WebView

Slide 41

Slide 41 text

https://jsconf.jp/2021 develop hosting https://liff.line.me/{liff_id} register You can open your web app on LINE app open link Example: LIFF

Slide 42

Slide 42 text

https://jsconf.jp/2021 develop hosting register ● LINE Login ● send messages ● get friends list ● ... https://liff.line.me/{liff_id} Your web app can use a variety of LINE-specific APIs Introduction to LIFF/LINE Mini App - Characteristics and Practical Development https://linedevday.linecorp.com/2021/en/sessions/151/ Example: LIFF

Slide 43

Slide 43 text

https://jsconf.jp/2021 develop hosting register ● LINE Login ● send messages ● get friends list ● ... https://liff.line.me/{liff_id} Your web app can use a variety of LINE-specific APIs You can’t debug your web app running on LIFF Browser 😢 Android: setWebContentsDebuggingEnabled(false) iOS: Production build (downloaded from App Store) Example: LIFF For Improvement of Developer Experience of All LIFF App Developers https://linedevday.linecorp.com/2021/en/sessions/142/

Slide 44

Slide 44 text

1. Anatomy of DevTools 2. Debug methods for mobile devices 3. Problem 4. Universal DevTools 5. Summary

Slide 45

Slide 45 text

- Production build - setWebContentsDebuggingEnabled(false) DevTools can’t get information from JS engine. console, network logs, DOM tree, memory,... How debug a web app on WebView with debug options disabled? WebView

Slide 46

Slide 46 text

How debug a web app on WebView with debug options disabled? - Production build - setWebContentsDebuggingEnabled(false) Well, then, Let’s send debug information using only JavaScript API! DevTools can’t get information from JS engine. console, network logs, DOM tree, memory,... WebView

Slide 47

Slide 47 text

Reproduce client-server model with JavaScript DevTools client DevTools Protocol Server Client WebView Send events when an API is called (console, network request/response)

Slide 48

Slide 48 text

Reproduce client-server model with JavaScript DevTools client DevTools Protocol Send events when an API is called (console, network request/response) DevTools Protocol Interpret commands (DOM manipulation) Server Client WebView

Slide 49

Slide 49 text

WebView Architecture devtools://devtools/bundled/inspector.html?ws=localhost:9222 WebSocket server ws://localhost:9222 Server Client

Slide 50

Slide 50 text

Send events to the client DevTools client 1. Intercept API calls WebSocket server 2. Convert to DevTools Protocol WebView

Slide 51

Slide 51 text

DevTools client 1. Intercept API calls 2. Convert to DevTools Protocol WebView WebSocket server

Slide 52

Slide 52 text

Chrome DevTools blink V8 index.js index.js How intercept API calls?

Slide 53

Slide 53 text

Chrome DevTools blink V8 index.js index.js index.js doesn’t have access to JS engine’s information How intercept API calls?

Slide 54

Slide 54 text

Chrome DevTools blink V8 index.js index.js Intercept API calls with JavaScript fetch API is called! console API is called! DOM tree is changed! OK, let’s use JavaScript! index.js doesn’t have access to JS engine’s information

Slide 55

Slide 55 text

How intercept API calls with JavaScript? index.js console.log fetch Execution Execution Send to DevTools Send to DevTools

Slide 56

Slide 56 text

Trap a function call with Proxy object index.js console.log fetch Execution Execution Send to DevTools Proxy object

Slide 57

Slide 57 text

Trap a function call with Proxy object index.js console.log fetch Execution Execution Send to DevTools Proxy object No side effects 😊

Slide 58

Slide 58 text

Trap a function call with Proxy object index.js console.log fetch Execution Execution Send to DevTools Send to DevTools

Slide 59

Slide 59 text

Headless Inspector emits the corresponding event in response to the API call. https://github.com/cola119/universal-devtools-poc/tree/master/packages

Slide 60

Slide 60 text

Headless Inspector emits the corresponding event in response to the API call. https://github.com/cola119/universal-devtools-poc/tree/master/packages

Slide 61

Slide 61 text

Headless Inspector detect and notify DOM tree changes with MutationObserver

Slide 62

Slide 62 text

Headless Inspector index.js headless-inspector-core Console fetch XHR DOM Storage sendBeacon Each API is wrapped by a Proxy object and emits events Network Elements Console Application

Slide 63

Slide 63 text

Headless Inspector index.js headless-inspector-core Console fetch XHR DOM Storage sendBeacon Each API is wrapped by a Proxy object and emits events Network Elements Console Application

Slide 64

Slide 64 text

Convert to DevTools protocol index.js headless-inspector-core DevTools client DevTools Protocol

Slide 65

Slide 65 text

Convert to DevTools protocol index.js headless-inspector-core Chrome DevTools Safari Web Inspector Chrome DevTools Protocol WebKit Inspector Protocol headless-inspector-cdp headless-inspector-wip

Slide 66

Slide 66 text

Convert to DevTools protocol index.js headless-inspector-core Chrome DevTools Safari Web Inspector Chrome DevTools Protocol WebKit Inspector Protocol headless-inspector-cdp headless-inspector-wip I/F: https://www.npmjs.com/package/devtools-protocol

Slide 67

Slide 67 text

DevTools client Handle commands Command Command WebView WebSocket server

Slide 68

Slide 68 text

Inspect DOM Give me the DOM information blink V8 Chrome DevTools index.js index.js

Slide 69

Slide 69 text

Inspect DOM Give me the DOM information blink V8 Chrome DevTools index.js index.js OK, just a moment.

Slide 70

Slide 70 text

Inspect DOM DOM information Give me the DOM information blink V8 Chrome DevTools index.js index.js

Slide 71

Slide 71 text

Architecture DevTools client Command Command Event Event Intercept API calls Convert to DevTools Protocol headless-inspector-cdp headless-inspector-core headless-inspector-cdp WebView WebSocket server

Slide 72

Slide 72 text

DEMO PoC: https://github.com/cola119/universal-inspector-poc

Slide 73

Slide 73 text

1. Anatomy of DevTools 2. Debug methods for mobile devices 3. Problem 4. Universal DevTools 5. Summary

Slide 74

Slide 74 text

Summary 80% of native apps use WebViews 10+% of Web content is rendered on WebViews WebView But you cannot debug WebView 😢

Slide 75

Slide 75 text

Summary 80% of native apps use WebViews 10+% of Web content is rendered on WebViews WebView But you cannot debug WebView 😢 All web applications can be debugged using any DevTools (Chrome DevTools, Safari Web Inspector, etc.) without any side effects 😄 Universal DevTools https://github.com/cola119/universal-devtools-poc

Slide 76

Slide 76 text

Limitation, Challenges - Call Stack - Breakpoints - Variables - Analyze runtime performance There are some limitations because Universal DevTools is implemented in JavaScript. But at the same time, there are many challenges - Edit source files - Memory Usage - Record - Extension etc...

Slide 77

Slide 77 text

Thank you! GitHub: https://github.com/cola119/universal-devtools-poc - https://www.w3.org/2021/10/TPAC/breakouts.html - https://chromedevtools.github.io/devtools-protocol/ - https://docs.google.com/document/d/1WNF-KqRSzPLUUfZqQG5AFeU_Ll8TfWYcJasa_XGf7ro/edit - https://linedevday.linecorp.com/2021/en/sessions/151/ - https://linedevday.linecorp.com/2021/en/sessions/142/