Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Universal DevTools for all environments

91f6ef52b846ca5f00325b05f3e3547d?s=47 Kohei Ueno
November 27, 2021

Universal DevTools for all environments

Universal DevTools enables use to debug WebViews with Chrome DevTools.

DEMO: https://youtu.be/5H3Sswp5qYg?t=5100

https://github.com/cola119/universal-devtools-poc

91f6ef52b846ca5f00325b05f3e3547d?s=128

Kohei Ueno

November 27, 2021
Tweet

Other Decks in Technology

Transcript

  1. Universal DevTools for all environments ্໺߁ฏKohei Ueno 2021/11/27

  2. ্໺ ߁ฏ / Kohei Ueno Software Engineer@LINE https://twitter.com/cola119_ https://github.com/cola119 👶

  3. What I want to talk about today Chrome (PC) Safari

    (PC) Chrome DevTools Web Inspector
  4. What I want to talk about today Chrome (PC) Chrome

    (Android) Safari (iOS) Safari (PC) Chrome DevTools Web Inspector
  5. What I want to talk about today Chrome (PC) Chrome

    (Android) Safari (iOS) Safari (PC) Chrome DevTools Web Inspector WebView WebView WebView WebView
  6. What I want to talk about today Chrome (PC) Chrome

    (Android) Safari (iOS) Safari (PC) Chrome DevTools Web Inspector WebView WebView WebView WebView Cannot debug WebView 😢
  7. WebViews are widely used in native apps to render web

    content Chrome (PC) Chrome (Android) Safari (iOS) Safari (PC) WebView WebView WebView WebView Making WebViews Work for the Web Dominique Hazael-Massieux, TPAC2021 https://www.w3.org/2021/Talks/dhm-tpac-webviews/#5
  8. Universal DevTools WebView WebView MINI App Super App LIFF All

    environments DevTools Debug
  9. Contents 1. Anatomy of DevTools 2. Debug methods for mobile

    devices 3. Problem 4. Universal DevTools 5. Summary
  10. 1. Anatomy of DevTools 2. Debug methods for mobile devices

    3. Problem 4. Universal DevTools 5. Summary
  11. DevTools Chrome DevTools Web Inspector Firefox Developer Tools DevTools

  12. Client-server model Chrome DevTools

  13. Client-server model Chrome DevTools Server

  14. Client-server model Server Client ChromeDevTools/devtools-frontend Chrome DevTools

  15. Client-server model Chrome DevTools Server Client Command Event Tell me

    the DOM tree of your site! Sure, here you are.
  16. Communication with DevTools Chrome DevTools blink V8 index.js index.js

  17. Chrome DevTools blink V8 console API is called! index.js index.js

    Communication with DevTools JS engines notify API calls to DevTools (Event) OK, I’ll display them! Event
  18. Chrome DevTools blink V8 console API is called! fetch API

    is called! index.js index.js Communication with DevTools JS engines notify API calls to DevTools (Event) DOM tree is changed! OK, I’ll display them!
  19. Chrome DevTools blink index.js V8 Give me the DOM tree!

    index.js Communication with DevTools DevTools send commands to the browser
  20. Chrome DevTools blink index.js V8 DOM tree Give me the

    DOM tree! index.js Communication with DevTools JS engines send the response to the command Thank you!
  21. Chrome DevTools Protocol

  22. Chrome DevTools Protocol Hi, could you change <h1> text from

    “JSConf JP” to “Hello World”?
  23. Chrome DevTools Protocol Hi, could you change <h1> text from

    “JSConf JP” to “Hello World”?
  24. Chrome DevTools Protocol Hi, could you change <h1> text from

    “JSConf JP” to “Hello World”? OK, I changed.
  25. Protocol monitor ⚙ Settings → Experiments → ☑ Protocol Monitor

  26. 1. Anatomy of DevTools 2. Debug methods for mobile devices

    3. Problem 4. Universal DevTools 5. Summary
  27. Simulator $ open /Applications/Xcode.app/Contents/Developer/Applications/Simulator.app $ ~/Library/Android/sdk/emulator -avd ${AVD_NAME}

  28. Remote debug USB WiFi

  29. Remote debug USB WiFi Server Client WebKit Inspector Protocol https://github.com/WebKit/webkit/tree/main/Source/JavaScriptCore/inspector/protocol

  30. 1. Anatomy of DevTools 2. Debug methods for mobile devices

    3. Problem 4. Universal DevTools 5. Summary
  31. WebView Your website is broken when I view it in

    Twitter’s in-app browser! <
  32. WebView Your website is broken when I view it in

    Twitter’s in-app browser! < > Oops, let me check. > Wait, how can I debug in-app browser? I can’t inspect it with DevTools...
  33. In-app browser, LIFF, Mini app, Super app How do you

    debug WebView-based applications? WebView Your website is broken when I view it in Twitter’s in-app browser! < > Oops, let me check. > Wait, how can I debug in-app browser? I can’t inspect it with DevTools...
  34. In-app browser, LIFF, Mini app, Super app How do you

    debug WebView-based applications? WebView Your website is broken when I view it in Twitter’s in-app browser! < > Oops, let me check. > Wait, how can I debug in-app browser? I can’t inspect it with DevTools... Simulator? Remote debug?
  35. Simulator WKWebView WebView If you can run an app using

    WebView on your simulator, you can debug without any problem. Good! But you can’t run Facebook, LINE, Twitter on your simulator. (unless you are working there) Simulator
  36. Remote debug WKWebView WebView Your iPhone

  37. Remote debug WKWebView WebView debugging must be enabled! Android: setWebContentsDebuggingEnabled(true)

    iOS: Debug build (beta version) WebView Your iPhone
  38. Remote debug WKWebView WebView debugging must be enabled! Android: setWebContentsDebuggingEnabled(true)

    iOS: Debug build (beta version) Android: setWebContentsDebuggingEnabled(false) iOS: Production build (downloaded from App Store) WebView Your iPhone
  39. Example: LIFF LINE Front-end Framework

  40. LIFF Browser LIFF Browser WebView extended native WebView

  41. https://jsconf.jp/2021 develop hosting https://liff.line.me/{liff_id} register You can open your web

    app on LINE app open link Example: LIFF
  42. https://jsconf.jp/2021 develop hosting register • LINE Login • send messages

    • get friends list • ... https://liff.line.me/{liff_id} Your web app can use a variety of LINE-specific APIs Introduction to LIFF/LINE Mini App - Characteristics and Practical Development https://linedevday.linecorp.com/2021/en/sessions/151/ Example: LIFF
  43. https://jsconf.jp/2021 develop hosting register • LINE Login • send messages

    • get friends list • ... https://liff.line.me/{liff_id} Your web app can use a variety of LINE-specific APIs You can’t debug your web app running on LIFF Browser 😢 Android: setWebContentsDebuggingEnabled(false) iOS: Production build (downloaded from App Store) Example: LIFF For Improvement of Developer Experience of All LIFF App Developers https://linedevday.linecorp.com/2021/en/sessions/142/
  44. 1. Anatomy of DevTools 2. Debug methods for mobile devices

    3. Problem 4. Universal DevTools 5. Summary
  45. - Production build - setWebContentsDebuggingEnabled(false) DevTools can’t get information from

    JS engine. console, network logs, DOM tree, memory,... How debug a web app on WebView with debug options disabled? WebView
  46. How debug a web app on WebView with debug options

    disabled? - Production build - setWebContentsDebuggingEnabled(false) Well, then, Let’s send debug information using only JavaScript API! DevTools can’t get information from JS engine. console, network logs, DOM tree, memory,... WebView
  47. Reproduce client-server model with JavaScript DevTools client DevTools Protocol Server

    Client WebView Send events when an API is called (console, network request/response)
  48. Reproduce client-server model with JavaScript DevTools client DevTools Protocol Send

    events when an API is called (console, network request/response) DevTools Protocol Interpret commands (DOM manipulation) Server Client WebView
  49. WebView Architecture devtools://devtools/bundled/inspector.html?ws=localhost:9222 WebSocket server ws://localhost:9222 Server Client

  50. Send events to the client DevTools client 1. Intercept API

    calls WebSocket server 2. Convert to DevTools Protocol WebView
  51. DevTools client 1. Intercept API calls 2. Convert to DevTools

    Protocol WebView WebSocket server
  52. Chrome DevTools blink V8 index.js index.js How intercept API calls?

  53. Chrome DevTools blink V8 index.js index.js index.js doesn’t have access

    to JS engine’s information How intercept API calls?
  54. Chrome DevTools blink V8 index.js index.js Intercept API calls with

    JavaScript fetch API is called! console API is called! DOM tree is changed! OK, let’s use JavaScript! index.js doesn’t have access to JS engine’s information
  55. How intercept API calls with JavaScript? index.js console.log fetch Execution

    Execution Send to DevTools Send to DevTools
  56. Trap a function call with Proxy object index.js console.log fetch

    Execution Execution Send to DevTools Proxy object
  57. Trap a function call with Proxy object index.js console.log fetch

    Execution Execution Send to DevTools Proxy object No side effects 😊
  58. Trap a function call with Proxy object index.js console.log fetch

    Execution Execution Send to DevTools Send to DevTools
  59. Headless Inspector emits the corresponding event in response to the

    API call. https://github.com/cola119/universal-devtools-poc/tree/master/packages
  60. Headless Inspector emits the corresponding event in response to the

    API call. https://github.com/cola119/universal-devtools-poc/tree/master/packages
  61. Headless Inspector detect and notify DOM tree changes with MutationObserver

  62. Headless Inspector index.js headless-inspector-core Console fetch XHR DOM Storage sendBeacon

    Each API is wrapped by a Proxy object and emits events Network Elements Console Application
  63. Headless Inspector index.js headless-inspector-core Console fetch XHR DOM Storage sendBeacon

    Each API is wrapped by a Proxy object and emits events Network Elements Console Application
  64. Convert to DevTools protocol index.js headless-inspector-core DevTools client DevTools Protocol

  65. Convert to DevTools protocol index.js headless-inspector-core Chrome DevTools Safari Web

    Inspector Chrome DevTools Protocol WebKit Inspector Protocol headless-inspector-cdp headless-inspector-wip
  66. Convert to DevTools protocol index.js headless-inspector-core Chrome DevTools Safari Web

    Inspector Chrome DevTools Protocol WebKit Inspector Protocol headless-inspector-cdp headless-inspector-wip I/F: https://www.npmjs.com/package/devtools-protocol
  67. DevTools client Handle commands Command Command WebView WebSocket server

  68. Inspect DOM Give me the DOM information blink V8 Chrome

    DevTools index.js index.js
  69. Inspect DOM Give me the DOM information blink V8 Chrome

    DevTools index.js index.js OK, just a moment.
  70. Inspect DOM DOM information Give me the DOM information blink

    V8 Chrome DevTools index.js index.js
  71. Architecture DevTools client Command Command Event Event Intercept API calls

    Convert to DevTools Protocol headless-inspector-cdp headless-inspector-core headless-inspector-cdp WebView WebSocket server
  72. DEMO PoC: https://github.com/cola119/universal-inspector-poc

  73. 1. Anatomy of DevTools 2. Debug methods for mobile devices

    3. Problem 4. Universal DevTools 5. Summary
  74. Summary 80% of native apps use WebViews 10+% of Web

    content is rendered on WebViews WebView But you cannot debug WebView 😢
  75. Summary 80% of native apps use WebViews 10+% of Web

    content is rendered on WebViews WebView But you cannot debug WebView 😢 All web applications can be debugged using any DevTools (Chrome DevTools, Safari Web Inspector, etc.) without any side effects 😄 Universal DevTools https://github.com/cola119/universal-devtools-poc
  76. Limitation, Challenges - Call Stack - Breakpoints - Variables -

    Analyze runtime performance There are some limitations because Universal DevTools is implemented in JavaScript. But at the same time, there are many challenges - Edit source files - Memory Usage - Record - Extension etc...
  77. Thank you! GitHub: https://github.com/cola119/universal-devtools-poc - https://www.w3.org/2021/10/TPAC/breakouts.html - https://chromedevtools.github.io/devtools-protocol/ - https://docs.google.com/document/d/1WNF-KqRSzPLUUfZqQG5AFeU_Ll8TfWYcJasa_XGf7ro/edit

    - https://linedevday.linecorp.com/2021/en/sessions/151/ - https://linedevday.linecorp.com/2021/en/sessions/142/