Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Universal DevTools for all environments

Kohei Ueno
November 27, 2021

Universal DevTools for all environments

Universal DevTools enables use to debug WebViews with Chrome DevTools.

DEMO: https://youtu.be/5H3Sswp5qYg?t=5100

https://github.com/cola119/universal-devtools-poc

Kohei Ueno

November 27, 2021
Tweet

Other Decks in Technology

Transcript

  1. Universal DevTools
    for all environments
    ্໺߁ฏKohei Ueno
    2021/11/27

    View full-size slide

  2. ্໺ ߁ฏ / Kohei Ueno
    Software Engineer@LINE
    https://twitter.com/cola119_
    https://github.com/cola119
    👶

    View full-size slide

  3. What I want to talk about today
    Chrome (PC) Safari (PC)
    Chrome DevTools Web Inspector

    View full-size slide

  4. What I want to talk about today
    Chrome (PC) Chrome (Android) Safari (iOS) Safari (PC)
    Chrome DevTools Web Inspector

    View full-size slide

  5. What I want to talk about today
    Chrome (PC) Chrome (Android) Safari (iOS) Safari (PC)
    Chrome DevTools Web Inspector
    WebView
    WebView
    WebView
    WebView

    View full-size slide

  6. What I want to talk about today
    Chrome (PC) Chrome (Android) Safari (iOS) Safari (PC)
    Chrome DevTools Web Inspector
    WebView
    WebView
    WebView
    WebView
    Cannot debug WebView 😢

    View full-size slide

  7. WebViews are widely used in native apps
    to render web content
    Chrome (PC) Chrome (Android) Safari (iOS) Safari (PC)
    WebView
    WebView
    WebView
    WebView
    Making WebViews Work for the Web
    Dominique Hazael-Massieux, TPAC2021
    https://www.w3.org/2021/Talks/dhm-tpac-webviews/#5

    View full-size slide

  8. Universal DevTools
    WebView WebView
    MINI
    App
    Super
    App LIFF
    All environments DevTools
    Debug

    View full-size slide

  9. Contents
    1. Anatomy of DevTools
    2. Debug methods for mobile devices
    3. Problem
    4. Universal DevTools
    5. Summary

    View full-size slide

  10. 1. Anatomy of DevTools
    2. Debug methods for mobile devices
    3. Problem
    4. Universal DevTools
    5. Summary

    View full-size slide

  11. DevTools
    Chrome DevTools
    Web Inspector
    Firefox Developer Tools
    DevTools

    View full-size slide

  12. Client-server model
    Chrome DevTools

    View full-size slide

  13. Client-server model
    Chrome DevTools
    Server

    View full-size slide

  14. Client-server model
    Server Client
    ChromeDevTools/devtools-frontend
    Chrome DevTools

    View full-size slide

  15. Client-server model
    Chrome DevTools
    Server Client
    Command
    Event
    Tell me the DOM tree of your site!
    Sure, here you are.

    View full-size slide

  16. Communication with DevTools
    Chrome DevTools
    blink
    V8
    index.js
    index.js

    View full-size slide

  17. Chrome DevTools
    blink
    V8
    console API is called!
    index.js
    index.js
    Communication with DevTools
    JS engines notify API calls to DevTools (Event)
    OK, I’ll display them!
    Event

    View full-size slide

  18. Chrome DevTools
    blink
    V8
    console API is called!
    fetch API is called!
    index.js
    index.js
    Communication with DevTools
    JS engines notify API calls to DevTools (Event)
    DOM tree is changed! OK, I’ll display them!

    View full-size slide

  19. Chrome DevTools
    blink
    index.js
    V8
    Give me the DOM tree!
    index.js
    Communication with DevTools
    DevTools send commands to the browser

    View full-size slide

  20. Chrome DevTools
    blink
    index.js
    V8
    DOM tree
    Give me the DOM tree!
    index.js
    Communication with DevTools
    JS engines send the response to the command
    Thank you!

    View full-size slide

  21. Chrome DevTools Protocol

    View full-size slide

  22. Chrome DevTools Protocol
    Hi, could you change text
    from “JSConf JP” to “Hello World”?

    View full-size slide

  23. Chrome DevTools Protocol
    Hi, could you change text
    from “JSConf JP” to “Hello World”?

    View full-size slide

  24. Chrome DevTools Protocol
    Hi, could you change text
    from “JSConf JP” to “Hello World”?
    OK, I changed.

    View full-size slide

  25. Protocol monitor
    ⚙ Settings → Experiments → ☑ Protocol Monitor

    View full-size slide

  26. 1. Anatomy of DevTools
    2. Debug methods for mobile devices
    3. Problem
    4. Universal DevTools
    5. Summary

    View full-size slide

  27. Simulator
    $ open /Applications/Xcode.app/Contents/Developer/Applications/Simulator.app
    $ ~/Library/Android/sdk/emulator -avd ${AVD_NAME}

    View full-size slide

  28. Remote debug
    USB
    WiFi

    View full-size slide

  29. Remote debug
    USB
    WiFi
    Server Client
    WebKit Inspector Protocol
    https://github.com/WebKit/webkit/tree/main/Source/JavaScriptCore/inspector/protocol

    View full-size slide

  30. 1. Anatomy of DevTools
    2. Debug methods for mobile devices
    3. Problem
    4. Universal DevTools
    5. Summary

    View full-size slide

  31. WebView
    Your website is broken when I view it in Twitter’s in-app browser! <

    View full-size slide

  32. WebView
    Your website is broken when I view it in Twitter’s in-app browser! <
    > Oops, let me check.
    > Wait, how can I debug in-app browser?
    I can’t inspect it with DevTools...

    View full-size slide

  33. In-app browser, LIFF, Mini app, Super app
    How do you debug WebView-based applications?
    WebView
    Your website is broken when I view it in Twitter’s in-app browser! <
    > Oops, let me check.
    > Wait, how can I debug in-app browser?
    I can’t inspect it with DevTools...

    View full-size slide

  34. In-app browser, LIFF, Mini app, Super app
    How do you debug WebView-based applications?
    WebView
    Your website is broken when I view it in Twitter’s in-app browser! <
    > Oops, let me check.
    > Wait, how can I debug in-app browser?
    I can’t inspect it with DevTools...
    Simulator? Remote debug?

    View full-size slide

  35. Simulator
    WKWebView
    WebView
    If you can run an app using WebView on your simulator,
    you can debug without any problem. Good!
    But you can’t run Facebook, LINE, Twitter on your simulator.
    (unless you are working there)
    Simulator

    View full-size slide

  36. Remote debug
    WKWebView
    WebView
    Your iPhone

    View full-size slide

  37. Remote debug
    WKWebView
    WebView debugging must be enabled!
    Android: setWebContentsDebuggingEnabled(true)
    iOS: Debug build (beta version)
    WebView
    Your iPhone

    View full-size slide

  38. Remote debug
    WKWebView
    WebView debugging must be enabled!
    Android: setWebContentsDebuggingEnabled(true)
    iOS: Debug build (beta version)
    Android: setWebContentsDebuggingEnabled(false)
    iOS: Production build (downloaded from App Store)
    WebView
    Your iPhone

    View full-size slide

  39. Example: LIFF
    LINE Front-end Framework

    View full-size slide

  40. LIFF Browser
    LIFF Browser
    WebView
    extended native WebView

    View full-size slide

  41. https://jsconf.jp/2021
    develop
    hosting
    https://liff.line.me/{liff_id}
    register
    You can open your web app on LINE app
    open link
    Example: LIFF

    View full-size slide

  42. https://jsconf.jp/2021
    develop
    hosting
    register
    ● LINE Login
    ● send messages
    ● get friends list
    ● ...
    https://liff.line.me/{liff_id}
    Your web app can use a variety of LINE-specific APIs
    Introduction to LIFF/LINE Mini App
    - Characteristics and Practical Development
    https://linedevday.linecorp.com/2021/en/sessions/151/
    Example: LIFF

    View full-size slide

  43. https://jsconf.jp/2021
    develop
    hosting
    register
    ● LINE Login
    ● send messages
    ● get friends list
    ● ...
    https://liff.line.me/{liff_id}
    Your web app can use a variety of LINE-specific APIs
    You can’t debug your web app
    running on LIFF Browser 😢
    Android: setWebContentsDebuggingEnabled(false)
    iOS: Production build (downloaded from App Store)
    Example: LIFF
    For Improvement of Developer Experience of All LIFF App Developers
    https://linedevday.linecorp.com/2021/en/sessions/142/

    View full-size slide

  44. 1. Anatomy of DevTools
    2. Debug methods for mobile devices
    3. Problem
    4. Universal DevTools
    5. Summary

    View full-size slide

  45. - Production build
    - setWebContentsDebuggingEnabled(false)
    DevTools can’t get information from JS engine.
    console, network logs, DOM tree, memory,...
    How debug a web app on WebView
    with debug options disabled?
    WebView

    View full-size slide

  46. How debug a web app on WebView
    with debug options disabled?
    - Production build
    - setWebContentsDebuggingEnabled(false)
    Well, then, Let’s send debug information
    using only JavaScript API!
    DevTools can’t get information from JS engine.
    console, network logs, DOM tree, memory,...
    WebView

    View full-size slide

  47. Reproduce client-server model with JavaScript
    DevTools
    client
    DevTools Protocol
    Server Client
    WebView
    Send events when an API is called
    (console, network request/response)

    View full-size slide

  48. Reproduce client-server model with JavaScript
    DevTools
    client
    DevTools Protocol
    Send events when an API is called
    (console, network request/response)
    DevTools Protocol
    Interpret commands
    (DOM manipulation)
    Server Client
    WebView

    View full-size slide

  49. WebView
    Architecture
    devtools://devtools/bundled/inspector.html?ws=localhost:9222
    WebSocket
    server
    ws://localhost:9222
    Server Client

    View full-size slide

  50. Send events to the client
    DevTools
    client
    1. Intercept API calls
    WebSocket
    server
    2. Convert to DevTools Protocol
    WebView

    View full-size slide

  51. DevTools
    client
    1. Intercept API calls 2. Convert to DevTools Protocol
    WebView
    WebSocket
    server

    View full-size slide

  52. Chrome DevTools
    blink
    V8
    index.js
    index.js
    How intercept API calls?

    View full-size slide

  53. Chrome DevTools
    blink
    V8
    index.js
    index.js
    index.js doesn’t have access to JS engine’s information
    How intercept API calls?

    View full-size slide

  54. Chrome DevTools
    blink
    V8
    index.js
    index.js
    Intercept API calls with JavaScript
    fetch API is called!
    console API is called!
    DOM tree is changed!
    OK, let’s use JavaScript!
    index.js doesn’t have access to JS engine’s information

    View full-size slide

  55. How intercept API calls with JavaScript?
    index.js
    console.log fetch
    Execution Execution
    Send to DevTools Send to DevTools

    View full-size slide

  56. Trap a function call with Proxy object
    index.js
    console.log fetch
    Execution Execution
    Send to DevTools
    Proxy object

    View full-size slide

  57. Trap a function call with Proxy object
    index.js
    console.log fetch
    Execution Execution
    Send to DevTools
    Proxy object
    No side effects 😊

    View full-size slide

  58. Trap a function call with Proxy object
    index.js
    console.log fetch
    Execution Execution
    Send to DevTools
    Send to DevTools

    View full-size slide

  59. Headless Inspector
    emits the corresponding event in response to the API call.
    https://github.com/cola119/universal-devtools-poc/tree/master/packages

    View full-size slide

  60. Headless Inspector
    emits the corresponding event in response to the API call.
    https://github.com/cola119/universal-devtools-poc/tree/master/packages

    View full-size slide

  61. Headless Inspector
    detect and notify DOM tree changes with MutationObserver

    View full-size slide

  62. Headless Inspector
    index.js
    headless-inspector-core
    Console
    fetch
    XHR DOM
    Storage
    sendBeacon
    Each API is wrapped by a Proxy object
    and emits events
    Network
    Elements
    Console
    Application

    View full-size slide

  63. Headless Inspector
    index.js
    headless-inspector-core
    Console
    fetch
    XHR DOM
    Storage
    sendBeacon
    Each API is wrapped by a Proxy object
    and emits events
    Network
    Elements
    Console
    Application

    View full-size slide

  64. Convert to DevTools protocol
    index.js
    headless-inspector-core
    DevTools client
    DevTools Protocol

    View full-size slide

  65. Convert to DevTools protocol
    index.js
    headless-inspector-core
    Chrome DevTools
    Safari Web Inspector
    Chrome DevTools Protocol
    WebKit Inspector Protocol
    headless-inspector-cdp
    headless-inspector-wip

    View full-size slide

  66. Convert to DevTools protocol
    index.js
    headless-inspector-core
    Chrome DevTools
    Safari Web Inspector
    Chrome DevTools Protocol
    WebKit Inspector Protocol
    headless-inspector-cdp
    headless-inspector-wip
    I/F: https://www.npmjs.com/package/devtools-protocol

    View full-size slide

  67. DevTools
    client
    Handle commands
    Command
    Command
    WebView
    WebSocket
    server

    View full-size slide

  68. Inspect DOM
    Give me the DOM information
    blink
    V8
    Chrome DevTools
    index.js
    index.js

    View full-size slide

  69. Inspect DOM
    Give me the DOM information
    blink
    V8
    Chrome DevTools
    index.js
    index.js
    OK, just a moment.

    View full-size slide

  70. Inspect DOM
    DOM information
    Give me the DOM information
    blink
    V8
    Chrome DevTools
    index.js
    index.js

    View full-size slide

  71. Architecture
    DevTools
    client
    Command
    Command
    Event Event
    Intercept API calls Convert to DevTools Protocol
    headless-inspector-cdp
    headless-inspector-core
    headless-inspector-cdp
    WebView
    WebSocket
    server

    View full-size slide

  72. DEMO
    PoC: https://github.com/cola119/universal-inspector-poc

    View full-size slide

  73. 1. Anatomy of DevTools
    2. Debug methods for mobile devices
    3. Problem
    4. Universal DevTools
    5. Summary

    View full-size slide

  74. Summary
    80% of native apps use WebViews
    10+% of Web content is rendered on WebViews
    WebView
    But you cannot debug WebView 😢

    View full-size slide

  75. Summary
    80% of native apps use WebViews
    10+% of Web content is rendered on WebViews
    WebView
    But you cannot debug WebView 😢
    All web applications can be debugged using any DevTools
    (Chrome DevTools, Safari Web Inspector, etc.)
    without any side effects 😄
    Universal
    DevTools
    https://github.com/cola119/universal-devtools-poc

    View full-size slide

  76. Limitation, Challenges
    - Call Stack
    - Breakpoints
    - Variables
    - Analyze runtime performance
    There are some limitations because Universal DevTools is implemented in JavaScript.
    But at the same time, there are many challenges
    - Edit source files
    - Memory Usage
    - Record
    - Extension etc...

    View full-size slide

  77. Thank you!
    GitHub: https://github.com/cola119/universal-devtools-poc
    - https://www.w3.org/2021/10/TPAC/breakouts.html
    - https://chromedevtools.github.io/devtools-protocol/
    - https://docs.google.com/document/d/1WNF-KqRSzPLUUfZqQG5AFeU_Ll8TfWYcJasa_XGf7ro/edit
    - https://linedevday.linecorp.com/2021/en/sessions/151/
    - https://linedevday.linecorp.com/2021/en/sessions/142/

    View full-size slide