Upgrade to Pro — share decks privately, control downloads, hide ads and more …

1000 JavaScript errors are as good as None

1000 JavaScript errors are as good as None

Does your site’s webdev console show thousands of errors on a user’s browser but none on yours? Do you receive complaints from users that they’re unable to complete a purchase and you can’t reproduce it on your end?

When studying JavaScript errors, we can easily get lost in a very large number of errors that all seem to say the same thing. How do we tell if different errors really refer to the same root cause? How do we tell which errors are the most important to address? If we had to pick just one error to fix, how do we tell which one has the most impact on our users and business?

In this talk, we look at ways to de-duplicate JavaScript errors across browsers and devices, and then how to identify which ones affect our users the most.

Do some errors block the page from rendering? Do other errors cause increased user frustration? Is there an error that prevents users from finishing a checkout flow?

Presented at: iJS Munich

Philip Tellis

October 27, 2022
Tweet

More Decks by Philip Tellis

Other Decks in Technology

Transcript

  1. 1000 JavaScript Errors are as
    good as None!
    Identifying the most important JavaScript Errors

    View full-size slide

  2. Philip Tellis
    Principal RUM Distiller @ Akamai
    ● Analyses real user performance data from mPulse
    ● Author of the OpenSource boomerang RUM library
    twitter:@bluesmoon ⦿ github:@bluesmoon
    speakerdeck:@bluesmoon

    View full-size slide

  3. Needles in a
    JavaScript
    Haystack(Trace)
    ★ Errors? What errors?
    ★ Too Much Information!
    ★ Ugh! Help me fix them 🙏

    View full-size slide

  4. Hi all, Customer complains that we are
    blocking the “Fortsetzung ohne Akzeptanz”
    button on the website… they see this error

    View full-size slide

  5. Developer:
    I can click on “Fortsetzung ohne
    Akzeptanz” and it works for me. I don’t
    see any error in the stack.

    View full-size slide

  6. User Machines are Strange Beasts!
    They are a mystery to us…

    View full-size slide

  7. Boomerang!
    ● A JavaScript library to mainly collect performance
    data from the browser.
    ● It also collects information about JavaScript errors.
    ★ onerror global event handler
    ★ XMLHttpRequest responses that were not successful
    ★ Any calls to window.console.error
    ★ Runtime errors that happen during a callback for addEventListener,
    setTimeout and setInterval
    ★ And more!
    Note: Make sure cross-origin scripts have the crossorigin="anonymous" attribute and Access-Control-Allow-Origin header set.
    Code: https://github.com/akamai/boomerang Docs: https://techdocs.akamai.com/mpulse-boomerang/docs/errors

    View full-size slide

  8. 25-120 million JavaScript
    errors every Day!

    View full-size slide

  9. e=null; e.length
    Uncaught TypeError: Cannot read properties of null (reading 'length')
    TypeError: null is not an object (evaluating 'e.length')
    Unable to get value of the property 'length': object is null or undefined
    Uncaught TypeError: e is null

    View full-size slide

  10. TypeError: A conexão de rede foi perdida
    TypeError: De netwerkverbinding is verbroken
    TypeError: Die Netzwerkverbindung wurde
    unterbrochen
    TypeError: La conexión de red se ha perdido
    TypeError: La conexión de red se perdió
    TypeError: La connexion réseau a été perdue
    TypeError: Mistet nettverksforbindelsen
    TypeError: Netværksforbindelsen gik tabt
    Grüß Gott Fehler, Salut erreur, नमस्ते गलती
    TypeError: Nätverksanslutningen förlorades
    TypeError: Połączenie sieciowe zostało
    utracone
    TypeError: Pripojenie k sieti bolo stratené
    TypeError: The network connection was lost
    TypeError: Сетевое соединение потеряно
    TypeError: ネットワーク接続が切れました。
    TypeError: 網路連線中斷。
    TypeError: 네트워크 연결이 유실되었습니다

    View full-size slide

  11. TypeError: Abgebrochen
    TypeError: Kumottu
    TypeError: annullato
    TypeError: annulleret
    TypeError: annulé
    TypeError: anulat
    TypeError: avbruten
    TypeError: avbrutt
    TypeError: cancelado
    Kumottu, Abgebrochen, キャンセルしました
    TypeError: cancelled
    TypeError: geannuleerd
    TypeError: vazgeçildi
    TypeError: visszavonva
    TypeError: отменено
    TypeError: ถูกยกเลิก
    TypeError: キャンセルしました
    TypeError: 中止
    TypeError: 已取消

    View full-size slide

  12. TypeError: A server with the specified hostname could not be found
    TypeError: Aucun serveur ayant le nom d’hôte précisé n’a pu être trouvé
    TypeError: Aucun serveur ayant le nom d’hôte précisé n’a été détecté
    TypeError: Es wurde kein Server mit dem angegebenen Hostnamen gefunden
    TypeError: The Internet connection appears to be offline
    TypeError: The internet connection appears to be offline
    Even different dialects

    View full-size slide

  13. e=null; e…
    Cannot read properties of undefined (reading 'appendChild')
    Cannot read properties of undefined (reading 'childNodes')
    Cannot read properties of null (reading 'classList')
    Cannot read properties of null (reading 'innerHTML')
    Cannot read properties of null (reading '__vue__')
    Cannot read properties of null (reading 'add')
    Cannot read properties of null (reading 'addEventListener')

    View full-size slide

  14. Script Error
    [object Object]
    zoid destroyed all components
    Error: Syntax error
    ReferenceError: 'google' is undefined
    window is not an object
    (empty string)
    Plugin errors
    🙀

    View full-size slide

  15. DS 101: clean the data
    ● Different browsers reports the same error differently
    ● Errors are reported as text messages rather than numeric codes
    ● A certain browser also localizes messages based on browser
    locale.
    ● For any given issue, there will be many errors reported – one for
    each side-effect of that error.
    ● And then some errors are just useless.

    View full-size slide

  16. Start in the Browser
    ● Browsers are consistent within the same page view
    ● We can pre-aggregate messages into groups before sending them
    to the server.
    Down to 10-30 million
    errors a Day 🎉

    View full-size slide

  17. Step 2: Deal with the L10N
    ● Localized strings are tackled with a large mapping table.
    ● Normalizes all strings to their en-US equivalent.
    ● In some cases there are immutable parts, like variable identifiers
    that must remain unchanged, but change position.
    Down to 5-15 million
    errors a Day

    View full-size slide

  18. Step 3: Common Error Names
    ● List of 300 regular expressions to munge errors down to a
    common format.
    ● Some of the regexes are chained.
    Down to 2-8 million
    errors a Day

    View full-size slide

  19. Step 4: Uniqueness
    ● We can now look at unique errors for analysis, and what we see is
    surprising…
    Between 300-1700
    unique errors a Day 🎉

    View full-size slide

  20. Copyright Randall Munroe https://xkcd.com/2343/

    View full-size slide

  21. Clustering
    ● We have several parameters to cluster on.
    ● Filename & line number are useful.
    ● In most cases, the error text is the best indicator of its cluster.
    Down to 20-100 groups

    View full-size slide

  22. Methods of Clustering
    ● Density-based spatial clustering of applications with noise
    (DBSCAN)
    ● K-Nearest-Neighbors (KNN)
    ● K-Means
    ● K-Medoids
    You’ll need to figure out a distance function for the dataset
    Martin Ester, Hans-Peter Kriegel, Jiirg Sander, Xiaowei Xu
    A density based approach for discovering Clusters in Large Spatial Databases with Noise

    View full-size slide

  23. Error: Expected ')'
    Error: Expected ':'
    Error: Expected ';'
    Error: Expected '}'
    Error: Expected ']'
    Error: Expected '('
    Error: Expected identifier
    Error: Expected hexadecimal digit
    Error: Expected identifier, string or number
    Example Clusters
    Cannot read properties of null (reading '1')
    Cannot read properties of null (reading '0')
    Cannot read properties of null (reading 'add')
    Cannot read properties of null (reading 'blur')
    Cannot read properties of null (reading '7dvn')
    Cannot read properties of null (reading 'width')
    Cannot read properties of null (reading 'click')
    Cannot read properties of null (reading 'style')
    Cannot read properties of null (reading 'value')
    Cannot read properties of null (reading 'remove')
    Cannot read properties of null (reading '__vue__')
    Cannot read properties of null (reading 'getItem')
    Cannot read properties of null (reading 'setItem')
    Cannot read properties of null (reading 'nodeType')
    Cannot read properties of null (reading 'errorCode')
    Cannot read properties of null (reading 'startsWith')
    Cannot read properties of null (reading 'postMessage')
    Cannot read properties of null (reading 'removeChild')

    View full-size slide

  24. Error Impact
    Identifying which errors have an impact

    View full-size slide

  25. Which one do I fix first?
    ● Let’s look at errors that impact the user journey.
    ● Focus on errors that affect a user’s ability to convert.
    ● Can we tell if certain errors frustrate the user?
    ● What’s the performance impact of errors?

    View full-size slide


  26. Reload Rate
    ● Do users hit the reload button when this error
    occurs?
    ● Use the type property from the
    PerformanceNavigationTiming object.
    ● Make sure sample size is large enough,
    i.e. 100/500 reloads is more reliable than 3/6
    reloads even though it’s 20% v/s 50%

    View full-size slide

  27. 😠
    Rage Click Rate
    ● Does the existence of a particular
    error correspond to a high number of
    Rage Clicks on the page with the
    error?

    View full-size slide

  28. Rage Clicks occur when users rapid-fire click (or
    tap) on your site or app.
    Rage clicking is the digital equivalent of
    cursing to release frustration.
    https://www.fullstory.com/resources/guide-to-understanding-frustrating-user-experiences-online/
    https://www.psychologytoday.com/us/blog/hide-and-seek/201205/hell-yes-the-7-best-reasons-swearing
    Rage Clicks

    View full-size slide

  29. 😠
    Rage Click Rate
    ● Does the existence of a particular
    error correspond to a high number of
    Rage Clicks on the page with the
    error?
    ● Points to user frustration

    View full-size slide

  30. Truncated User Journey
    ● Some errors cause a user session to terminate immediately.
    ● Others seem to allow the user to proceed.
    ● It’s also instructive to split this report by converted and/or “might
    convert” status.

    View full-size slide

  31. Errors affecting Performance
    ● Some errors make page load time worse.
    ● In this case inability to access localStorage probably
    requires extra network calls.
    Access denied for ‘localStorage’

    View full-size slide

  32. Errors affecting Performance
    ● Some errors make page load time better.
    ● In some cases, it causes page load to fail resulting in partial or
    no-content.
    There was an error setting cookie .* Please check domain and path

    View full-size slide

  33. If an error blocks content and makes
    the page faster, without reloads or
    truncated sessions, then is that
    content really needed?

    View full-size slide

  34. Debugging Aids
    Information to identify root causes

    View full-size slide

  35. How do I fix them?
    ● When did it start happening?
    related to a new product release, browser release, etc.
    ● Where does it happen?
    Browser, OS, specific page, Geo region, etc.
    ● What are the conditions when it happens?
    Specific scripts/css loaded/not loaded, load time, http/h2, etc.
    ● What was the user journey like when it happens?
    Previous page, SPA lifetime, Session Length

    View full-size slide

  36. Errors over Time
    ● Look at total error count as well as average
    errors per page for a given error.
    ● Check if new spikes or fixes correlate with a new
    product release or a new browser release (we
    generally annotate our charts with product
    releases).
    ● For existing errors, look at trends in the average
    error count.

    View full-size slide

  37. Where does the error happen?
    OS Browser Device
    The operation is insecure
    Country Page

    View full-size slide

  38. Conditions corresponding to Errors
    Some errors only show up when particular
    JavaScript files fail to load… for example, due to an
    Ad-Blocker or high packet loss

    View full-size slide

  39. Some errors show up when a particular JavaScript
    file loads up more than once!

    View full-size slide

  40. I had the same error, it seems to have been caused by loading
    the PayPal script more than once. I fixed it by loading the script
    just once in the head, then loading the two buttons where they
    should be.
    zoid destroyed all components

    RebeccaRey on github issues for paypal

    View full-size slide

  41. Some errors show up when the page is proxied
    through Google Translate

    View full-size slide

  42. CSRF token expired.
    Some errors often show up when particular
    resources were served from cache.

    View full-size slide

  43. Some errors show up when the client device
    is in low power mode (battery < 10%).

    View full-size slide

  44. Effects of User Journey
    ● Data corruption in cookies or localStorage might
    only show up after a user browses through a few
    pages.
    ● Memory leaks in Single Page Apps may only
    manifest as problems after several navigations.
    ● Cross-domain cookie issues could show up if the
    user had to pass through a different subdomain
    through their journey.
    🍪
    🥠

    View full-size slide

  45. Dealing with Script Error

    View full-size slide

  46. What’s the Problem?
    Script error 51,169,268
    [container] is not a valid DOM Element 2,845,511
    [object Object] 2,553,637
    ReferenceError: 'xxxxxx' is undefined 1,952,751
    Error: s:0 rs:4 1,445,856
    Attempting to change access mechanism for an unconfigurable property 1,083,304
    TypeError: [code] is not a function 809,376
    TypeError: undefined is not an object 789,812
    81%
    No information in Script Error

    View full-size slide

  47. Why does this happen?
    ● Cross-origin script files that may be credentialed.
    ● The browser doesn’t know if error information may be sensitive or
    not, so it hides it by default.
    ● For example, an error that looks like this in the web dev console…

    View full-size slide

  48. Looks like this in an onerror handler…
    ● No real error message, no filename, line number, stack, nothing.
    ● You only get a timestamp!

    View full-size slide

  49. Two options to fix it
    1. If you have control over the page & script domain, do the
    following:
    a. Set the crossorigin="anonymous" attribute on the element.<br/>This tells the browser not to send cookies, and that this request is<br/>non-credentialed.<br/>b. Set the Access-Control-Allow-Origin HTTP response header on the<br/>script to either * or the domain of your host page.<br/>

    View full-size slide

  50. Two options to fix it
    2. If you only have control over the page, do the following:
    ○ Wrap all your calls to 3rd party libraries in a try/catch block:
    try {
    libraryCall();
    }
    catch(e) {
    // e contains detailed information about the error
    }

    View full-size slide

  51. ★ Collect user errors
    ★ Normalize & De-duplicate
    ★ Cluster similar errors
    ★ Measure User Impact
    ★ Handle Script Error
    Summary

    View full-size slide

  52. ● boomerang error plugin: Source on github • Docs at Akamai
    ● null has no properties – MDN docs
    ● zoid destroyed all components – github issue
    ● A density based approach for discovering Clusters in Large Spatial Databases with
    Noise – Ester M, Kriegel H-P, Sander J, Xu X
    ● PerformanceNavigationTiming.type – MDN docs
    ● Rage Clicking – Fullstory
    ● Web Performance with Android's Battery Saver Mode – Goel U, Ludin S, Steiner M
    ● Side effects of boomerang’s JS Error tracking – Nic Jansma
    References

    View full-size slide