Upgrade to Pro — share decks privately, control downloads, hide ads and more …

1000 JavaScript errors are as good as None

1000 JavaScript errors are as good as None

Does your site’s webdev console show thousands of errors on a user’s browser but none on yours? Do you receive complaints from users that they’re unable to complete a purchase and you can’t reproduce it on your end?

When studying JavaScript errors, we can easily get lost in a very large number of errors that all seem to say the same thing. How do we tell if different errors really refer to the same root cause? How do we tell which errors are the most important to address? If we had to pick just one error to fix, how do we tell which one has the most impact on our users and business?

In this talk, we look at ways to de-duplicate JavaScript errors across browsers and devices, and then how to identify which ones affect our users the most.

Do some errors block the page from rendering? Do other errors cause increased user frustration? Is there an error that prevents users from finishing a checkout flow?

Presented at: iJS Munich

Philip Tellis

October 27, 2022
Tweet

More Decks by Philip Tellis

Other Decks in Technology

Transcript

  1. 1000 JavaScript Errors are as
    good as None!
    Identifying the most important JavaScript Errors

    View Slide

  2. Philip Tellis
    Principal RUM Distiller @ Akamai
    ● Analyses real user performance data from mPulse
    ● Author of the OpenSource boomerang RUM library
    twitter:@bluesmoon ⦿ github:@bluesmoon
    speakerdeck:@bluesmoon

    View Slide

  3. Needles in a
    JavaScript
    Haystack(Trace)
    ★ Errors? What errors?
    ★ Too Much Information!
    ★ Ugh! Help me fix them 🙏

    View Slide

  4. Hi all, Customer complains that we are
    blocking the “Fortsetzung ohne Akzeptanz”
    button on the website… they see this error

    View Slide

  5. Developer:
    I can click on “Fortsetzung ohne
    Akzeptanz” and it works for me. I don’t
    see any error in the stack.

    View Slide

  6. WFM!

    View Slide

  7. User Machines are Strange Beasts!
    They are a mystery to us…

    View Slide

  8. Boomerang!
    ● A JavaScript library to mainly collect performance
    data from the browser.
    ● It also collects information about JavaScript errors.
    ★ onerror global event handler
    ★ XMLHttpRequest responses that were not successful
    ★ Any calls to window.console.error
    ★ Runtime errors that happen during a callback for addEventListener,
    setTimeout and setInterval
    ★ And more!
    Note: Make sure cross-origin scripts have the crossorigin="anonymous" attribute and Access-Control-Allow-Origin header set.
    Code: https://github.com/akamai/boomerang Docs: https://techdocs.akamai.com/mpulse-boomerang/docs/errors

    View Slide

  9. 25-120 million JavaScript
    errors every Day!

    View Slide

  10. e=null; e.length
    Uncaught TypeError: Cannot read properties of null (reading 'length')
    TypeError: null is not an object (evaluating 'e.length')
    Unable to get value of the property 'length': object is null or undefined
    Uncaught TypeError: e is null

    View Slide

  11. TypeError: A conexão de rede foi perdida
    TypeError: De netwerkverbinding is verbroken
    TypeError: Die Netzwerkverbindung wurde
    unterbrochen
    TypeError: La conexión de red se ha perdido
    TypeError: La conexión de red se perdió
    TypeError: La connexion réseau a été perdue
    TypeError: Mistet nettverksforbindelsen
    TypeError: Netværksforbindelsen gik tabt
    Grüß Gott Fehler, Salut erreur, नमस्ते गलती
    TypeError: Nätverksanslutningen förlorades
    TypeError: Połączenie sieciowe zostało
    utracone
    TypeError: Pripojenie k sieti bolo stratené
    TypeError: The network connection was lost
    TypeError: Сетевое соединение потеряно
    TypeError: ネットワーク接続が切れました。
    TypeError: 網路連線中斷。
    TypeError: 네트워크 연결이 유실되었습니다

    View Slide

  12. TypeError: Abgebrochen
    TypeError: Kumottu
    TypeError: annullato
    TypeError: annulleret
    TypeError: annulé
    TypeError: anulat
    TypeError: avbruten
    TypeError: avbrutt
    TypeError: cancelado
    Kumottu, Abgebrochen, キャンセルしました
    TypeError: cancelled
    TypeError: geannuleerd
    TypeError: vazgeçildi
    TypeError: visszavonva
    TypeError: отменено
    TypeError: ถูกยกเลิก
    TypeError: キャンセルしました
    TypeError: 中止
    TypeError: 已取消

    View Slide

  13. TypeError: A server with the specified hostname could not be found
    TypeError: Aucun serveur ayant le nom d’hôte précisé n’a pu être trouvé
    TypeError: Aucun serveur ayant le nom d’hôte précisé n’a été détecté
    TypeError: Es wurde kein Server mit dem angegebenen Hostnamen gefunden
    TypeError: The Internet connection appears to be offline
    TypeError: The internet connection appears to be offline
    Even different dialects

    View Slide

  14. e=null; e…
    Cannot read properties of undefined (reading 'appendChild')
    Cannot read properties of undefined (reading 'childNodes')
    Cannot read properties of null (reading 'classList')
    Cannot read properties of null (reading 'innerHTML')
    Cannot read properties of null (reading '__vue__')
    Cannot read properties of null (reading 'add')
    Cannot read properties of null (reading 'addEventListener')

    View Slide

  15. Script Error
    [object Object]
    zoid destroyed all components
    Error: Syntax error
    ReferenceError: 'google' is undefined
    window is not an object
    (empty string)
    Plugin errors
    🙀

    View Slide

  16. DS 101: clean the data
    ● Different browsers reports the same error differently
    ● Errors are reported as text messages rather than numeric codes
    ● A certain browser also localizes messages based on browser
    locale.
    ● For any given issue, there will be many errors reported – one for
    each side-effect of that error.
    ● And then some errors are just useless.

    View Slide

  17. Start in the Browser
    ● Browsers are consistent within the same page view
    ● We can pre-aggregate messages into groups before sending them
    to the server.
    Down to 10-30 million
    errors a Day 🎉

    View Slide

  18. Step 2: Deal with the L10N
    ● Localized strings are tackled with a large mapping table.
    ● Normalizes all strings to their en-US equivalent.
    ● In some cases there are immutable parts, like variable identifiers
    that must remain unchanged, but change position.
    Down to 5-15 million
    errors a Day

    View Slide

  19. Step 3: Common Error Names
    ● List of 300 regular expressions to munge errors down to a
    common format.
    ● Some of the regexes are chained.
    Down to 2-8 million
    errors a Day

    View Slide

  20. Step 4: Uniqueness
    ● We can now look at unique errors for analysis, and what we see is
    surprising…
    Between 300-1700
    unique errors a Day 🎉

    View Slide

  21. Copyright Randall Munroe https://xkcd.com/2343/

    View Slide

  22. Clustering
    ● We have several parameters to cluster on.
    ● Filename & line number are useful.
    ● In most cases, the error text is the best indicator of its cluster.
    Down to 20-100 groups

    View Slide

  23. Methods of Clustering
    ● Density-based spatial clustering of applications with noise
    (DBSCAN)
    ● K-Nearest-Neighbors (KNN)
    ● K-Means
    ● K-Medoids
    You’ll need to figure out a distance function for the dataset
    Martin Ester, Hans-Peter Kriegel, Jiirg Sander, Xiaowei Xu
    A density based approach for discovering Clusters in Large Spatial Databases with Noise

    View Slide

  24. Error: Expected ')'
    Error: Expected ':'
    Error: Expected ';'
    Error: Expected '}'
    Error: Expected ']'
    Error: Expected '('
    Error: Expected identifier
    Error: Expected hexadecimal digit
    Error: Expected identifier, string or number
    Example Clusters
    Cannot read properties of null (reading '1')
    Cannot read properties of null (reading '0')
    Cannot read properties of null (reading 'add')
    Cannot read properties of null (reading 'blur')
    Cannot read properties of null (reading '7dvn')
    Cannot read properties of null (reading 'width')
    Cannot read properties of null (reading 'click')
    Cannot read properties of null (reading 'style')
    Cannot read properties of null (reading 'value')
    Cannot read properties of null (reading 'remove')
    Cannot read properties of null (reading '__vue__')
    Cannot read properties of null (reading 'getItem')
    Cannot read properties of null (reading 'setItem')
    Cannot read properties of null (reading 'nodeType')
    Cannot read properties of null (reading 'errorCode')
    Cannot read properties of null (reading 'startsWith')
    Cannot read properties of null (reading 'postMessage')
    Cannot read properties of null (reading 'removeChild')

    View Slide

  25. Error Impact
    Identifying which errors have an impact

    View Slide

  26. Which one do I fix first?
    ● Let’s look at errors that impact the user journey.
    ● Focus on errors that affect a user’s ability to convert.
    ● Can we tell if certain errors frustrate the user?
    ● What’s the performance impact of errors?

    View Slide


  27. Reload Rate
    ● Do users hit the reload button when this error
    occurs?
    ● Use the type property from the
    PerformanceNavigationTiming object.
    ● Make sure sample size is large enough,
    i.e. 100/500 reloads is more reliable than 3/6
    reloads even though it’s 20% v/s 50%

    View Slide

  28. 😠
    Rage Click Rate
    ● Does the existence of a particular
    error correspond to a high number of
    Rage Clicks on the page with the
    error?

    View Slide

  29. Rage Clicks occur when users rapid-fire click (or
    tap) on your site or app.
    Rage clicking is the digital equivalent of
    cursing to release frustration.
    https://www.fullstory.com/resources/guide-to-understanding-frustrating-user-experiences-online/
    https://www.psychologytoday.com/us/blog/hide-and-seek/201205/hell-yes-the-7-best-reasons-swearing
    Rage Clicks

    View Slide

  30. 😠
    Rage Click Rate
    ● Does the existence of a particular
    error correspond to a high number of
    Rage Clicks on the page with the
    error?
    ● Points to user frustration

    View Slide

  31. Truncated User Journey
    ● Some errors cause a user session to terminate immediately.
    ● Others seem to allow the user to proceed.
    ● It’s also instructive to split this report by converted and/or “might
    convert” status.

    View Slide

  32. Errors affecting Performance
    ● Some errors make page load time worse.
    ● In this case inability to access localStorage probably
    requires extra network calls.
    Access denied for ‘localStorage’

    View Slide

  33. Errors affecting Performance
    ● Some errors make page load time better.
    ● In some cases, it causes page load to fail resulting in partial or
    no-content.
    There was an error setting cookie .* Please check domain and path

    View Slide

  34. If an error blocks content and makes
    the page faster, without reloads or
    truncated sessions, then is that
    content really needed?

    View Slide

  35. Debugging Aids
    Information to identify root causes

    View Slide

  36. How do I fix them?
    ● When did it start happening?
    related to a new product release, browser release, etc.
    ● Where does it happen?
    Browser, OS, specific page, Geo region, etc.
    ● What are the conditions when it happens?
    Specific scripts/css loaded/not loaded, load time, http/h2, etc.
    ● What was the user journey like when it happens?
    Previous page, SPA lifetime, Session Length

    View Slide

  37. Errors over Time
    ● Look at total error count as well as average
    errors per page for a given error.
    ● Check if new spikes or fixes correlate with a new
    product release or a new browser release (we
    generally annotate our charts with product
    releases).
    ● For existing errors, look at trends in the average
    error count.

    View Slide

  38. Where does the error happen?
    OS Browser Device
    The operation is insecure
    Country Page

    View Slide

  39. Conditions corresponding to Errors
    Some errors only show up when particular
    JavaScript files fail to load… for example, due to an
    Ad-Blocker or high packet loss

    View Slide

  40. Some errors show up when a particular JavaScript
    file loads up more than once!

    View Slide

  41. I had the same error, it seems to have been caused by loading
    the PayPal script more than once. I fixed it by loading the script
    just once in the head, then loading the two buttons where they
    should be.
    zoid destroyed all components

    RebeccaRey on github issues for paypal

    View Slide

  42. Some errors show up when the page is proxied
    through Google Translate

    View Slide

  43. CSRF token expired.
    Some errors often show up when particular
    resources were served from cache.

    View Slide

  44. Some errors show up when the client device
    is in low power mode (battery < 10%).

    View Slide

  45. Effects of User Journey
    ● Data corruption in cookies or localStorage might
    only show up after a user browses through a few
    pages.
    ● Memory leaks in Single Page Apps may only
    manifest as problems after several navigations.
    ● Cross-domain cookie issues could show up if the
    user had to pass through a different subdomain
    through their journey.
    🍪
    🥠

    View Slide

  46. Dealing with Script Error

    View Slide

  47. What’s the Problem?
    Script error 51,169,268
    [container] is not a valid DOM Element 2,845,511
    [object Object] 2,553,637
    ReferenceError: 'xxxxxx' is undefined 1,952,751
    Error: s:0 rs:4 1,445,856
    Attempting to change access mechanism for an unconfigurable property 1,083,304
    TypeError: [code] is not a function 809,376
    TypeError: undefined is not an object 789,812
    81%
    No information in Script Error

    View Slide

  48. Why does this happen?
    ● Cross-origin script files that may be credentialed.
    ● The browser doesn’t know if error information may be sensitive or
    not, so it hides it by default.
    ● For example, an error that looks like this in the web dev console…

    View Slide

  49. Looks like this in an onerror handler…
    ● No real error message, no filename, line number, stack, nothing.
    ● You only get a timestamp!

    View Slide

  50. Two options to fix it
    1. If you have control over the page & script domain, do the
    following:
    a. Set the crossorigin="anonymous" attribute on the element.<br/>This tells the browser not to send cookies, and that this request is<br/>non-credentialed.<br/>b. Set the Access-Control-Allow-Origin HTTP response header on the<br/>script to either * or the domain of your host page.<br/>

    View Slide

  51. Two options to fix it
    2. If you only have control over the page, do the following:
    ○ Wrap all your calls to 3rd party libraries in a try/catch block:
    try {
    libraryCall();
    }
    catch(e) {
    // e contains detailed information about the error
    }

    View Slide

  52. ★ Collect user errors
    ★ Normalize & De-duplicate
    ★ Cluster similar errors
    ★ Measure User Impact
    ★ Handle Script Error
    Summary

    View Slide

  53. Thank you!

    View Slide

  54. ● boomerang error plugin: Source on github • Docs at Akamai
    ● null has no properties – MDN docs
    ● zoid destroyed all components – github issue
    ● A density based approach for discovering Clusters in Large Spatial Databases with
    Noise – Ester M, Kriegel H-P, Sander J, Xu X
    ● PerformanceNavigationTiming.type – MDN docs
    ● Rage Clicking – Fullstory
    ● Web Performance with Android's Battery Saver Mode – Goel U, Ludin S, Steiner M
    ● Side effects of boomerang’s JS Error tracking – Nic Jansma
    References

    View Slide