Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Alexa and Google Are Listening: How Much Are They Transmitting

Alexa and Google Are Listening: How Much Are They Transmitting

Presentation from BSidesIowa and BSidesKC 2018

Aaron Blythe

April 21, 2018
Tweet

More Decks by Aaron Blythe

Other Decks in Technology

Transcript

  1. ALEXA AND GOOGLE ARE LISTENING
    how much are they transmitting?

    View full-size slide

  2. @ablythe
    DEMO

    View full-size slide

  3. @ablythe
    RECORDING
    HTTPS://YOUTU.BE/OFPGL5Z5DFQ

    View full-size slide

  4. @ablythe
    https://www.youtube.com/watch?v=vmINGWsyWX0

    View full-size slide

  5. @ablythe
    Aaron Blythe (aaronblythe.com)
    • Lead Organizer

    @devopskc
    @devopsdayskc
    sizzle.devopsdayskc.org

    View full-size slide

  6. @ablythe
    https://youtu.be/r8wl5dcWg5w

    View full-size slide

  7. @ablythe
    Assistant/Device Cost Test Result
    Siri/iPhone
    Too Much/
    Have to have
    What sound does a
    pig make?
    Alexa/Echo Dot
    $49.95
    ($29.95 Black Friday)
    What does a pig say?
    I’m sorry. I only know
    English at the moment,
    I’ll try and learn some
    animal languages
    Hey Google/
    Google Home
    $49.00
    ($29.99 Black Friday)
    What sound does a
    pig make?
    What does a pig say?

    View full-size slide

  8. @ablythe
    Agreement (Legal)
    Router (Technical)
    Server
    Amazon
    Google
    App Services
    Other Services
    Client
    Pandora… etc
    ???

    View full-size slide

  9. @ablythe
    Agreement (Legal)
    Router (Technical)
    Server
    Amazon
    Google
    App Services
    Other Services
    Client
    Pandora… etc
    ???

    View full-size slide

  10. @ablythe
    Agreement (Legal)
    Router (Technical)
    Server
    Amazon
    Google
    App Services
    Other Services
    Client
    Pandora… etc
    ???

    View full-size slide

  11. @ablythe
    Agreement (Legal)
    Router (Technical)
    Server
    Amazon
    Google
    App Services
    Other Services
    Client
    Pandora… etc
    ???
    WHAT TRAFFIC IS LEAVING MY
    HOUSE?

    View full-size slide

  12. AMAZON ECHO DOT
    @ablythe

    View full-size slide

  13. @ablythe
    DOMAINS
    • api.amazon.com
    • api.amazonalexa.com
    • *.amazon.com
    • *.amazonaws.com
    • *.pandora.com
    • *.spotify.com
    • *.amazoncrl.com

    View full-size slide

  14. @ablythe
    DOMAINS
    • *.cloudfront.com
    • clients3.google.com
    • *.amcs-tachyon.com

    View full-size slide

  15. @ablythe
    DOMAINS
    • *.cloudfront.com —> Needs authentication - Amazon CDN
    • clients3.google.com —> ???
    • *.amcs-tachyon.com —> Registered to Amazon

    View full-size slide

  16. GOOGLE HOME MINI

    View full-size slide

  17. @ablythe
    DOMAINS
    • *.google.com
    • *.googleapis.com
    • *.googleanalytics.com
    • *.googleusercontent.com
    • *.gstatic.com —> Registered to Google
    • pandora.com

    View full-size slide

  18. @ablythe
    DOMAINS
    • redirector.gvt1.com
    • *.tritondigital.com
    • *p-cdn.com
    • tunein.com
    • sb.scorecardresearch.com

    View full-size slide

  19. @ablythe
    DOMAINS
    • redirector.gvt1.com —> Registered to google
    • *p-cdn.com -> Pandora
    • *.tritondigital.com
    • tunein.com
    • sb.scorecardresearch.com

    View full-size slide

  20. @ablythe
    DOMAINS
    • redirector.gvt1.com —> Google
    • *p-cdn.com —> Pandora
    • *.tritondigital.com —> Podcasts/Streaming
    • tunein.com —> Podcasts/Streaming
    • sb.scorecardresearch.com —> ??

    View full-size slide

  21. @ablythe
    Data to Home
    Company
    Data to 3rd Parties
    I Expect
    Data to 3rd Parties
    that Surprise Me

    View full-size slide

  22. @ablythe
    Agreement (Legal)
    Router (Technical)
    Server
    Amazon
    Google
    App Services
    Other Services
    Client
    Pandora… etc
    ???

    View full-size slide

  23. @ablythe
    Agreement (Legal)
    Router (Technical)
    Server
    Amazon
    Google
    App Services
    Other Services
    Client
    Pandora… etc
    ???
    HOW MUCH? HOW OFTEN?

    View full-size slide

  24. @ablythe
    • Plugin Ethernet
    • Create a Network
    • Connect Device (Echo Dot or Google Home Mini)
    • Reference
    • https://support.apple.com/kb/PH25327?locale=en_US
    • https://stackoverflow.com/questions/7166044/how-can-i-configure-wireshark-to-see-https-
    traffic

    View full-size slide

  25. AMAZON ECHO DOT
    @ablythe

    View full-size slide

  26. @ablythe
    Alexa, What’s the Weather

    View full-size slide

  27. @ablythe
    Alexa, what sound does a pig make?

    View full-size slide

  28. @ablythe
    Silent TV On

    View full-size slide

  29. GOOGLE HOME MINI
    @ablythe

    View full-size slide

  30. @ablythe
    Ok Google, What’s the Weather?

    View full-size slide

  31. @ablythe
    216.58.216.68 -> 1e100.net -> 1e100 is scientific notation for 1 googol -> Name “Google”

    View full-size slide

  32. @ablythe
    OK Google, what sound does a pig make?

    View full-size slide

  33. @ablythe
    Silent TV On

    View full-size slide

  34. @ablythe
    Echo Dot with TV On Google Home with TV On

    View full-size slide

  35. @ablythe
    Data when
    interacting
    ~ 5 to 10 seconds
    Data when silent
    ~ 3 mins
    Data when tv on
    ~3 mins
    46,665 7,728 2,035
    187,967 36,176 33,276

    View full-size slide

  36. @ablythe
    Agreement (Legal)
    Router (Technical)
    Server
    Amazon
    Google
    App Services
    Other Services
    Client
    Pandora… etc
    ???

    View full-size slide

  37. @ablythe
    Agreement (Legal)
    Router (Technical)
    Server
    Amazon
    Google
    App Services
    Other Services
    Client
    Pandora… etc
    ???
    WHAT HAVE I AGREED TOO?

    View full-size slide

  38. AMAZON ECHO DOT
    @ablythe

    View full-size slide

  39. @ablythe
    AMAZON ECHO
    • Alexa Terms of Use:
    • https://www.amazon.com/gp/help/customer/display.html?
    nodeId=201809740
    • Amazon Privacy Notice:
    • https://www.amazon.com/gp/help/customer/display.html?
    nodeId=201909010

    View full-size slide

  40. GOOGLE HOME MINI
    @ablythe

    View full-size slide

  41. @ablythe
    GOOGLE HOME
    • Google Home Privacy:
    • https://support.google.com/googlehome/answer/7072285?hl=en
    • Google Privacy Policy:
    • https://www.google.com/policies/privacy/
    • Archive
    • https://www.google.com/policies/privacy/archive/

    View full-size slide

  42. @ablythe
    Creates Voice Profile Allows Deletion
    Easy to find archive
    of agreement

    View full-size slide

  43. @ablythe
    Creates Voice Profile Allows Deletion
    Easy to find archive
    of agreement
    ?
    ?

    View full-size slide

  44. @ablythe
    Agreement (Legal)
    Router (Technical)
    Server
    Amazon
    Google
    App Services
    Other Services
    Client
    Pandora… etc
    ???

    View full-size slide

  45. @ablythe
    Agreement (Legal)
    Router (Technical)
    Server
    Amazon
    Google
    App Services
    Other Services
    Client
    Pandora… etc
    ???
    WHAT’S STORED ON
    THE SERVER SIDE?

    View full-size slide

  46. AMAZON ECHO DOT
    @ablythe

    View full-size slide

  47. @ablythe
    www.amazon.com/mycd

    View full-size slide

  48. GOOGLE HOME MINI
    @ablythe

    View full-size slide

  49. @ablythe
    https://myactivity.google.com/myactivity

    View full-size slide

  50. @ablythe
    https://www.google.com/maps/timeline

    View full-size slide

  51. @ablythe
    https://twitter.com/iamdylancurran/status/977559925680467968

    View full-size slide

  52. @ablythe
    Deletion is clear and
    what you would
    expect

    View full-size slide

  53. @ablythe
    Agreement (Legal)
    Router (Technical)
    Server
    Amazon
    Google
    App Services
    Other Services
    Client
    Pandora… etc
    ???

    View full-size slide

  54. @ablythe
    https://www.nytimes.com/2016/03/10/technology/the-echo-from-amazon-brims-with-
    groundbreaking-promise.html

    View full-size slide

  55. @ablythe
    http://www.techtimes.com/articles/207375/20170516/amazon-wants-to-make-sure-alexa-
    responds-only-when-you-actually-call-her.htm

    View full-size slide

  56. @ablythe
    https://twitter.com/iamdylancurran/status/984123375537844231

    View full-size slide

  57. @ablythe
    http://appft.uspto.gov/netacgi/nph-Parser?
    Sect1=PTO1&Sect2=HITOFF&d=PG01&p=1&u=%2Fnetahtml%2FPTO%2Fsrchnum.html&r=1&
    f=G&l=50&s1=%2220170323645%22.PGNR.&OS=DN/20170323645&RS=DN/20170323645

    View full-size slide

  58. @ablythe
    SMART TV’S

    View full-size slide

  59. @ablythe
    HTTP://AARONBLYTHE.ORG/

    View full-size slide