Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Alexa and Google Are Listening: How Much Are They Transmitting

Alexa and Google Are Listening: How Much Are They Transmitting

Presentation from BSidesIowa and BSidesKC 2018

Aaron Blythe

April 21, 2018
Tweet

More Decks by Aaron Blythe

Other Decks in Technology

Transcript

  1. ALEXA AND GOOGLE ARE LISTENING
    how much are they transmitting?

    View Slide

  2. @ablythe
    DEMO

    View Slide

  3. @ablythe
    RECORDING
    HTTPS://YOUTU.BE/OFPGL5Z5DFQ

    View Slide

  4. @ablythe
    https://www.youtube.com/watch?v=vmINGWsyWX0

    View Slide

  5. @ablythe

    View Slide

  6. @ablythe

    View Slide

  7. @ablythe

    View Slide

  8. @ablythe

    View Slide

  9. @ablythe

    View Slide

  10. @ablythe

    View Slide

  11. @ablythe

    View Slide

  12. @ablythe
    Aaron Blythe (aaronblythe.com)
    • Lead Organizer

    @devopskc
    @devopsdayskc
    sizzle.devopsdayskc.org

    View Slide

  13. @ablythe
    https://youtu.be/r8wl5dcWg5w

    View Slide

  14. @ablythe
    Assistant/Device Cost Test Result
    Siri/iPhone
    Too Much/
    Have to have
    What sound does a
    pig make?
    Alexa/Echo Dot
    $49.95
    ($29.95 Black Friday)
    What does a pig say?
    I’m sorry. I only know
    English at the moment,
    I’ll try and learn some
    animal languages
    Hey Google/
    Google Home
    $49.00
    ($29.99 Black Friday)
    What sound does a
    pig make?
    What does a pig say?

    View Slide

  15. @ablythe
    Agreement (Legal)
    Router (Technical)
    Server
    Amazon
    Google
    App Services
    Other Services
    Client
    Pandora… etc
    ???

    View Slide

  16. @ablythe
    Agreement (Legal)
    Router (Technical)
    Server
    Amazon
    Google
    App Services
    Other Services
    Client
    Pandora… etc
    ???

    View Slide

  17. @ablythe
    Agreement (Legal)
    Router (Technical)
    Server
    Amazon
    Google
    App Services
    Other Services
    Client
    Pandora… etc
    ???

    View Slide

  18. @ablythe
    Agreement (Legal)
    Router (Technical)
    Server
    Amazon
    Google
    App Services
    Other Services
    Client
    Pandora… etc
    ???
    WHAT TRAFFIC IS LEAVING MY
    HOUSE?

    View Slide

  19. AMAZON ECHO DOT
    @ablythe

    View Slide

  20. @ablythe

    View Slide

  21. @ablythe

    View Slide

  22. @ablythe
    DOMAINS
    • api.amazon.com
    • api.amazonalexa.com
    • *.amazon.com
    • *.amazonaws.com
    • *.pandora.com
    • *.spotify.com
    • *.amazoncrl.com

    View Slide

  23. @ablythe

    View Slide

  24. @ablythe

    View Slide

  25. @ablythe
    DOMAINS
    • *.cloudfront.com
    • clients3.google.com
    • *.amcs-tachyon.com

    View Slide

  26. @ablythe

    View Slide

  27. @ablythe
    DOMAINS
    • *.cloudfront.com —> Needs authentication - Amazon CDN
    • clients3.google.com —> ???
    • *.amcs-tachyon.com —> Registered to Amazon

    View Slide

  28. GOOGLE HOME MINI

    View Slide

  29. @ablythe

    View Slide

  30. @ablythe
    DOMAINS
    • *.google.com
    • *.googleapis.com
    • *.googleanalytics.com
    • *.googleusercontent.com
    • *.gstatic.com —> Registered to Google
    • pandora.com

    View Slide

  31. @ablythe
    DOMAINS
    • redirector.gvt1.com
    • *.tritondigital.com
    • *p-cdn.com
    • tunein.com
    • sb.scorecardresearch.com

    View Slide

  32. @ablythe

    View Slide

  33. @ablythe

    View Slide

  34. @ablythe
    DOMAINS
    • redirector.gvt1.com —> Registered to google
    • *p-cdn.com -> Pandora
    • *.tritondigital.com
    • tunein.com
    • sb.scorecardresearch.com

    View Slide

  35. @ablythe

    View Slide

  36. @ablythe

    View Slide

  37. @ablythe

    View Slide

  38. @ablythe

    View Slide

  39. @ablythe

    View Slide

  40. @ablythe
    DOMAINS
    • redirector.gvt1.com —> Google
    • *p-cdn.com —> Pandora
    • *.tritondigital.com —> Podcasts/Streaming
    • tunein.com —> Podcasts/Streaming
    • sb.scorecardresearch.com —> ??

    View Slide

  41. @ablythe

    View Slide

  42. @ablythe

    View Slide

  43. @ablythe
    Data to Home
    Company
    Data to 3rd Parties
    I Expect
    Data to 3rd Parties
    that Surprise Me

    View Slide

  44. @ablythe
    Agreement (Legal)
    Router (Technical)
    Server
    Amazon
    Google
    App Services
    Other Services
    Client
    Pandora… etc
    ???

    View Slide

  45. @ablythe
    Agreement (Legal)
    Router (Technical)
    Server
    Amazon
    Google
    App Services
    Other Services
    Client
    Pandora… etc
    ???
    HOW MUCH? HOW OFTEN?

    View Slide

  46. @ablythe
    • Plugin Ethernet
    • Create a Network
    • Connect Device (Echo Dot or Google Home Mini)
    • Reference
    • https://support.apple.com/kb/PH25327?locale=en_US
    • https://stackoverflow.com/questions/7166044/how-can-i-configure-wireshark-to-see-https-
    traffic

    View Slide

  47. AMAZON ECHO DOT
    @ablythe

    View Slide

  48. @ablythe
    Alexa, What’s the Weather

    View Slide

  49. @ablythe

    View Slide

  50. @ablythe
    Alexa, what sound does a pig make?

    View Slide

  51. @ablythe
    Silent TV On

    View Slide

  52. GOOGLE HOME MINI
    @ablythe

    View Slide

  53. @ablythe
    Ok Google, What’s the Weather?

    View Slide

  54. @ablythe
    216.58.216.68 -> 1e100.net -> 1e100 is scientific notation for 1 googol -> Name “Google”

    View Slide

  55. @ablythe
    OK Google, what sound does a pig make?

    View Slide

  56. @ablythe
    Silent TV On

    View Slide

  57. @ablythe
    Echo Dot with TV On Google Home with TV On

    View Slide

  58. @ablythe
    Data when
    interacting
    ~ 5 to 10 seconds
    Data when silent
    ~ 3 mins
    Data when tv on
    ~3 mins
    46,665 7,728 2,035
    187,967 36,176 33,276

    View Slide

  59. @ablythe
    Agreement (Legal)
    Router (Technical)
    Server
    Amazon
    Google
    App Services
    Other Services
    Client
    Pandora… etc
    ???

    View Slide

  60. @ablythe
    Agreement (Legal)
    Router (Technical)
    Server
    Amazon
    Google
    App Services
    Other Services
    Client
    Pandora… etc
    ???
    WHAT HAVE I AGREED TOO?

    View Slide

  61. AMAZON ECHO DOT
    @ablythe

    View Slide

  62. @ablythe
    AMAZON ECHO
    • Alexa Terms of Use:
    • https://www.amazon.com/gp/help/customer/display.html?
    nodeId=201809740
    • Amazon Privacy Notice:
    • https://www.amazon.com/gp/help/customer/display.html?
    nodeId=201909010

    View Slide

  63. @ablythe

    View Slide

  64. @ablythe

    View Slide

  65. @ablythe

    View Slide

  66. GOOGLE HOME MINI
    @ablythe

    View Slide

  67. @ablythe
    GOOGLE HOME
    • Google Home Privacy:
    • https://support.google.com/googlehome/answer/7072285?hl=en
    • Google Privacy Policy:
    • https://www.google.com/policies/privacy/
    • Archive
    • https://www.google.com/policies/privacy/archive/

    View Slide

  68. @ablythe

    View Slide

  69. @ablythe
    Creates Voice Profile Allows Deletion
    Easy to find archive
    of agreement

    View Slide

  70. @ablythe
    Creates Voice Profile Allows Deletion
    Easy to find archive
    of agreement
    ?
    ?

    View Slide

  71. @ablythe
    Agreement (Legal)
    Router (Technical)
    Server
    Amazon
    Google
    App Services
    Other Services
    Client
    Pandora… etc
    ???

    View Slide

  72. @ablythe
    Agreement (Legal)
    Router (Technical)
    Server
    Amazon
    Google
    App Services
    Other Services
    Client
    Pandora… etc
    ???
    WHAT’S STORED ON
    THE SERVER SIDE?

    View Slide

  73. AMAZON ECHO DOT
    @ablythe

    View Slide

  74. @ablythe
    www.amazon.com/mycd

    View Slide

  75. @ablythe

    View Slide

  76. @ablythe

    View Slide

  77. GOOGLE HOME MINI
    @ablythe

    View Slide

  78. @ablythe
    https://myactivity.google.com/myactivity

    View Slide

  79. @ablythe
    https://www.google.com/maps/timeline

    View Slide

  80. @ablythe

    View Slide

  81. @ablythe

    View Slide

  82. @ablythe

    View Slide

  83. @ablythe
    https://twitter.com/iamdylancurran/status/977559925680467968

    View Slide

  84. @ablythe

    View Slide

  85. @ablythe
    Deletion is clear and
    what you would
    expect

    View Slide

  86. @ablythe
    Agreement (Legal)
    Router (Technical)
    Server
    Amazon
    Google
    App Services
    Other Services
    Client
    Pandora… etc
    ???

    View Slide

  87. @ablythe
    https://www.nytimes.com/2016/03/10/technology/the-echo-from-amazon-brims-with-
    groundbreaking-promise.html

    View Slide

  88. @ablythe
    http://www.techtimes.com/articles/207375/20170516/amazon-wants-to-make-sure-alexa-
    responds-only-when-you-actually-call-her.htm

    View Slide

  89. @ablythe

    View Slide

  90. @ablythe
    https://twitter.com/iamdylancurran/status/984123375537844231

    View Slide

  91. @ablythe
    http://appft.uspto.gov/netacgi/nph-Parser?
    Sect1=PTO1&Sect2=HITOFF&d=PG01&p=1&u=%2Fnetahtml%2FPTO%2Fsrchnum.html&r=1&
    f=G&l=50&s1=%2220170323645%22.PGNR.&OS=DN/20170323645&RS=DN/20170323645

    View Slide

  92. @ablythe

    View Slide

  93. @ablythe

    View Slide

  94. @ablythe
    SMART TV’S

    View Slide

  95. @ablythe

    View Slide

  96. @ablythe
    HTTP://AARONBLYTHE.ORG/

    View Slide