Alexa and Google Are Listening: How Much Are They Transmitting

Presentation from BSidesIowa and BSidesKC 2018

Transcript

1. ALEXA AND GOOGLE ARE LISTENING
   how much are they transmitting?
   

   View Slide

2. @ablythe
   DEMO
   

   View Slide

3. @ablythe
   RECORDING
   HTTPS://YOUTU.BE/OFPGL5Z5DFQ
   

   View Slide

4. @ablythe
   https://www.youtube.com/watch?v=vmINGWsyWX0
   

   View Slide

5. @ablythe
   

   View Slide

6. @ablythe
   

   View Slide

7. @ablythe
   

   View Slide

8. @ablythe
   

   View Slide

9. @ablythe
   

   View Slide

10. @ablythe
    

    View Slide

11. @ablythe
    

    View Slide

12. @ablythe
    Aaron Blythe (aaronblythe.com)
    • Lead Organizer
    •
    @devopskc
    @devopsdayskc
    sizzle.devopsdayskc.org
    

    View Slide

13. @ablythe
    https://youtu.be/r8wl5dcWg5w
    

    View Slide

14. @ablythe
    Assistant/Device Cost Test Result
    Siri/iPhone
    Too Much/
    Have to have
    What sound does a
    pig make?
    Alexa/Echo Dot
    $49.95
    ($29.95 Black Friday)
    What does a pig say?
    I’m sorry. I only know
    English at the moment,
    I’ll try and learn some
    animal languages
    Hey Google/
    Google Home
    $49.00
    ($29.99 Black Friday)
    What sound does a
    pig make?
    What does a pig say?
    

    View Slide

15. @ablythe
    Agreement (Legal)
    Router (Technical)
    Server
    Amazon
    Google
    App Services
    Other Services
    Client
    Pandora… etc
    ???
    

    View Slide

16. @ablythe
    Agreement (Legal)
    Router (Technical)
    Server
    Amazon
    Google
    App Services
    Other Services
    Client
    Pandora… etc
    ???
    

    View Slide

17. @ablythe
    Agreement (Legal)
    Router (Technical)
    Server
    Amazon
    Google
    App Services
    Other Services
    Client
    Pandora… etc
    ???
    

    View Slide

18. @ablythe
    Agreement (Legal)
    Router (Technical)
    Server
    Amazon
    Google
    App Services
    Other Services
    Client
    Pandora… etc
    ???
    WHAT TRAFFIC IS LEAVING MY
    HOUSE?
    

    View Slide

19. AMAZON ECHO DOT
    @ablythe
    

    View Slide

20. @ablythe
    

    View Slide

21. @ablythe
    

    View Slide

22. @ablythe
    DOMAINS
    • api.amazon.com
    • api.amazonalexa.com
    • *.amazon.com
    • *.amazonaws.com
    • *.pandora.com
    • *.spotify.com
    • *.amazoncrl.com
    

    View Slide

23. @ablythe
    

    View Slide

24. @ablythe
    

    View Slide

25. @ablythe
    DOMAINS
    • *.cloudfront.com
    • clients3.google.com
    • *.amcs-tachyon.com
    

    View Slide

26. @ablythe
    

    View Slide

27. @ablythe
    DOMAINS
    • *.cloudfront.com —> Needs authentication - Amazon CDN
    • clients3.google.com —> ???
    • *.amcs-tachyon.com —> Registered to Amazon
    

    View Slide

28. GOOGLE HOME MINI
    

    View Slide

29. @ablythe
    

    View Slide

30. @ablythe
    DOMAINS
    • *.google.com
    • *.googleapis.com
    • *.googleanalytics.com
    • *.googleusercontent.com
    • *.gstatic.com —> Registered to Google
    • pandora.com
    

    View Slide

31. @ablythe
    DOMAINS
    • redirector.gvt1.com
    • *.tritondigital.com
    • *p-cdn.com
    • tunein.com
    • sb.scorecardresearch.com
    

    View Slide

32. @ablythe
    

    View Slide

33. @ablythe
    

    View Slide

34. @ablythe
    DOMAINS
    • redirector.gvt1.com —> Registered to google
    • *p-cdn.com -> Pandora
    • *.tritondigital.com
    • tunein.com
    • sb.scorecardresearch.com
    

    View Slide

35. @ablythe
    

    View Slide

36. @ablythe
    

    View Slide

37. @ablythe
    

    View Slide

38. @ablythe
    

    View Slide

39. @ablythe
    

    View Slide

40. @ablythe
    DOMAINS
    • redirector.gvt1.com —> Google
    • *p-cdn.com —> Pandora
    • *.tritondigital.com —> Podcasts/Streaming
    • tunein.com —> Podcasts/Streaming
    • sb.scorecardresearch.com —> ??
    

    View Slide

41. @ablythe
    

    View Slide

42. @ablythe
    

    View Slide

43. @ablythe
    Data to Home
    Company
    Data to 3rd Parties
    I Expect
    Data to 3rd Parties
    that Surprise Me
    

    View Slide

44. @ablythe
    Agreement (Legal)
    Router (Technical)
    Server
    Amazon
    Google
    App Services
    Other Services
    Client
    Pandora… etc
    ???
    

    View Slide

45. @ablythe
    Agreement (Legal)
    Router (Technical)
    Server
    Amazon
    Google
    App Services
    Other Services
    Client
    Pandora… etc
    ???
    HOW MUCH? HOW OFTEN?
    

    View Slide

46. @ablythe
    • Plugin Ethernet
    • Create a Network
    • Connect Device (Echo Dot or Google Home Mini)
    • Reference
    • https://support.apple.com/kb/PH25327?locale=en_US
    • https://stackoverflow.com/questions/7166044/how-can-i-configure-wireshark-to-see-https-
    traffic
    

    View Slide

47. AMAZON ECHO DOT
    @ablythe
    

    View Slide

48. @ablythe
    Alexa, What’s the Weather
    

    View Slide

49. @ablythe
    

    View Slide

50. @ablythe
    Alexa, what sound does a pig make?
    

    View Slide

51. @ablythe
    Silent TV On
    

    View Slide

52. GOOGLE HOME MINI
    @ablythe
    

    View Slide

53. @ablythe
    Ok Google, What’s the Weather?
    

    View Slide

54. @ablythe
    216.58.216.68 -> 1e100.net -> 1e100 is scientific notation for 1 googol -> Name “Google”
    

    View Slide

55. @ablythe
    OK Google, what sound does a pig make?
    

    View Slide

56. @ablythe
    Silent TV On
    

    View Slide

57. @ablythe
    Echo Dot with TV On Google Home with TV On
    

    View Slide

58. @ablythe
    Data when
    interacting
    ~ 5 to 10 seconds
    Data when silent
    ~ 3 mins
    Data when tv on
    ~3 mins
    46,665 7,728 2,035
    187,967 36,176 33,276
    

    View Slide

59. @ablythe
    Agreement (Legal)
    Router (Technical)
    Server
    Amazon
    Google
    App Services
    Other Services
    Client
    Pandora… etc
    ???
    

    View Slide

60. @ablythe
    Agreement (Legal)
    Router (Technical)
    Server
    Amazon
    Google
    App Services
    Other Services
    Client
    Pandora… etc
    ???
    WHAT HAVE I AGREED TOO?
    

    View Slide

61. AMAZON ECHO DOT
    @ablythe
    

    View Slide

62. @ablythe
    AMAZON ECHO
    • Alexa Terms of Use:
    • https://www.amazon.com/gp/help/customer/display.html?
    nodeId=201809740
    • Amazon Privacy Notice:
    • https://www.amazon.com/gp/help/customer/display.html?
    nodeId=201909010
    

    View Slide

63. @ablythe
    

    View Slide

64. @ablythe
    

    View Slide

65. @ablythe
    

    View Slide

66. GOOGLE HOME MINI
    @ablythe
    

    View Slide

67. @ablythe
    GOOGLE HOME
    • Google Home Privacy:
    • https://support.google.com/googlehome/answer/7072285?hl=en
    • Google Privacy Policy:
    • https://www.google.com/policies/privacy/
    • Archive
    • https://www.google.com/policies/privacy/archive/
    

    View Slide

68. @ablythe
    

    View Slide

69. @ablythe
    Creates Voice Profile Allows Deletion
    Easy to find archive
    of agreement
    

    View Slide

70. @ablythe
    Creates Voice Profile Allows Deletion
    Easy to find archive
    of agreement
    ?
    ?
    

    View Slide

71. @ablythe
    Agreement (Legal)
    Router (Technical)
    Server
    Amazon
    Google
    App Services
    Other Services
    Client
    Pandora… etc
    ???
    

    View Slide

72. @ablythe
    Agreement (Legal)
    Router (Technical)
    Server
    Amazon
    Google
    App Services
    Other Services
    Client
    Pandora… etc
    ???
    WHAT’S STORED ON
    THE SERVER SIDE?
    

    View Slide

73. AMAZON ECHO DOT
    @ablythe
    

    View Slide

74. @ablythe
    www.amazon.com/mycd
    

    View Slide

75. @ablythe
    

    View Slide

76. @ablythe
    

    View Slide

77. GOOGLE HOME MINI
    @ablythe
    

    View Slide

78. @ablythe
    https://myactivity.google.com/myactivity
    

    View Slide

79. @ablythe
    https://www.google.com/maps/timeline
    

    View Slide

80. @ablythe
    

    View Slide

81. @ablythe
    

    View Slide

82. @ablythe
    

    View Slide

83. @ablythe
    https://twitter.com/iamdylancurran/status/977559925680467968
    

    View Slide

84. @ablythe
    

    View Slide

85. @ablythe
    Deletion is clear and
    what you would
    expect
    

    View Slide

86. @ablythe
    Agreement (Legal)
    Router (Technical)
    Server
    Amazon
    Google
    App Services
    Other Services
    Client
    Pandora… etc
    ???
    

    View Slide

87. @ablythe
    https://www.nytimes.com/2016/03/10/technology/the-echo-from-amazon-brims-with-
    groundbreaking-promise.html
    

    View Slide

88. @ablythe
    http://www.techtimes.com/articles/207375/20170516/amazon-wants-to-make-sure-alexa-
    responds-only-when-you-actually-call-her.htm
    

    View Slide

89. @ablythe
    

    View Slide

90. @ablythe
    https://twitter.com/iamdylancurran/status/984123375537844231
    

    View Slide

91. @ablythe
    http://appft.uspto.gov/netacgi/nph-Parser?
    Sect1=PTO1&Sect2=HITOFF&d=PG01&p=1&u=%2Fnetahtml%2FPTO%2Fsrchnum.html&r=1&
    f=G&l=50&s1=%2220170323645%22.PGNR.&OS=DN/20170323645&RS=DN/20170323645
    

    View Slide

92. @ablythe
    

    View Slide

93. @ablythe
    

    View Slide

94. @ablythe
    SMART TV’S
    

    View Slide

95. @ablythe
    

    View Slide

96. @ablythe
    HTTP://AARONBLYTHE.ORG/
    

    View Slide