Consensual Software - QCon London 2017

Consensual Software - QCon London 2017

Online safety has become a huge problem in the world of oversharing. Real name policies, automatic geolocation tracking, and photo tagging increase user adoption rates, but these features can be quickly abused by bad actors. This talk will discuss how to apply a "consent filter" to product decisions to make a safer user experience and to help protect your most vulnerable users from harm.
https://qconlondon.com/london-2017/speakers/danielle-leong

15d8946b2d4141db558701c2327ceb25?s=128

Danielle Leong

March 07, 2017
Tweet

Transcript

  1. @tsunamino ! " Consensual Software: Prioritizing Trust & Safety

  2. Consensual Software: Prioritizing Trust & Safety ! @tsunamino 2 @tsunamino

    http://www.wocintechchat.com/blog/wocintechphotos
  3. Consensual Software: Prioritizing Trust & Safety ! @tsunamino 3 @tsunamino

    http://www.wocintechchat.com/blog/wocintechphotos
  4. Consensual Software: Prioritizing Trust & Safety ! @tsunamino 4 @tsunamino

    http://www.wocintechchat.com/blog/wocintechphotos
  5. Consensual Software: Prioritizing Trust & Safety ! @tsunamino 5 @tsunamino

    http://www.wocintechchat.com/blog/wocintechphotos
  6. Consensual Software: Prioritizing Trust & Safety ! @tsunamino 6 @tsunamino

    http://www.wocintechchat.com/blog/wocintechphotos
  7. Consensual Software: Prioritizing Trust & Safety ! @tsunamino 7 @tsunamino

  8. Consensual Software: Prioritizing Trust & Safety ! @tsunamino 8 @tsunamino

    http://www.wocintechchat.com/blog/wocintechphotos
  9. Consensual Software: Prioritizing Trust & Safety ! @tsunamino 9 @tsunamino

  10. Consensual Software: Prioritizing Trust & Safety ! @tsunamino 10 @tsunamino

    http://www.wocintechchat.com/blog/wocintechphotos
  11. Consensual Software: Prioritizing Trust & Safety ! @tsunamino 11 @tsunamino

    http://www.wocintechchat.com/blog/wocintechphotos
  12. Consensual Software: Prioritizing Trust & Safety ! @tsunamino 12

  13. Consensual Software: Prioritizing Trust & Safety ! @tsunamino 13 @tsunamino

    http://www.wocintechchat.com/blog/wocintechphotos
  14. Consensual Software: Prioritizing Trust & Safety ! @tsunamino 14 @tsunamino

  15. Consensual Software: Prioritizing Trust & Safety ! @tsunamino 15

  16. Consensual Software: Prioritizing Trust & Safety ! @tsunamino 16 Danielle

    Leong Engineer on GitHub’s Community & Safety team @tsunamino
  17. Title Text @tsunamino

  18. Consensual Software: Prioritizing Trust & Safety ! @tsunamino 18 GitHub

    is a web-based version-control and collaboration platform for software developers. @tsunamino
  19. Consensual Software: Prioritizing Trust & Safety ! @tsunamino 19 Collaboration

    in text form is hard.
  20. Consensual Software: Prioritizing Trust & Safety ! @tsunamino 20 Build

    systems that empower inclusive and healthy communities and encourage good online citizenship. Discourage behavior that is destructive or threatens personal safety.
  21. Consensual Software: Prioritizing Trust & Safety ! @tsunamino 21 Some

    things we’ve built • Repository Invitations • User blocking API • Organizations have the ability to block users • Contributor badges • Community Health management tools • Transparency around comment edit history
  22. Consensual Software: Prioritizing Trust & Safety ! @tsunamino 22 On

    any platform, any time you have user-to-user interactions you have the possibility of harassment.
  23. Consensual Software: Prioritizing Trust & Safety ! @tsunamino 23 Types

    of online harassment • Posting personal info (doxing) • Hate speech • Sexual or pornographic content • Impersonation • SWATing • Spam • Identity theft • Exclusionary behavior • Death and rape threats • Dog piling • Personal attacks • Threatening and inappropriate images • So many more
  24. Consensual Software: Prioritizing Trust & Safety ! @tsunamino 24 Racist

    repository names led to repository invitations
  25. Consensual Software: Prioritizing Trust & Safety ! @tsunamino 25

  26. Consensual Software: Prioritizing Trust & Safety ! @tsunamino 26 Women,

    women of color, and trans women experience harassment daily. We are experts in online safety. Not to say that men don’t experience harassment, but women tend to get the worst of it
  27. Consensual Software: Prioritizing Trust & Safety ! @tsunamino 27 70%

    women 40% women of color 30% trans women 30% white men 4 engineers 1 designer 1 product manager 1 qa 1 year of work to address 7 years of backlog
  28. Consensual Software: Prioritizing Trust & Safety ! @tsunamino 28 Some

    tactics women use to avoid online harassment • Not using full names on social media sites • Not using real photos for avatars • Turning off location tracking data • Regularly Googling name, address, & phone numbers • Using a burner phone number service & PO boxes • Turning off direct messages • Not live posting on social media during vacation
  29. Consensual Software: Prioritizing Trust & Safety ! @tsunamino 29 @tsunamino

    http://www.wocintechchat.com/blog/wocintechphotos
  30. Consensual Software: Prioritizing Trust & Safety ! @tsunamino 30 @tsunamino

  31. Consensual Software: Prioritizing Trust & Safety ! @tsunamino 31 It’s

    our job to write software that protects vulnerable people.
  32. Consensual Software: Prioritizing Trust & Safety ! @tsunamino 32 http://www.huffingtonpost.com/aran-khanna/stalking-your-friends-on-facebook-messenger-_b_7452912.html

  33. Consensual Software: Prioritizing Trust & Safety ! @tsunamino 33 http://www.huffingtonpost.com/aran-khanna/stalking-your-friends-on-facebook-messenger-_b_7452912.html

  34. Consensual Software: Prioritizing Trust & Safety ! @tsunamino 34 http://www.huffingtonpost.com/aran-khanna/venmo-money_b_8418130.html

  35. Consensual Software: Prioritizing Trust & Safety ! @tsunamino 35 "

    " " " BUT FREE SPEECH BUT FREE SPEECH https://www.flickr.com/photos/epw/348448681/in/photolist-wMTDT-eMnoZK-ahqLSQ-dWv9vQ-dK4Hnd-eLPF8d-DKNYyX-jnCGNf-eNgckU-388Mus-eKGUs3- eR26Ui-58nGwR-idg5CJ-9Jwyxb-639vTn-FmvThd-4e5Fff-ah1Dse-eiTkAT-9Jwyzh-pvMWjr-ovCHpM-dWtVVp-i2aAxc-5mDSDM-DrXkrE-eciQvg-c9811Y-4t5Tcr-dbzN7K- r1WnoK-qKrNny-qKrueE-dWtJ4T-hYCB6D-hDr4y9-bq3RAJ-92ee1-dTuCwi-Ji54Ed-eMa6Du-9yqLWZ-7PoVXq-b2Ev8R-aho1T2-9TR1s8-9E6usg-nzmYLL-dNEzjC
  36. Consensual Software: Prioritizing Trust & Safety ! @tsunamino 36 “Free

    speech” does not mean you get to say whatever you want without consequences
  37. Consensual Software: Prioritizing Trust & Safety ! @tsunamino 37 I

    mean, you can, but it’ll cost you a bunch of money http://www.businessinsider.com/twitter-trolls-caused-salesforce-to-walk-away-from-deal-2016-10 http://sfist.com/2016/10/18/disney_dropped_twitter_bid_because.php https://www.bloomberg.com/news/articles/2016-10-17/disney-said-to-have-dropped-twitter-pursuit-partly-over-image http://money.cnn.com/2016/10/14/technology/salesforce-not-buying-twitter/
  38. Consensual Software: Prioritizing Trust & Safety ! @tsunamino 38 https://www.theguardian.com/commentisfree/2017/jan/03/ive-left-twitter-unusable-anyone-but-trolls-robots-dictators-lindy-west

  39. Consensual Software: Prioritizing Trust & Safety ! @tsunamino 39 More

    than half of app users have uninstalled or decided to not install an app due to concerns about personal information Pew Research Center http://www.pewinternet.org/2012/09/05/privacy-and-data-management-on-mobile-devices/
  40. Consensual Software: Prioritizing Trust & Safety ! @tsunamino 40 Safety

    features shouldn’t be an afterthought. This leads to years of technical debt.
  41. Consensual Software: Prioritizing Trust & Safety ! @tsunamino 41 70%

    women 40% women of color 30% trans women 30% white men 4 engineers 1 designer 1 product manager 1 qa 1 year of work to address 7 years of backlog
  42. Consensual Software: Prioritizing Trust & Safety ! @tsunamino 42 Design

    software with trust & safety in mind before it becomes a problem.
  43. Consensual Software: Prioritizing Trust & Safety ! @tsunamino 43 Have

    a definition of harassment.
  44. Consensual Software: Prioritizing Trust & Safety ! @tsunamino 44 GitHub’s

    definition of harassment is destructive behavior that undermines productivity.
  45. Consensual Software: Prioritizing Trust & Safety ! @tsunamino 45 While

    it is not forbidden to broach topics such as age, body size, disability, ethnicity, gender identity and expression, level of experience, nationality, personal appearance, race, religion, or sexual identity and orientation, we do not tolerate speech that attacks a person or group of people on the basis of who they are.
  46. Consensual Software: Prioritizing Trust & Safety ! @tsunamino 46 https://mobile.twitter.com/TweetAnnaMarie/status/789957313649967104

  47. Consensual Software: Prioritizing Trust & Safety ! @tsunamino 47 Build

    Trust & Safety into new features • Ensure explicit consent • Keep an audit log trail • Strive to minimize abuse https://githubengineering.com/community-and-safety-feature-reviews/
  48. Consensual Software: Prioritizing Trust & Safety ! @tsunamino 48 Build

    consensual software.
  49. Consensual Software: Prioritizing Trust & Safety ! @tsunamino 49

  50. Consensual Software: Prioritizing Trust & Safety ! @tsunamino 50 Consent

    means all parties are aware and able to agree to an interaction.
  51. Consensual Software: Prioritizing Trust & Safety ! @tsunamino 51 https://www.youtube.com/watch?v=oQbei5JGiT8

  52. Consensual Software: Prioritizing Trust & Safety ! @tsunamino 52

  53. Consensual Software: Prioritizing Trust & Safety ! @tsunamino 53

  54. Consensual Software: Prioritizing Trust & Safety ! @tsunamino 54 Consensual

    software means all users are aware and explicitly consenting to every interaction and disclosure of their personal data on your site or service.
  55. Consensual Software: Prioritizing Trust & Safety ! @tsunamino 55

  56. Consensual Software: Prioritizing Trust & Safety ! @tsunamino 56 More

    than half of app users have uninstalled or decided to not install an app due to concerns about personal information Pew Research Center http://www.pewinternet.org/2012/09/05/privacy-and-data-management-on-mobile-devices/
  57. Consensual Software: Prioritizing Trust & Safety ! @tsunamino 57 %

    &
  58. Consensual Software: Prioritizing Trust & Safety ! @tsunamino 58 Ensure

    explicit consent
  59. Consensual Software: Prioritizing Trust & Safety ! @tsunamino 59

  60. Consensual Software: Prioritizing Trust & Safety ! @tsunamino 60 %•

    Does Alice want to be in this interaction? • Is it safe to block Bob? • How much of Alice’s information is revealed if she reports Bob?
  61. Consensual Software: Prioritizing Trust & Safety ! @tsunamino 61

  62. Consensual Software: Prioritizing Trust & Safety ! @tsunamino 62 https://www.vg247.com/2015/06/03/the-sad-saga-of-lego-universes-dong-detection/

  63. Consensual Software: Prioritizing Trust & Safety ! @tsunamino 63 https://techcrunch.com/2016/03/24/microsoft-silences-its-new-a-i-bot-tay-after-twitter-users-teach-it-racism/

  64. Consensual Software: Prioritizing Trust & Safety ! @tsunamino 64 https://qz.com/818151/virtual-assistant-bots-like-siri-alexa-and-cortana-spend-much-of-their-time-fending-off-sexual-harassment/

  65. Consensual Software: Prioritizing Trust & Safety ! @tsunamino 65 •

    Is it easy to create sock puppet accounts? • Can you introduce friction to discourage destructive behavior? • Is it easy to learn from mistakes? &
  66. Consensual Software: Prioritizing Trust & Safety ! @tsunamino 66 Keep

    an audit log trail
  67. Consensual Software: Prioritizing Trust & Safety ! @tsunamino 67 )•

    Is there an audit log for your support folks? • If you can edit content, do you know who did what change? • How many people have reported Bob?
  68. Consensual Software: Prioritizing Trust & Safety ! @tsunamino 68 Strive

    to minimize abuse
  69. Consensual Software: Prioritizing Trust & Safety ! @tsunamino 69 *•

    Are you limiting 0-day account activity? • Are you rate limiting potentially abusive accounts? • If you have a reputation score, how is it calculated?
  70. Consensual Software: Prioritizing Trust & Safety ! @tsunamino 70 https://githubengineering.com/community-and-safety-

    feature-reviews/
  71. Consensual Software: Prioritizing Trust & Safety ! @tsunamino 71 Smart

    Girl’s Guide to Privacy by Violet Blue HATERS: Harassment, Abuse, and Violence Online by Bailey Poland The Internet of Garbage by Sarah Jeong
  72. Consensual Software: Prioritizing Trust & Safety ! @tsunamino 72 Have

    a definition of harassment.
  73. Consensual Software: Prioritizing Trust & Safety ! @tsunamino 73 Trust

    & Safety guidelines for new features • Ensure explicit consent • Keep an audit log trail • Strive to minimize abuse https://githubengineering.com/community-and-safety-feature-reviews/
  74. Consensual Software: Prioritizing Trust & Safety ! @tsunamino 74 Make

    sure your users are aware and explicitly consenting to every interaction and disclosure of their personal data on your site or service.
  75. Consensual Software: Prioritizing Trust & Safety ! @tsunamino 75 Hire

    more women of color and trans women. We are experts in avoiding online harassment.
  76. Consensual Software: Prioritizing Trust & Safety ! @tsunamino 76 Questions?

  77. How people build software ! " Thanks!