Consensual Software - QCon London 2017

Consensual Software - QCon London 2017

Online safety has become a huge problem in the world of oversharing. Real name policies, automatic geolocation tracking, and photo tagging increase user adoption rates, but these features can be quickly abused by bad actors. This talk will discuss how to apply a "consent filter" to product decisions to make a safer user experience and to help protect your most vulnerable users from harm.
https://qconlondon.com/london-2017/speakers/danielle-leong

15d8946b2d4141db558701c2327ceb25?s=128

Danielle Leong

March 07, 2017
Tweet

Transcript

  1. 2.

    Consensual Software: Prioritizing Trust & Safety ! @tsunamino 2 @tsunamino

    http://www.wocintechchat.com/blog/wocintechphotos
  2. 3.

    Consensual Software: Prioritizing Trust & Safety ! @tsunamino 3 @tsunamino

    http://www.wocintechchat.com/blog/wocintechphotos
  3. 4.

    Consensual Software: Prioritizing Trust & Safety ! @tsunamino 4 @tsunamino

    http://www.wocintechchat.com/blog/wocintechphotos
  4. 5.

    Consensual Software: Prioritizing Trust & Safety ! @tsunamino 5 @tsunamino

    http://www.wocintechchat.com/blog/wocintechphotos
  5. 6.

    Consensual Software: Prioritizing Trust & Safety ! @tsunamino 6 @tsunamino

    http://www.wocintechchat.com/blog/wocintechphotos
  6. 8.

    Consensual Software: Prioritizing Trust & Safety ! @tsunamino 8 @tsunamino

    http://www.wocintechchat.com/blog/wocintechphotos
  7. 10.

    Consensual Software: Prioritizing Trust & Safety ! @tsunamino 10 @tsunamino

    http://www.wocintechchat.com/blog/wocintechphotos
  8. 11.

    Consensual Software: Prioritizing Trust & Safety ! @tsunamino 11 @tsunamino

    http://www.wocintechchat.com/blog/wocintechphotos
  9. 13.

    Consensual Software: Prioritizing Trust & Safety ! @tsunamino 13 @tsunamino

    http://www.wocintechchat.com/blog/wocintechphotos
  10. 16.

    Consensual Software: Prioritizing Trust & Safety ! @tsunamino 16 Danielle

    Leong Engineer on GitHub’s Community & Safety team @tsunamino
  11. 18.

    Consensual Software: Prioritizing Trust & Safety ! @tsunamino 18 GitHub

    is a web-based version-control and collaboration platform for software developers. @tsunamino
  12. 20.

    Consensual Software: Prioritizing Trust & Safety ! @tsunamino 20 Build

    systems that empower inclusive and healthy communities and encourage good online citizenship. Discourage behavior that is destructive or threatens personal safety.
  13. 21.

    Consensual Software: Prioritizing Trust & Safety ! @tsunamino 21 Some

    things we’ve built • Repository Invitations • User blocking API • Organizations have the ability to block users • Contributor badges • Community Health management tools • Transparency around comment edit history
  14. 22.

    Consensual Software: Prioritizing Trust & Safety ! @tsunamino 22 On

    any platform, any time you have user-to-user interactions you have the possibility of harassment.
  15. 23.

    Consensual Software: Prioritizing Trust & Safety ! @tsunamino 23 Types

    of online harassment • Posting personal info (doxing) • Hate speech • Sexual or pornographic content • Impersonation • SWATing • Spam • Identity theft • Exclusionary behavior • Death and rape threats • Dog piling • Personal attacks • Threatening and inappropriate images • So many more
  16. 24.

    Consensual Software: Prioritizing Trust & Safety ! @tsunamino 24 Racist

    repository names led to repository invitations
  17. 26.

    Consensual Software: Prioritizing Trust & Safety ! @tsunamino 26 Women,

    women of color, and trans women experience harassment daily. We are experts in online safety. Not to say that men don’t experience harassment, but women tend to get the worst of it
  18. 27.

    Consensual Software: Prioritizing Trust & Safety ! @tsunamino 27 70%

    women 40% women of color 30% trans women 30% white men 4 engineers 1 designer 1 product manager 1 qa 1 year of work to address 7 years of backlog
  19. 28.

    Consensual Software: Prioritizing Trust & Safety ! @tsunamino 28 Some

    tactics women use to avoid online harassment • Not using full names on social media sites • Not using real photos for avatars • Turning off location tracking data • Regularly Googling name, address, & phone numbers • Using a burner phone number service & PO boxes • Turning off direct messages • Not live posting on social media during vacation
  20. 29.

    Consensual Software: Prioritizing Trust & Safety ! @tsunamino 29 @tsunamino

    http://www.wocintechchat.com/blog/wocintechphotos
  21. 31.

    Consensual Software: Prioritizing Trust & Safety ! @tsunamino 31 It’s

    our job to write software that protects vulnerable people.
  22. 35.

    Consensual Software: Prioritizing Trust & Safety ! @tsunamino 35 "

    " " " BUT FREE SPEECH BUT FREE SPEECH https://www.flickr.com/photos/epw/348448681/in/photolist-wMTDT-eMnoZK-ahqLSQ-dWv9vQ-dK4Hnd-eLPF8d-DKNYyX-jnCGNf-eNgckU-388Mus-eKGUs3- eR26Ui-58nGwR-idg5CJ-9Jwyxb-639vTn-FmvThd-4e5Fff-ah1Dse-eiTkAT-9Jwyzh-pvMWjr-ovCHpM-dWtVVp-i2aAxc-5mDSDM-DrXkrE-eciQvg-c9811Y-4t5Tcr-dbzN7K- r1WnoK-qKrNny-qKrueE-dWtJ4T-hYCB6D-hDr4y9-bq3RAJ-92ee1-dTuCwi-Ji54Ed-eMa6Du-9yqLWZ-7PoVXq-b2Ev8R-aho1T2-9TR1s8-9E6usg-nzmYLL-dNEzjC
  23. 36.

    Consensual Software: Prioritizing Trust & Safety ! @tsunamino 36 “Free

    speech” does not mean you get to say whatever you want without consequences
  24. 37.

    Consensual Software: Prioritizing Trust & Safety ! @tsunamino 37 I

    mean, you can, but it’ll cost you a bunch of money http://www.businessinsider.com/twitter-trolls-caused-salesforce-to-walk-away-from-deal-2016-10 http://sfist.com/2016/10/18/disney_dropped_twitter_bid_because.php https://www.bloomberg.com/news/articles/2016-10-17/disney-said-to-have-dropped-twitter-pursuit-partly-over-image http://money.cnn.com/2016/10/14/technology/salesforce-not-buying-twitter/
  25. 39.

    Consensual Software: Prioritizing Trust & Safety ! @tsunamino 39 More

    than half of app users have uninstalled or decided to not install an app due to concerns about personal information Pew Research Center http://www.pewinternet.org/2012/09/05/privacy-and-data-management-on-mobile-devices/
  26. 40.

    Consensual Software: Prioritizing Trust & Safety ! @tsunamino 40 Safety

    features shouldn’t be an afterthought. This leads to years of technical debt.
  27. 41.

    Consensual Software: Prioritizing Trust & Safety ! @tsunamino 41 70%

    women 40% women of color 30% trans women 30% white men 4 engineers 1 designer 1 product manager 1 qa 1 year of work to address 7 years of backlog
  28. 42.

    Consensual Software: Prioritizing Trust & Safety ! @tsunamino 42 Design

    software with trust & safety in mind before it becomes a problem.
  29. 44.

    Consensual Software: Prioritizing Trust & Safety ! @tsunamino 44 GitHub’s

    definition of harassment is destructive behavior that undermines productivity.
  30. 45.

    Consensual Software: Prioritizing Trust & Safety ! @tsunamino 45 While

    it is not forbidden to broach topics such as age, body size, disability, ethnicity, gender identity and expression, level of experience, nationality, personal appearance, race, religion, or sexual identity and orientation, we do not tolerate speech that attacks a person or group of people on the basis of who they are.
  31. 47.

    Consensual Software: Prioritizing Trust & Safety ! @tsunamino 47 Build

    Trust & Safety into new features • Ensure explicit consent • Keep an audit log trail • Strive to minimize abuse https://githubengineering.com/community-and-safety-feature-reviews/
  32. 50.

    Consensual Software: Prioritizing Trust & Safety ! @tsunamino 50 Consent

    means all parties are aware and able to agree to an interaction.
  33. 54.

    Consensual Software: Prioritizing Trust & Safety ! @tsunamino 54 Consensual

    software means all users are aware and explicitly consenting to every interaction and disclosure of their personal data on your site or service.
  34. 56.

    Consensual Software: Prioritizing Trust & Safety ! @tsunamino 56 More

    than half of app users have uninstalled or decided to not install an app due to concerns about personal information Pew Research Center http://www.pewinternet.org/2012/09/05/privacy-and-data-management-on-mobile-devices/
  35. 60.

    Consensual Software: Prioritizing Trust & Safety ! @tsunamino 60 %•

    Does Alice want to be in this interaction? • Is it safe to block Bob? • How much of Alice’s information is revealed if she reports Bob?
  36. 65.

    Consensual Software: Prioritizing Trust & Safety ! @tsunamino 65 •

    Is it easy to create sock puppet accounts? • Can you introduce friction to discourage destructive behavior? • Is it easy to learn from mistakes? &
  37. 67.

    Consensual Software: Prioritizing Trust & Safety ! @tsunamino 67 )•

    Is there an audit log for your support folks? • If you can edit content, do you know who did what change? • How many people have reported Bob?
  38. 69.

    Consensual Software: Prioritizing Trust & Safety ! @tsunamino 69 *•

    Are you limiting 0-day account activity? • Are you rate limiting potentially abusive accounts? • If you have a reputation score, how is it calculated?
  39. 71.

    Consensual Software: Prioritizing Trust & Safety ! @tsunamino 71 Smart

    Girl’s Guide to Privacy by Violet Blue HATERS: Harassment, Abuse, and Violence Online by Bailey Poland The Internet of Garbage by Sarah Jeong
  40. 73.

    Consensual Software: Prioritizing Trust & Safety ! @tsunamino 73 Trust

    & Safety guidelines for new features • Ensure explicit consent • Keep an audit log trail • Strive to minimize abuse https://githubengineering.com/community-and-safety-feature-reviews/
  41. 74.

    Consensual Software: Prioritizing Trust & Safety ! @tsunamino 74 Make

    sure your users are aware and explicitly consenting to every interaction and disclosure of their personal data on your site or service.
  42. 75.

    Consensual Software: Prioritizing Trust & Safety ! @tsunamino 75 Hire

    more women of color and trans women. We are experts in avoiding online harassment.