Identifying Abuse Vectors in Web Applications

Transcript

1. Identifying Abuse Vectors Terian Koscik @spine_cone www.pineconedoesthings.com/abuse-vectors
2. (cw)
3. Agenda for today • Talking about abuse vectors (20 minutes) • Getting into groups • Identifying abuse vectors (15 minutes) • Check in (5 minutes) • Take a break (5 minutes) • Solutions to abuse vectors (15 minutes) • Solutions to abuse vectors (part 2) (15 minutes) • Presentations (5 minutes)
4. Things we’ll be learning about • What is an abuse vector? • How can you tell when something is an abuse vector? • What are some ways of handling them?
6. Hello it’s me! www.pineconedoesthings.com www.twitter.com/spine_cone www.github.com/spinecone www.djangogirls.org/portland
7. • What’s your name? • Where are you from? • What are you most excited for at RailsConf?
9. https://www.wired.com/2015/10/brief-history-of-the-demise-of-the-comments-timeline/
14. http://www.pewinternet.org/2017/07/11/online-harassment-2017/
15. http://www.pewinternet.org/2017/07/11/online-harassment-2017/
16. http://www.pewinternet.org/2017/07/11/online-harassment-2017/
17. http://www.pewinternet.org/2017/07/11/online-harassment-2017/ !!!!!!
18. “Bad Actor” fallacy • Bad people do bad things • Good people do not do bad things • If we find all the bad people, bad things will never happen
19. “Bad behaviors” vs “bad actors” • All people are capable of good and bad actions • There is a huge variety of the kinds of things humans are willing and motivated to do • If it’s at all possible to do a certain action on your website, assume that eventually, some (most likely very bored) person will do it
20. Anyone could become a troll at any time “For nearly three years Steve Smith has spewed over-the-top conservative blather on Twitter, luring Senator Claire McCaskill, Christiane Amanpour and Rosie O'Donnell into arguments. A 40-year-old dad and lawyer who lives outside Tampa, he says he has become addicted to the attention. "I was totally ruined when I started this. My ex-wife and I had just separated. She decided to start a new, more exciting life without me," he says. Then his best friend, who he used to do pranks with as a kid, killed himself. Marty says his trolling has been empowering.” http://time.com/4457110/internet-trolls/
21. And trolls can be rehabilitated “In 2015, Reddit closed several subreddits—foremost among them r/fatpeoplehate and r/CoonTown—due to violations of Reddit’s anti-harassment policy. We find that the ban worked for Reddit. More accounts than expected discontinued using the site; those that stayed drastically decreased their hate speech usage—by at least 80%. Though many subreddits saw an influx of r/fatpeoplehate and r/CoonTown “migrants,” those subreddits saw no significant changes in hate speech usage. In other words, other subreddits did not inherit the problem. ” http://comp.social.gatech.edu/papers/cscw18-chand-hate.pdf
30. If you’re feeling...
31. Take a minute to be nice to yourself!
32. Getting into groups
33. workshop instructions: www.pineconedoesthings.com/abuse-vectors
34. Trust and Safety Engineer Role Playing Game! You have been hired as a software engineer on an exciting new product! It’s your job to check the existing features of the product for potential abuse vectors, and determine options for handling them. Your new employer has most likely never considered the possibility of harassment on their platform and is eager to hear any feedback you might have.
35. Part 1 Identifying Abuse Vectors
36. Is this an abuse vector? Could someone use this feature to… • Communicate with someone without their consent? • Show something to a person without their consent? • Learn something about a person without their consent?
37. Is this an abuse vector? Could this be used to hurt someone… • Physically? • Emotionally? • Their career?
38. Some examples:
40. ?
43. ?
46. ?
47. HEY ARE YOU A GITHUB STAFF MEMBER BECAUSE I WAS WONDERING IF MAYBE YOU COULD TELL ME WHY I WAS BANNED IS IT BECAUSE I’M ANNOYING BECAUSE I REALLY DON’T THINK YOU’RE QUALIFIED TO MAKE THOSE KINDS OF JUDGMENTS AND FURTHERMORE
48. First steps Pick one of the “Use Cases” sections from the workshop instructions. Your group will be looking for abuse vectors in one of these applications, and thinking of ways to fix them. You can change your mind later on if you decide you don’t like the one you picked! (P.S.: the Pokedex is a little more challenging than the others)
49. Next steps Familiarize yourself with the features described for the application you’ll be working on, then think about: • What potential abuse vectors does the application have? • What might motivate someone to take advantage of them? • Note: harassment and abuse are not the same as hacking You can review everything we’ve talked about so far in the “Addressing Abuse Vectors” section of the workshop instructions. We’ll check in after 15 minutes
50. Check in! • What abuse vectors did your group find? • Is there anything that’s unclear about the product you’re investigating?
51. Take a break!
52. Part 2 Implicitly Preventing Abuse
53. How to deal with an abuse vector • Take out the feature • Reduce interaction • Reduce visibility • Don't keep data you don't need • Intervene before, during, and after harassment • Make it opt-in • Add moderation
54. How to deal with an abuse vector • Take out the feature • Reduce interaction • Reduce visibility • Don't keep data you don't need • Intervene before, during, and after harassment • Make it opt-in • Add moderation
55. Take out the feature https://www.wired.com/2015/10/brief-history-of-the-demise-of-the-comments-timeline/
56. Reduce interaction https://hearthstone.gamepedia.com/Emote
57. Reduce visibility https://help.twitch.tv/customer/portal/articles/2401004-partner-settings-guide
58. Don’t keep data you don’t need https://protonvpn.com/privacy-policy
59. If you’re not sure what to do... • What are other websites doing? • What does research (or google) say to do? • What would stop a cat?
60. What a proposed solution looks like • How does this address an abuse vector? • What are the changes to the UI? • What are the tradeoffs being made? • How does this affect the business? (revenue, PR, software engineering time, etc.) • What alternatives are there?
61. ?
66. ?
67. Next Steps Now that you’ve identified some abuse vectors, decide where you can use these principles: • Take out the feature • Reduce interaction • Reduce visibility • Don't keep data you don't need We’ll move on to the next section in 15 minutes
68. Part 3 Empowering Humans to Prevent Abuse
69. How to deal with an abuse vector • Take out the feature • Reduce interaction • Reduce visibility • Don't keep data you don't need • Intervene before, during, and after harassment • Make it opt-in • Add moderation
70. Intervene before, during, and after harassment https://www.reddit.com/
71. Make it opt-in https://twitter.com/settings/safety
72. Add moderation https://www.mediawiki.org/wiki/Extension:Moderation
73. Last Steps! Your employer has decided they aren’t comfortable with any of the options you’ve come up with :C Decide where you can use these principles instead: • Add moderation • Make it opt-in • Intervene before, during, and after harassment Each group will give a 1-2 minute presentation in 15 minutes
74. HEY ARE YOU A GITHUB STAFF MEMBER BECAUSE I WAS WONDERING IF MAYBE YOU COULD TELL ME WHY I WAS BANNED IS IT BECAUSE I’M ANNOYING BECAUSE I REALLY DON’T THINK YOU’RE QUALIFIED TO MAKE THOSE KINDS OF JUDGMENTS AND FURTHERMORE ?
75. ?
76. Check in (again)! • What abuse vectors did your group decide to address? • What solutions do you propose to address them? • What are the tradeoffs to these solutions?
77. Taco Bell won’t let me send them messages about how many burritos I can fit in my mouth at once :C
81. teriankoscik@gmail.com www.twitter.com/spine_cone www.pineconedoesthings.com