Pro Yearly is on sale from $80 to $50! »

Identifying Abuse Vectors in Web Applications

13074675eac6f0ee1599b341d4a95342?s=47 Terian Koscik
April 17, 2018
120

Identifying Abuse Vectors in Web Applications

13074675eac6f0ee1599b341d4a95342?s=128

Terian Koscik

April 17, 2018
Tweet

Transcript

  1. Identifying Abuse Vectors Terian Koscik @spine_cone www.pineconedoesthings.com/abuse-vectors

  2. (cw)

  3. Agenda for today • Talking about abuse vectors (20 minutes)

    • Getting into groups • Identifying abuse vectors (15 minutes) • Check in (5 minutes) • Take a break (5 minutes) • Solutions to abuse vectors (15 minutes) • Solutions to abuse vectors (part 2) (15 minutes) • Presentations (5 minutes)
  4. Things we’ll be learning about • What is an abuse

    vector? • How can you tell when something is an abuse vector? • What are some ways of handling them?
  5. None
  6. Hello it’s me! www.pineconedoesthings.com www.twitter.com/spine_cone www.github.com/spinecone www.djangogirls.org/portland

  7. • What’s your name? • Where are you from? •

    What are you most excited for at RailsConf?
  8. None
  9. https://www.wired.com/2015/10/brief-history-of-the-demise-of-the-comments-timeline/

  10. None
  11. None
  12. None
  13. None
  14. http://www.pewinternet.org/2017/07/11/online-harassment-2017/

  15. http://www.pewinternet.org/2017/07/11/online-harassment-2017/

  16. http://www.pewinternet.org/2017/07/11/online-harassment-2017/

  17. http://www.pewinternet.org/2017/07/11/online-harassment-2017/ !!!!!!

  18. “Bad Actor” fallacy • Bad people do bad things •

    Good people do not do bad things • If we find all the bad people, bad things will never happen
  19. “Bad behaviors” vs “bad actors” • All people are capable

    of good and bad actions • There is a huge variety of the kinds of things humans are willing and motivated to do • If it’s at all possible to do a certain action on your website, assume that eventually, some (most likely very bored) person will do it
  20. Anyone could become a troll at any time “For nearly

    three years Steve Smith has spewed over-the-top conservative blather on Twitter, luring Senator Claire McCaskill, Christiane Amanpour and Rosie O'Donnell into arguments. A 40-year-old dad and lawyer who lives outside Tampa, he says he has become addicted to the attention. "I was totally ruined when I started this. My ex-wife and I had just separated. She decided to start a new, more exciting life without me," he says. Then his best friend, who he used to do pranks with as a kid, killed himself. Marty says his trolling has been empowering.” http://time.com/4457110/internet-trolls/
  21. And trolls can be rehabilitated “In 2015, Reddit closed several

    subreddits—foremost among them r/fatpeoplehate and r/CoonTown—due to violations of Reddit’s anti-harassment policy. We find that the ban worked for Reddit. More accounts than expected discontinued using the site; those that stayed drastically decreased their hate speech usage—by at least 80%. Though many subreddits saw an influx of r/fatpeoplehate and r/CoonTown “migrants,” those subreddits saw no significant changes in hate speech usage. In other words, other subreddits did not inherit the problem. ” http://comp.social.gatech.edu/papers/cscw18-chand-hate.pdf
  22. None
  23. None
  24. None
  25. None
  26. None
  27. None
  28. None
  29. None
  30. If you’re feeling...

  31. Take a minute to be nice to yourself!

  32. Getting into groups

  33. workshop instructions: www.pineconedoesthings.com/abuse-vectors

  34. Trust and Safety Engineer Role Playing Game! You have been

    hired as a software engineer on an exciting new product! It’s your job to check the existing features of the product for potential abuse vectors, and determine options for handling them. Your new employer has most likely never considered the possibility of harassment on their platform and is eager to hear any feedback you might have.
  35. Part 1 Identifying Abuse Vectors

  36. Is this an abuse vector? Could someone use this feature

    to… • Communicate with someone without their consent? • Show something to a person without their consent? • Learn something about a person without their consent?
  37. Is this an abuse vector? Could this be used to

    hurt someone… • Physically? • Emotionally? • Their career?
  38. Some examples:

  39. None
  40. ?

  41. None
  42. None
  43. ?

  44. None
  45. None
  46. ?

  47. HEY ARE YOU A GITHUB STAFF MEMBER BECAUSE I WAS

    WONDERING IF MAYBE YOU COULD TELL ME WHY I WAS BANNED IS IT BECAUSE I’M ANNOYING BECAUSE I REALLY DON’T THINK YOU’RE QUALIFIED TO MAKE THOSE KINDS OF JUDGMENTS AND FURTHERMORE
  48. First steps Pick one of the “Use Cases” sections from

    the workshop instructions. Your group will be looking for abuse vectors in one of these applications, and thinking of ways to fix them. You can change your mind later on if you decide you don’t like the one you picked! (P.S.: the Pokedex is a little more challenging than the others)
  49. Next steps Familiarize yourself with the features described for the

    application you’ll be working on, then think about: • What potential abuse vectors does the application have? • What might motivate someone to take advantage of them? • Note: harassment and abuse are not the same as hacking You can review everything we’ve talked about so far in the “Addressing Abuse Vectors” section of the workshop instructions. We’ll check in after 15 minutes
  50. Check in! • What abuse vectors did your group find?

    • Is there anything that’s unclear about the product you’re investigating?
  51. Take a break!

  52. Part 2 Implicitly Preventing Abuse

  53. How to deal with an abuse vector • Take out

    the feature • Reduce interaction • Reduce visibility • Don't keep data you don't need • Intervene before, during, and after harassment • Make it opt-in • Add moderation
  54. How to deal with an abuse vector • Take out

    the feature • Reduce interaction • Reduce visibility • Don't keep data you don't need • Intervene before, during, and after harassment • Make it opt-in • Add moderation
  55. Take out the feature https://www.wired.com/2015/10/brief-history-of-the-demise-of-the-comments-timeline/

  56. Reduce interaction https://hearthstone.gamepedia.com/Emote

  57. Reduce visibility https://help.twitch.tv/customer/portal/articles/2401004-partner-settings-guide

  58. Don’t keep data you don’t need https://protonvpn.com/privacy-policy

  59. If you’re not sure what to do... • What are

    other websites doing? • What does research (or google) say to do? • What would stop a cat?
  60. What a proposed solution looks like • How does this

    address an abuse vector? • What are the changes to the UI? • What are the tradeoffs being made? • How does this affect the business? (revenue, PR, software engineering time, etc.) • What alternatives are there?
  61. ?

  62. None
  63. None
  64. None
  65. None
  66. ?

  67. Next Steps Now that you’ve identified some abuse vectors, decide

    where you can use these principles: • Take out the feature • Reduce interaction • Reduce visibility • Don't keep data you don't need We’ll move on to the next section in 15 minutes
  68. Part 3 Empowering Humans to Prevent Abuse

  69. How to deal with an abuse vector • Take out

    the feature • Reduce interaction • Reduce visibility • Don't keep data you don't need • Intervene before, during, and after harassment • Make it opt-in • Add moderation
  70. Intervene before, during, and after harassment https://www.reddit.com/

  71. Make it opt-in https://twitter.com/settings/safety

  72. Add moderation https://www.mediawiki.org/wiki/Extension:Moderation

  73. Last Steps! Your employer has decided they aren’t comfortable with

    any of the options you’ve come up with :C Decide where you can use these principles instead: • Add moderation • Make it opt-in • Intervene before, during, and after harassment Each group will give a 1-2 minute presentation in 15 minutes
  74. HEY ARE YOU A GITHUB STAFF MEMBER BECAUSE I WAS

    WONDERING IF MAYBE YOU COULD TELL ME WHY I WAS BANNED IS IT BECAUSE I’M ANNOYING BECAUSE I REALLY DON’T THINK YOU’RE QUALIFIED TO MAKE THOSE KINDS OF JUDGMENTS AND FURTHERMORE ?
  75. ?

  76. Check in (again)! • What abuse vectors did your group

    decide to address? • What solutions do you propose to address them? • What are the tradeoffs to these solutions?
  77. Taco Bell won’t let me send them messages about how

    many burritos I can fit in my mouth at once :C
  78. None
  79. None
  80. None
  81. teriankoscik@gmail.com www.twitter.com/spine_cone www.pineconedoesthings.com