Mutually Assured Destruction and the Impending AI Apocalypse

USENIX Workshop on Offensive Technologies 2018
Opening Keynote
Baltimore, Maryland
13 August 2018

The history of security includes a long series of arms races, where a new technology emerges and is subsequently developed and exploited by both defenders and attackers. Over the past few years, "Artificial Intelligence" has re-emerged as a potentially transformative technology, and deep learning in particular has produced a barrage of amazing results. We are in the very early stages of understanding the potential of this technology in security, but more worryingly, seeing how it may be exploited by malicious individuals and powerful organizations. In this talk, I'll look at what lessons might be learned from previous security arms races, consider how asymmetries in AI may be exploited by attackers and defenders, touch on some recent work in adversarial machine learning, and hopefully help progress-loving Luddites figure out how to survive in a world overrun by AI doppelgängers, GAN gangs, and gibbon-impersonating pandas.

David Evans is a Professor of Computer Science at the University of Virginia where he leads the Security Research Group. He is the author of an open computer science textbook and a children's book on combinatorics and computability. He won the Outstanding Faculty Award from the State Council of Higher Education for Virginia, and was Program Co-Chair for the 24th ACM Conference on Computer and Communications Security (CCS 2017) and the 30th (2009) and 31st (2010) IEEE Symposia on Security and Privacy. He has SB, SM and PhD degrees in Computer Science from MIT and has been a faculty member at the University of Virginia since 1999.

Transcript

1. Mutually
   Assured
   Destruction
   and the
   Impending AI
   Apocalypse
   David Evans
   University of Virginia
   evadeML.org
   USENIX Workshop on
   Offensive Technologies
   13 August 2018
   Baltimore, MD
   

   View Slide

2. AI Arms Races
   and How to
   End Them
   David Evans
   University of Virginia
   evadeML.org
   USENIX Workshop on
   Offensive Technologies
   13 August 2018
   Baltimore, MD
   

   View Slide

3. 2
   All technologies
   are (potentially)
   offensive
   Artificial
   Intelligence is an
   encompassing,
   disruptive
   technology
   

   View Slide

4. Plan for Talk
   1. What is AI?
   Definitions
   2. What should (and shouldn’t) we be afraid of?
   Harmful use of AI
   3. What can we learn from previous arms races?
   Evasive malware
   4. What (if anything) can we do?
   3
   

   View Slide

5. What is Artificial Intelligence?
   4
   

   View Slide

6. 5
   Doesn’t distinguish
   from computing in
   general
   Unclear target
   

   View Slide

7. 6
   Cognitive Task Human Machine (2018)
   Adding 4-digit numbers ü
   Adding 5-digit numbers ü
   ... ü
   Adding 8923-digit numbers ü
   Spelling ü
   Sorting alphabetically ü
   Sorting numerically ü
   Factoring big numbers ü
   Playing chess ü
   Playing poker ü
   Playing go ü
   Face recognition ü
   

   View Slide

8. 7
   Cognitive Task Human Machine (2018)
   Adding 4-digit numbers ü
   Adding 5-digit numbers ü
   ... ü
   Adding 8923-digit numbers ü
   Spelling ü
   Sorting alphabetically ü
   Sorting numerically ü
   Factoring big numbers ü
   Playing chess ü
   Playing poker ü
   Playing go ü
   Face recognition ü
   Giving talks at WOOT ?
   

   View Slide

9. Preparation for 1st Grade
   8
   

   View Slide

10. Cognitive Tasks
    9
    Typical 6-Year Old
    

    View Slide

11. Cognitive Tasks
    10
    Typical 6-Year Old
    Typical Adult
    

    View Slide

12. Cognitive Tasks
    11
    Typical 6-Year Old
    Typical Adult
    Median WOOT Attendee
    

    View Slide

13. Cognitive Tasks
    12
    Typical 6-Year Old
    Typical Adult
    Any Human Alive
    Median WOOT Attendee
    

    View Slide

14. Humanity
    Cognitive Tasks
    13
    

    View Slide

15. Humanity
    Cognitive Tasks
    14
    Machines (2018)
    

    View Slide

16. Humanity
    Cognitive Tasks
    15
    Machines (2018)
    Machines (202x)
    

    View Slide

17. View Slide

18. View Slide

19. More Ambition
    18
    “The human race will have a new
    kind of instrument which will
    increase the power of the mind
    much more than optical lenses
    strengthen the eyes and which
    will be as far superior to
    microscopes or telescopes as
    reason is superior to sight.”
    

    View Slide

20. More Ambition
    19
    “The human race will have a new
    kind of instrument which will
    increase the power of the mind
    much more than optical lenses
    strengthen the eyes and which
    will be as far superior to
    microscopes or telescopes as
    reason is superior to sight.”
    Gottfried Wilhelm Leibniz (1679)
    

    View Slide

21. 20
    Gottfried Wilhelm Leibniz (Universitat Altdorf, 1666) who advised:
    Jacob Bernoulli (Universitdt Basel, 1684) who advised:
    Johann Bernoulli (Universitdt Basel, 1694) who advised:
    Leonhard Euler (Universitat Basel, 1726) who advised:
    Joseph Louis Lagrange who advised:
    Simeon Denis Poisson who advised:
    Michel Chasles (Ecole Polytechnique, 1814) who advised:
    H. A. (Hubert Anson) Newton (Yale, 1850) who advised:
    E. H. Moore (Yale, 1885) who advised:
    Oswald Veblen (U. of Chicago, 1903) who advised:
    Philip Franklin (Princeton 1921) who advised:
    Alan Perlis (MIT Math PhD 1950) who advised:
    Jerry Feldman (CMU Math 1966) who advised:
    Jim Horning (Stanford CS PhD 1969) who advised:
    John Guttag (U. of Toronto CS PhD 1975) who advised:
    David Evans (MIT CS PhD 2000)
    my academic great-
    great-great-great-
    great-great-great-
    great-great-great-
    great-great-great-
    great-great-
    grandparent!
    

    View Slide

22. More Precision
    21
    “The human race will have a new
    kind of instrument which will
    increase the power of the mind
    much more than optical lenses
    strengthen the eyes and which
    will be as far superior to
    microscopes or telescopes as
    reason is superior to sight.”
    Gottfried Wilhelm Leibniz (1679)
    Normal computing amplifies
    (quadrillions of times faster)
    and aggregates (enables
    millions of humans to work
    together) human cognitive
    abilities; AI goes beyond
    what humans can do.
    

    View Slide

23. 22
    (Cover story by Steve Levy)
    May 5, 1997
    

    View Slide

24. 23
    The history of computer chess is the history of
    artificial intelligence. After their
    disappointments in trying to reverse-
    engineer the brain, computer scientists
    narrowed their sights. Abandoning their
    pursuit of human-like intelligence, they
    began to concentrate on accomplishing
    sophisticated, but limited, analytical tasks by
    capitalizing on the inhuman speed of the
    modern computer’s calculations. This less
    ambitious but more pragmatic approach has
    paid off in areas ranging from medical
    diagnosis to self-driving cars. Computers are
    replicating the results of human thought
    without replicating thought itself.
    Nicolas Carr, A Brutal Intelligence: AI, Chess,
    and the Human Mind, 2017
    

    View Slide

25. View Slide

26. 25
    Claude Shannon, 1948
    Reinforcement Learning
    Image: Mark Chang, AlphaGo in Depth
    

    View Slide

27. Operational Definition
    “Artificial Intelligence”
    means making
    computers do things
    their programmers
    don’t understand well
    enough to program
    explicitly.
    26
    If it is
    explainable,
    its not AI!
    

    View Slide

28. Plan for Talk
    1. What is AI?
    Definitions
    2. What should (and shouldn’t) we be afraid of?
    Harmful use of AI
    3. What can we learn from previous arms races?
    Evasive malware
    4. What (if anything) can we do?
    27
    

    View Slide

29. Making Predictions
    28
    Paul Gascoigne
    I never predict
    anything
    

    View Slide

30. Making Predictions
    29
    Paul Gascoigne
    I never predict
    anything, and I
    never will.
    

    View Slide

31. Harmful AI
    Benign developers and operators
    AI out of control
    AI inadvertently causes harm
    Malicious operators
    Build AI to do harm
    30
    

    View Slide

32. Harmful AI
    Benign developers and operators
    AI out of control
    AI inadvertently causes harm
    Malicious operators
    Build AI to do harm
    31
    

    View Slide

33. Out-of-Control AI
    32
    HAL, 2001: A Space Odyssey SkyNet, The Terminator
    

    View Slide

34. Alignment Problem
    33
    Bostrom’s Paperclip Maximizer
    

    View Slide

35. Harmful AI
    Benign developers and operators
    AI out of control
    AI inadvertently causes harm to humanity
    Malicious operators
    Build AI to do harm
    34
    

    View Slide

36. Lost Jobs and Dignity
    35
    

    View Slide

37. Lost Jobs and Dignity
    36
    

    View Slide

38. 37
    On Robots
    Joe Berger and Pascal Wyse
    (The Guardian, 21 July 2018)
    Human Jobs
    of the Future
    

    View Slide

39. Inadvertent Bias and Discrimination
    38
    

    View Slide

40. Inadvertent Bias and Discrimination
    39
    

    View Slide

41. Harmful AI
    Benign developers
    AI out of control
    AI causes harm (without creators objecting)
    Malicious developers
    Using AI to do harm
    40
    Malice is (often) in the eye of the beholder
    (e.g., mass surveillance, pop-up ads, etc.)
    

    View Slide

42. 41
    “The future has arrived —
    it’s just not evenly
    distributed yet.”
    (William Gibson, 1990s)
    Photo: Christopher J. Morris/Corbis
    

    View Slide

43. 42
    “The future has arrived —
    it’s just not evenly
    distributed yet.”
    (William Gibson, 1990s)
    Expanding victims:
    Attacks that are only cost-effective
    for high-value, easy-compromise
    targets, become cost-effective
    against everyone
    Expanding adversaries:
    Attacks only available to nation-state
    level adversaries, become accessible
    to everyone
    

    View Slide

44. Malicious Uses of AI
    43
    Malware
    Automated Vulnerability Finding, Exploit Generation
    Social Engineering
    Mass-market Spear Phishing
    Fake content generation
    Virtual-physical attacks
    

    View Slide

45. Software Vulnerabilities and Exploits
    44
    IEEE S&P 2013
    DARPA Cyber Grand Challenge 2016
    1996
    

    View Slide

46. View Slide

47. Strategy 1: Deception Arms Race!
    46
    

    View Slide

48. Strategy 2:
    Build Less Vulnerable Systems
    47
    Rust
    Project Everest
    We actually know how to build much less vulnerable
    software, it just costs too much for everyday use.
    

    View Slide

49. Malicious Uses of AI
    48
    Malware
    Automated Vulnerability Finding, Exploit Generation
    Social Engineering
    Mass-market Spear Phishing
    Fake content generation
    Virtual-physical attacks
    

    View Slide

50. 49
    WEIS 2012
    Automated, low cost: sending out initial scam email
    Human, high effort: conversing with potential victims
    What happens when the conversing with potential victims part is automated also?
    

    View Slide

51. Automated Spear Phishing
    50
    “It’s slightly less effective [than manually generated] but it’s
    dramatically more efficient” (John Seymour)
    

    View Slide

52. Asymmetry of Automated Spear Phishing
    51
    AI Classifier
    “99.9% accurate”
    AI Spear
    Phishing
    Generator +
    Botnet
    ...
    Victim
    

    View Slide

53. (Long-Term) Solution to Spear Phishing
    52
    Better Authentication Mechanisms Better Software
    

    View Slide

54. Malicious Uses of AI
    53
    Malware
    Automated Vulnerability Finding, Exploit Generation
    Social Engineering
    Mass-market Spear Phishing
    Fake content generation
    Virtual-physical attacks
    

    View Slide

55. Fake Content
    54
    Deep Video Portraits (SIGGRAPH 2018)
    

    View Slide

56. Fake Content
    55
    Deep Video Portraits (SIGGRAPH 2018)
    

    View Slide

57. Detection-Generation Arms Race
    56
    Forgery
    Technique
    Detection
    Classifier
    Forgery
    Technique
    Detection
    Classifier
    If you know the forgery technique,
    detection (by machines) has advantage.
    

    View Slide

58. Plan for Talk
    1. What is AI?
    Definitions
    2. What should (and shouldn’t) we be afraid of?
    Harmful use of AI
    3. What can we learn from previous arms races?
    Evasive malware
    4. What (if anything) can we do?
    57
    

    View Slide

59. Trojan Horse Arms Race
    58
    Or do you think any Greek
    gift’s free of treachery? Is that
    Ulysses’s reputation? Either
    there are Greeks in hiding,
    concealed by the wood, or it’s
    been built as a machine to use
    against our walls, or spy on
    our homes, or fall on the city
    from above, or it hides some
    other trick: Trojans, don’t trust
    this horse. Whatever it is, I’m
    afraid of Greeks even those
    bearing gifts.’
    Virgil, The Aenid (Book II)
    

    View Slide

60. Evasive Malware
    Péter Ször (1970-2013)
    

    View Slide

61. Adversarial Examples before Deep Learning
    60
    

    View Slide

62. Labelled
    Training Data
    ML
    Algorithm
    Feature
    Extraction
    Vectors
    Deployment
    Malicious / Benign
    Operational Data
    Trained Classifier
    Training
    (supervised learning)
    Assumption: Training Data is Representative
    

    View Slide

63. Deployment
    Adversaries Don’t Cooperate
    Assumption: Training Data is Representative
    Training
    Poisoning
    

    View Slide

64. Adversaries Don’t Cooperate
    Assumption: Training Data is Representative
    Evading
    Deployment
    Training
    

    View Slide

65. Domain: PDF Malware Classifiers
    

    View Slide

66. PDF Malware Classifiers
    Random Forest
    Features
    Object counts,
    lengths,
    positions, …
    Manual Features
    PDFrate
    [ACSA 2012]
    

    View Slide

67. PDF Malware Classifiers
    Random Forest Random Forest
    Support Vector Machine
    Features
    Object counts,
    lengths,
    positions, …
    Object structural paths
    Very robust against “strongest
    conceivable mimicry attack”.
    Automated Features
    Manual Features
    PDFrate
    [ACSA 2012]
    Hidost16
    [JIS 2016]
    Hidost13
    [NDSS 2013]
    

    View Slide

68. Adversarial Examples across Domains
    67
    Domain Classifier Space “Reality” Space
    Trojan Wars
    Judgment of Trojans
    !(#) = “gift”
    Physical Reality
    !∗(#) = invading army
    Malware
    Malware Detector
    !(#) = “benign”
    Victim’s Execution
    !∗(#) = malicious behavior
    Image
    Classification,
    Detection
    DNN Classifier
    !(#) = )
    Human Perception
    !∗(#) = *
    Next
    Next 2 talks!
    

    View Slide

69. “Oracle” Definition
    68
    Given seed sample, !, !" is an adversarial example iff:
    # !" = % Class is % (for malware, %= “benign”)
    ℬ !′) = ℬ(! Behavior we care about is the same
    Malware: evasive variant preserves malicious
    behavior of seed, but is classified as benign
    No requirement that ! ~ !′ except through ℬ.
    

    View Slide

70. Finding Evasive Malware
    69
    Given seed sample, !, !" is an adversarial example iff:
    # !" = % Class is % (for malware, %= “benign”)
    ℬ !′) = ℬ(! Behavior we care about is the same
    Generic attack: heuristically explore input
    space for !′ that satisfies definition.
    

    View Slide

71. Variants
    Evolutionary Search
    Clone
    Benign PDFs
    Malicious PDF
    Mutation
    01011001101
    Variants
    Variants
    Select
    Variants
    ✓
    ✓
    ✗
    ✓
    Found
    Evasive?
    Benign
    Oracle
    Weilin Xu Yanjun Qi
    Fitness
    Selection
    Mutant
    Generation
    

    View Slide

72. Variants
    Generating Variants
    Clone
    Benign PDFs
    Malicious PDF
    Mutation
    01011001101
    Variants
    Variants
    Select
    Variants
    ✓
    ✓
    ✗
    ✓
    Found
    Evasive?
    Fitness
    Selection
    Mutant
    Generation
    

    View Slide

73. PDF Structure
    

    View Slide

74. Variants
    Generating Variants
    Clone
    Benign PDFs
    Malicious PDF
    Mutation
    01011001101
    Variants
    Variants
    Select
    Variants
    ✓
    ✓
    ✗
    ✓
    Found
    Evasive?
    Fitness
    Selection
    Mutant
    Generation
    

    View Slide

75. Variants
    Generating Variants
    Clone
    Benign PDFs
    Malicious PDF
    Mutation
    01011001101
    Variants
    Variants
    Select
    Variants
    ✓
    ✓
    ✗
    ✓
    Found
    Evasive?
    Found
    Evasive
    ?
    0
    /JavaScript
    eval(‘…’);
    /Root
    /Catalog
    /Pages
    Select random node
    Randomly transform: delete, insert, replace
    

    View Slide

76. Variants
    Generating Variants
    Clone
    Benign PDFs
    Malicious PDF
    Mutation
    01011001101
    Variants
    Variants
    Select
    Variants
    Found
    Evasive?
    Found
    Evasive
    ?
    Select random node
    Randomly transform: delete, insert, replace
    Nodes from
    Benign PDFs
    0
    /JavaScript
    eval(‘…’);
    /Root
    /Catalog
    /Pages
    128
    546
    7
    63
    128
    

    View Slide

77. Variants
    Selecting Promising Variants
    Clone
    Benign PDFs
    Malicious PDF
    Mutation
    01011001101
    Variants
    Variants
    Select
    Variants
    ✓
    ✓
    ✗
    ✓
    Found
    Evasive?
    Fitness
    Selection
    Mutant
    Generation
    

    View Slide

78. Variants
    Selecting Promising Variants
    Clone
    Benign PDFs
    Malicious PDF
    Mutation
    01011001101
    Variants
    Variants
    Select
    Variants
    ✓
    ✓
    ✗
    ✓
    Found
    Evasive?
    Fitness Function
    Candidate Variant
    !(#$%&'()
    , #'(&++
    )
    Score
    Malicious
    0
    /JavaScript
    eval(‘…’);
    /Root
    /Catalog
    /Pages
    128
    Oracle
    Target Classifier
    

    View Slide

79. Oracle: ℬ "′) = ℬ(" ?
    Execute candidate in
    vulnerable Adobe Reader in
    virtual environment
    Behavioral signature:
    malicious if signature matches
    https://github.com/cuckoosandbox
    Simulated network: INetSim
    Cuckoo
    HTTP_URL + HOST
    extracted from API traces
    

    View Slide

80. Fitness Function
    Assumes lost malicious behavior will not be
    recovered
    !itness '′ = *
    1 − classi!ier_score '3 if ℬ '′) = ℬ('
    −∞ otherwise
    

    View Slide

81. 0
    100
    200
    300
    400
    500
    0 100 200 300
    Seeds Evaded
    (out of 500)
    PDFRate
    Number of Mutations
    Hidost
    

    View Slide

82. 0
    100
    200
    300
    400
    500
    0 100 200 300
    Seeds Evaded
    (out of 500)
    PDFRate
    Number of Mutations
    Hidost
    Simple
    transformations
    often worked
    

    View Slide

83. 0
    100
    200
    300
    400
    500
    0 100 200 300
    Seeds Evaded
    (out of 500)
    PDFRate
    Number of Mutations
    Hidost
    (insert, /Root/Pages/Kids,
    3:/Root/Pages/Kids/4/Kids/5/)
    Works on 162/500 seeds
    

    View Slide

84. 0
    100
    200
    300
    400
    500
    0 100 200 300
    Seeds Evaded
    (out of 500)
    PDFRate
    Number of Mutations
    Hidost
    Some seeds
    required complex
    transformations
    

    View Slide

85. Malicious Label
    Threshold
    Original Malicious Seeds
    Evading
    PDFrate
    Classification Score
    Malware Seed (sorted by original score)
    Discovered Evasive Variants
    

    View Slide

86. Discovered Evasive Variants
    Malicious Label
    Threshold
    Original Malicious Seeds
    Adjust threshold?
    Charles Smutz, Angelos
    Stavrou. When a Tree Falls:
    Using Diversity in Ensemble
    Classifiers to Identify
    Evasion in Malware
    Detectors. NDSS 2016.
    Classification Score
    Malware Seed (sorted by original score)
    

    View Slide

87. Variants found with threshold = 0.25
    Variants found with threshold = 0.50
    Adjust threshold?
    Classification Score
    Malware Seed (sorted by original score)
    

    View Slide

88. Variants
    Hide the Classifier Score?
    Clone
    Benign PDFs
    Malicious PDF
    Mutation
    01011001101
    Variants
    Variants
    Select
    Variants
    ✓
    ✓
    ✗
    ✓
    Found
    Evasive?
    Fitness Function
    Candidate Variant
    !(#$%&'()
    , #'(&++
    )
    Score
    Malicious
    0
    /JavaScript
    eval(‘…’);
    /Root
    /Catalog
    /Pages
    128
    Oracle
    Target Classifier
    

    View Slide

89. Variants
    Binary Classifier Output is Enough
    Clone
    Benign PDFs
    Malicious PDF
    Mutation
    01011001101
    Variants
    Variants
    Select
    Variants
    ✓
    ✓
    ✗
    ✓
    Found
    Evasive?
    Fitness Function
    Candidate Variant
    !(#$%&'()
    , #'(&++
    )
    Score
    Malicious
    0
    /JavaScript
    eval(‘…’);
    /Root
    /Catalog
    /Pages
    128
    Oracle
    Target Classifier
    ACM CCS 2017
    

    View Slide

90. Labelled
    Training Data
    ML
    Algorithm
    Feature
    Extraction
    Vectors
    Deployment
    Malicious / Benign
    Operational Data
    Trained Classifier
    Training
    (supervised learning)
    Retrain Classifier
    

    View Slide

91. Labelled
    Training Data
    ML
    Algorithm
    Feature
    Extraction
    Vectors
    Training
    (supervised learning)
    Clone
    01011001
    101
    EvadeML
    Deployment
    

    View Slide

92. 0
    100
    200
    300
    400
    500
    0 200 400 600 800
    Seeds Evaded (out of 500)
    Generations
    Hidost16
    Original classifier:
    Takes 614 generations
    to evade all seeds
    

    View Slide

93. 0
    100
    200
    300
    400
    500
    0 200 400 600 800
    HidostR1
    Seeds Evaded (out of 500)
    Generations
    Hidost16
    

    View Slide

94. 0
    100
    200
    300
    400
    500
    0 200 400 600 800
    HidostR1
    Seeds Evaded (out of 500)
    Generations
    Hidost16
    

    View Slide

95. 0
    100
    200
    300
    400
    500
    0 200 400 600 800
    HidostR1
    HidostR2
    Seeds Evaded (out of 500)
    Generations
    Hidost16
    

    View Slide

96. 0
    100
    200
    300
    400
    500
    0 200 400 600 800
    HidostR1
    HidostR2
    Seeds Evaded (out of 500)
    Generations
    Hidost16
    

    View Slide

97. 0
    100
    200
    300
    400
    500
    0 200 400 600 800
    Hidost16
    Genome Contagio Benign
    Hidost16 0.00 0.00
    HidostR1 0.78 0.30
    HidostR2 0.85 0.53
    False Positive Rates
    HidostR1
    Seeds Evaded (out of 500)
    Generations
    HidostR2
    

    View Slide

98. 97
    Only 8/6987 robust features (Hidost)
    Robust classifier
    High false positives
    /Names
    /Names /JavaScript
    /Names /JavaScript /Names
    /Names /JavaScript /JS
    /OpenAction
    /OpenAction /JS
    /OpenAction /S
    /Pages
    

    View Slide

99. AI Arms Races
    AI-based defenses are at-best temporary
    98
    “Artificial Intelligence”
    means making
    computers do things
    their programmers don’t
    understand well enough
    to program explicitly.
    Can be effective against
    current adversaries
    Asymmetries benefit
    attackers
    Motivated adversary with
    any access to defense can
    learn to thwart it
    

    View Slide

100. AI Arms Races
     AI-based defenses are at-best temporary
     99
     “Artificial Intelligence”
     means making
     computers do things
     their programmers don’t
     understand well enough
     to program explicitly.
     Can be effective against
     current adversaries
     Asymmetries benefit
     attackers
     Motivated adversary with
     any access to defense can
     learn to thwart it
     Can only work reliably, if we
     are using robust features
     that are strong signals – but
     then, don’t need AI!
     

     View Slide

101. Real Solution
     to Malicious
     PDFs
     100
     Better Software
     

     View Slide

102. Plan for Talk
     1. What is AI?
     Definitions
     2. What should (and shouldn’t) we be afraid of?
     Harmful use of AI
     3. What can we learn from previous arms races?
     Evasive malware
     4. What (if anything) can we do?
     101
     

     View Slide

103. 102
     https://maliciousaireport.com/
     

     View Slide

104. 103
     https://maliciousaireport.com/
     

     View Slide

105. AI-Based Attacks
     Low-cost, low-risk automation of attacks
     New types of attacks
     Humans will be easily fooled
     104
     

     View Slide

106. 105
     In defense
     of Luddites?
     

     View Slide

107. 106
     In defense
     of Luddites?
     

     View Slide

108. View Slide

109. “Made by Human” Labels
     Certified:
     Human
     Made
     

     View Slide

110. Google’s
     Duplex
     Demo
     

     View Slide

111. 110
     @_youhadonejob1
     

     View Slide

112. David Evans
     University of Virginia
     evans@virginia.edu
     EvadeML.org
     

     View Slide