$30 off During Our Annual Pro Sale. View Details »

Han Xiao - Adversarial and and Robust Machine Learning

Han Xiao - Adversarial and and Robust Machine Learning

Talk by Han Xiao (http://home.in.tum.de/~xiaoh/) at the first meetup of Munich Data Geeks (http://www.meetup.com/Munich-Datageeks)
Date: 2013.07.02

MunichDataGeeks

July 02, 2013
Tweet

More Decks by MunichDataGeeks

Other Decks in Research

Transcript

  1. Technische Universit¨
    at M¨
    unchen
    Adversarial and Robust Machine
    Learning
    Han Xiao
    Department of Informatics
    Technische Universit¨
    at M¨
    unchen
    [email protected]
    July 2, 2013

    View Slide

  2. Outline Examples Adversarial and Robust Learning Other Work Take Home Messages
    Outline
    1 Examples
    2 Adversarial and Robust Learning
    Attack
    Defense
    3 Other Work
    4 Take Home Messages

    View Slide

  3. Outline Examples Adversarial and Robust Learning Other Work Take Home Messages
    Outline
    1 Examples
    2 Adversarial and Robust Learning
    Attack
    Defense
    3 Other Work
    4 Take Home Messages

    View Slide

  4. Outline Examples Adversarial and Robust Learning Other Work Take Home Messages
    Spam Filtering
    Three steps of machine learning: representation→ optimization → evaluation

    View Slide

  5. Outline Examples Adversarial and Robust Learning Other Work Take Home Messages
    Spam Disguise
    Adding noise to the junk mail

    View Slide

  6. Outline Examples Adversarial and Robust Learning Other Work Take Home Messages
    Spam Disguise
    Introducing feature noise

    View Slide

  7. Outline Examples Adversarial and Robust Learning Other Work Take Home Messages
    Exploratory Attack: Spam Disguise in Practice
    Question
    Given a spam, how do you disguise it to evade from being
    detected?
    How?
    • Create [email protected]
    • Generate disguised spams
    and send to
    [email protected]
    • Select the most desired
    modification from the inbox.

    View Slide

  8. Outline Examples Adversarial and Robust Learning Other Work Take Home Messages
    Exploratory Attack: Spam Disguise in Practice
    Question
    Given a spam, how do you disguise it to evade from being
    detected?
    How?
    • Create [email protected]
    • Generate disguised spams
    and send to
    [email protected]
    • Select the most desired
    modification from the inbox.
    Questions:
    • What is the “most desired”
    mail?
    • How to generate efficiently?

    View Slide

  9. Outline Examples Adversarial and Robust Learning Other Work Take Home Messages
    Training Data from Online Services
    Users may vary in expertise, dedication and motivation

    View Slide

  10. Outline Examples Adversarial and Robust Learning Other Work Take Home Messages
    Training Data from Online Services
    Users may vary in expertise, dedication and motivation

    View Slide

  11. Outline Examples Adversarial and Robust Learning Other Work Take Home Messages
    Training Data from Online Services
    Users may vary in expertise, dedication and motivation
    What if haters dominate?
    Are they going to subvert the learning algorithm?
    How to recover the unbiased labels/ratings?

    View Slide

  12. Outline Examples Adversarial and Robust Learning Other Work Take Home Messages
    Causative Attack: Poisoning the Spam Filter
    Introducing label noise to training data

    View Slide

  13. Outline Examples Adversarial and Robust Learning Other Work Take Home Messages
    Outline
    1 Examples
    2 Adversarial and Robust Learning
    Attack
    Defense
    3 Other Work
    4 Take Home Messages

    View Slide

  14. Outline Examples Adversarial and Robust Learning Other Work Take Home Messages
    Modeling the Adversary
    Adversarial settings
    The adversary manipulates instances to mislead the decision of the
    classifier in their favor.
    Exploratory attack
    • in the test phrase;
    • disguise a malicious instance to evade from being detected;
    • e.g. disguise a spam, mutate a virus.

    View Slide

  15. Outline Examples Adversarial and Robust Learning Other Work Take Home Messages
    Modeling the Adversary
    Adversarial settings
    The adversary manipulates instances to mislead the decision of the
    classifier in their favor.
    Exploratory attack
    • in the test phrase;
    • disguise a malicious instance to evade from being detected;
    • e.g. disguise a spam, mutate a virus.
    Causative attack
    • in the training phrase;
    • manipulate the training set to subvert the learning process;
    • e.g. poisoning the spam filter, unfair rating on SNS.

    View Slide

  16. Outline Examples Adversarial and Robust Learning Other Work Take Home Messages
    Why Adversarial Learning is Interesting?
    (1) Because social network and crowdsourcing platform (e.g.
    Amazon mechanical turk) are popular.

    View Slide

  17. Outline Examples Adversarial and Robust Learning Other Work Take Home Messages
    Why Adversarial Learning is Interesting?
    (1) Because social network and crowdsourcing platform (e.g.
    Amazon mechanical turk) are popular.
    (2) Know your enemies and
    yourself, you will not be
    imperiled in a hundred battles.
    –Sun Tzu, The Art of War, 544 BC

    View Slide

  18. Outline Examples Adversarial and Robust Learning Other Work Take Home Messages
    Why Adversarial Learning is Interesting?
    (1) Because social network and crowdsourcing platform (e.g.
    Amazon mechanical turk) are popular.
    (2) Know your enemies and
    yourself, you will not be
    imperiled in a hundred battles.
    –Sun Tzu, The Art of War, 544 BC
    Secure learning
    The ultimate goal is to develop robust learning algorithms, which
    are resilient to the adversarial noise.

    View Slide

  19. Outline Examples Adversarial and Robust Learning Other Work Take Home Messages
    Research Directions
    Optimal attack strategies
    knowing the worst-case performance
    • Exploratory Attack
    • Causative Attack
    Robust learning algorithms
    improving the worst-case performance
    • Learning from crowds

    View Slide

  20. Outline Examples Adversarial and Robust Learning Other Work Take Home Messages
    Binary Classification
    Formalize the problem in math
    Term Notation Real world
    Input space X ⊆ RD All possible mails
    Response space Y := {−1, 1} All possible labels
    Instance x ∈ X, i.e. a D-dimensional vector A mail

    View Slide

  21. Outline Examples Adversarial and Robust Learning Other Work Take Home Messages
    Binary Classification
    Formalize the problem in math
    Term Notation Real world
    Input space X ⊆ RD All possible mails
    Response space Y := {−1, 1} All possible labels
    Instance x ∈ X, i.e. a D-dimensional vector A mail
    Hypothesis space H All possible filters
    Classifier f : X → Y, f ∈ H A filter

    View Slide

  22. Outline Examples Adversarial and Robust Learning Other Work Take Home Messages
    Binary Classification
    Formalize the problem in math
    Term Notation Real world
    Input space X ⊆ RD All possible mails
    Response space Y := {−1, 1} All possible labels
    Instance x ∈ X, i.e. a D-dimensional vector A mail
    Hypothesis space H All possible filters
    Classifier f : X → Y, f ∈ H A filter
    Positive set X+ := {x ∈ X | f(x) = +1} All possible spams
    Negative set X− := {x ∈ X | f(x) = −1} All possible legit mails

    View Slide

  23. Outline Examples Adversarial and Robust Learning Other Work Take Home Messages
    Binary Classification
    Formalize the problem in math
    Term Notation Real world
    Input space X ⊆ RD All possible mails
    Response space Y := {−1, 1} All possible labels
    Instance x ∈ X, i.e. a D-dimensional vector A mail
    Hypothesis space H All possible filters
    Classifier f : X → Y, f ∈ H A filter
    Positive set X+ := {x ∈ X | f(x) = +1} All possible spams
    Negative set X− := {x ∈ X | f(x) = −1} All possible legit mails
    Loss function V : Y × Y → R0+
    Cost of misclassification

    View Slide

  24. Outline Examples Adversarial and Robust Learning Other Work Take Home Messages
    Training the Classifier
    Solving an optimization problem
    Classification
    Given a training set S := {(xi, yi
    ) | xi
    ∈ X, yi
    ∈ Y}n
    i=1
    . Find the
    classifier fS
    ∈ H that performs best on some test set T.
    Solving an optimization problem:
    fS
    := arg min
    f
    γ
    n
    i=1
    V (yi, f(xi
    )) + f 2
    H
    ,
    where γ ∈ R0+
    is a fixed parameter for quantifying the trade off.

    View Slide

  25. Outline Examples Adversarial and Robust Learning Other Work Take Home Messages
    Spam Disguise
    Problem formulation
    Disguise a spam from being detected by a filter. Be efficient.
    Problem Formulation
    Given
    • a trained classifier f;
    • a positive (malicious) instance xA ∈ X+;
    • a random negative (benign) instance x− ∈ X−.
    Find an instance x∗ ∈ X−
    f
    such that
    • x∗ should be similar to xA;
    • issuing as few queries to f as possible.
    Han, Thomas, Claudia. Evasion Attack of Multi-Class Linear Classifiers PAKDD 2012 16 of 46

    View Slide

  26. Outline Examples Adversarial and Robust Learning Other Work Take Home Messages
    Assumptions
    Assumption Real world
    Know the dimension of X Know how many features
    Attack a fixed f Spam filter is not updated
    Observe f(x) by a membership query Observe the label of a sent mail
    Design a cost function Know the cost of misclassification
    Han, Thomas, Claudia. Evasion Attack of Multi-Class Linear Classifiers PAKDD 2012 17 of 46

    View Slide

  27. Outline Examples Adversarial and Robust Learning Other Work Take Home Messages
    Exploratory Attack as ℓp
    -norm Minimization
    Exploratory Attack
    Given xA, f, and a cost function g : X × X → R0+
    , solve
    min
    x
    g(x, xA) subject to x ∈ X−,
    where X− is specified by the membership oracle f.
    For example, g(x, xA) := x − xA
    ℓ1
    Han, Thomas, Claudia. Evasion Attack of Multi-Class Linear Classifiers PAKDD 2012 18 of 46

    View Slide

  28. Outline Examples Adversarial and Robust Learning Other Work Take Home Messages
    Illustration of the Problem
    g(x) := x − xA
    ℓ1
    X+
    f
    X−
    f
    xA
    x−
    Han, Thomas, Claudia. Evasion Attack of Multi-Class Linear Classifiers PAKDD 2012 19 of 46

    View Slide

  29. Outline Examples Adversarial and Robust Learning Other Work Take Home Messages
    Illustration of the Problem
    g(x) := x − xA
    ℓ1
    X+
    f
    X−
    f
    xA
    x−

    x∗
    Han, Thomas, Claudia. Evasion Attack of Multi-Class Linear Classifiers PAKDD 2012 19 of 46

    View Slide

  30. Outline Examples Adversarial and Robust Learning Other Work Take Home Messages
    Face Camouflage
    Considering a suspect tries to disguise herself as innocent.
    Han, Thomas, Claudia. Evasion Attack of Multi-Class Linear Classifiers PAKDD 2012 20 of 46

    View Slide

  31. Outline Examples Adversarial and Robust Learning Other Work Take Home Messages
    Label Flips Attack
    Given a training set, the adversary contaminates the training data
    through flipping labels.
    Han, Huang, Claudia. Adversarial Label Flips Attack on Support Vector Machines ECAI 2012 21 of 46

    View Slide

  32. Outline Examples Adversarial and Robust Learning Other Work Take Home Messages
    Adversarial Label Flips Attack
    Adversarial Label Flip Attack
    Find a combination of label flips under a given budget so that a
    classifier trained on such data will have maximal classification error
    on some test data.
    Han, Huang, Claudia. Adversarial Label Flips Attack on Support Vector Machines ECAI 2012 22 of 46

    View Slide

  33. Outline Examples Adversarial and Robust Learning Other Work Take Home Messages
    Adversarial Label Flips Attack
    Adversarial Label Flip Attack
    Find a combination of label flips under a given budget so that a
    classifier trained on such data will have maximal classification error
    on some test data.
    Training set: S := {(xi, yi
    ) | xi
    ∈ X, yi
    ∈ Y}n
    i=1
    ;
    Indicator: zi
    ∈ {0: normal, 1: flipped}, i = 1, . . . , n;
    Flipping cost: ci
    ∈ R0+, i = 1, . . . , n;
    Han, Huang, Claudia. Adversarial Label Flips Attack on Support Vector Machines ECAI 2012 22 of 46

    View Slide

  34. Outline Examples Adversarial and Robust Learning Other Work Take Home Messages
    Adversarial Label Flips Attack
    Adversarial Label Flip Attack
    Find a combination of label flips under a given budget so that a
    classifier trained on such data will have maximal classification error
    on some test data.
    Training set: S := {(xi, yi
    ) | xi
    ∈ X, yi
    ∈ Y}n
    i=1
    ;
    Indicator: zi
    ∈ {0: normal, 1: flipped}, i = 1, . . . , n;
    Flipping cost: ci
    ∈ R0+, i = 1, . . . , n;
    Tainted label: y′
    i
    := yi
    (1 − 2zi
    );
    Tainted training set: S′ := {(xi, y′
    i
    )}.
    Han, Huang, Claudia. Adversarial Label Flips Attack on Support Vector Machines ECAI 2012 22 of 46

    View Slide

  35. Outline Examples Adversarial and Robust Learning Other Work Take Home Messages
    A Bilevel Formulation
    Finding the optimal label flips
    Given S, a test set T and a budget C, solve
    max
    z
    (x,y)∈T
    V (y, fS′
    (x)) ,
    s.t. fS′
    ∈ arg min
    f
    γ
    n
    i=1
    V y′
    i
    , f(xi
    ) + f 2
    H
    ,
    n
    i=1
    cizi
    ≤ C, zi
    ∈ {0, 1}.
    Han, Huang, Claudia. Adversarial Label Flips Attack on Support Vector Machines ECAI 2012 23 of 46

    View Slide

  36. Outline Examples Adversarial and Robust Learning Other Work Take Home Messages
    A Bilevel Formulation
    Finding the optimal label flips
    Given S, a test set T and a budget C, solve
    max
    z
    (x,y)∈T
    V (y, fS′
    (x)) ,
    s.t. fS′
    ∈ arg min
    f
    γ
    n
    i=1
    V y′
    i
    , f(xi
    ) + f 2
    H
    ,
    n
    i=1
    cizi
    ≤ C, zi
    ∈ {0, 1}.
    Defender
    Han, Huang, Claudia. Adversarial Label Flips Attack on Support Vector Machines ECAI 2012

    View Slide

  37. Outline Examples Adversarial and Robust Learning Other Work Take Home Messages
    A Bilevel Formulation
    Finding the optimal label flips
    Given S, a test set T and a budget C, solve
    max
    z
    (x,y)∈T
    V (y, fS′
    (x)) ,
    s.t. fS′
    ∈ arg min
    f
    γ
    n
    i=1
    V y′
    i
    , f(xi
    ) + f 2
    H
    ,
    n
    i=1
    cizi
    ≤ C, zi
    ∈ {0, 1}.
    Defender
    Attacker
    Han, Huang, Claudia. Adversarial Label Flips Attack on Support Vector Machines ECAI 2012 23 of 46

    View Slide

  38. Outline Examples Adversarial and Robust Learning Other Work Take Home Messages
    Label Flips Attack Against SVM
    Train: 100, flip:20, test 800
    (a) Synthetic data
    Linear pattern
    (b) No Flips
    1.8%
    Linear SVM
    (c) Random
    1.9%
    (d) Nearst
    6.9%
    (e) Furthest
    9.5%
    (f) ALFA
    21.8%
    3.2%
    RBF−SVM
    4.0% 3.5% 26.5% 32.4%
    Parabolic pattern
    23.5%
    Linear SVM
    28.8% 29.2% 40.5% 48.0%
    5.1%
    RBF−SVM
    9.4% 10.1% 12.9% 40.8%
    Han, Huang, Claudia. Adversarial Label Flips Attack on Support Vector Machines ECAI 2012 24 of 46

    View Slide

  39. Outline Examples Adversarial and Robust Learning Other Work Take Home Messages
    An Endless Game between Adversary and Defender
    Escher. Drawing Hands 1948 25 of 46

    View Slide

  40. Outline Examples Adversarial and Robust Learning Other Work Take Home Messages
    Detecting Exploratory Attack
    Convergence pattern
    50
    100
    150
    200
    250
    300
    350
    400
    450
    500
    550
    Original
    Disguised
    Initial
    Time
    Start
    Dim 1
    Dim 2
    malcious
    benign
    Han, Thomas, Claudia. Evasion Attack of Multi-Class Linear Classifiers PAKDD 2012 26 of 46

    View Slide

  41. Outline Examples Adversarial and Robust Learning Other Work Take Home Messages
    Subjective opinions from crowds
    Learning objective assessment from subjective opinions
    Han, Huang, Claudia. Learning from Multiple Observers with Unknown Expertise PAKDD 2013
    Meyyar. Leveraging the Wisdom of Crowds for Reputation Management Master’s thesis 27 of 46

    View Slide

  42. Outline Examples Adversarial and Robust Learning Other Work Take Home Messages
    Subjective opinions from crowds
    Learning objective assessment from subjective opinions
    Han, Huang, Claudia. Learning from Multiple Observers with Unknown Expertise PAKDD 2013
    Meyyar. Leveraging the Wisdom of Crowds for Reputation Management Master’s thesis 27 of 46

    View Slide

  43. Outline Examples Adversarial and Robust Learning Other Work Take Home Messages
    Subjective opinions from crowds
    Learning objective assessment from subjective opinions
    Fair rating (Groundtruth)?
    Han, Huang, Claudia. Learning from Multiple Observers with Unknown Expertise PAKDD 2013
    Meyyar. Leveraging the Wisdom of Crowds for Reputation Management Master’s thesis 27 of 46

    View Slide

  44. Outline Examples Adversarial and Robust Learning Other Work Take Home Messages
    Subjective opinions from crowds
    Learning objective assessment from subjective opinions
    Fair rating (Groundtruth)?
    What’s wrong with “majority vote” and “take average”?
    They completely ignore the individual expertise and may fail in the
    settings with non-Gaussian or adversarial noise!
    Han, Huang, Claudia. Learning from Multiple Observers with Unknown Expertise PAKDD 2013
    Meyyar. Leveraging the Wisdom of Crowds for Reputation Management Master’s thesis 27 of 46

    View Slide

  45. Outline Examples Adversarial and Robust Learning Other Work Take Home Messages
    Unreliable readings from sensors
    237◦
    229◦
    240◦
    236◦
    −13◦
    Groundtruth?
    Questions
    1. How to integrate readings from multiple sensors?
    2. How accurate is each sensor?
    Han, Huang, Claudia. Learning from Multiple Observers with Unknown Expertise PAKDD 2013
    Meyyar. Leveraging the Wisdom of Crowds for Reputation Management Master’s thesis 28 of 46

    View Slide

  46. Outline Examples Adversarial and Robust Learning Other Work Take Home Messages
    Learning from Multiple Observers
    Problems
    • How to learn a regression function to predict the ground truth
    precluding the prior knowledge of observers?
    • How to estimate the expertise of each observer without
    knowing the ground truth?
    Han, Huang, Claudia. Learning from Multiple Observers with Unknown Expertise PAKDD 2013
    Meyyar. Leveraging the Wisdom of Crowds for Reputation Management Master’s thesis 29 of 46

    View Slide

  47. Outline Examples Adversarial and Robust Learning Other Work Take Home Messages
    Intuition behind
    Leveraging the neighborhood information
    Instance space X
    x2
    x1
    x3
    Han, Huang, Claudia. Learning from Multiple Observers with Unknown Expertise PAKDD 2013
    Meyyar. Leveraging the Wisdom of Crowds for Reputation Management Master’s thesis 30 of 46

    View Slide

  48. Outline Examples Adversarial and Robust Learning Other Work Take Home Messages
    Intuition behind
    Leveraging the neighborhood information
    Instance space X
    x2
    x1
    x3
    f(x1
    )
    f(x2
    )
    f(x3
    )
    Han, Huang, Claudia. Learning from Multiple Observers with Unknown Expertise PAKDD 2013
    Meyyar. Leveraging the Wisdom of Crowds for Reputation Management Master’s thesis 30 of 46

    View Slide

  49. Outline Examples Adversarial and Robust Learning Other Work Take Home Messages
    Intuition behind
    Leveraging the neighborhood information
    Instance space X
    x2
    x1
    x3
    f(x1
    )
    f(x2
    )
    f(x3
    )
    Groundtruth space Z
    z1 z2
    z3
    (Latent)
    Han, Huang, Claudia. Learning from Multiple Observers with Unknown Expertise PAKDD 2013
    Meyyar. Leveraging the Wisdom of Crowds for Reputation Management Master’s thesis 30 of 46

    View Slide

  50. Outline Examples Adversarial and Robust Learning Other Work Take Home Messages
    Intuition behind
    Leveraging the neighborhood information
    Instance space X
    x2
    x1
    x3
    f(x1
    )
    f(x2
    )
    f(x3
    )
    Groundtruth space Z
    z1 z2
    z3
    (Latent) g1
    (z3
    )
    g1
    (z2
    )
    gM
    (z3
    )
    gM
    (z1
    )
    gM
    (z2
    )
    Han, Huang, Claudia. Learning from Multiple Observers with Unknown Expertise PAKDD 2013
    Meyyar. Leveraging the Wisdom of Crowds for Reputation Management Master’s thesis 30 of 46

    View Slide

  51. Outline Examples Adversarial and Robust Learning Other Work Take Home Messages
    Intuition behind
    Leveraging the neighborhood information
    Instance space X
    x2
    x1
    x3
    f(x1
    )
    f(x2
    )
    f(x3
    )
    Groundtruth space Z
    z1 z2
    z3
    (Latent) g1
    (z3
    )
    g1
    (z2
    )
    gM
    (z3
    )
    gM
    (z1
    )
    gM
    (z2
    )
    y1,1 y2,1
    y3,1
    Response space Y
    1st Observer
    y1,M y2,M
    y3,M
    Response space Y
    Mth Observer
    Han, Huang, Claudia. Learning from Multiple Observers with Unknown Expertise PAKDD 2013
    Meyyar. Leveraging the Wisdom of Crowds for Reputation Management Master’s thesis 30 of 46

    View Slide

  52. Outline Examples Adversarial and Robust Learning Other Work Take Home Messages
    Nonparametric probabilistic model
    xn yn,m
    zn
    M
    N
    p(Y, Z, X) = p(Z | X)p(Y | Z, X)p(X).
    Gaussian process: a less-parametric approach for modeling a
    function.
    Han, Huang, Claudia. Learning from Multiple Observers with Unknown Expertise PAKDD 2013
    Meyyar. Leveraging the Wisdom of Crowds for Reputation Management Master’s thesis 31 of 46

    View Slide

  53. Outline Examples Adversarial and Robust Learning Other Work Take Home Messages
    Nonparametric probabilistic model
    xn yn,m
    zn
    M
    N
    p(Y, Z, X) = p(Z | X)p(Y | Z, X)p(X).
    Gaussian process: a less-parametric approach for modeling a
    function.
    Maximizing the posterior, which gives
    log p(Z, Θ | Y, X) = log p(Y | Z, X, Θ)+log p(Z | X, Θ)+constant.
    Deriving the gradient w.r.t. z, κ, φ, η, respectively.
    Feed the gradients to L-BFGS method for finding the stationary
    point.
    Han, Huang, Claudia. Learning from Multiple Observers with Unknown Expertise PAKDD 2013
    Meyyar. Leveraging the Wisdom of Crowds for Reputation Management Master’s thesis 31 of 46

    View Slide

  54. Outline Examples Adversarial and Robust Learning Other Work Take Home Messages
    1-D example
    Groundtruth function: f(t) = 10 sin(6t) sin( t
    2
    ),
    0 1 2 3 4 5 6
    0
    0.5
    1
    Han, Huang, Claudia. Learning from Multiple Observers with Unknown Expertise PAKDD 2013
    Meyyar. Leveraging the Wisdom of Crowds for Reputation Management Master’s thesis 32 of 46

    View Slide

  55. Outline Examples Adversarial and Robust Learning Other Work Take Home Messages
    1-D example
    Groundtruth function: f(t) = 10 sin(6t) sin( t
    2
    ),
    Randomly sample responses at t ∈ [0, 6] from four sensors.
    0 1 2 3 4 5 6
    0
    0.5
    1
    0 0.5 1
    0
    0.5
    1
    Ground truth
    Ob.1 resp.
    Han, Huang, Claudia. Learning from Multiple Observers with Unknown Expertise PAKDD 2013
    Meyyar. Leveraging the Wisdom of Crowds for Reputation Management Master’s thesis 32 of 46

    View Slide

  56. Outline Examples Adversarial and Robust Learning Other Work Take Home Messages
    1-D example
    Groundtruth function: f(t) = 10 sin(6t) sin( t
    2
    ),
    Randomly sample responses at t ∈ [0, 6] from four sensors.
    0 1 2 3 4 5 6
    0
    0.5
    1
    0 0.5 1
    0
    0.5
    1
    Ground truth
    Ob.1 resp.
    0 0.5 1
    0
    0.5
    1
    Ground truth
    Ob.2 resp.
    Han, Huang, Claudia. Learning from Multiple Observers with Unknown Expertise PAKDD 2013
    Meyyar. Leveraging the Wisdom of Crowds for Reputation Management Master’s thesis 32 of 46

    View Slide

  57. Outline Examples Adversarial and Robust Learning Other Work Take Home Messages
    1-D example
    Groundtruth function: f(t) = 10 sin(6t) sin( t
    2
    ),
    Randomly sample responses at t ∈ [0, 6] from four sensors.
    0 1 2 3 4 5 6
    0
    0.5
    1
    0 0.5 1
    0
    0.5
    1
    Ground truth
    Ob.1 resp.
    0 0.5 1
    0
    0.5
    1
    Ground truth
    Ob.2 resp.
    0 0.5 1
    0
    0.5
    1
    Ground truth
    Ob.3 resp.
    Han, Huang, Claudia. Learning from Multiple Observers with Unknown Expertise PAKDD 2013
    Meyyar. Leveraging the Wisdom of Crowds for Reputation Management Master’s thesis 32 of 46

    View Slide

  58. Outline Examples Adversarial and Robust Learning Other Work Take Home Messages
    1-D example
    Groundtruth function: f(t) = 10 sin(6t) sin( t
    2
    ),
    Randomly sample responses at t ∈ [0, 6] from four sensors.
    0 1 2 3 4 5 6
    0
    0.5
    1
    0 0.5 1
    0
    0.5
    1
    Ground truth
    Ob.1 resp.
    0 0.5 1
    0
    0.5
    1
    Ground truth
    Ob.2 resp.
    0 0.5 1
    0
    0.5
    1
    Ground truth
    Ob.3 resp.
    0 0.5 1
    0
    0.5
    1
    Ground truth
    Ob.4 resp.
    Han, Huang, Claudia. Learning from Multiple Observers with Unknown Expertise PAKDD 2013
    Meyyar. Leveraging the Wisdom of Crowds for Reputation Management Master’s thesis 32 of 46

    View Slide

  59. Outline Examples Adversarial and Robust Learning Other Work Take Home Messages
    1-D example
    Groundtruth function: f(t) = 10 sin(6t) sin( t
    2
    ),
    Randomly sample responses at t ∈ [0, 6] from four sensors.
    0 1 2 3 4 5 6
    0
    0.5
    1
    0 0.5 1
    0
    0.5
    1
    Ground truth
    Ob.1 resp.
    0 0.5 1
    0
    0.5
    1
    Ground truth
    Ob.2 resp.
    0 0.5 1
    0
    0.5
    1
    Ground truth
    Ob.3 resp.
    0 0.5 1
    0
    0.5
    1
    Ground truth
    Ob.4 resp.
    Han, Huang, Claudia. Learning from Multiple Observers with Unknown Expertise PAKDD 2013
    Meyyar. Leveraging the Wisdom of Crowds for Reputation Management Master’s thesis 32 of 46

    View Slide

  60. Outline Examples Adversarial and Robust Learning Other Work Take Home Messages
    1-D example
    What do we know?
    Only the readings from each sensor
    0 1 2 3 4 5 6
    0
    0.5
    1
    Han, Huang, Claudia. Learning from Multiple Observers with Unknown Expertise PAKDD 2013
    Meyyar. Leveraging the Wisdom of Crowds for Reputation Management Master’s thesis 33 of 46

    View Slide

  61. Outline Examples Adversarial and Robust Learning Other Work Take Home Messages
    1-D example
    What do we want to know?
    1. Groundtruth function, i.e. f(t).
    2. Response function of each sensor.
    0 1 2 3 4 5 6
    0
    0.5
    1
    0 0.5 1
    0
    0.5
    1
    Ground truth
    Ob.1 resp.
    0 0.5 1
    0
    0.5
    1
    Ground truth
    Ob.2 resp.
    0 0.5 1
    0
    0.5
    1
    Ground truth
    Ob.4 resp.
    0 0.5 1
    0
    0.5
    1
    Ground truth
    Ob.3 resp.
    Han, Huang, Claudia. Learning from Multiple Observers with Unknown Expertise PAKDD 2013
    Meyyar. Leveraging the Wisdom of Crowds for Reputation Management Master’s thesis 33 of 46

    View Slide

  62. Outline Examples Adversarial and Robust Learning Other Work Take Home Messages
    Synthetic data set
    Recover f(t) = 10 sin(6t) sin( t
    2
    ) and g1
    , g2
    , g3
    , g4
    .
    0 1 2 3 4 5 6
    0
    0.5
    1
    a
    0 0.5 1
    0
    0.5
    1
    Ob.1 resp.
    (a)
    0 0.5 1
    0
    0.5
    1
    Ob.2 resp.
    0 0.5 1
    0
    0.5
    1
    Ob.3 resp.
    Ground truth
    0 0.5 1
    0
    0.5
    1
    Ground truth
    Ob.4 resp.
    Ground truth Ob.1 Ob.2 Ob.3 Ob.4
    Han, Huang, Claudia. Learning from Multiple Observers with Unknown Expertise PAKDD 2013
    Meyyar. Leveraging the Wisdom of Crowds for Reputation Management Master’s thesis 34 of 46

    View Slide

  63. Outline Examples Adversarial and Robust Learning Other Work Take Home Messages
    Synthetic data set
    Recover f(t) = 10 sin(6t) sin( t
    2
    ) and g1
    , g2
    , g3
    , g4
    .
    0 1 2 3 4 5 6
    0
    0.5
    1
    a
    0 0.5 1
    0
    0.5
    1
    Ob.1 resp.
    (a)
    0 0.5 1
    0
    0.5
    1
    Ob.2 resp.
    0 0.5 1
    0
    0.5
    1
    Ob.3 resp.
    Ground truth
    0 0.5 1
    0
    0.5
    1
    Ground truth
    Ob.4 resp.
    Ground truth Ob.1 Ob.2 Ob.3 Ob.4
    0 2 4 6
    MANE:0.38, PCC:0.00
    0 2 4 6
    0
    0.5
    1
    MANE:0.29, PCC:0.50
    0 2 4 6
    0
    0.5
    1
    MANE:0.13, PCC:0.73
    (d) LOB
    (b) SVR−AVG (c) GPR−AVG
    Han, Huang, Claudia. Learning from Multiple Observers with Unknown Expertise PAKDD 2013
    Meyyar. Leveraging the Wisdom of Crowds for Reputation Management Master’s thesis 34 of 46

    View Slide

  64. Outline Examples Adversarial and Robust Learning Other Work Take Home Messages
    Synthetic data set
    Recover f(t) = 10 sin(6t) sin( t
    2
    ) and g1
    , g2
    , g3
    , g4
    .
    0 1 2 3 4 5 6
    0
    0.5
    1
    a
    0 0.5 1
    0
    0.5
    1
    Ob.1 resp.
    (a)
    0 0.5 1
    0
    0.5
    1
    Ob.2 resp.
    0 0.5 1
    0
    0.5
    1
    Ob.3 resp.
    Ground truth
    0 0.5 1
    0
    0.5
    1
    Ground truth
    Ob.4 resp.
    Ground truth Ob.1 Ob.2 Ob.3 Ob.4
    0 1 2 3 4 5 6
    0
    0.5
    1
    MANE:0.09, PCC:0.89
    (e)
    0 0.5 1
    0
    0.5
    1
    Ob.1 resp.
    0 0.5 1
    0
    0.5
    1
    Ob.2 resp.
    0 0.5 1
    0
    0.5
    1
    Ground truth
    Ob.4 resp.
    0 0.5 1
    0
    0.5
    1
    Ground truth
    Ob.3 resp.
    NLOB
    Han, Huang, Claudia. Learning from Multiple Observers with Unknown Expertise PAKDD 2013
    Meyyar. Leveraging the Wisdom of Crowds for Reputation Management Master’s thesis 34 of 46

    View Slide

  65. Outline Examples Adversarial and Robust Learning Other Work Take Home Messages
    TUM1002 photo rating data set
    Contributed by Huang http://ml.sec.in.tum.de/opars
    Huang, Han, Claudia. OPARS: Objective Photo Aesthetics Ranking System (demo paper). ECIR 2013
    Han, Huang, Claudia. Learning from Multiple Observers with Unknown Expertise PAKDD 2013
    Meyyar. Leveraging the Wisdom of Crowds for Reputation Management Master’s thesis 35 of 46

    View Slide

  66. Outline Examples Adversarial and Robust Learning Other Work Take Home Messages
    Sparse rating matrix from 34 users
    50 100 150 200 250 300 350 400
    5
    10
    15
    20
    25
    30
    −1 0 1 2 3 4 5
    Impressive
    Poor
    Missing Value
    Photos
    Missing
    Ratings
    Users
    Huang, Han, Claudia. OPARS: Objective Photo Aesthetics Ranking System (demo paper). ECIR 2013
    Han, Huang, Claudia. Learning from Multiple Observers with Unknown Expertise PAKDD 2013
    Meyyar. Leveraging the Wisdom of Crowds for Reputation Management Master’s thesis 36 of 46

    View Slide

  67. Outline Examples Adversarial and Robust Learning Other Work Take Home Messages
    Results
    Top-5 and bottom-5 ranked photos
    5.00
    4.86 4.79
    4.65 4.60
    0.84 0.83 0.64 0.44 0.00
    5.00
    4.97
    4.55 4.18
    4.10
    0.27 0.25 0.22
    0.14
    0.00
    5.00 3.77 3.66 3.55 3.48
    0.29 0.29 0.29 0.12 0.00
    5.00
    4.45 3.75 3.75 3.56
    0.21 0.19 0.18 0.07 0.00
    GPR−AVG
    Raykar
    LOB
    NLOB
    Top−5 Bottom−5
    Huang, Han, Claudia. OPARS: Objective Photo Aesthetics Ranking System (demo paper). ECIR 2013
    Han, Huang, Claudia. Learning from Multiple Observers with Unknown Expertise PAKDD 2013
    Meyyar. Leveraging the Wisdom of Crowds for Reputation Management Master’s thesis 37 of 46

    View Slide

  68. Outline Examples Adversarial and Robust Learning Other Work Take Home Messages
    Outline
    1 Examples
    2 Adversarial and Robust Learning
    Attack
    Defense
    3 Other Work
    4 Take Home Messages

    View Slide

  69. Outline Examples Adversarial and Robust Learning Other Work Take Home Messages
    Scalable Online Learning Algorithms
    Incrementally finding a good enough solution as fast as possible
    −1.5 −1 −0.5 0.5 1 1.5
    −1
    −0.5
    0.5
    1
    1.5
    2
    <0.000000s: execve
    0.000512s: brk
    0.000757s: mmap
    0.001707s: stat
    0.002275s: stat
    0.002679s: stat
    0.003077s: stat
    0.003498s: stat
    0.003811s: stat
    0.003999s:
    stat
    0.004030s:
    open
    0.004053s:
    fstat
    0.004077s:
    mmap
    0.004100s:
    close
    0.004147s:
    open
    0.004170s:
    read
    0.004198s:
    fstat
    0.004224s:
    mmap
    0.004245s:
    mprotect
    0.004268s:
    mmap
    0.004298s:
    mmap
    0.004322s:
    close
    0.004531s:
    open
    0.004555s:
    read
    0.004579s: fstat
    0.004605s: mmap
    0.004628s: mmap
    0.004648s: mprotect
    0.004670s: mmap
    0.004696s: close
    0.004908s: open
    0.004932s: read
    0.004956s: fstat
    0.004982s: mmap
    0.005002s: mprotect
    0.005024s: mmap
    0.005050s: close
    0.005258s: open
    0.005282s: read
    0.005305s: fstat
    0.005330s: mmap
    0.005351s:
    mprotect
    0.005372s:
    mmap
    0.005397s:
    mmap
    0.005421s:
    close
    0.006004s:
    open
    0.006052s:
    read
    0.006102s:
    fstat
    0.006153s:
    mmap
    0.006203s:
    mmap
    0.006244s:
    mprotect
    0.006292s:
    mmap
    0.006344s:
    close
    0.006756s:
    open
    0.006801s:
    read
    0.006847s:
    fstat
    0.006919s: mmap
    0.006968s: mprotect
    0.007018s: mmap
    0.007073s: mmap
    0.007125s: close
    0.007584s: open
    0.007635s: read
    0.007686s: fstat
    0.007743s: mmap
    0.007786s: mprotect
    0.007834s: mmap
    0.007890s: close
    0.007940s: mmap
    0.008013s: mmap
    0.008063s: arch_prctl
    0.008209s: mprotect
    0.008262s: mprotect
    0.008319s:
    mprotect
    0.008371s:
    mprotect
    0.008420s:
    mprotect
    0.008467s:
    mprotect
    0.008522s:
    mprotect
    0.008573s:
    mprotect
    0.008622s:
    mprotect
    0.008665s:
    munmap
    0.008719s:
    set_tid_address
    0.008760s:
    set_robust_list
    0.008857s:
    rt_sigaction
    0.008920s:
    rt_sigaction
    0.008973s:
    rt_sigprocmask
    0.009023s:
    getrlimit
    0.009179s:
    statfs
    0.009301s: brk
    0.009342s: brk
    0.009396s: open
    0.009463s: fstat
    0.009586s: mmap
    0.009669s: read
    0.009775s: read
    0.009847s: close
    0.009903s: munmap
    0.009993s: open
    0.010039s: fstat
    0.010091s: mmap
    0.010134s: close
    0.010222s: ioctl
    0.010301s: ioctl
    0.010388s: openat
    0.010450s: getdents
    0.010822s:
    getdents
    0.010871s:
    close
    0.011114s:
    fstat
    0.011179s:
    mmap
    0.011241s:
    write
    0.011308s:
    write
    0.011366s:
    write
    0.011423s:
    write
    0.011479s:
    write
    0.011537s:
    write
    0.011595s:
    write
    0.011654s:
    write
    0.011710s:
    write
    0.011765s:
    write
    0.011821s:
    write
    0.011877s: write
    0.011933s: write
    0.011989s: write
    0.012044s: write
    0.012112s: close
    0.012151s: munmap
    0.012203s: close
    >0.012269s: exit_group
    • Han, Claudia. Lazy Gaussian Process
    Committee for Real-Time Online
    Regression. AAAI 2013.
    • Han, Claudia. Efficient Online
    Sequence Prediction with Side
    Information. Submitted to ICDM 2013.

    View Slide

  70. Outline Examples Adversarial and Robust Learning Other Work Take Home Messages
    Ubiquitous Anomaly Detection
    • Huang Xiao. Indicative Support Vector Clustering with
    Its Application in Anomaly Detection.
    • Chih-Ta Lin. Behavior Based Malware Detection, 2013.
    • Sami Ghawi. Spatio-Temporal Anomaly detection for
    Tracking Mobile Devices. Master’s thesis, 2013.
    • Siddhant Goel. Utilizing Crowd Intelligence for Online
    Detection of Emotional Distress. Master’s thesis, 2013.
    • Fernando Hernandez Montoya. Predicting Malicious
    Linking Behavior on a Microblogging Service. Master’s
    thesis, 2012.

    View Slide

  71. Outline Examples Adversarial and Robust Learning Other Work Take Home Messages
    Group
    • Prof. Claudia Eckert
    • Huang Xiao (Ph.D. student)
    • Han Xiao (Ph.D. student)
    • Chih-Ta Lin (visitor)
    • Sami Ghawi (Master student)
    • Meyyar Palaniappan (graduated)
    • Fernando Hernandez Montoya (graduated)
    • Siddhant Goel (graduated)

    View Slide

  72. Outline Examples Adversarial and Robust Learning Other Work Take Home Messages
    Outline
    1 Examples
    2 Adversarial and Robust Learning
    Attack
    Defense
    3 Other Work
    4 Take Home Messages

    View Slide

  73. Outline Examples Adversarial and Robust Learning Other Work Take Home Messages

    View Slide

  74. Outline Examples Adversarial and Robust Learning Other Work Take Home Messages

    View Slide

  75. Outline Examples Adversarial and Robust Learning Other Work Take Home Messages

    View Slide

  76. Technische Universit¨
    at M¨
    unchen
    Q&A
    http://home.in.tum.de/~xiaoh

    View Slide