Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Hacking Facial Recognition With Beards

Hacking Facial Recognition With Beards

I gave a streaming session on the IBMDeveloper Twitch about how to perform facial recognition, and the human processes involved.

David Okun

August 25, 2018
Tweet

More Decks by David Okun

Other Decks in Programming

Transcript

  1. @dokun24
    Hacking Facial Recognition With
    Beards
    David Okun, IBM
    dokun24

    View Slide

  2. @dokun24
    Agenda
    • Ethics In Machine Learning
    • Vernacular
    • Doing The Facial Recognition
    • Demo
    • Existing Challenges
    • Q & A

    View Slide

  3. @dokun24

    View Slide

  4. @dokun24

    View Slide

  5. @dokun24
    First, Some Vocabulary
    • Face Detection
    • Face Verification
    • Face Identification

    View Slide

  6. @dokun24
    The Highest Level Process
    • Face Detection
    • Image Normalization
    • Feature Extraction
    • Feature Matching

    View Slide

  7. @dokun24
    Face Detection

    View Slide

  8. @dokun24
    What is OpenCV?
    • Open(source) Computer Vision
    • Normalizes computer vision
    applications & infrastructure
    • Target detection, texture mapping,
    etc

    View Slide

  9. @dokun24
    What is dlib?
    • C++ library for machine learning
    algorithms
    • Here, mostly for facial detection
    • 68 landmark points

    View Slide

  10. View Slide

  11. View Slide

  12. @dokun24
    Image Normalization

    View Slide

  13. @dokun24

    View Slide

  14. View Slide

  15. View Slide

  16. View Slide

  17. @dokun24
    Feature Extraction

    View Slide

  18. @dokun24
    What is TensorFlow?
    • High performance computation library
    for machine learning
    • Open source, heavily adopted
    • The lowest level of code needed for
    training CNNs

    View Slide

  19. @dokun24
    c = tf.constant([[1.0, 2.0, 3.0], [4.0, 5.0, 6.0]])
    print(c.shape)
    ==> TensorShape([Dimension(2), Dimension(3)])
    d = tf.constant([[1.0, 0.0], [0.0, 1.0], [1.0, 0.0], [0.0, 1.0]])
    print(d.shape)
    ==> TensorShape([Dimension(4), Dimension(2)])
    # Raises a ValueError, because `c` and `d` do not have compatible
    # inner dimensions.
    e = tf.matmul(c, d)
    f = tf.matmul(c, d, transpose_a=True, transpose_b=True)
    print(f.shape)
    ==> TensorShape([Dimension(3), Dimension(4)])

    View Slide

  20. @dokun24
    What is Keras?
    • A neural network library written in
    Python
    • Can run on top of TensorFlow
    • Creates the layers that help create a
    feature vector

    View Slide

  21. @dokun24
    from keras.layers import Input, Dense
    from keras.models import Model
    # This returns a tensor
    inputs = Input(shape=(784,))
    # a layer instance is callable on a tensor, and returns a tensor
    x = Dense(64, activation='relu')(inputs)
    x = Dense(64, activation='relu')(x)
    predictions = Dense(10, activation='softmax')(x)
    # This creates a model that includes
    # the Input layer and three Dense layers
    model = Model(inputs=inputs, outputs=predictions)
    model.compile(optimizer='rmsprop',
    loss='categorical_crossentropy',
    metrics=['accuracy'])
    model.fit(data, labels) # starts training

    View Slide

  22. View Slide

  23. @dokun24
    Feature Matching

    View Slide

  24. @dokun24
    [0.0109382765, 0.0727260783, -0.0886565521, 0.106995322, -0.0263014287, -0.0352396965, -0.0471194535, 0.0224863011, 0.00886561163,
    -0.136294395, -0.00985514186, -0.0441077091, 0.0644643679, 0.0119109377, -0.00304533541, 0.00841313601, -0.0451855995, 0.0409480296,
    -0.0849511996, -0.046876207, 0.00489062304, -0.100049019, -0.0260294266, 0.0340725258, -0.0513369851, -0.00715692108, -0.138269156,
    -0.0447790548, -0.0971052274, 0.016863659, 0.0200845413, 0.0345470943, 0.0226635113, 0.0210720059, 0.0939424559, -0.0567186847, -0.0420572162,
    -0.00359278591, 0.0274323691, -0.0161195938, 0.0396690778, 0.0509826653, 0.100426823, -0.0316316895, -0.0500608087, -0.00339857256,
    -0.0342332497, 0.0790704489, -0.0289952923, 0.0568330586, -0.0285114124, 0.0588419661, -0.0434439555, 0.0621240847, 0.0360112451,
    0.00799505785, -0.0279962141, -0.0449286103, 0.0152444597, 0.0455824099, -0.0581656098, -0.00988157, -0.024159437, 0.0274357516, -0.0862255767,
    -0.00760430517, -0.102911048, -0.0202399883, 0.00621778751, -0.0181367081, 0.0223715473, -0.125922918, -0.0999212191, -0.0126653658,
    -0.0358478688, -0.0665559843, 0.0375230871, -0.0261705182, -0.0212064162, 0.0475422479, -0.0623819679, 0.0129780034, 0.0282707643,
    0.0232121553, -0.00730743492, -0.0821457431, 0.0655974671, -0.0265328269, 0.0388734452, 0.0616755709, -0.0121487472, -0.0232637692,
    -0.0545362122, 0.0236765929, -0.0611603297, 0.0797719285, -0.0404306911, 0.0323628858, -0.00949066877, -0.0609771982, -0.00158646447,
    0.0596057661, -0.0802996904, 0.0247787572, 0.0387842879, 0.0258943904, -0.093511194, 0.0587848015, -0.0104612159, -0.108764656, -0.0245255344,
    -0.00470191566, 0.0061077862, -0.0946708396, 0.0128557365, 0.123939671, 0.0517629161, 0.0203773696, 0.0309179667, 0.0296497084, -0.0960420221,
    0.0165317804, 0.0315312482, 0.0090330299, 0.0824666694, 0.137421414, 0.00069823768, -0.0312179867, 0.0248888023, -0.00145759375, -0.0291704088,
    -0.0118671609, 0.0213795807, -0.0371772498, 0.00247276342, 0.0654902682, -0.0687400624, 0.00264206412, 0.0854841322, -0.0100153023,
    -0.0529452562, 0.0973913893, 0.0627576113, 0.00176332367, -0.0661887601, -0.080224067, 0.0554779172, -0.0210913122, 0.0315915756, 0.0259354841,
    -0.0917198285, -0.0626271218, -0.0229110811, -0.0031353659, -0.0217538457, 0.057530541, 0.0180884395, -0.116396718, 0.0102889976,
    -0.0272365212, -0.0515930578, 0.0503248908, -0.0153394509, 0.0429311357, 0.0498886444, -0.0364963003, -0.00377800176, 0.0172923729,
    -0.0085753873, -0.000616554695, -0.0112605086, -0.0504184701, -0.0347453021, -0.0306184422, 0.0429552235, -0.126647428, 0.0414417945,
    0.0330664888, 0.0490230545, -0.00483355578, -0.0539604612, -0.00565166539, -0.120982081, -0.00506902765, -0.0661799386, 0.0654867887,
    -0.0254629534, -0.00545939198, 0.112354159, -0.0514094941, 0.0167419966, 0.0574088842, 0.0635244325, 0.0998285115, 0.014563757, 0.0446437597,
    -0.0102947541, 0.0601763278, -0.022337636, 0.037583936, -0.00868016109, 0.0387439467, -0.0472361892, -0.00683514262, -0.0536096953,
    0.0930362642, -0.0444846824, 0.0863161162, -0.0145008266, -0.0109270848, -0.0247354154, 0.0888869762, 0.0915196687, -0.0189450141, 0.157319754,
    -0.074196659, -0.0373945273, -0.0393407792, 0.110559419, -0.123502225, -0.0390469283, 0.0392427184, -0.0211585611, 0.029190179, 0.0259871911,
    -0.0924885496, -0.0496961176, 0.0109286346, -0.0429181717, 0.0285253581, -0.0200652219, -0.188982397, -0.0164047889, 0.0247660689,
    0.0287661962, -0.0118430201, 0.0300309248, 0.0160504375, -0.00699294591, 0.0520862937, -0.0729718357, -0.0837474763, -0.0414310731,
    -0.096074976, 0.0275698956, -0.051039014, 0.084851712, 0.0742572099, -0.0493934005, 0.0458364189, 0.055183582, -0.0109172817, -0.0432627872,
    -0.0828055739, -0.0384820662, 0.0220153034, -0.00765768997, 0.0994410664, 0.017342262, -0.0428088047, -0.0226635933, 0.0442144275,
    -0.0242784154, -0.0128913475, 0.00506109418, 0.0339680836, 0.0699482784, 0.0170274191, 0.0268076807, -0.0130135585, -0.131615028, 0.10316924,
    -0.0259890705, 0.122296281, -0.0297779553, -0.0306672305, -0.0287104975, -0.048643548, -0.0360500105, -0.0858685449, -0.00986277591,
    -0.0646833256, -0.0840798244, 0.0136408471, 0.0169043299, -0.0971477106, -0.016923707, 0.0805660486, 0.0159345381, 0.0525551066, -0.0761455074,
    -0.136946559, 0.0588576943, -0.0372881182, 0.0313418806, 0.0984985977, -0.0552069917, 0.0313524827, 0.0150029277, 0.0668970719, 0.0640067905,
    0.0310357977, 0.0117677432, -0.0163922533, -0.0199124962, -0.0404609703, 0.0657613128, 0.0340500884, -0.0149180656, 0.0291028358, 0.0193162505,
    -0.0158343688, 0.103551552, -0.0468648039, -0.0689977854, 0.0592100658, 0.037243735, 0.0348685384, -0.0724523813, 0.00524123944]

    View Slide

  25. @dokun24

    View Slide

  26. @dokun24
    A Binary Result Told Three Ways
    • Match
    • Undetermined
    • No Match

    View Slide

  27. @dokun24

    View Slide

  28. @dokun24

    View Slide

  29. @dokun24

    View Slide

  30. @dokun24
    DEMO

    View Slide

  31. @dokun24
    Existing Challenges
    • Landmark detection with enough light
    • Different poses / insufficient training data
    • Occlusion / facial expressions

    View Slide

  32. @dokun24
    https://github.com/dokun1/
    CallForCodeFacialRecognition

    View Slide