Upgrade to Pro — share decks privately, control downloads, hide ads and more …

TensorFlow Neural Networks on iOS

TensorFlow Neural Networks on iOS

Slides from my 360iDev presentation on neural networks, CoreML and TensorFlow on iOS: https://360idev.com/sessions/tensorflow-neural-networks-ios/

Taylan Pince

August 15, 2017
Tweet

More Decks by Taylan Pince

Other Decks in Technology

Transcript

  1. TensorFlow on iOS
    Taylan Pince

    View Slide

  2. Using TensorFlow, CoreML, Metal
    Performance Shaders, Accelerate
    BNNs, Keras and What the Heck is a
    Neural Network Anyway?
    Taylan Pince

    View Slide

  3. View Slide

  4. View Slide

  5. View Slide

  6. View Slide

  7. View Slide

  8. View Slide

  9. View Slide

  10. View Slide

  11. View Slide

  12. View Slide

  13. View Slide

  14. View Slide

  15. View Slide

  16. View Slide

  17. View Slide

  18. What is a Neural Network
    Anyway?

    View Slide

  19. View Slide

  20. if x + y > b {
    return blue
    } else {
    return orange
    }

    View Slide

  21. if (xWeight * x) + (yWeight * y) > b {
    return blue
    } else {
    return orange
    }

    View Slide

  22. View Slide

  23. View Slide

  24. View Slide

  25. View Slide

  26. View Slide

  27. View Slide

  28. View Slide

  29. Network Training Basics

    View Slide

  30. 1. Data Gathering & Balancing
    2. Preprocessing
    3. Training
    4. Testing Results

    View Slide

  31. View Slide

  32. View Slide

  33. View Slide

  34. View Slide

  35. View Slide

  36. View Slide

  37. View Slide

  38. View Slide

  39. import tensorflow as tf
    matrix_size = 224 * 224
    category_size = 150
    with tf.name_scope("data"):
    d1 = tf.placeholder(tf.float32, [None, matrix_size], name="image_data")
    d2 = tf.placeholder(tf.float32, [None, category_size], name="category_data")
    with tf.name_scope("model"):
    weights = tf.Variable(tf.zeros([matrix_size, category_size]), name="weights")
    bias = tf.Variable(tf.zeros([category_size]), name="bias")

    View Slide

  40. View Slide

  41. pb

    View Slide

  42. Neural Networks on iOS

    View Slide

  43. TensorFlow
    CoreML
    Metal Performance Shaders
    Accelerate

    View Slide

  44. C++ library
    Adds around 40MB to
    final app size
    Cannot use Bitcode
    Cannot use GPU

    View Slide

  45. Use freeze_graph &
    optimize_for_inference
    Import final pb file into
    Xcode project

    View Slide

  46. tensorflow::GraphDef graph;
    tensorflow::Session *session;
    ReadBinaryProto(tensorflow::Env::Default(), path, &graph);
    tensorflow::NewSession(options, &session);
    session->Create(graph);
    tensorflow::Tensor x(
    tensorflow::DT_FLOAT,
    tensorflow::TensorShape({ 1, 224 * 224 })
    );
    std::vector outputs;
    session->Run(inputs, nodes, {}, &outputs);

    View Slide

  47. Limited support for
    training engines and
    layer types
    Custom models need
    conversion
    Picks CPU or GPU
    automatically

    View Slide

  48. Pretrained Models
    Inception v3
    VGG16
    MobileNet
    SqueezeNet

    View Slide

  49. Custom Models
    Convert Caffe or Keras
    models with coremltools
    Import mlmodel file into
    Xcode project

    View Slide

  50. let model = VNCoreMLModel(for: graph().model)
    let request = VNCoreMLRequest(model: model) {
    [unowned self] request, error in
    results.forEach({ (result) in
    print("\(result.identifier)")
    })
    }
    }
    let handler = VNImageRequestHandler(ciImage: image)
    DispatchQueue.global(qos: .userInitiated).async {
    do {
    try handler.perform([request])
    } catch {
    print(error)
    }
    }

    View Slide

  51. Low-level API behind
    CoreML
    Always runs on GPU
    Got tons of love with
    iOS11 updates

    View Slide

  52. Convert pb file into a
    binary Metal can read: A
    list of floating point
    numbers

    View Slide

  53. Delivering Updates

    View Slide

  54. Recap
    Train with TensorFlow + Keras
    Use CoreML if you can
    Use TF if you need multi-platform

    View Slide

  55. Further Reading
    Matthijs Hollemans
    (MachineThink.net)
    Reza Shirazian (reza.codes)

    View Slide

  56. Further Reading
    Apple samples
    Google TensorFlow docs & samples

    View Slide

  57. Thank you!
    @taylanpince
    [email protected]

    View Slide