TensorFlow Neural Networks on iOS

Slides from my 360iDev presentation on neural networks, CoreML and TensorFlow on iOS: https://360idev.com/sessions/tensorflow-neural-networks-ios/

Transcript

1. TensorFlow on iOS Taylan Pince
2. Using TensorFlow, CoreML, Metal Performance Shaders, Accelerate BNNs, Keras and What the Heck is a Neural Network Anyway? Taylan Pince
18. What is a Neural Network Anyway?
20. if x + y > b { return blue } else { return orange }
21. if (xWeight * x) + (yWeight * y) > b { return blue } else { return orange }
29. Network Training Basics
30. 1. Data Gathering & Balancing 2. Preprocessing 3. Training 4. Testing Results
39. import tensorflow as tf matrix_size = 224 * 224 category_size = 150 with tf.name_scope("data"): d1 = tf.placeholder(tf.float32, [None, matrix_size], name="image_data") d2 = tf.placeholder(tf.float32, [None, category_size], name="category_data") with tf.name_scope("model"): weights = tf.Variable(tf.zeros([matrix_size, category_size]), name="weights") bias = tf.Variable(tf.zeros([category_size]), name="bias")
41. pb
42. Neural Networks on iOS
43. TensorFlow CoreML Metal Performance Shaders Accelerate
44. C++ library Adds around 40MB to final app size Cannot use Bitcode Cannot use GPU
45. Use freeze_graph & optimize_for_inference Import final pb file into Xcode project
46. tensorflow::GraphDef graph; tensorflow::Session *session; ReadBinaryProto(tensorflow::Env::Default(), path, &graph); tensorflow::NewSession(options, &session); session->Create(graph); tensorflow::Tensor x( tensorflow::DT_FLOAT, tensorflow::TensorShape({ 1, 224 * 224 }) ); std::vector<tensorflow::Tensor> outputs; session->Run(inputs, nodes, {}, &outputs);
47. Limited support for training engines and layer types Custom models need conversion Picks CPU or GPU automatically
48. Pretrained Models Inception v3 VGG16 MobileNet SqueezeNet
49. Custom Models Convert Caffe or Keras models with coremltools Import mlmodel file into Xcode project
50. let model = VNCoreMLModel(for: graph().model) let request = VNCoreMLRequest(model: model) { [unowned self] request, error in results.forEach({ (result) in print("\(result.identifier)") }) } } let handler = VNImageRequestHandler(ciImage: image) DispatchQueue.global(qos: .userInitiated).async { do { try handler.perform([request]) } catch { print(error) } }
51. Low-level API behind CoreML Always runs on GPU Got tons of love with iOS11 updates
52. Convert pb file into a binary Metal can read: A list of floating point numbers
53. Delivering Updates
54. Recap Train with TensorFlow + Keras Use CoreML if you can Use TF if you need multi-platform
55. Further Reading Matthijs Hollemans (MachineThink.net) Reza Shirazian (reza.codes)
56. Further Reading Apple samples Google TensorFlow docs & samples
57. Thank you! @taylanpince taylan@hipolabs.com