Erik Hellman
October 26, 2017
320

# My App Is Smarter Than Your App

DroidCon London 2017 talk about how to apply Machine Learning to make your app smarter.

October 26, 2017

## Transcript

1. My App is Smarter
Erik Hellman

2. Personalisation

3. Personalisation

4. Automation

5. User Assistance

6. User Assistance

7. What is this Machine Learning thing?!?

8. Deep Learning vs. Machine Learning

9. TensorFlow
www.tensorflow.org

10. Simple example
import tensorflow as tf
# Model parameters
W = tf.Variable([.3], dtype=tf.float32)
b = tf.Variable([-.3], dtype=tf.float32)
# Model input and output
x = tf.placeholder(tf.float32)
linear_model = W * x + b
y = tf.placeholder(tf.float32)
# loss
loss = tf.reduce_sum(tf.square(linear_model - y)) # sum of the squares
# optimizer
train = optimizer.minimize(loss)

11. Simple example
# training data
x_train = [1, 2, 3, 4]
y_train = [0, -1, -2, -3]
# training loop
init = tf.global_variables_initializer()
sess = tf.Session()
sess.run(init) # reset values to wrong
for i in range(1000):
sess.run(train, {x: x_train, y: y_train})
# evaluate training accuracy
curr_W, curr_b, curr_loss = sess.run([W, b, loss], {x: x_train, y: y_train})
print("W: %s b: %s loss: %s"%(curr_W, curr_b, curr_loss))

12. TensorBoard

13. Tensor?

14. Vector

15. Matrix

16. Tensor
3 # a rank 0 tensor; this is a scalar with shape []
[1., 2., 3.] # a rank 1 tensor; this is a vector with shape [3]
[[1., 2., 3.], [4., 5., 6.]] # a rank 2 tensor; a matrix with shape [2, 3]
[[[1., 2., 3.]], [[7., 8., 9.]]] # a rank 3 tensor with shape [2, 1, 3]

17. But wait, there is more!

19. Cloud Vision API

20. Cloud Vision API

21. Cloud Vision API
interface CloudVisionApi {
@POST("images:annotate")
fun annotateImage(cloudVisionRequest: CloudVisionRequest): Call
}

22. Cloud Vision API
data class CloudVisionRequest(val requests:List)
data class AnnotateImageRequest(val image:Image, val features: List,
val imageContext:ImageContext)
data class Image(val content:String?, val source:ImageSource?)
data class ImageSource(val gcsImageUri:String?, val imageUri:String?)
data class Feature(...)
data class ImageContext(...)

23. Cloud Vision API
data class CloudVisionResponse(val responses:List)
data class AnnotateImageResponse(val faceAnnotations:List,
val landmarkAnnotations:List,
val logoAnnotations:List,
val labelAnnotations:List,
val textAnnotations:List,
val fullTextAnnotation:FullTextAnnotation,
val safeSearchAnnotation:SafeSearchAnnotation,
val imagePropertiesAnnotation:ImagePropertiesAnnotation,
val cropHintsAnnotation:CropHintsAnnotation,
val webDetection:WebDetection,
val error:Status)

24. Cloud Vision API

25. Video Intelligence API
{
"inputUri": string,
"inputContent": string,
"features": [
enum(Feature)
],
"videoContext": {
object(VideoContext)
},
"outputUri": string,
"locationId": string,
}

26. Video Intelligence API

27. Video Intelligence API
{
"inputUri": string,
"segmentLabelAnnotations": [
{ object(LabelAnnotation) }
],
"shotLabelAnnotations": [
{ object(LabelAnnotation) }
],
"frameLabelAnnotations": [
{ object(LabelAnnotation) }
],
"shotAnnotations": [
{ object(VideoSegment) }
],
"explicitAnnotation": { object(ExplicitContentAnnotation) },
"error": { object(Status) },
}

28. Video Intelligence API

29. Natural Language API

30. ChaChi app by Luis G. Valle (unreleased)

31. TensorFlow for Android

32. implementation 'org.tensorflow:tensorflow-android:1.4.0-rc1'

33. // load the model into a TensorFlowInferenceInterface.
c.inferenceInterface = new TensorFlowInferenceInterface(
assetManager, modelFilename);
// Get the tensorflow node
final Operation operation = c.inferenceInterface.graphOperation(outputName);
// Inspect its shape
final int numClasses = (int) operation.output(0).shape().size(1);
// Build the output array with the correct size.
c.outputs = new float[numClasses];

34. inferenceInterface.feed(
inputName, // The name of the node to feed.
floatValues, // The array to feed
1, inputSize, inputSize, 3 ); // The shape of the array
inferenceInterface.run(
outputNames, // Names of all the nodes to calculate.
logStats); // Bool, enable stat logging.
inferenceInterface.fetch(
outputName, // Fetch this output.
outputs); // Into the prepared array.

35. Thank you for listening!