TechSummit Amsterdam 2017: Enhance! Deploying Image Recognition with TensorFlow and Kubernetes

“Enhance… enhance… enhance…” Have you ever wondered how image recognition works in the movies, or how you can take advantage of it? In this talk you’ll find out. I’ll explain the basics of Machine Learning and Image Recognition and demonstrate how it works with TensorFlow, and Open Source library for machine intelligence. Once we have a working image recognition system I’ll show you how to deploy it in production on Kubernetes, an open source container management system.

Transcript

1. Enhance!
   Deploying Image Recognition with
   TensorFlow and Kubernetes
   Casey West
   ! @caseywest @googlecloud #techsummitio #amsterdam #tensorflow #kubernetes 1
   

   View Slide

2. I am…
   —Dad, software architect, technical lead, change agent, programmer,
   beverage enthusiast, rock climber, cyclist, acro yoga person…
   —An Architecture Advocate at Google.
   I am not…
   —A data scientist.
   —An expert in machine learning.
   ! @caseywest @googlecloud #techsummitio #amsterdam #tensorflow #kubernetes 2
   

   View Slide

3. however…
   ! @caseywest @googlecloud #techsummitio #amsterdam #tensorflow #kubernetes 3
   

   View Slide

4. I know enough to be
   dangerous.
   ! @caseywest @googlecloud #techsummitio #amsterdam #tensorflow #kubernetes 4
   

   View Slide

5. In 28 minutes you will also
   know enough to be
   dangerous. !
   ! @caseywest @googlecloud #techsummitio #amsterdam #tensorflow #kubernetes 5
   

   View Slide

6. Let's get dangerous.
   ! @caseywest @googlecloud #techsummitio #amsterdam #tensorflow #kubernetes 6
   

   View Slide

7. Project Plan
   —Phase 0: Understand ML Basics
   —Phase 1: Create TensorFlow Serving model
   —Phase 2: Create TensorFlow Serving client app
   —Phase 3: Deploy on Kubernetes
   ! @caseywest @googlecloud #techsummitio #amsterdam #tensorflow #kubernetes 7
   

   View Slide

8. Understand ML Basics
   ! @caseywest @googlecloud #techsummitio #amsterdam #tensorflow #kubernetes 8
   

   View Slide

9. Understand ML Basics
   ¯\_(ϑ)_/¯
   ! @caseywest @googlecloud #techsummitio #amsterdam #tensorflow #kubernetes 9
   

   View Slide

10. Training a Model
    1. Data
    2. Time
    3. Procedure
    ! @caseywest @googlecloud #techsummitio #amsterdam #tensorflow #kubernetes 10
    

    View Slide

11. Training a Model
    1. Data
    Historical or available data.
    ! @caseywest @googlecloud #techsummitio #amsterdam #tensorflow #kubernetes 11
    

    View Slide

12. Training a Model
    2. Time
    Iterations you have time to make.
    ! @caseywest @googlecloud #techsummitio #amsterdam #tensorflow #kubernetes 12
    

    View Slide

13. Training a Model
    3. Procedure
    Discover data preparation procedures, an algorithm
    to use, and how to configure it.
    ! @caseywest @googlecloud #techsummitio #amsterdam #tensorflow #kubernetes 13
    

    View Slide

14. Training a Model: Steps
    1. Get a data set.
    2. Split it into training and testing data.
    3. Train the algorithm.
    4. Test the algorithm.
    5. Ship it!
    ! @caseywest @googlecloud #techsummitio #amsterdam #tensorflow #kubernetes 14
    

    View Slide

15. Training a Model: Know More in Five Minutes
    https://www.youtube.com/watch?v=2FmcHiLCwTU
    — Saraj Javal
    ! @caseywest @googlecloud #techsummitio #amsterdam #tensorflow #kubernetes 15
    

    View Slide

16. Image Recognition: Inception v3
    https://github.com/tensorflow/models/tree/master/inception
    ! @caseywest @googlecloud #techsummitio #amsterdam #tensorflow #kubernetes 16
    

    View Slide

17. Serving the model with TensorFlow Serving
    —Continuous training pipeline
    —gRPC interface
    —Production ready
    https://tensorflow.github.io/serving/
    ! @caseywest @googlecloud #techsummitio #amsterdam #tensorflow #kubernetes 17
    

    View Slide

18. Pre-trained models exist! !
    Let's containerize them before they get away!
    ! @caseywest @googlecloud #techsummitio #amsterdam #tensorflow #kubernetes 18
    

    View Slide

19. Create a Docker Image: TensorFlow Serving Inception v3
    Docker Machine
    docker-machine create \
    --driver google \
    --google-project goog-caseywest \
    --google-zone us-central1-a \
    --google-machine-type n1-standard-4 \
    docker-machine
    eval $(docker-machine env docker-machine)
    ! @caseywest @googlecloud #techsummitio #amsterdam #tensorflow #kubernetes 19
    

    View Slide

20. Create a Docker Image: TensorFlow Serving Inception v3
    Run developer container
    git clone git@github.com:tensorflow/serving.git
    cd serving
    docker build --pull \
    -t $USER/tensorflow-serving-devel \
    -f tensorflow_serving/tools/docker/Dockerfile.devel .
    docker run --name=inception_container -it \
    $USER/tensorflow-serving-devel
    ! @caseywest @googlecloud #techsummitio #amsterdam #tensorflow #kubernetes 20
    

    View Slide

21. Create a Docker Image: TensorFlow Serving Inception v3
    Build in the devel the container
    git clone --recurse-submodules \
    https://github.com/tensorflow/serving
    cd serving/tensorflow
    ./configure # interactive :-(
    cd ..
    bazel build -c opt tensorflow_serving/...
    ! @caseywest @googlecloud #techsummitio #amsterdam #tensorflow #kubernetes 21
    

    View Slide

22. Create a Docker Image: TensorFlow Serving Inception v3
    Export inception in the devel the container
    curl -LO http://cwe.st/tf-inception-v3
    tar xzf inception-v3-2016-03-01.tar.gz
    bazel-bin/tensorflow_serving/example/inception_saved_model \
    --checkpoint_dir=inception-v3 \
    --output_dir=inception-export
    ! @caseywest @googlecloud #techsummitio #amsterdam #tensorflow #kubernetes 22
    

    View Slide

23. Create a Docker Image: TensorFlow Serving Inception v3
    Commit and deploy container
    docker commit inception_container \
    $USER/inception_serving
    docker tag $USER/inception_serving \
    gcr.io/goog-caseywest/inception
    gcloud docker -- push \
    gcr.io/goog-caseywest/inception
    ! @caseywest @googlecloud #techsummitio #amsterdam #tensorflow #kubernetes 23
    

    View Slide

24. Deploy Inception Service on Kubernetes
    Create a cluster
    gcloud container clusters create \
    image-recognition-cluster \
    --num-nodes 5
    ! @caseywest @googlecloud #techsummitio #amsterdam #tensorflow #kubernetes 24
    

    View Slide

25. Let's do it live…
    ! @caseywest @googlecloud #techsummitio #amsterdam #tensorflow #kubernetes 25
    

    View Slide

26. We did it!
    !
    ! @caseywest @googlecloud #techsummitio #amsterdam #tensorflow #kubernetes 26
    

    View Slide

27. Takeaways
    —Training models is hard.
    —Serving models is kind of easy.
    —gRPC and protobuf have steep learning curves.
    —therefore, client libraries are challenging.
    ! @caseywest @googlecloud #techsummitio #amsterdam #tensorflow #kubernetes 27
    

    View Slide

28. Want the code?
    —DM me @caseywest
    —Email me caseywest@google.com
    —Try a similar tutorial
    —https://tensorflow.github.io/serving/
    serving_inception
    ! @caseywest @googlecloud #techsummitio #amsterdam #tensorflow #kubernetes 28
    

    View Slide

29. Want to try more ML?
    —TensorFlow
    —https://www.tensorflow.org/
    —MXNet
    —http://mxnet.io/
    —Google Cloud Playground
    —https://cloud.google.com/vision/
    ! @caseywest @googlecloud #techsummitio #amsterdam #tensorflow #kubernetes 29
    

    View Slide

30. Let's be friends
    @caseywest
    !
    ! @caseywest @googlecloud #techsummitio #amsterdam #tensorflow #kubernetes 30
    

    View Slide