Voxxed Days Belgrade 2017: Deploying Image Recognition with TensorFlow and Kubernetes

Transcript

1. Enhance!
   Deploying Image Recognition with
   TensorFlow and Kubernetes
   Casey West
   ! @caseywest @googlecloud @VoxxedBelgrade #" #VBD17 #machinelearning #tensorflow #kubernetes 1
   

   View Slide

2. I am…
   —Dad, software architect, technical lead, change agent, programmer,
   beverage enthusiast, rock climber, cyclist, acro yoga person…
   —An Architecture Advocate at Google.
   I am not…
   —A data scientist.
   —An expert in machine learning.
   ! @caseywest @googlecloud @VoxxedBelgrade #" #VBD17 #machinelearning #tensorflow #kubernetes 2
   

   View Slide

3. however…
   ! @caseywest @googlecloud @VoxxedBelgrade #" #VBD17 #machinelearning #tensorflow #kubernetes 3
   

   View Slide

4. I know enough to be
   dangerous.
   ! @caseywest @googlecloud @VoxxedBelgrade #" #VBD17 #machinelearning #tensorflow #kubernetes 4
   

   View Slide

5. In 40 minutes you will also
   know enough to be
   dangerous. !
   ! @caseywest @googlecloud @VoxxedBelgrade #" #VBD17 #machinelearning #tensorflow #kubernetes 5
   

   View Slide

6. Let's get dangerous.
   ! @caseywest @googlecloud @VoxxedBelgrade #" #VBD17 #machinelearning #tensorflow #kubernetes 6
   

   View Slide

7. Project Plan
   —Phase 0: Understand ML Basics
   —Phase 1: Create TensorFlow Serving model
   —Phase 2: Create TensorFlow Serving client app
   —Phase 3: Deploy on Kubernetes
   —Phase 4: Play!
   ! @caseywest @googlecloud @VoxxedBelgrade #" #VBD17 #machinelearning #tensorflow #kubernetes 7
   

   View Slide

8. Understand ML Basics
   ! @caseywest @googlecloud @VoxxedBelgrade #" #VBD17 #machinelearning #tensorflow #kubernetes 8
   

   View Slide

9. Understand ML Basics
   ¯\_(ϑ)_/¯
   ! @caseywest @googlecloud @VoxxedBelgrade #" #VBD17 #machinelearning #tensorflow #kubernetes 9
   

   View Slide

10. Training a Model
    1. Data
    2. Time
    3. Procedure
    ! @caseywest @googlecloud @VoxxedBelgrade #" #VBD17 #machinelearning #tensorflow #kubernetes 10
    

    View Slide

11. Training a Model
    1. Data
    Historical or available data.
    ! @caseywest @googlecloud @VoxxedBelgrade #" #VBD17 #machinelearning #tensorflow #kubernetes 11
    

    View Slide

12. Training a Model
    2. Time
    Iterations you have time to make.
    ! @caseywest @googlecloud @VoxxedBelgrade #" #VBD17 #machinelearning #tensorflow #kubernetes 12
    

    View Slide

13. Training a Model
    3. Procedure
    Discover data preparation procedures, an algorithm
    to use, and how to configure it.
    ! @caseywest @googlecloud @VoxxedBelgrade #" #VBD17 #machinelearning #tensorflow #kubernetes 13
    

    View Slide

14. Training a Model: Steps
    1. Get a data set.
    2. Split it into training and testing data.
    3. Train the algorithm.
    4. Test the algorithm.
    5. Ship it!
    ! @caseywest @googlecloud @VoxxedBelgrade #" #VBD17 #machinelearning #tensorflow #kubernetes 14
    

    View Slide

15. Training a Model: Know More in Five Minutes
    https://www.youtube.com/watch?v=2FmcHiLCwTU
    — Siraj Raval
    ! @caseywest @googlecloud @VoxxedBelgrade #" #VBD17 #machinelearning #tensorflow #kubernetes 15
    

    View Slide

16. Image Recognition: Inception v3
    https://github.com/tensorflow/models/tree/master/research/inception
    ! @caseywest @googlecloud @VoxxedBelgrade #" #VBD17 #machinelearning #tensorflow #kubernetes 16
    

    View Slide

17. Serving the model with TensorFlow Serving
    —Continuous training pipeline
    —gRPC interface
    —Production ready
    https://www.tensorflow.org/serving/
    ! @caseywest @googlecloud @VoxxedBelgrade #" #VBD17 #machinelearning #tensorflow #kubernetes 17
    

    View Slide

18. Pre-trained models exist! !
    Let's containerize them before they get away!
    ! @caseywest @googlecloud @VoxxedBelgrade #" #VBD17 #machinelearning #tensorflow #kubernetes 18
    

    View Slide

19. Create a Docker Image: TensorFlow Serving Inception v3
    Docker Machine
    docker-machine create \
    --driver google \
    --google-project goog-caseywest \
    --google-zone us-central1-a \
    --google-machine-type n1-standard-4 \
    docker-machine
    eval $(docker-machine env docker-machine)
    ! @caseywest @googlecloud @VoxxedBelgrade #" #VBD17 #machinelearning #tensorflow #kubernetes 19
    

    View Slide

20. Create a Docker Image: TensorFlow Serving Inception v3
    Run developer container
    git clone git@github.com:tensorflow/serving.git
    cd serving
    docker build --pull \
    -t $USER/tensorflow-serving-devel \
    -f tensorflow_serving/tools/docker/Dockerfile.devel .
    docker run --name=inception_container -it \
    $USER/tensorflow-serving-devel
    ! @caseywest @googlecloud @VoxxedBelgrade #" #VBD17 #machinelearning #tensorflow #kubernetes 20
    

    View Slide

21. Create a Docker Image: TensorFlow Serving Inception v3
    Build in the devel the container
    git clone --recurse-submodules \
    https://github.com/tensorflow/serving
    cd serving/tensorflow
    ./configure # interactive :-(
    cd ..
    bazel build -c opt tensorflow_serving/...
    ! @caseywest @googlecloud @VoxxedBelgrade #" #VBD17 #machinelearning #tensorflow #kubernetes 21
    

    View Slide

22. Create a Docker Image: TensorFlow Serving Inception v3
    Export inception in the devel the container
    curl -LO http://cwe.st/tf-inception-v3
    tar xzf inception-v3-2016-03-01.tar.gz
    bazel-bin/tensorflow_serving/example/inception_saved_model \
    --checkpoint_dir=inception-v3 \
    --output_dir=inception-export
    ! @caseywest @googlecloud @VoxxedBelgrade #" #VBD17 #machinelearning #tensorflow #kubernetes 22
    

    View Slide

23. Create a Docker Image: TensorFlow Serving Inception v3
    Commit and deploy container
    docker commit inception_container \
    $USER/inception_serving
    docker tag $USER/inception_serving \
    gcr.io/goog-caseywest/inception
    gcloud docker -- push \
    gcr.io/goog-caseywest/inception
    ! @caseywest @googlecloud @VoxxedBelgrade #" #VBD17 #machinelearning #tensorflow #kubernetes 23
    

    View Slide

24. The Machines Have Already Learned
    Using ML isn't doing ML.
    ! @caseywest @googlecloud @VoxxedBelgrade #" #VBD17 #machinelearning #tensorflow #kubernetes 24
    

    View Slide

25. Deploy Inception Service on Kubernetes
    Create a cluster
    gcloud container clusters create \
    image-recognition-cluster \
    --num-nodes 5
    ! @caseywest @googlecloud @VoxxedBelgrade #" #VBD17 #machinelearning #tensorflow #kubernetes 25
    

    View Slide

26. Let's do it live…
    ! @caseywest @googlecloud @VoxxedBelgrade #" #VBD17 #machinelearning #tensorflow #kubernetes 26
    

    View Slide

27. We did it!
    !
    ! @caseywest @googlecloud @VoxxedBelgrade #" #VBD17 #machinelearning #tensorflow #kubernetes 27
    

    View Slide

28. Takeaways
    —Training models is hard.
    —Serving models is kind of easy.
    —gRPC and protobuf have steep learning curves.
    —therefore, client libraries are challenging.
    ! @caseywest @googlecloud @VoxxedBelgrade #" #VBD17 #machinelearning #tensorflow #kubernetes 28
    

    View Slide

29. Want the code?
    —DM me @caseywest
    —Email me caseywest@google.com
    —Try a similar tutorial
    —https://tensorflow.github.io/serving/
    serving_inception
    ! @caseywest @googlecloud @VoxxedBelgrade #" #VBD17 #machinelearning #tensorflow #kubernetes 29
    

    View Slide

30. Want to try more ML?
    —TensorFlow
    —https://www.tensorflow.org/
    —MXNet
    —http://mxnet.io/
    —Google Cloud Playground
    —https://cloud.google.com/vision/
    ! @caseywest @googlecloud @VoxxedBelgrade #" #VBD17 #machinelearning #tensorflow #kubernetes 30
    

    View Slide

31. Let's be friends
    @caseywest
    !
    ! @caseywest @googlecloud @VoxxedBelgrade #" #VBD17 #machinelearning #tensorflow #kubernetes 31
    

    View Slide