KerasCV for the Young and Curious

For the eager beginners looking to start with Computer Vision and for the people who want to start building right away.

The session covers the import use cases and models such as StableDiffusion (v1 & 2), YOLO, CSPDarknet, EfficientNet, etc. that can be used in CV.

The session is introductory in nature and is meant for beginners and intermediate.

Transcript

1. KerasCV for the Young and Curious Wesley Kambale @weskambale kambale.dev
2. • Machine Learning Engineer with 3 years of experience • Community Builder for 3 years • Explore ML Facilitator with Crowdsource by Google for 2 years • Flutter Dev Profile Interests Experience • Research in TinyML, TTS & LLM • DevRel & Technical Writing
3. Things to cover today What is Computer Vision? Convoluting your way into CV Pay A ention. You’re going to need it Going into the 3 pool with Keras Who is KerasCV? Diving into the 6 pool with KerasCV Taking KerasCV for a spin Maps for your next trip to KerasCV 0 50 75 25
4. Computer Vision is just CNNs and Deep Learning. - Someone trying to understand CV
5. Algorithmic methods exist to perform most tasks for Computer Vision that we see today. They used to do a good job at it, too. Source: University of Waterloo
6. Then they failed when we asked them to do this. Source: Towards Data Science
7. Things to cover today What is Computer Vision? Convoluting your way into CV Pay A ention. You’re going to need it Going into the 3 pool with Keras Who is KerasCV? Diving into the 6 pool with KerasCV Taking KerasCV for a spin Maps for your next trip to KerasCV 0 50 75 25
9. Source: Stanford CS Convolutional Neural Networks Recognizing images better than before • Make the kernel learnable instead of leaving it fixed • Kernels learn to recognize edges and distinct features on their own • Pass these learnings as a vector-space representation Source: Stanford CS
10. Source: Keras TPU Codelabs
11. Things to cover today What is Computer Vision? Convoluting your way into CV Pay A ention. You’re going to need it Going into the 3 pool with Keras Who is KerasCV? Diving into the 6 pool with KerasCV Taking KerasCV for a spin Maps for your next trip to KerasCV 0 50 75 25
12. Attention. Building a simple intuition - Imagine having 10 locks and 10 corresponding keys. Each lock opens a certain box. - Being the chaotic evil person I am, I mix the keys up and give them to you. Your job is to unlock each box - How do you unlock it?
13. Attention. Building a simple intuition Solution: Brute Force - Try each key with each lock - Combinations = 10 x 10 = 100 tries - What would this solution possibly look like?
14. Building a simple intuition - Now, imagine pixels of an image as the boxes, locks, and the keys - Run the same brute force experiment - You’re gonna focus on the right lock-key combination, correct? The instance where you get the right lock with the right key, that is where you focus the most - Each of the 10 instances for you is where your attention will lie (I know that neither is it this simple, nor is it mathematically sound. Bear with me.) Source: MDPI
17. Things to cover today What is Computer Vision? Convoluting your way into CV Pay A ention. You’re going to need it Going into the 3 pool with Keras Who is KerasCV? Diving into the 6 pool with KerasCV Taking KerasCV for a spin Maps for your next trip to KerasCV 0 50 75 25
18. Accelerating ML with Keras - Developed by François Chollet in 2015 - Started as part of a research project - High-level API built on top of TensorFlow, Theano, Caffe etc. - Default TensorFlow backend - Fully Free and Open Source
19. Features in Keras Why consider it as your first (and last) ML Framework? - Large Community: Built from the ground up by the community, folks at Google and beyond - Strong documentation support: Among the most concise documentations on the internet, along with guides and examples for all use-cases (literally.) - Production ready: Made for full compatibility with TFX ecosystem, leading to use with TF.js, TFLite, TensorFlow Serving, and more - Extensible: You can implement your own activations, layers, models, and more using existing base building blocks for everything!
20. Things to cover today What is Computer Vision? Convoluting your way into CV Pay A ention. You’re going to need it Going into the 3 pool with Keras Who is KerasCV? Diving into the 6 pool with KerasCV Taking KerasCV for a spin Maps for your next trip to KerasCV 0 50 75 25
21. KerasCV for the Young For the eager beginners looking to start with Computer Vision - Library of Vision-oriented Keras components - Supports large number of latest features like Mixed-precision training, XLA-compiling, TPUs, and more - Make Vision tools for everything, be it industry-grade deployments, tech for onboard GPUs, mobile phones or edge-devices - Get pre-trained components with out-of-the-box support and awesome documentation
22. KerasCV for the Curious For the people who want to start building right away - Support for models like StableDiffusion (v1 & 2), YOLO, CSPDarknet, EfficientNet, and more - Other vision tasks are now first-class citizens with full support for bounding boxes and segmentation map utilities. - Modular API to mix-and-match components - Extensive guides and examples available to start with
23. Things to cover today What is Computer Vision? Convoluting your way into CV Pay A ention. You’re going to need it Going into the 3 pool with Keras Who is KerasCV? Diving into the 6 pool with KerasCV Taking KerasCV for a spin Maps for your next trip to KerasCV 0 50 75 25
24. KerasCV for Image Augmentation Making your models robust to change. - Latest layers like AugMix, CutMix, FourierMix and more - Confused? Use RandAugment! - Support for bounding boxes!
25. KerasCV for Object Detection Detect as far as the camera can see. - Full support for YOLOv8 and RetinaNet - 2D and 3D modalities supported - Newer models in the pipeline!
26. KerasCV for Image Segmentation Everything everywhere all at once. - Rapid development towards supporting Segmentation models - Powerful visualization APIs already present (shoutout to Aritra for this!)
27. KerasCV for Image Generation Let your imagination run wild. - Support for pre-trained Stable Diffusion (v1 and v2) - Elaborate documentation on how to play with it - Extend to Textual Inversion or DreamBooth with support from 🤗
28. Things to cover today What is Computer Vision? Convoluting your way into CV Pay A ention. You’re going to need it Going into the 3 pool with Keras Who is KerasCV? Diving into the 6 pool with KerasCV Taking KerasCV for a spin Maps for your next trip to KerasCV 0 50 75 25
29. Generating high-fidelity images with Stable Diffusion Because who wants to just see boxes in images, right? Right? Colab Notebook here Weights & Biases dashboard here
30. Things to cover today What is Computer Vision? Convoluting your way into CV Pay A ention. You’re going to need it Going into the 3 pool with Keras Who is KerasCV? Diving into the 6 pool with KerasCV Taking KerasCV for a spin Maps for your next trip to KerasCV 0 50 75 25
31. Resources Because these slides clearly aren’t enough to become a ‘VisionPro’ For the Young :- - keras.io Examples - PyImageSearch - Towards Data Science For the Curious :- - First Principles of Computer Vision (Columbia University) - KerasCV Documentation - Hugging Face Computer Vision Tasks
32. Things to cover today What is Computer Vision? Convoluting your way into CV Pay A ention. You’re going to need it Going into the 3 pool with Keras Who is KerasCV? Diving into the 6 pool with KerasCV Taking KerasCV for a spin Maps for your next trip with KerasCV 0 50 75 25
33. Thank you! Any questions? Wesley Kambale @weskambale kambale.dev
34. Original slides contributed by Suvaditya Mukherjee ML GDE | GDG MAD @halcyonrayes