A Short History of Computer Vision

This Slide Deck takes us through the history of Computer Vision by tracking one task through time: classifying and detecting a bird in an image. Starting from KeyPoint Detection, through Convolutional Neural Networks, Transformers all the way to modern multimodal models and their capabibilites, we can see a steady development in the models' abilities.

Transcript

1. A Short History of Computer Vision by Johannes Kolbe 09/11/2024
2. About Me Johannes Kolbe • Data Scientist at ◦ Focus on Computer Vision ◦ Some Expertise in NLP • M.Sc. in Computer Science at TU Berlin • Automotive Systems Research Background • Conference Speaker (PyCon Africa, EuroPython, …) • Hugging Face Fellow @johko@sigmoid.social linkedin.com/in/johko github.com/johko huggingface.co/johko
3. Overview 1. From SIFT to AlexNet 2. Convolutional Neural Networks 3. Transformers 4. Multimodality 5. The Future
4. The Bird Problem - 2014
5. The Bird Problem - 2024 Classify 9000 animals Share Photos/Videos via App Take HD Photos/Videos Swarovski Optik AX Visio
7. The “Old” Computer Vision SVM Keypoints Clustering Feature Vector
8. Haar Cascades - Object Detection Source: https://www.cs.cmu.edu/~efros/courses/LBMV07/Papers/viola-cvpr-01.pdf
9. Haar Cascades in Action
10. Keypoint Detectors Haar Cascades Generalization Flexibility Robustness Problems
11. The Revolution - AlexNet (2012)
12. A Short History of CNNs 1979 Necognitron 1989 LeNet-1 Kunihiko Fukushima Yann LeCun 1994 Convolutional Dynamic Systems 1998 LeNet-5 Yann LeCun 2012 AlexNet Krizhevsky, Sutskever, Hinton Rummelhart, Hinton, Williams
13. General CNN Idea Source: https://github.com/helblazer811/ManimML
14. LeNet-5 Source: http://vision.stanford.edu/cs598_spring07/papers/Lecun98.pdf
15. AlexNet - Architecture Source: https://papers.nips.cc/paper_files/paper/2012/file/c399862d3b9d6b76c8436e924a68c45b-Paper.pdf
16. Convolutional Neural Networks Source: https://papers.nips.cc/paper_files/paper/2012/file/c399862d3b9d6b76c8436e924a68c45b-Paper.pdf Haar Cascades:
17. Alex and the Bird
18. Scaling
19. The CNN Boom 2014 GoogLeNet 2014 VGG Christian Szegedy et al. Simonyan, Zisserman 2015 ResNet 2017 MobileNet Howard Andrew et al. 2019 EfficientNet Tan, Le Kaiming He et al. 2022 ConvNext Zhuang Liu et al.
20. Model Powerhouses Source: https://paperswithcode.com/method/resnet
21. Resnet Maybe?
22. CNNs for everyone! Classification Detection Segmentation Bird 0.965
23. CNNs for everyone! Classification Detection Segmentation VGG ResNet Efficient Net Mobile Net ConvNext Mask R-CNN DeepLab UNet Faster R-CNN YOLO SSD RetinaNet FCN
24. Which Domain? Every Domain! CNN Augmented and Virtual Reality Aerial and Satellite Imagery Analysis Manufacturing and Industrial Automation Security and Surveillance Autonomous Vehicles Medical Imaging Multimedia Content Analysis Social Media Analytics Financial Services Retail and E-commerce Text Mining and Natural Language Processing
25. NLP “Natural language processing (NLP) is the discipline of building machines that can manipulate human language — or data that resembles human language — in the way that it is written, spoken, and organized.” deeplearning.ai
26. Transformers
27. Transformers CV People
28. Self Attention I like kea parrots q1 k1 v1 q2 k2 v2 q3 k3 v3 q4 k4 v4 z3
29. ViT - Vision Transformer
30. ViT - Vision Transformer Linear Projection CLS Transformer Encoder MLP Class
31. ViT - Vision Transformer
32. ViT - The Takeover Source: https://paperswithcode.com/method/vision-transformer
33. Transformers for everyone! Classification Detection Segmentation ViT Swin DEiT MAE DiNO Mask Former SAM DETR Mask DiNO YOLOS Deformable DETR SegFormer
34. Transformers for Everything Source: https://paperswithcode.com/method/swin-transformer
35. CNN Trans former Parameter Efficiency Global Context Few-Shot Adaption Interpretability Multimodality Industry Adoption Local Receptive Fields Training time
36. Multimodality created with ideogram.ai
37. Contrastive Language Image Pre-Training (CLIP) Source: https://arxiv.org/pdf/2103.00020.pdf
38. Contrastive Language Image Pre-Training (CLIP)
39. The Rise of CLIP Source: https://paperswithcode.com/method/clip
40. OwlViT / OwlV2 Source: https://arxiv.org/pdf/2205.06230
41. OwlViT / OwlV2
42. Multimodal Large Language Models Source: https://arxiv.org/pdf/2405.08603
43. Qwen2-VL Source: https://arxiv.org/pdf/2409.12191
44. Qwen2-VL Source: https://arxiv.org/pdf/2409.12191
45. Qwen2-VL
47. KOSMOS-2 Source: https://arxiv.org/pdf/2306.14824
48. KOSMOS-2
49. Vision Transformer + LLM =
51. Florence-2
52. Florence-2
53. Florence-2
54. Florence-2
55. Florence and the (Bird) Machine
56. THE FUTURE
57. The Future Source: https://arxiv.org/abs/2210.00881
58. The Future Source: https://arxiv.org/abs/2210.00881
59. JEPA RTM MAMBA Capsule Networks GFlow Networks
60. Thank You!