Slide 1

Slide 1 text

SEMANTIC IMAGE SEARCH Ade Oshineye

Slide 2

Slide 2 text

www.oshineye.com/+

Slide 3

Slide 3 text

www.oshineye.com/+

Slide 4

Slide 4 text

WHAT IS GOOGLE?

Slide 5

Slide 5 text

THE MISSION & THE TOOLS “Google’s mission is to organize the world’s information and make it universally accessible and useful.”

Slide 6

Slide 6 text

MACHINE LEARNING Learning by example not learning by programming Here are some features Classify and generalise

Slide 7

Slide 7 text

MACHINE LEARNING: TERMINOLOGY 1 Features 2 Training 3 Clustering 4 Classification

Slide 8

Slide 8 text

MACHINE LEARNING: TERMINOLOGY 1 Supervised learning 2 Unsupervised learning 3 Semi-supervised learning 4 Deep learning

Slide 9

Slide 9 text

HOW WOULD YOU BUILD IMAGE SEARCH?

Slide 10

Slide 10 text

SEARCHING FOR RED PHOTOS

Slide 11

Slide 11 text

SEARCHING FOR GREEN PHOTOS

Slide 12

Slide 12 text

SEARCHING FOR YELLOW PHOTOS

Slide 13

Slide 13 text

CONVENTIONALLY 1 Identify feature 2 Write feature extractor

Slide 14

Slide 14 text

SEARCHING FOR BORIS BIKES Californians wouldn’t know about these

Slide 15

Slide 15 text

SEARCHING FOR PARIS & OTHER CITIES 3400 cities with 100k population

Slide 16

Slide 16 text

!SCALE The world’s information...

Slide 17

Slide 17 text

DEEP LEARNING: PROBLEM STATEMENT 1 Too many features 2 Too many labels 3 Too much data

Slide 18

Slide 18 text

DEEP LEARNING: SOLUTION 1 Unsupervised (mostly) 2 Machine learning for feature extractor 3 Hierarchy of features and pre-training 4 Machine learning for classifier Based on applying academic work at unprecedented scale

Slide 19

Slide 19 text

2K classes from Freebase 5K images per class for training

Slide 20

Slide 20 text

GENERALISE Infer structure and patterns

Slide 21

Slide 21 text

Skipping over a ton of other work by other teams and systems to turn this into a useful production service

Slide 22

Slide 22 text

CHALLENGES 1 Feedback 2 Finding applications 3 The Highlander test and face recognition 4 The screwdriver test and ethics

Slide 23

Slide 23 text

ACCESSIBLE AND USEFUL “We systematically overestimate the value of access to information and underestimate the value of access to each other” Clay Shirky

Slide 24

Slide 24 text

THE FUTURE IS UNWRITTEN

Slide 25

Slide 25 text

THANK YOU