What does your Instagram say about you? Exploring Google Cloud Vision

@tech_christine @ryanhos What does your Christine Seeman and Ryan Hochstetler
say about you?

@tech_christine @ryanhos What is in an image?

@tech_christine @ryanhos What do you focus on?

@tech_christine @ryanhos What is in it that you see?

@tech_christine @ryanhos But what does an application see in them?

@tech_christine @ryanhos Meet Google Cloud Vision

@tech_christine @ryanhos

@tech_christine @ryanhos Little bit different vision

@tech_christine @ryanhos Cloud Vision API Landmarks Face detection Image labeling
Optical character recognition Explicit content detection Logo detection Web entities Crop hint detection

@tech_christine @ryanhos But how can we use it?

@tech_christine @ryanhos Let's look at the client library

@tech_christine @ryanhos Let's combine accounts with

@tech_christine @ryanhos So that got us from this

@tech_christine @ryanhos To this

@tech_christine @ryanhos Aggregate the results into histograms Labels Landmarks Logos

@tech_christine @ryanhos Then finally the tag cloud

@tech_christine @ryanhos Let's explore the different detections

@tech_christine @ryanhos Face detection

@tech_christine @ryanhos What’s a “gnathion”?

@tech_christine @ryanhos Sentiment Detection Probably should be called “Sentiment Best
Guestimation”

@tech_christine @ryanhos Joy

@tech_christine @ryanhos Surprise

@tech_christine @ryanhos Anger?

@tech_christine @ryanhos Sorrow?

@tech_christine @ryanhos Image Composition

@tech_christine @ryanhos Label detection

@tech_christine @ryanhos @expertvagabond

@tech_christine @ryanhos @ripleyandrue

@tech_christine @ryanhos Safe Search detection

@tech_christine @ryanhos Yes it knows if it is a hotdog

@tech_christine @ryanhos ...and when it's not

@tech_christine @ryanhos Seems accurate…

@tech_christine @ryanhos Oh, there might be a bit of bias…

@tech_christine @ryanhos Violence is difficult too…

@tech_christine @ryanhos Racy but not violent?!?

@tech_christine @ryanhos Doctored images are hit and miss We chose
not to include the image of a former US president photoshopped to look like a dictator and war criminal. Google labeled it “Possibly” a spoof; i.e. altered to be funny or oﬀensive

@tech_christine @ryanhos Text detection

@tech_christine @ryanhos @letterfolk

@tech_christine @ryanhos @letterfolk MY BRAIN HAS TO0 MANY TABS OPEN
Dechtlatte THIS WEEK pt de C CAit s pg

@tech_christine @ryanhos @letterfolk YOURE NOT THE BOSS OF ME.. IWHISPER
UNDER MY BREATH AS I CLEAN UP ALL OF MY CHILDREN'S MESSES. TRE

@tech_christine @ryanhos @letterfolk Seth Elizabeth Mat IT'S BEGINNING TO amazon
Prime LOOK A LOT LIKE... vmazon Prime echo I GOT MY MONEY'S WORTH FROM MY 1t\nare en a ton Prime AMAZON PRIME MEMBERSHIP Nany echo eciro

@tech_christine @ryanhos Logo detection

@tech_christine @ryanhos @nike

@tech_christine @ryanhos @nike 1. nike 2. nike plus 3. nike
azul 4. blue nike 5. nike store

@tech_christine @ryanhos @christine_seeman

@tech_christine @ryanhos @christine_seeman 1. etelä suomen sanomat 2. rogue status
3. american horror story 4. delta skymiles 5. deník !"!"

@tech_christine @ryanhos Landmark detection

@tech_christine @ryanhos @visit_nebraska

@tech_christine @ryanhos @iloveny

@tech_christine @ryanhos @expertvagabond

@tech_christine @ryanhos Have you been to London, NE?

@tech_christine @ryanhos The British Museum there is lovely

@tech_christine @ryanhos British Museum, alternate photo

@tech_christine @ryanhos So much alike

@tech_christine @ryanhos This wasn’t even a building…

@tech_christine @ryanhos But what does my instagram say about me?

@tech_christine @ryanhos @christineseeman

@tech_christine @ryanhos Blooper cloud

@tech_christine @ryanhos Cloud AutoML Vision •Train ML models to classify
images •Use your own deﬁned labels. •Graphical user interface to train, evaluate, improve, and deploy models

@tech_christine @ryanhos @visit_nebraska

@tech_christine @ryanhos Vs @visitcalifornia

@tech_christine @ryanhos Vs @iloveny

@tech_christine @ryanhos Sydney, Australia  Bangkok, Thailand Paris, France

@tech_christine @ryanhos Evaluating Model Accuracy

@tech_christine @ryanhos Confusion Matrix

@tech_christine @ryanhos Skyline Predictions

@tech_christine @ryanhos More training required…

@tech_christine @ryanhos Urban Scenery Predictions

@tech_christine @ryanhos Outdoor Scenery Predictions

@tech_christine @ryanhos Where to go from here? What to do
next with Google Cloud Vision

@tech_christine @ryanhos Explore bias in Machine Learning?

@tech_christine @ryanhos How much will this cost you? For us,
about $65

@tech_christine @ryanhos 905.33!

@tech_christine @ryanhos $40 for the Training Compute Hours 

@tech_christine @ryanhos Technology Used •Ruby (No Rails) •Rspec •Ruby Vision
API Client Libraries •Google Cloud Storage •Rmagick (Ruby binding to Imagemagick) •MagicCloud tag cloud gem

@tech_christine @ryanhos All the code https://github.com/hi-christine/cloud-vision-insta

@tech_christine @ryanhos Reference Links •https://cloud.google.com/vision/ •https://cloud.google.com/vision/docs/ •https://googleapis.github.io/google-cloud-ruby/docs/ •https://github.com/GoogleCloudPlatform/ruby-docs- samples/blob/master/vision/quickstart.rb •https://github.com/zverok/magic_cloud

@tech_christine @ryanhos All the instagram accounts* @ripleyandrue @visit_nebraska @wolffolins @expertvagabond
@iloveny @letterfolk @myraswim @visitcalifornia @nationalportraitgallery @nike * and none were harmed in the making of this talk

@tech_christine @ryanhos getﬂywheel.com/about/careers

@tech_christine @ryanhos Thank you for attending!

@tech_christine @ryanhos For your evaluation consideration...

@tech_christine @ryanhos Bonus Code Time

@tech_christine @ryanhos image_annotator_client.rb

@tech_christine @ryanhos analysis_reader.rb

@tech_christine @ryanhos raster_annotator.rb

What does your Instagram say about you? Exploring Google Cloud Vision

What does your Instagram say about you? Exploring Google Cloud Vision

More Decks by Christine

Other Decks in Technology

Featured

Transcript