What does your Instagram say about you? Exploring Google Cloud Vision

@tech_christine @ryanhos What does your Christine Seeman and Ryan Hochstetler
say about you?

@tech_christine @ryanhos What is in an image?

@tech_christine @ryanhos What do you focus on?

@tech_christine @ryanhos What is in it that you see?

@tech_christine @ryanhos But what does an application see in them?

@tech_christine @ryanhos Meet Google Cloud Vision

@tech_christine @ryanhos

@tech_christine @ryanhos Little bit different vision

@tech_christine @ryanhos Cloud Vision API Landmarks Face detection Image labeling
Optical character recognition Explicit content detection Logo detection Web entities Crop hint detection

@tech_christine @ryanhos But how can we use it?

@tech_christine @ryanhos Let's look at the client library

@tech_christine @ryanhos Let's combine accounts with

@tech_christine @ryanhos So that got us from this

@tech_christine @ryanhos To this

@tech_christine @ryanhos Aggregate the results into histograms Labels Landmarks Logos

@tech_christine @ryanhos Then finally the tag cloud

@tech_christine @ryanhos Let's explore the different detections

@tech_christine @ryanhos Face detection

@tech_christine @ryanhos What’s a “gnathion”?

@tech_christine @ryanhos Sentiment Detection Probably should be called “Sentiment Best
Guestimation”

@tech_christine @ryanhos Joy

@tech_christine @ryanhos Surprise

@tech_christine @ryanhos Anger?

@tech_christine @ryanhos Sorrow?

@tech_christine @ryanhos Image Composition

@tech_christine @ryanhos Label detection

@tech_christine @ryanhos @expertvagabond

@tech_christine @ryanhos @ripleyandrue

@tech_christine @ryanhos Safe Search detection

@tech_christine @ryanhos Yes it knows if it is a hotdog

@tech_christine @ryanhos ...and when it's not

@tech_christine @ryanhos Seems accurate…

@tech_christine @ryanhos Oh, there might be a bit of bias…

@tech_christine @ryanhos Violence is difficult too…

@tech_christine @ryanhos Racy but not violent?!?

@tech_christine @ryanhos Doctored images are hit and miss We chose
not to include the image of a former US president photoshopped to look like a dictator and war criminal. Google labeled it “Possibly” a spoof; i.e. altered to be funny or oﬀensive

@tech_christine @ryanhos Text detection

@tech_christine @ryanhos @letterfolk

@tech_christine @ryanhos @letterfolk MY BRAIN HAS TO0 MANY TABS OPEN
Dechtlatte THIS WEEK pt de C CAit s pg

@tech_christine @ryanhos @letterfolk YOURE NOT THE BOSS OF ME.. IWHISPER
UNDER MY BREATH AS I CLEAN UP ALL OF MY CHILDREN'S MESSES. TRE

@tech_christine @ryanhos @letterfolk Seth Elizabeth Mat IT'S BEGINNING TO amazon
Prime LOOK A LOT LIKE... vmazon Prime echo I GOT MY MONEY'S WORTH FROM MY 1t\nare en a ton Prime AMAZON PRIME MEMBERSHIP Nany echo eciro

@tech_christine @ryanhos Logo detection

@tech_christine @ryanhos @nike

@tech_christine @ryanhos @nike 1. nike 2. nike plus 3. nike
azul 4. blue nike 5. nike store

@tech_christine @ryanhos @christine_seeman

@tech_christine @ryanhos @christine_seeman 1. etelä suomen sanomat 2. rogue status
3. american horror story 4. delta skymiles 5. deník !"!"

@tech_christine @ryanhos Landmark detection

@tech_christine @ryanhos @visit_nebraska

@tech_christine @ryanhos @iloveny

@tech_christine @ryanhos @expertvagabond

@tech_christine @ryanhos Have you been to London, NE?

@tech_christine @ryanhos The British Museum there is lovely

@tech_christine @ryanhos British Museum, alternate photo

@tech_christine @ryanhos So much alike

@tech_christine @ryanhos This wasn’t even a building…

@tech_christine @ryanhos But what does my instagram say about me?

@tech_christine @ryanhos @christineseeman

@tech_christine @ryanhos Blooper cloud

@tech_christine @ryanhos Cloud AutoML Vision •Train ML models to classify
images •Use your own deﬁned labels. •Graphical user interface to train, evaluate, improve, and deploy models

@tech_christine @ryanhos @visit_nebraska

@tech_christine @ryanhos Vs @visitcalifornia

@tech_christine @ryanhos Vs @iloveny

@tech_christine @ryanhos Sydney, Australia  Bangkok, Thailand Paris, France

@tech_christine @ryanhos Evaluating Model Accuracy

@tech_christine @ryanhos Confusion Matrix

@tech_christine @ryanhos Skyline Predictions

@tech_christine @ryanhos More training required…

@tech_christine @ryanhos Urban Scenery Predictions

@tech_christine @ryanhos Outdoor Scenery Predictions

@tech_christine @ryanhos Where to go from here? What to do
next with Google Cloud Vision

@tech_christine @ryanhos Explore bias in Machine Learning?

@tech_christine @ryanhos How much will this cost you? For us,
about $65

@tech_christine @ryanhos 905.33!

@tech_christine @ryanhos $40 for the Training Compute Hours 

@tech_christine @ryanhos Technology Used •Ruby (No Rails) •Rspec •Ruby Vision
API Client Libraries •Google Cloud Storage •Rmagick (Ruby binding to Imagemagick) •MagicCloud tag cloud gem

@tech_christine @ryanhos All the code https://github.com/hi-christine/cloud-vision-insta

@tech_christine @ryanhos Reference Links •https://cloud.google.com/vision/ •https://cloud.google.com/vision/docs/ •https://googleapis.github.io/google-cloud-ruby/docs/ •https://github.com/GoogleCloudPlatform/ruby-docs- samples/blob/master/vision/quickstart.rb •https://github.com/zverok/magic_cloud

@tech_christine @ryanhos All the instagram accounts* @ripleyandrue @visit_nebraska @wolffolins @expertvagabond
@iloveny @letterfolk @myraswim @visitcalifornia @nationalportraitgallery @nike * and none were harmed in the making of this talk

@tech_christine @ryanhos getﬂywheel.com/about/careers

@tech_christine @ryanhos Thank you for attending!

@tech_christine @ryanhos For your evaluation consideration...

@tech_christine @ryanhos Bonus Code Time

@tech_christine @ryanhos image_annotator_client.rb

@tech_christine @ryanhos analysis_reader.rb

@tech_christine @ryanhos raster_annotator.rb

What does your Instagram say about you? Explori...

What does your Instagram say about you? Exploring Google Cloud Vision

More Decks by Christine

Other Decks in Technology

Featured

Transcript