What does your Instagram say about you? Exploring Google Cloud Vision

@tech_christine @ryanhos What does your Christine Seeman and Ryan Hochstetler
say about you?

@tech_christine @ryanhos What is in an image?

@tech_christine @ryanhos What do you focus on?

@tech_christine @ryanhos What is in it that you see?

@tech_christine @ryanhos But what does an application see in them?

@tech_christine @ryanhos Meet Google Cloud Vision

@tech_christine @ryanhos

@tech_christine @ryanhos Little bit different vision

@tech_christine @ryanhos Cloud Vision API Landmarks Face detection Image labeling
Optical character recognition Explicit content detection Logo detection Object localization Crop hint detection

@tech_christine @ryanhos But how can we use it?

@tech_christine @ryanhos Let's look at the client library

@tech_christine @ryanhos Let's combine accounts with

@tech_christine @ryanhos Onto the code

@tech_christine @ryanhos image_annotator_client.rb

@tech_christine @ryanhos So that got us from this

@tech_christine @ryanhos To this

@tech_christine @ryanhos analysis_reader.rb

@tech_christine @ryanhos After histogram generation Labels Landmarks Logos

@tech_christine @ryanhos Then finally the tag cloud

@tech_christine @ryanhos Let's explore the different detections

@tech_christine @ryanhos Face detection

@tech_christine @ryanhos What’s a “gnathion”?

@tech_christine @ryanhos raster_annotator.rb

@tech_christine @ryanhos Sentiment Detection Probably should be called “Sentiment Best
Guestimation”

@tech_christine @ryanhos Joy

@tech_christine @ryanhos Surprise

@tech_christine @ryanhos Anger?

@tech_christine @ryanhos Sorrow?

@tech_christine @ryanhos Image Composition

@tech_christine @ryanhos Label detection

@tech_christine @ryanhos @expertvagabond

@tech_christine @ryanhos @ripleyandrue

@tech_christine @ryanhos Safe Search detection

@tech_christine @ryanhos Yes it knows if it is a hotdog

@tech_christine @ryanhos ...and when it's not

@tech_christine @ryanhos Seems accurate…

@tech_christine @ryanhos Oh, there might be a bit of bias…

@tech_christine @ryanhos Violence is difficult too…

@tech_christine @ryanhos Racy but not violent?!?

@tech_christine @ryanhos Doctored images are hit and miss We chose
not to include the image of a former US president photoshopped to look like a dictator and war criminal. Google labeled it “Possibly” a spoof; i.e. altered to be funny or oﬀensive

@tech_christine @ryanhos Text detection

@tech_christine @ryanhos @letterfolk

@tech_christine @ryanhos @letterfolk "MY\nBRAIN\nHAS\nTO0\nMANY\nTABS\nOPEN\nDechtlatte\nTHIS WEEK\npt de C\nCAit s pg\n" MY
BRAIN HAS TO0 MANY TABS OPEN Dechtlatte THIS WEEK pt de C CAit s pg

@tech_christine @ryanhos @letterfolk YOURE NOT THE\nBOSS OF ME..\nIWHISPER UNDER\nMY BREATH
AS I\nCLEAN UP ALL OF\nMY CHILDREN'S\nMESSES.\nTRE\n YOURE NOT THE BOSS OF ME.. IWHISPER UNDER MY BREATH AS I CLEAN UP ALL OF MY CHILDREN'S MESSES. TRE

@tech_christine @ryanhos @letterfolk Seth\nElizabeth\nMat\nIT'S BEGINNING TO\namazon Prime\nLOOK A LOT LIKE...\nvmazon
Prime\necho\nI GOT MY MONEY'S\nWORTH FROM MY\n1t\nare en\na ton Prime\nAMAZON PRIME\nMEMBERSHIP\nNany\necho\neciro\n Seth Elizabeth Mat IT'S BEGINNING TO amazon Prime LOOK A LOT LIKE... vmazon Prime echo I GOT MY MONEY'S WORTH FROM MY 1t\nare en a ton Prime AMAZON PRIME MEMBERSHIP Nany echo eciro

@tech_christine @ryanhos Logo detection

@tech_christine @ryanhos @nike

@tech_christine @ryanhos @nike 1. nike 2. nike plus 3. nike
azul 4. blue nike 5. nike store

@tech_christine @ryanhos @christine_seeman

@tech_christine @ryanhos @christine_seeman 1. etelä suomen sanomat 2. rogue status
3. american horror story 4. delta skymiles 5. deník !"!"

@tech_christine @ryanhos Landmark detection

@tech_christine @ryanhos @visit_nebraska

@tech_christine @ryanhos @iloveny

@tech_christine @ryanhos @expertvagabond

@tech_christine @ryanhos Have you been to London, NE?

@tech_christine @ryanhos The British Museum there is lovely

@tech_christine @ryanhos British Museum, alternate photo

@tech_christine @ryanhos So much alike

@tech_christine @ryanhos This wasn’t even a building…

@tech_christine @ryanhos But what does my instagram say about me?

@tech_christine @ryanhos @christineseeman

@tech_christine @ryanhos Blooper cloud

@tech_christine @ryanhos Where to go from here? What to do
next with Google Cloud Vision

@tech_christine @ryanhos @visit_nebraska

@tech_christine @ryanhos Vs @visitcalifornia

@tech_christine @ryanhos Vs @iloveny

@tech_christine @ryanhos Can we teach a machine to know if
a pic is from NY vs CA vs NE

@tech_christine @ryanhos Explore bias in Machine Learning?

@tech_christine @ryanhos How much will this cost you? For us,
about $65

@tech_christine @ryanhos Technology Used •Ruby (No Rails) •Rspec •Ruby Vision
API Client Libraries •Google Cloud Storage •Rmagick (Ruby binding to Imagemagick) •MagicCloud tag cloud gem

@tech_christine @ryanhos Reference Links •https://cloud.google.com/vision/ •https://cloud.google.com/vision/docs/ •https://googleapis.github.io/google-cloud-ruby/docs/ •https://github.com/GoogleCloudPlatform/ruby-docs- samples/blob/master/vision/quickstart.rb •https://github.com/zverok/magic_cloud

@tech_christine @ryanhos All the instagram accounts* @ripleyandrue @visit_nebraska @wolffolins @expertvagabond
@iloveny @letterfolk @myraswim @visitcalifornia @nationalportraitgallery @nike * and none were harmed in the making of this talk

@tech_christine @ryanhos getﬂywheel.com/about/careers

@tech_christine @ryanhos Thank you for attending!

@tech_christine @ryanhos For your evaluation consideration...

What does your Instagram say about you? Explori...

What does your Instagram say about you? Exploring Google Cloud Vision

More Decks by Christine

Other Decks in Programming

Featured

Transcript