What does your Instagram say about you? Exploring Google Cloud Vision

9d7385d9e0c45a30c081fbbf53136e2d?s=47 Christine
September 12, 2019

What does your Instagram say about you? Exploring Google Cloud Vision

Prairie.code() Des Moines, IA September 12th 2019

Explore application development with Google Cloud Vision API, which can evaluate a photo and extract categories with what is going in it. From there, the sky's the limit with exploring the detected labels. We'll check out different photo sharing accounts, and see what we can know about them all through their photos. This will use Google Cloud Platform, and Ruby, with JSON formatted labels that are translated into tag cloud with what labels are being displayed through the images and shows a great cross-section of different technologies.

9d7385d9e0c45a30c081fbbf53136e2d?s=128

Christine

September 12, 2019
Tweet

Transcript

  1. @tech_christine @ryanhos What does your Christine Seeman and Ryan Hochstetler

    say about you?
  2. @tech_christine @ryanhos What is in an image?

  3. @tech_christine @ryanhos What is in an image?

  4. @tech_christine @ryanhos What do you focus on?

  5. @tech_christine @ryanhos What do you focus on?

  6. @tech_christine @ryanhos What is in it that you see?

  7. @tech_christine @ryanhos What is in it that you see?

  8. @tech_christine @ryanhos But what does an application see in them?

  9. @tech_christine @ryanhos Meet Google Cloud Vision

  10. @tech_christine @ryanhos

  11. @tech_christine @ryanhos

  12. @tech_christine @ryanhos Little bit different vision

  13. @tech_christine @ryanhos Cloud Vision API Landmarks Face detection Image labeling

    Optical character recognition Explicit content detection Logo detection Web entities Crop hint detection
  14. @tech_christine @ryanhos But how can we use it?

  15. @tech_christine @ryanhos Let's look at the client library

  16. @tech_christine @ryanhos Let's combine accounts with

  17. @tech_christine @ryanhos

  18. @tech_christine @ryanhos So that got us from this

  19. @tech_christine @ryanhos To this

  20. @tech_christine @ryanhos Aggregate the results into histograms Labels Landmarks Logos

  21. @tech_christine @ryanhos Then finally the tag cloud

  22. @tech_christine @ryanhos Let's explore the different detections

  23. @tech_christine @ryanhos Face detection

  24. @tech_christine @ryanhos What’s a “gnathion”?

  25. @tech_christine @ryanhos Sentiment Detection Probably should be called “Sentiment Best

    Guestimation”
  26. @tech_christine @ryanhos Joy

  27. @tech_christine @ryanhos Surprise

  28. @tech_christine @ryanhos Anger?

  29. @tech_christine @ryanhos Sorrow?

  30. @tech_christine @ryanhos Image Composition

  31. @tech_christine @ryanhos Label detection

  32. @tech_christine @ryanhos @expertvagabond

  33. @tech_christine @ryanhos @expertvagabond

  34. @tech_christine @ryanhos @ripleyandrue

  35. @tech_christine @ryanhos @ripleyandrue

  36. @tech_christine @ryanhos Safe Search detection

  37. @tech_christine @ryanhos Yes it knows if it is a hotdog

  38. @tech_christine @ryanhos ...and when it's not

  39. @tech_christine @ryanhos Seems accurate…

  40. @tech_christine @ryanhos Oh, there might be a bit of bias…

  41. @tech_christine @ryanhos Violence is difficult too…

  42. @tech_christine @ryanhos Racy but not violent?!?

  43. @tech_christine @ryanhos Doctored images are hit and miss We chose

    not to include the image of a former US president photoshopped to look like a dictator and war criminal. Google labeled it “Possibly” a spoof; i.e. altered to be funny or offensive
  44. @tech_christine @ryanhos Text detection

  45. @tech_christine @ryanhos @letterfolk

  46. @tech_christine @ryanhos @letterfolk MY BRAIN HAS TO0 MANY TABS OPEN

    Dechtlatte THIS WEEK pt de C CAit s pg
  47. @tech_christine @ryanhos @letterfolk

  48. @tech_christine @ryanhos @letterfolk YOURE NOT THE BOSS OF ME.. IWHISPER

    UNDER MY BREATH AS I CLEAN UP ALL OF MY CHILDREN'S MESSES. TRE
  49. @tech_christine @ryanhos @letterfolk

  50. @tech_christine @ryanhos @letterfolk Seth Elizabeth Mat IT'S BEGINNING TO amazon

    Prime LOOK A LOT LIKE... vmazon Prime echo I GOT MY MONEY'S WORTH FROM MY 1t\nare en a ton Prime AMAZON PRIME MEMBERSHIP Nany echo eciro
  51. @tech_christine @ryanhos Logo detection

  52. @tech_christine @ryanhos

  53. @tech_christine @ryanhos @nike

  54. @tech_christine @ryanhos @nike

  55. @tech_christine @ryanhos @nike 1. nike 2. nike plus 3. nike

    azul 4. blue nike 5. nike store
  56. @tech_christine @ryanhos @christine_seeman

  57. @tech_christine @ryanhos @christine_seeman

  58. @tech_christine @ryanhos @christine_seeman 1. etelä suomen sanomat 2. rogue status

    3. american horror story 4. delta skymiles 5. deník !"!"
  59. @tech_christine @ryanhos Landmark detection

  60. @tech_christine @ryanhos @visit_nebraska

  61. @tech_christine @ryanhos @iloveny

  62. @tech_christine @ryanhos @iloveny

  63. @tech_christine @ryanhos @expertvagabond

  64. @tech_christine @ryanhos Have you been to London, NE?

  65. @tech_christine @ryanhos The British Museum there is lovely

  66. @tech_christine @ryanhos British Museum, alternate photo

  67. @tech_christine @ryanhos So much alike

  68. @tech_christine @ryanhos This wasn’t even a building…

  69. @tech_christine @ryanhos But what does my instagram say about me?

  70. @tech_christine @ryanhos

  71. @tech_christine @ryanhos @christineseeman

  72. @tech_christine @ryanhos Blooper cloud

  73. @tech_christine @ryanhos Cloud AutoML Vision •Train ML models to classify

    images •Use your own defined labels. •Graphical user interface to train, evaluate, improve, and deploy models
  74. @tech_christine @ryanhos @visit_nebraska

  75. @tech_christine @ryanhos Vs @visitcalifornia

  76. @tech_christine @ryanhos Vs @iloveny

  77. @tech_christine @ryanhos Sydney, Australia
 Bangkok, Thailand Paris, France

  78. @tech_christine @ryanhos

  79. @tech_christine @ryanhos Evaluating Model Accuracy

  80. @tech_christine @ryanhos Confusion Matrix

  81. @tech_christine @ryanhos Skyline Predictions

  82. @tech_christine @ryanhos More training required…

  83. @tech_christine @ryanhos Urban Scenery Predictions

  84. @tech_christine @ryanhos Outdoor Scenery Predictions

  85. @tech_christine @ryanhos Where to go from here? What to do

    next with Google Cloud Vision
  86. @tech_christine @ryanhos Explore bias in Machine Learning?

  87. @tech_christine @ryanhos How much will this cost you? For us,

    about $65
  88. @tech_christine @ryanhos 905.33!

  89. @tech_christine @ryanhos $40 for the Training Compute Hours


  90. @tech_christine @ryanhos Technology Used •Ruby (No Rails) •Rspec •Ruby Vision

    API Client Libraries •Google Cloud Storage •Rmagick (Ruby binding to Imagemagick) •MagicCloud tag cloud gem
  91. @tech_christine @ryanhos All the code https://github.com/hi-christine/cloud-vision-insta

  92. @tech_christine @ryanhos Reference Links •https://cloud.google.com/vision/ •https://cloud.google.com/vision/docs/ •https://googleapis.github.io/google-cloud-ruby/docs/ •https://github.com/GoogleCloudPlatform/ruby-docs- samples/blob/master/vision/quickstart.rb •https://github.com/zverok/magic_cloud

  93. @tech_christine @ryanhos All the instagram accounts* @ripleyandrue @visit_nebraska @wolffolins @expertvagabond

    @iloveny @letterfolk @myraswim @visitcalifornia @nationalportraitgallery @nike * and none were harmed in the making of this talk
  94. @tech_christine @ryanhos getflywheel.com/about/careers

  95. @tech_christine @ryanhos Thank you for attending!

  96. @tech_christine @ryanhos For your evaluation consideration...

  97. @tech_christine @ryanhos Bonus Code Time

  98. @tech_christine @ryanhos image_annotator_client.rb

  99. @tech_christine @ryanhos image_annotator_client.rb

  100. @tech_christine @ryanhos image_annotator_client.rb

  101. @tech_christine @ryanhos image_annotator_client.rb

  102. @tech_christine @ryanhos analysis_reader.rb

  103. @tech_christine @ryanhos analysis_reader.rb

  104. @tech_christine @ryanhos raster_annotator.rb

  105. @tech_christine @ryanhos raster_annotator.rb