$30 off During Our Annual Pro Sale. View Details »

What does your Instagram say about you? Exploring Google Cloud Vision

Christine
August 15, 2019

What does your Instagram say about you? Exploring Google Cloud Vision

Explore application development with Google Cloud Vision API which can categorize photos, detect objects, identify landmarks, and extract corporate logos from images. From there, the sky's the limit with exploring the detected labels. We'll check out different photo sharing accounts, and see what we can know about them all through their photos. This will use Google Cloud Platform, Ruby, with JSON formatted labels that are translated into a tag cloud with what activities are being displayed through the images and shows a cross-section of different technologies.

Christine

August 15, 2019
Tweet

More Decks by Christine

Other Decks in Programming

Transcript

  1. @tech_christine @ryanhos
    What does your
    Christine Seeman and Ryan Hochstetler
    say about you?

    View Slide

  2. @tech_christine @ryanhos
    What is in an image?

    View Slide

  3. @tech_christine @ryanhos
    What is in an image?

    View Slide

  4. @tech_christine @ryanhos
    What do you focus on?

    View Slide

  5. @tech_christine @ryanhos
    What do you focus on?

    View Slide

  6. @tech_christine @ryanhos
    What is in it that you see?

    View Slide

  7. @tech_christine @ryanhos
    What is in it that you see?

    View Slide

  8. @tech_christine @ryanhos
    But what does an
    application see in them?

    View Slide

  9. @tech_christine @ryanhos
    Meet Google Cloud
    Vision

    View Slide

  10. @tech_christine @ryanhos

    View Slide

  11. @tech_christine @ryanhos

    View Slide

  12. @tech_christine @ryanhos
    Little bit different
    vision

    View Slide

  13. @tech_christine @ryanhos
    Cloud Vision API
    Landmarks
    Face detection Image labeling
    Optical character recognition
    Explicit content detection
    Logo detection
    Object localization
    Crop hint detection

    View Slide

  14. @tech_christine @ryanhos
    But how can we use it?

    View Slide

  15. @tech_christine @ryanhos
    Let's look at the client library

    View Slide

  16. @tech_christine @ryanhos
    Let's combine
    accounts with

    View Slide

  17. @tech_christine @ryanhos

    View Slide

  18. @tech_christine @ryanhos
    Onto the code

    View Slide

  19. @tech_christine @ryanhos
    image_annotator_client.rb

    View Slide

  20. @tech_christine @ryanhos
    image_annotator_client.rb

    View Slide

  21. @tech_christine @ryanhos
    image_annotator_client.rb

    View Slide

  22. @tech_christine @ryanhos
    image_annotator_client.rb

    View Slide

  23. @tech_christine @ryanhos
    So that got us from this

    View Slide

  24. @tech_christine @ryanhos
    To this

    View Slide

  25. @tech_christine @ryanhos
    analysis_reader.rb

    View Slide

  26. @tech_christine @ryanhos
    analysis_reader.rb

    View Slide

  27. @tech_christine @ryanhos
    After histogram generation
    Labels
    Landmarks
    Logos

    View Slide

  28. @tech_christine @ryanhos
    Then finally the tag cloud

    View Slide

  29. @tech_christine @ryanhos
    Let's explore the
    different detections

    View Slide

  30. @tech_christine @ryanhos
    Face detection

    View Slide

  31. @tech_christine @ryanhos
    What’s a “gnathion”?

    View Slide

  32. @tech_christine @ryanhos
    raster_annotator.rb

    View Slide

  33. @tech_christine @ryanhos
    raster_annotator.rb

    View Slide

  34. @tech_christine @ryanhos
    Sentiment Detection
    Probably should be called “Sentiment Best Guestimation”

    View Slide

  35. @tech_christine @ryanhos
    Joy

    View Slide

  36. @tech_christine @ryanhos
    Surprise

    View Slide

  37. @tech_christine @ryanhos
    Anger?

    View Slide

  38. @tech_christine @ryanhos
    Sorrow?

    View Slide

  39. @tech_christine @ryanhos
    Image Composition

    View Slide

  40. @tech_christine @ryanhos
    Label detection

    View Slide

  41. @tech_christine @ryanhos
    @expertvagabond

    View Slide

  42. @tech_christine @ryanhos
    @expertvagabond

    View Slide

  43. @tech_christine @ryanhos
    @ripleyandrue

    View Slide

  44. @tech_christine @ryanhos
    @ripleyandrue

    View Slide

  45. @tech_christine @ryanhos
    Safe Search detection

    View Slide

  46. @tech_christine @ryanhos
    Yes it knows if it is a hotdog

    View Slide

  47. @tech_christine @ryanhos
    ...and when it's not

    View Slide

  48. @tech_christine @ryanhos
    Seems accurate…

    View Slide

  49. @tech_christine @ryanhos
    Oh, there might be a bit of bias…

    View Slide

  50. @tech_christine @ryanhos
    Violence is difficult too…

    View Slide

  51. @tech_christine @ryanhos
    Racy but not violent?!?

    View Slide

  52. @tech_christine @ryanhos
    Doctored images are hit and miss
    We chose not to include the image of a former US
    president photoshopped to look like a dictator and
    war criminal.
    Google labeled it “Possibly” a spoof; i.e. altered to be
    funny or offensive

    View Slide

  53. @tech_christine @ryanhos
    Text detection

    View Slide

  54. @tech_christine @ryanhos
    @letterfolk

    View Slide

  55. @tech_christine @ryanhos
    @letterfolk
    "MY\nBRAIN\nHAS\nTO0\nMANY\nTABS\nOPEN\nDechtlatte\nTHIS WEEK\npt de C\nCAit s pg\n"
    MY
    BRAIN
    HAS
    TO0
    MANY
    TABS
    OPEN
    Dechtlatte
    THIS WEEK
    pt de C
    CAit s pg

    View Slide

  56. @tech_christine @ryanhos
    @letterfolk

    View Slide

  57. @tech_christine @ryanhos
    @letterfolk
    YOURE NOT THE\nBOSS OF ME..\nIWHISPER UNDER\nMY BREATH AS I\nCLEAN UP ALL OF\nMY
    CHILDREN'S\nMESSES.\nTRE\n
    YOURE NOT THE
    BOSS OF ME..
    IWHISPER UNDER
    MY BREATH AS I
    CLEAN UP ALL OF
    MY CHILDREN'S
    MESSES.
    TRE

    View Slide

  58. @tech_christine @ryanhos
    @letterfolk

    View Slide

  59. @tech_christine @ryanhos
    @letterfolk
    Seth\nElizabeth\nMat\nIT'S BEGINNING TO\namazon Prime\nLOOK A LOT LIKE...\nvmazon
    Prime\necho\nI GOT MY MONEY'S\nWORTH FROM MY\n1t\nare en\na ton Prime\nAMAZON
    PRIME\nMEMBERSHIP\nNany\necho\neciro\n
    Seth
    Elizabeth
    Mat
    IT'S BEGINNING TO
    amazon Prime
    LOOK A LOT LIKE...
    vmazon Prime
    echo
    I GOT MY MONEY'S
    WORTH FROM MY
    1t\nare en
    a ton Prime
    AMAZON PRIME
    MEMBERSHIP
    Nany
    echo
    eciro

    View Slide

  60. @tech_christine @ryanhos
    Logo detection

    View Slide

  61. @tech_christine @ryanhos

    View Slide

  62. @tech_christine @ryanhos
    @nike

    View Slide

  63. @tech_christine @ryanhos
    @nike

    View Slide

  64. @tech_christine @ryanhos
    @nike
    1. nike
    2. nike plus
    3. nike azul
    4. blue nike
    5. nike store

    View Slide

  65. @tech_christine @ryanhos
    @christine_seeman

    View Slide

  66. @tech_christine @ryanhos
    @christine_seeman

    View Slide

  67. @tech_christine @ryanhos
    @christine_seeman
    1. etelä suomen sanomat
    2. rogue status
    3. american horror story
    4. delta skymiles
    5. deník
    !"!"

    View Slide

  68. @tech_christine @ryanhos
    Landmark detection

    View Slide

  69. @tech_christine @ryanhos
    @visit_nebraska

    View Slide

  70. @tech_christine @ryanhos
    @iloveny

    View Slide

  71. @tech_christine @ryanhos
    @iloveny

    View Slide

  72. @tech_christine @ryanhos
    @expertvagabond

    View Slide

  73. @tech_christine @ryanhos
    Have you been to London, NE?

    View Slide

  74. @tech_christine @ryanhos
    The British Museum there is lovely

    View Slide

  75. @tech_christine @ryanhos
    British Museum, alternate photo

    View Slide

  76. @tech_christine @ryanhos
    So much alike

    View Slide

  77. @tech_christine @ryanhos
    This wasn’t even a building…

    View Slide

  78. @tech_christine @ryanhos
    But what does my
    instagram say about me?

    View Slide

  79. @tech_christine @ryanhos

    View Slide

  80. @tech_christine @ryanhos
    @christineseeman

    View Slide

  81. @tech_christine @ryanhos
    Blooper cloud

    View Slide

  82. @tech_christine @ryanhos
    Where to go from here?
    What to do next with Google Cloud Vision

    View Slide

  83. @tech_christine @ryanhos
    @visit_nebraska

    View Slide

  84. @tech_christine @ryanhos
    Vs @visitcalifornia

    View Slide

  85. @tech_christine @ryanhos
    Vs @iloveny

    View Slide

  86. @tech_christine @ryanhos
    Can we teach a machine
    to know if a pic is from
    NY vs CA vs NE

    View Slide

  87. @tech_christine @ryanhos
    Explore bias in Machine
    Learning?

    View Slide

  88. @tech_christine @ryanhos
    How much will this cost you?
    For us, about $65

    View Slide

  89. @tech_christine @ryanhos
    Technology Used
    •Ruby (No Rails)
    •Rspec
    •Ruby Vision API Client Libraries
    •Google Cloud Storage
    •Rmagick (Ruby binding to Imagemagick)
    •MagicCloud tag cloud gem

    View Slide

  90. @tech_christine @ryanhos
    Reference Links
    •https://cloud.google.com/vision/
    •https://cloud.google.com/vision/docs/
    •https://googleapis.github.io/google-cloud-ruby/docs/
    •https://github.com/GoogleCloudPlatform/ruby-docs-
    samples/blob/master/vision/quickstart.rb
    •https://github.com/zverok/magic_cloud

    View Slide

  91. @tech_christine @ryanhos
    All the instagram accounts*
    @ripleyandrue
    @visit_nebraska
    @wolffolins
    @expertvagabond
    @iloveny
    @letterfolk
    @myraswim
    @visitcalifornia
    @nationalportraitgallery
    @nike
    * and none were harmed in the making of this talk

    View Slide

  92. @tech_christine @ryanhos
    getflywheel.com/about/careers

    View Slide

  93. @tech_christine @ryanhos
    Thank you for attending!

    View Slide

  94. @tech_christine @ryanhos
    For your evaluation consideration...

    View Slide