Golang + Cloud Vision API + GAE - GoCon Spring

by Shintaro Kaneko

Slide 1

Slide 1 text

Golang + Google Cloud Platform Presented by @kaneshin 04/23/2016 Go Conference

Slide 2

Slide 2 text

@kaneshin (Shintaro Kaneko) - Principal Engineering Manager at eureka, Inc. - Photographer when Not Coding - MacBook Air 11 Inch ʻ Camera+Lens

Slide 3

Slide 3 text

recruit.eure.jp

Slide 4

Slide 4 text

• Online Dating Service • Responsible for dates, relationships and marriages. • Development • Language: PHP -> Golang • AWS: EC2/Aurora /DynamoDB/ElastiCache/… • GCP: BigQuery • Microservices

Slide 5

Slide 5 text

Golang for • Web Application Framework • Revel -> Gin • O/R Mapper/DB • Xorm/Wizard (DB Shard Library) • Vendor Package Management • Glide • etc…

Slide 6

Slide 6 text

What is Google Cloud Platform?

Slide 7

Slide 7 text

Google Cloud Platform • Compute • Compute Engine • Container Engine • App Engine • Storage • Cloud Storage (Nearline) • Cloud Datastore • Cloud Bigtable • Networking • Cloud CDN • Cloud DNS • Big Data • BigQuery • Machine Learning • Cloud Machine Learning • Cloud Vision API • Operations, Tools, …

Slide 8

Slide 8 text

Slide 9

Slide 9 text

Slide 10

Slide 10 text

Cloud Vision API

Slide 11

Slide 11 text

• Understands the content of submitted images. • Encapsulating powerful machine learning models. • Classifies images into thousands of categories. • detects individual objects • faces within images • finds and reads printed words contained within images. • Official Documentation • https://cloud.google.com/vision/docs/ Cloud Vision API

Slide 12

Slide 12 text

pigeon - Cloud Vision API on Golang

Slide 13

Slide 13 text

pigeon • A package for the Google Cloud Vision API on Golang. • It provides command and web app. • Repository • https://github.com/kaneshin/pigeon

Slide 14

Slide 14 text

Getting Started • Enabling the Google Cloud Vision API • API Manager > Overview > Cloud Vision API • Setting Up a Service Account • API Manager > Credentials > Create Credentials • Get service account credentials json ﬁle $ export GOOGLE_APPLICATION_CREDENTIALS =~/.credentials/service_account.json

Slide 15

Slide 15 text

Installation • The pigeon provides the command-line tools. • a • Make sure that pigeon was installed correctly: $ go get github.com/kaneshin/pigeon/tools/cmd/...

Slide 16

Slide 16 text

Usage - command # Default Detection is LabelDetection. $ pigeon assets/lenna.jpg $ pigeon -face gs://bucket_name/lenna.jpg $ pigeon -label https://httpbin.org/image/jpeg

Slide 17

Slide 17 text

Usage - command # Default Detection is LabelDetection. $ pigeon assets/lenna.jpg $ pigeon -face gs://bucket_name/lenna.jpg $ pigeon -label https://httpbin.org/image/jpeg This flag is specified detection of Cloud Vision API Given image file path (OK: GCS, Over http/s)

Slide 18

Slide 18 text

Label Detection

Slide 19

Slide 19 text

Text Detection (OCR)

Slide 20

Slide 20 text

Safe Search

Slide 21

Slide 21 text

Usage - web app # Default port is 8080. # Default Detection is LabelDetection. $ pigeon-app $ pigeon-app -port=8000 -- -face -label -safe-search

Slide 22

Slide 22 text

No content

Slide 23

Slide 23 text

Deploy to GAE

Slide 24

Slide 24 text

Deploy - pigeon # Requisite appengine go sdk and direnv $ cd /path/to/pigeon/tools/appengine $ direnv $ goapp deploy -application [your-project-id] ./src/app

Slide 25

Slide 25 text

How to use Cloud Vision API on GAE After That.

Slide 26

Slide 26 text

Customizable

Slide 27

Slide 27 text

import "github.com/kaneshin/pigeon" import "github.com/kaneshin/pigeon/credentials" func main() { // Initialize vision service by a credentials json. creds := credentials.NewApplicationCredentials("credentials.json") config := NewConfig().WithCredentials(creds) client, err := pigeon.New(config) if err != nil { panic(err) } // To call multiple image annotation requests. feature := pigeon.NewFeature(pigeon.LabelDetection) batch, err := pigeon.NewBatchAnnotateImageRequest([]string{"lenna.jpg"}, feature) if err != nil { panic(err) http://kaneshin.github.io/2016/02/27/pigeon/

Slide 28

Slide 28 text

pigeon is good to learn GAE, API :)

Slide 29

Slide 29 text

App Engine

Slide 30

Slide 30 text

For what purpose?

Slide 31

Slide 31 text

Slack Slash Command

Slide 32

Slide 32 text

kaonashi

Slide 33

Slide 33 text

human and bot are

Slide 34

Slide 34 text

Scheduled Tasks

Slide 35

Slide 35 text

Notify Today’s Scheduled

Slide 36

Slide 36 text

Time Tone

Slide 37

Slide 37 text

Slack Integration • Private Repository • Logging (importance) • Clean Architecture • Easy to add new integration

Slide 38

Slide 38 text

How to use API on GAE

Slide 39

Slide 39 text

How to use API on GAE • Cloud Vision API • Calendar API • Cloud Storage API • Youtube API • …

Slide 40

Slide 40 text

Non GAE config, err := google.JWTConfigFromJSON(b, scope) if err != nil { return nil, err } client := config.Client(context.Background()) srv, err := vision.New(client) if err != nil { return nil, err }

Slide 41

Slide 41 text

http.DefaultTransport and http.DefaultClient are not available in App Engine. See https:// developers.google.com/appengine/docs/go/ urlfetch/overview

Slide 42

Slide 42 text

http.DefaultTransport and http.DefaultClient are not available in App Engine. See https:// developers.google.com/appengine/docs/go/ urlfetch/overview

Slide 43

Slide 43 text

Use urlfetch.Client/Transport

Slide 44

Slide 44 text

Why urlfetch? • Using Google's network infrastructure • For efﬁciency and scaling purposes.

Slide 45

Slide 45 text

http.Client on GAE via OAuth2 httpClient := &http.Client{ Transport: &oauth2.Transport{ Source: google.AppEngineTokenSource(ctx, scope), Base: &urlfetch.Transport{Context: ctx}, }, }

Slide 46

Slide 46 text

GAE Oriented • Unavailable packages including Sirupsen/logrus • PR: https://github.com/Sirupsen/logrus/pull/343 • Some packages can’t access to http.Client to send a request. • Web Application Framework • Double Context Problem. (Complicated)

Slide 47

Slide 47 text

GAE Oriented - http.Client Best Practice type ( // A Config provides service configuration for service clients. // all clients will use the {defaults.DefaultConfig} structure. Config struct { // The credentials object to use when signing requests. // Defaults to application credentials file. Credentials *credentials.Credentials // The HTTP client to use when sending requests. // Defaults to `http.DefaultClient`. HTTPClient *http.Client } )