Golang + Cloud Vision API + GAE - GoCon Spring

Transcript

1. Golang + Google Cloud Platform Presented by @kaneshin 04/23/2016 Go Conference
2. @kaneshin (Shintaro Kaneko) - Principal Engineering Manager at eureka, Inc. - Photographer when Not Coding - MacBook Air 11 Inch ʻ Camera+Lens
3. recruit.eure.jp
4. • Online Dating Service • Responsible for dates, relationships and marriages. • Development • Language: PHP -> Golang • AWS: EC2/Aurora /DynamoDB/ElastiCache/… • GCP: BigQuery • Microservices
5. Golang for • Web Application Framework • Revel -> Gin • O/R Mapper/DB • Xorm/Wizard (DB Shard Library) • Vendor Package Management • Glide • etc…
6. What is Google Cloud Platform?
7. Google Cloud Platform • Compute • Compute Engine • Container Engine • App Engine • Storage • Cloud Storage (Nearline) • Cloud Datastore • Cloud Bigtable • Networking • Cloud CDN • Cloud DNS • Big Data • BigQuery • Machine Learning • Cloud Machine Learning • Cloud Vision API • Operations, Tools, …
8. Google Cloud Platform • Compute • Compute Engine • Container Engine • App Engine • Storage • Cloud Storage (Nearline) • Cloud Datastore • Cloud Bigtable • Networking • Cloud CDN • Cloud DNS • Big Data • BigQuery • Machine Learning • Cloud Machine Learning • Cloud Vision API • Operations, Tools, …
9. Google Cloud Platform • Compute • Compute Engine • Container Engine • App Engine • Storage • Cloud Storage (Nearline) • Cloud Datastore • Cloud Bigtable • Networking • Cloud CDN • Cloud DNS • Big Data • BigQuery • Machine Learning • Cloud Machine Learning • Cloud Vision API • Operations, Tools, …
10. Cloud Vision API
11. • Understands the content of submitted images. • Encapsulating powerful machine learning models. • Classifies images into thousands of categories. • detects individual objects • faces within images • finds and reads printed words contained within images. • Official Documentation • https://cloud.google.com/vision/docs/ Cloud Vision API
12. pigeon - Cloud Vision API on Golang
13. pigeon • A package for the Google Cloud Vision API on Golang. • It provides command and web app. • Repository • https://github.com/kaneshin/pigeon
14. Getting Started • Enabling the Google Cloud Vision API • API Manager > Overview > Cloud Vision API • Setting Up a Service Account • API Manager > Credentials > Create Credentials • Get service account credentials json file $ export GOOGLE_APPLICATION_CREDENTIALS =~/.credentials/service_account.json
15. Installation • The pigeon provides the command-line tools. • a • Make sure that pigeon was installed correctly: $ go get github.com/kaneshin/pigeon/tools/cmd/...
16. Usage - command # Default Detection is LabelDetection. $ pigeon assets/lenna.jpg $ pigeon -face gs://bucket_name/lenna.jpg $ pigeon -label https://httpbin.org/image/jpeg
17. Usage - command # Default Detection is LabelDetection. $ pigeon assets/lenna.jpg $ pigeon -face gs://bucket_name/lenna.jpg $ pigeon -label https://httpbin.org/image/jpeg This flag is specified detection of Cloud Vision API Given image file path (OK: GCS, Over http/s)
18. Label Detection
19. Text Detection (OCR)
20. Safe Search
21. Usage - web app # Default port is 8080. # Default Detection is LabelDetection. $ pigeon-app $ pigeon-app -port=8000 -- -face -label -safe-search
23. Deploy to GAE
24. Deploy - pigeon # Requisite appengine go sdk and direnv $ cd /path/to/pigeon/tools/appengine $ direnv $ goapp deploy -application [your-project-id] ./src/app
25. How to use Cloud Vision API on GAE After That.
26. Customizable
27. import "github.com/kaneshin/pigeon" import "github.com/kaneshin/pigeon/credentials" func main() { // Initialize vision service by a credentials json. creds := credentials.NewApplicationCredentials("credentials.json") config := NewConfig().WithCredentials(creds) client, err := pigeon.New(config) if err != nil { panic(err) } // To call multiple image annotation requests. feature := pigeon.NewFeature(pigeon.LabelDetection) batch, err := pigeon.NewBatchAnnotateImageRequest([]string{"lenna.jpg"}, feature) if err != nil { panic(err) http://kaneshin.github.io/2016/02/27/pigeon/
28. pigeon is good to learn GAE, API :)
29. App Engine
30. For what purpose?
31. Slack Slash Command
32. kaonashi
33. human and bot are
34. Scheduled Tasks
35. Notify Today’s Scheduled
36. Time Tone
37. Slack Integration • Private Repository • Logging (importance) • Clean Architecture • Easy to add new integration
38. How to use API on GAE
39. How to use API on GAE • Cloud Vision API • Calendar API • Cloud Storage API • Youtube API • …
40. Non GAE config, err := google.JWTConfigFromJSON(b, scope) if err != nil { return nil, err } client := config.Client(context.Background()) srv, err := vision.New(client) if err != nil { return nil, err }
41. http.DefaultTransport and http.DefaultClient are not available in App Engine. See https:// developers.google.com/appengine/docs/go/ urlfetch/overview
42. http.DefaultTransport and http.DefaultClient are not available in App Engine. See https:// developers.google.com/appengine/docs/go/ urlfetch/overview
43. Use urlfetch.Client/Transport
44. Why urlfetch? • Using Google's network infrastructure • For efficiency and scaling purposes.
45. http.Client on GAE via OAuth2 httpClient := &http.Client{ Transport: &oauth2.Transport{ Source: google.AppEngineTokenSource(ctx, scope), Base: &urlfetch.Transport{Context: ctx}, }, }
46. GAE Oriented • Unavailable packages including Sirupsen/logrus • PR: https://github.com/Sirupsen/logrus/pull/343 • Some packages can’t access to http.Client to send a request. • Web Application Framework • Double Context Problem. (Complicated)
47. GAE Oriented - http.Client Best Practice type ( // A Config provides service configuration for service clients. // all clients will use the {defaults.DefaultConfig} structure. Config struct { // The credentials object to use when signing requests. // Defaults to application credentials file. Credentials *credentials.Credentials // The HTTP client to use when sending requests. // Defaults to `http.DefaultClient`. HTTPClient *http.Client } )
48. recruit.eure.jp
49. Thank you Credit: NASA Earth Observatory/NOAA NGDC