Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Golang + Cloud Vision API + GAE - GoCon Spring

Golang + Cloud Vision API + GAE - GoCon Spring

A75c909334d4dc6c7dca7bd915eebdbf?s=128

Shintaro Kaneko

April 23, 2016
Tweet

Transcript

  1. Golang + Google Cloud Platform Presented by @kaneshin 04/23/2016 Go

    Conference
  2. @kaneshin (Shintaro Kaneko) - Principal Engineering Manager at eureka, Inc.

    - Photographer when Not Coding - MacBook Air 11 Inch ʻ Camera+Lens
  3. recruit.eure.jp

  4. • Online Dating Service • Responsible for dates, relationships and

    marriages. • Development • Language: PHP -> Golang • AWS: EC2/Aurora /DynamoDB/ElastiCache/… • GCP: BigQuery • Microservices
  5. Golang for • Web Application Framework • Revel -> Gin

    • O/R Mapper/DB • Xorm/Wizard (DB Shard Library) • Vendor Package Management • Glide • etc…
  6. What is Google Cloud Platform?

  7. Google Cloud Platform • Compute • Compute Engine • Container

    Engine • App Engine • Storage • Cloud Storage (Nearline) • Cloud Datastore • Cloud Bigtable • Networking • Cloud CDN • Cloud DNS • Big Data • BigQuery • Machine Learning • Cloud Machine Learning • Cloud Vision API • Operations, Tools, …
  8. Google Cloud Platform • Compute • Compute Engine • Container

    Engine • App Engine • Storage • Cloud Storage (Nearline) • Cloud Datastore • Cloud Bigtable • Networking • Cloud CDN • Cloud DNS • Big Data • BigQuery • Machine Learning • Cloud Machine Learning • Cloud Vision API • Operations, Tools, …
  9. Google Cloud Platform • Compute • Compute Engine • Container

    Engine • App Engine • Storage • Cloud Storage (Nearline) • Cloud Datastore • Cloud Bigtable • Networking • Cloud CDN • Cloud DNS • Big Data • BigQuery • Machine Learning • Cloud Machine Learning • Cloud Vision API • Operations, Tools, …
  10. Cloud Vision API

  11. • Understands the content of submitted images. • Encapsulating powerful

    machine learning models. • Classifies images into thousands of categories. • detects individual objects • faces within images • finds and reads printed words contained within images. • Official Documentation • https://cloud.google.com/vision/docs/ Cloud Vision API
  12. pigeon - Cloud Vision API on Golang

  13. pigeon • A package for the Google Cloud Vision API

    on Golang. • It provides command and web app. • Repository • https://github.com/kaneshin/pigeon
  14. Getting Started • Enabling the Google Cloud Vision API •

    API Manager > Overview > Cloud Vision API • Setting Up a Service Account • API Manager > Credentials > Create Credentials • Get service account credentials json file $ export GOOGLE_APPLICATION_CREDENTIALS =~/.credentials/service_account.json
  15. Installation • The pigeon provides the command-line tools. • a

    • Make sure that pigeon was installed correctly: $ go get github.com/kaneshin/pigeon/tools/cmd/...
  16. Usage - command # Default Detection is LabelDetection. $ pigeon

    assets/lenna.jpg $ pigeon -face gs://bucket_name/lenna.jpg $ pigeon -label https://httpbin.org/image/jpeg
  17. Usage - command # Default Detection is LabelDetection. $ pigeon

    assets/lenna.jpg $ pigeon -face gs://bucket_name/lenna.jpg $ pigeon -label https://httpbin.org/image/jpeg This flag is specified detection of Cloud Vision API Given image file path (OK: GCS, Over http/s)
  18. Label Detection

  19. Text Detection (OCR)

  20. Safe Search

  21. Usage - web app # Default port is 8080. #

    Default Detection is LabelDetection. $ pigeon-app $ pigeon-app -port=8000 -- -face -label -safe-search
  22. None
  23. Deploy to GAE

  24. Deploy - pigeon # Requisite appengine go sdk and direnv

    $ cd /path/to/pigeon/tools/appengine $ direnv $ goapp deploy -application [your-project-id] ./src/app
  25. How to use Cloud Vision API on GAE After That.

  26. Customizable

  27. import "github.com/kaneshin/pigeon" import "github.com/kaneshin/pigeon/credentials" func main() { // Initialize vision

    service by a credentials json. creds := credentials.NewApplicationCredentials("credentials.json") config := NewConfig().WithCredentials(creds) client, err := pigeon.New(config) if err != nil { panic(err) } // To call multiple image annotation requests. feature := pigeon.NewFeature(pigeon.LabelDetection) batch, err := pigeon.NewBatchAnnotateImageRequest([]string{"lenna.jpg"}, feature) if err != nil { panic(err) http://kaneshin.github.io/2016/02/27/pigeon/
  28. pigeon is good to learn GAE, API :)

  29. App Engine

  30. For what purpose?

  31. Slack Slash Command

  32. kaonashi

  33. human and bot are

  34. Scheduled Tasks

  35. Notify Today’s Scheduled

  36. Time Tone

  37. Slack Integration • Private Repository • Logging (importance) • Clean

    Architecture • Easy to add new integration
  38. How to use API on GAE

  39. How to use API on GAE • Cloud Vision API

    • Calendar API • Cloud Storage API • Youtube API • …
  40. Non GAE config, err := google.JWTConfigFromJSON(b, scope) if err !=

    nil { return nil, err } client := config.Client(context.Background()) srv, err := vision.New(client) if err != nil { return nil, err }
  41. http.DefaultTransport and http.DefaultClient are not available in App Engine. See

    https:// developers.google.com/appengine/docs/go/ urlfetch/overview
  42. http.DefaultTransport and http.DefaultClient are not available in App Engine. See

    https:// developers.google.com/appengine/docs/go/ urlfetch/overview
  43. Use urlfetch.Client/Transport

  44. Why urlfetch? • Using Google's network infrastructure • For efficiency

    and scaling purposes.
  45. http.Client on GAE via OAuth2 httpClient := &http.Client{ Transport: &oauth2.Transport{

    Source: google.AppEngineTokenSource(ctx, scope), Base: &urlfetch.Transport{Context: ctx}, }, }
  46. GAE Oriented • Unavailable packages including Sirupsen/logrus • PR: https://github.com/Sirupsen/logrus/pull/343

    • Some packages can’t access to http.Client to send a request. • Web Application Framework • Double Context Problem. (Complicated)
  47. GAE Oriented - http.Client Best Practice type ( // A

    Config provides service configuration for service clients. // all clients will use the {defaults.DefaultConfig} structure. Config struct { // The credentials object to use when signing requests. // Defaults to application credentials file. Credentials *credentials.Credentials // The HTTP client to use when sending requests. // Defaults to `http.DefaultClient`. HTTPClient *http.Client } )
  48. recruit.eure.jp

  49. Thank you Credit: NASA Earth Observatory/NOAA NGDC