Golang + Cloud Vision API + GAE - GoCon Spring

Transcript

1. Golang + Google Cloud Platform Presented by @kaneshin 04/23/2016 Go

   Conference

2. @kaneshin (Shintaro Kaneko) - Principal Engineering Manager at eureka, Inc.

   - Photographer when Not Coding - MacBook Air 11 Inch ʻ Camera+Lens

3. recruit.eure.jp

4. • Online Dating Service • Responsible for dates, relationships and

   marriages. • Development • Language: PHP -> Golang • AWS: EC2/Aurora /DynamoDB/ElastiCache/… • GCP: BigQuery • Microservices

5. Golang for • Web Application Framework • Revel -> Gin

   • O/R Mapper/DB • Xorm/Wizard (DB Shard Library) • Vendor Package Management • Glide • etc…

6. What is Google Cloud Platform?

7. Google Cloud Platform • Compute • Compute Engine • Container

   Engine • App Engine • Storage • Cloud Storage (Nearline) • Cloud Datastore • Cloud Bigtable • Networking • Cloud CDN • Cloud DNS • Big Data • BigQuery • Machine Learning • Cloud Machine Learning • Cloud Vision API • Operations, Tools, …

8. Google Cloud Platform • Compute • Compute Engine • Container

   Engine • App Engine • Storage • Cloud Storage (Nearline) • Cloud Datastore • Cloud Bigtable • Networking • Cloud CDN • Cloud DNS • Big Data • BigQuery • Machine Learning • Cloud Machine Learning • Cloud Vision API • Operations, Tools, …

9. Google Cloud Platform • Compute • Compute Engine • Container

   Engine • App Engine • Storage • Cloud Storage (Nearline) • Cloud Datastore • Cloud Bigtable • Networking • Cloud CDN • Cloud DNS • Big Data • BigQuery • Machine Learning • Cloud Machine Learning • Cloud Vision API • Operations, Tools, …

10. Cloud Vision API

11. • Understands the content of submitted images. • Encapsulating powerful

    machine learning models. • Classifies images into thousands of categories. • detects individual objects • faces within images • finds and reads printed words contained within images. • Official Documentation • https://cloud.google.com/vision/docs/ Cloud Vision API

12. pigeon - Cloud Vision API on Golang

13. pigeon • A package for the Google Cloud Vision API

    on Golang. • It provides command and web app. • Repository • https://github.com/kaneshin/pigeon

14. Getting Started • Enabling the Google Cloud Vision API •

    API Manager > Overview > Cloud Vision API • Setting Up a Service Account • API Manager > Credentials > Create Credentials • Get service account credentials json file $ export GOOGLE_APPLICATION_CREDENTIALS =~/.credentials/service_account.json

15. Installation • The pigeon provides the command-line tools. • a

    • Make sure that pigeon was installed correctly: $ go get github.com/kaneshin/pigeon/tools/cmd/...

16. Usage - command # Default Detection is LabelDetection. $ pigeon

    assets/lenna.jpg $ pigeon -face gs://bucket_name/lenna.jpg $ pigeon -label https://httpbin.org/image/jpeg

17. Usage - command # Default Detection is LabelDetection. $ pigeon

    assets/lenna.jpg $ pigeon -face gs://bucket_name/lenna.jpg $ pigeon -label https://httpbin.org/image/jpeg This flag is specified detection of Cloud Vision API Given image file path (OK: GCS, Over http/s)

18. Label Detection

19. Text Detection (OCR)

20. Safe Search

21. Usage - web app # Default port is 8080. #

    Default Detection is LabelDetection. $ pigeon-app $ pigeon-app -port=8000 -- -face -label -safe-search
22. None

23. Deploy to GAE

24. Deploy - pigeon # Requisite appengine go sdk and direnv

    $ cd /path/to/pigeon/tools/appengine $ direnv $ goapp deploy -application [your-project-id] ./src/app

25. How to use Cloud Vision API on GAE After That.

26. Customizable

27. import "github.com/kaneshin/pigeon" import "github.com/kaneshin/pigeon/credentials" func main() { // Initialize vision

    service by a credentials json. creds := credentials.NewApplicationCredentials("credentials.json") config := NewConfig().WithCredentials(creds) client, err := pigeon.New(config) if err != nil { panic(err) } // To call multiple image annotation requests. feature := pigeon.NewFeature(pigeon.LabelDetection) batch, err := pigeon.NewBatchAnnotateImageRequest([]string{"lenna.jpg"}, feature) if err != nil { panic(err) http://kaneshin.github.io/2016/02/27/pigeon/

28. pigeon is good to learn GAE, API :)

29. App Engine

30. For what purpose?

31. Slack Slash Command

32. kaonashi

33. human and bot are

34. Scheduled Tasks

35. Notify Today’s Scheduled

36. Time Tone

37. Slack Integration • Private Repository • Logging (importance) • Clean

    Architecture • Easy to add new integration

38. How to use API on GAE

39. How to use API on GAE • Cloud Vision API

    • Calendar API • Cloud Storage API • Youtube API • …

40. Non GAE config, err := google.JWTConfigFromJSON(b, scope) if err !=

    nil { return nil, err } client := config.Client(context.Background()) srv, err := vision.New(client) if err != nil { return nil, err }

41. http.DefaultTransport and http.DefaultClient are not available in App Engine. See

    https:// developers.google.com/appengine/docs/go/ urlfetch/overview

42. http.DefaultTransport and http.DefaultClient are not available in App Engine. See

    https:// developers.google.com/appengine/docs/go/ urlfetch/overview

43. Use urlfetch.Client/Transport

44. Why urlfetch? • Using Google's network infrastructure • For efficiency

    and scaling purposes.

45. http.Client on GAE via OAuth2 httpClient := &http.Client{ Transport: &oauth2.Transport{

    Source: google.AppEngineTokenSource(ctx, scope), Base: &urlfetch.Transport{Context: ctx}, }, }

46. GAE Oriented • Unavailable packages including Sirupsen/logrus • PR: https://github.com/Sirupsen/logrus/pull/343

    • Some packages can’t access to http.Client to send a request. • Web Application Framework • Double Context Problem. (Complicated)

47. GAE Oriented - http.Client Best Practice type ( // A

    Config provides service configuration for service clients. // all clients will use the {defaults.DefaultConfig} structure. Config struct { // The credentials object to use when signing requests. // Defaults to application credentials file. Credentials *credentials.Credentials // The HTTP client to use when sending requests. // Defaults to `http.DefaultClient`. HTTPClient *http.Client } )

48. recruit.eure.jp

49. Thank you Credit: NASA Earth Observatory/NOAA NGDC