Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Building a Go microservice from 0 to Hero

Building a Go microservice from 0 to Hero

Slides of "Building a Go microservice from 0 to Hero" talk made at GoLang Meetup Marseille, France (October 10, 2018) #golang #microservices #gcp #k8s



October 10, 2018

More Decks by Fedir RYKHTIK

Other Decks in Technology


  1. Building a Go microservice from 0 to Hero GoLang Marseille

    Meetup @FedirFr
  2. Fedir RYKHTIK • Building open source web since 2007 ◦

    Back-end developer ◦ Independent scientist ◦ DevOps / SA • CTO @AgenceStratis since 2015 • Gopher
  3. Plan • Introduction to microservices • Go microservices frameworks &

    toolkits • Packaging • Hosting & Deployment • Q&A
  4. Introduction to microservices

  5. • Complex applications are composed of small, independent processes communicating

    with each other using language-agnostic APIs • Highly decoupled and focus on doing a small task Microservices
  6. http://martinfowler.com/articles/microservices.html

  7. “There are certain things you need to get sorted out

    before you can put your first microservices system into production: monitoring, provisioning, and a devops culture.” Martin Fowler http://martinfowler.com/bliki/MicroservicePrerequisites.html
  8. (micro)service oriented architecture (SOA) typical problems • Service discovery •

    Monitoring • Logging • Load balancing • Circuit breaking • Scalability
  9. Go microservices frameworks & toolkits

  10. Go microsevice toolkits statistics rating • The most popular project

    is go-kit/kit • The newest project is NYTimes/gizmo • The project with more commits is koding/kite • The project with biggest average contribution period is rsms/gotalk • The project with more tags is micro/go-micro • The project made by most notable top contributors is koding/kite • The project with best errors resolving rate is micro/go-micro • The project with more commits by day is koding/kite • The project with the most active number of forkers is NYTimes/gizmo • The project with biggest number of returning contributors is go-kit/kit • The best project (taking in account placements in all competitions) is go-kit/kit Details: https://docs.google.com/spreadsheets/d/1d0WcI5ahmJdysyyHeZB9CVlolXF027UlY1fTPbyiFCE/edit?usp=sharing *Statistics made with github.com/fedir/ghstat comparator
  11. Go Micro Great for beginners, more opionionated, than others. Code

    generator. Enforces gRPC https://github.com/micro/go-micro Gizmo Good for pub/sub use cases gRPC, HTTP https://github.com/NYTimes/gizmo Kite Good for beginners, difficult to integrate with other solutions Websockets + RPC https://github.com/koding/kite Go microservices toolkits Go-Kit Very much production ready. Flexible. Patterns are a bit complex, especially for beginners (SOLID, DDD, hexagonal). gRPC, HTTP, Thrift https://github.com/go-kit/kit
  12. go-micro example package main import ( "context" "log" proto "github.com/micro/examples/helloworld/proto"

    "github.com/micro/go-micro" ) type Greeter struct{} func (g *Greeter) Hello(ctx context.Context, req *proto.HelloRequest, rsp *proto.HelloResponse) error { rsp.Greeting = "Hello " + req.Name return nil } func main() { service := micro.NewService( micro.Name("greeter"), ) service.Init() proto.RegisterGreeterHandler(service.Server(), new(Greeter)) if err := service.Run(); err != nil { log.Fatal(err) } } Ref.: https://github.com/micro/examples/blob/master/helloworld/main.go
  13. go-kit example package main import ( "context" "encoding/json" "errors" "log"

    "net/http" "strings" "github.com/go-kit/kit/endpoint" httptransport "github.com/go-kit/kit/transport/http" ) // StringService provides operations on strings. type StringService interface { Uppercase(string) (string, error) Count(string) int } // stringService is a concrete implementation of StringService type stringService struct{} func (stringService) Uppercase(s string) (string, error) { if s == "" { return "", ErrEmpty } return strings.ToUpper(s), nil } func (stringService) Count(s string) int { return len(s) } // ErrEmpty is returned when an input string is empty. var ErrEmpty = errors.New("empty string") // For each method, we define request and response structs type uppercaseRequest struct { S string `json:"s"` } type uppercaseResponse struct { V string `json:"v"` Err string `json:"err,omitempty"` // errors don't define JSON marshaling } type countRequest struct { S string `json:"s"` } type countResponse struct { V int `json:"v"` } // Endpoints are a primary abstraction in go-kit. An endpoint represents a single RPC (method in our service interface) func makeUppercaseEndpoint(svc StringService) endpoint.Endpoint { return func(_ context.Context, request interface{}) (interface{}, error) { req := request.(uppercaseRequest) v, err := svc.Uppercase(req.S) if err != nil { return uppercaseResponse{v, err.Error()}, nil } return uppercaseResponse{v, ""}, nil } } func makeCountEndpoint(svc StringService) endpoint.Endpoint { return func(_ context.Context, request interface{}) (interface{}, error) { req := request.(countRequest) v := svc.Count(req.S) return countResponse{v}, nil } } // Transports expose the service to the network. In this first example we utilize JSON over HTTP. func main() { svc := stringService{} uppercaseHandler := httptransport.NewServer( makeUppercaseEndpoint(svc), decodeUppercaseRequest, encodeResponse, ) countHandler := httptransport.NewServer( makeCountEndpoint(svc), decodeCountRequest, encodeResponse, ) http.Handle("/uppercase", uppercaseHandler) http.Handle("/count", countHandler) log.Fatal(http.ListenAndServe(":8080", nil)) } func decodeUppercaseRequest(_ context.Context, r *http.Request) (interface{}, error) { var request uppercaseRequest if err := json.NewDecoder(r.Body).Decode(&request); err != nil { return nil, err } return request, nil } func decodeCountRequest(_ context.Context, r *http.Request) (interface{}, error) { var request countRequest if err := json.NewDecoder(r.Body).Decode(&request); err != nil { return nil, err } return request, nil } func encodeResponse(_ context.Context, w http.ResponseWriter, response interface{}) error { return json.NewEncoder(w).Encode(response) } Ref.: https://github.com/go-kit/kit/blob/master/examples/stringsvc1/main.go
  14. Please choose Your framework now The red pill and its

    opposite, the blue pill, are a metaphor representing the choice between: • Knowledge, freedom and the brutal truths of reality (red pill) • Happiness, beauty, and the blissful ignorance of illusion (blue pill)
  15. Packaging

  16. packaging > github Dockerfile

  17. Dockerfile FROM golang:1.8-alpine ADD . /go/src/hello-app RUN go install hello-app

    FROM alpine:latest COPY --from=0 /go/bin/hello-app . ENV PORT 8080 CMD ["./hello-app"] Ref.: https://github.com/GoogleCloudPlatform/kubernetes-engine-samples/blob/master/hello-app/Dockerfile
  18. Hosting & Deployment

  19. google cloud + k8s

  20. Dockerfile deployment schema Docker image git k8s cluster

  21. 1. https://console.cloud.google.com/ 2. Creation a new project "meetup-golang-marseille-03" 3. Creation

    a new Kubernetes cluster "gmm03" (take some time) 4. Use of Cloud shell 5. Cloneing of the project from the git(hub/lab/...) repository 6. Building the container & Deploying it on the cluster 7. Checking status and getting public IP 8. Testing via HTTP from the developer's PC 9. Changing cluster size 10. Testing via HTTP again from the developer's PC 11. App modification 12. Redeploying it 13. Testing via HTTP again from the developer's PC k8s demo steps
  22. Tools - kubectl kubectl controls the Kubernetes cluster manager. Used

    Commands : run Run a particular image on the cluster expose Take a replication controller, service, deployment or pod and expose it as a new Kubernetes Service get Display one or many resources set Set specific features on objects Find more information at: https://kubernetes.io/docs/reference/kubectl/overview/
  23. gcloud manage Google Cloud Platform resources and developer workflow. Used

    commands : container deploy and manage clusters of machines for running containers docker enable Docker CLI access to Google Container Registry Find more information at: https://cloud.google.com/sdk/gcloud/reference/container/ Tools - gcloud
  24. Create cluster and deploy the app git clone https://github.com/GoogleCloudPlatform/kubernetes-engine-samples cd

    kubernetes-engine-samples/hello-app gcloud container clusters get-credentials gmm03 --zone europe-west1-b docker build -t gcr.io/meetup-golang-marseille-03/hello-node:v1 $PWD gcloud docker -- push gcr.io/meetup-golang-marseille-03/hello-node:v1 kubectl run hello-node --image=gcr.io/meetup-golang-marseille-03/hello-node:v1 --port=8080 kubectl expose deployment hello-node --type="LoadBalancer"
  25. Get external IP (takes some time) kubectl get service hello-node

  26. Modify and redeploy the app sed -i -e 's/1.0.0/2.0.0/g' main.go

    docker build -t gcr.io/meetup-golang-marseille-03/hello-node:v2 $PWD gcloud docker -- push gcr.io/meetup-golang-marseille-03/hello-node:v2 kubectl set image deployment/hello-node hello-node=gcr.io/meetup-golang-marseille-03/hello-node:v2 && echo 'image updated' kubectl get service hello-node
  27. Q&A Any questions ?

  28. Recommended books

  29. Ressources & related materials • microservices in go ◦ https://medium.com/seek-blog/microservices-in-go-2fc1570f6800

    ◦ https://mycodesmells.com/post/little-bit-about-go-kit ◦ https://www.youtube.com/watch?v=1AjaZi4QuGo ◦ https://www.youtube.com/watch?v=NX0sHF8ZZgw ◦ https://www.youtube.com/watch?v=1ScP5DyS1_g ◦ https://www.youtube.com/watch?v=OcjMi9cXItY ◦ https://dev.to/plutov/microservices-with-go-kit-part-1-13dd ◦ https://dev.to/plutov/packagemain-13-microservices-with-go-kit-part-2-4lgh ◦ https://about.sourcegraph.com/go/gophercon-2018-from-prototype-to-production-lessons-from-building-and/ ◦ https://github.com/ru-rocker/gokit-playground ◦ https://www.reddit.com/r/golang/comments/5tesl9/why_i_recommend_to_avoid_using_the_gokit_library/ ◦ http://gokit.io/examples/stringsvc.html ◦ http://www.ru-rocker.com/2017/02/17/micro-services-using-go-kit-http-endpoint/ ◦ https://github.com/micro/examples ◦ https://github.com/udacity/ud615 ◦ https://github.com/matryer/goblueprints • docker ◦ https://docs.docker.com/engine/reference/builder/ https://github.com/GoogleCloudPlatform/kubernetes-engine-samples/blob/master/hello-app/Dockerfile • google cloud & k8s ◦ https://about.sourcegraph.com/go/gophercon-2018-from-prototype-to-production-lessons-from-building-and/ ◦ https://cloud.google.com/ ◦ https://cloud.google.com/kubernetes-engine/ ◦ https://cloud.google.com/shell/ ◦ https://github.com/GoogleCloudPlatform/kubernetes-engine-samples/ ◦ https://medium.com/google-cloud/deploy-go-application-to-kubernetes-in-30-seconds-ebff0f51d67b • istio ◦ https://istio.io/ ◦ https://github.com/thesandlord/Istio101
  30. Merci :) @FedirFR