Building a Go microservice from 0 to Hero

Building a Go microservice from 0 to Hero

Slides of "Building a Go microservice from 0 to Hero" talk made at GoLang Meetup Marseille, France (October 10, 2018) #golang #microservices #gcp #k8s

Fdeabaecd044c39cb7d75b104b0aff7c?s=128

Fedir RYKHTIK

October 10, 2018
Tweet

Transcript

  1. 2.

    Fedir RYKHTIK • Building open source web since 2007 ◦

    Back-end developer ◦ Independent scientist ◦ DevOps / SA • CTO @AgenceStratis since 2015 • Gopher
  2. 3.

    Plan • Introduction to microservices • Go microservices frameworks &

    toolkits • Packaging • Hosting & Deployment • Q&A
  3. 5.

    • Complex applications are composed of small, independent processes communicating

    with each other using language-agnostic APIs • Highly decoupled and focus on doing a small task Microservices
  4. 7.

    “There are certain things you need to get sorted out

    before you can put your first microservices system into production: monitoring, provisioning, and a devops culture.” Martin Fowler http://martinfowler.com/bliki/MicroservicePrerequisites.html
  5. 8.

    (micro)service oriented architecture (SOA) typical problems • Service discovery •

    Monitoring • Logging • Load balancing • Circuit breaking • Scalability
  6. 10.

    Go microsevice toolkits statistics rating • The most popular project

    is go-kit/kit • The newest project is NYTimes/gizmo • The project with more commits is koding/kite • The project with biggest average contribution period is rsms/gotalk • The project with more tags is micro/go-micro • The project made by most notable top contributors is koding/kite • The project with best errors resolving rate is micro/go-micro • The project with more commits by day is koding/kite • The project with the most active number of forkers is NYTimes/gizmo • The project with biggest number of returning contributors is go-kit/kit • The best project (taking in account placements in all competitions) is go-kit/kit Details: https://docs.google.com/spreadsheets/d/1d0WcI5ahmJdysyyHeZB9CVlolXF027UlY1fTPbyiFCE/edit?usp=sharing *Statistics made with github.com/fedir/ghstat comparator
  7. 11.

    Go Micro Great for beginners, more opionionated, than others. Code

    generator. Enforces gRPC https://github.com/micro/go-micro Gizmo Good for pub/sub use cases gRPC, HTTP https://github.com/NYTimes/gizmo Kite Good for beginners, difficult to integrate with other solutions Websockets + RPC https://github.com/koding/kite Go microservices toolkits Go-Kit Very much production ready. Flexible. Patterns are a bit complex, especially for beginners (SOLID, DDD, hexagonal). gRPC, HTTP, Thrift https://github.com/go-kit/kit
  8. 12.

    go-micro example package main import ( "context" "log" proto "github.com/micro/examples/helloworld/proto"

    "github.com/micro/go-micro" ) type Greeter struct{} func (g *Greeter) Hello(ctx context.Context, req *proto.HelloRequest, rsp *proto.HelloResponse) error { rsp.Greeting = "Hello " + req.Name return nil } func main() { service := micro.NewService( micro.Name("greeter"), ) service.Init() proto.RegisterGreeterHandler(service.Server(), new(Greeter)) if err := service.Run(); err != nil { log.Fatal(err) } } Ref.: https://github.com/micro/examples/blob/master/helloworld/main.go
  9. 13.

    go-kit example package main import ( "context" "encoding/json" "errors" "log"

    "net/http" "strings" "github.com/go-kit/kit/endpoint" httptransport "github.com/go-kit/kit/transport/http" ) // StringService provides operations on strings. type StringService interface { Uppercase(string) (string, error) Count(string) int } // stringService is a concrete implementation of StringService type stringService struct{} func (stringService) Uppercase(s string) (string, error) { if s == "" { return "", ErrEmpty } return strings.ToUpper(s), nil } func (stringService) Count(s string) int { return len(s) } // ErrEmpty is returned when an input string is empty. var ErrEmpty = errors.New("empty string") // For each method, we define request and response structs type uppercaseRequest struct { S string `json:"s"` } type uppercaseResponse struct { V string `json:"v"` Err string `json:"err,omitempty"` // errors don't define JSON marshaling } type countRequest struct { S string `json:"s"` } type countResponse struct { V int `json:"v"` } // Endpoints are a primary abstraction in go-kit. An endpoint represents a single RPC (method in our service interface) func makeUppercaseEndpoint(svc StringService) endpoint.Endpoint { return func(_ context.Context, request interface{}) (interface{}, error) { req := request.(uppercaseRequest) v, err := svc.Uppercase(req.S) if err != nil { return uppercaseResponse{v, err.Error()}, nil } return uppercaseResponse{v, ""}, nil } } func makeCountEndpoint(svc StringService) endpoint.Endpoint { return func(_ context.Context, request interface{}) (interface{}, error) { req := request.(countRequest) v := svc.Count(req.S) return countResponse{v}, nil } } // Transports expose the service to the network. In this first example we utilize JSON over HTTP. func main() { svc := stringService{} uppercaseHandler := httptransport.NewServer( makeUppercaseEndpoint(svc), decodeUppercaseRequest, encodeResponse, ) countHandler := httptransport.NewServer( makeCountEndpoint(svc), decodeCountRequest, encodeResponse, ) http.Handle("/uppercase", uppercaseHandler) http.Handle("/count", countHandler) log.Fatal(http.ListenAndServe(":8080", nil)) } func decodeUppercaseRequest(_ context.Context, r *http.Request) (interface{}, error) { var request uppercaseRequest if err := json.NewDecoder(r.Body).Decode(&request); err != nil { return nil, err } return request, nil } func decodeCountRequest(_ context.Context, r *http.Request) (interface{}, error) { var request countRequest if err := json.NewDecoder(r.Body).Decode(&request); err != nil { return nil, err } return request, nil } func encodeResponse(_ context.Context, w http.ResponseWriter, response interface{}) error { return json.NewEncoder(w).Encode(response) } Ref.: https://github.com/go-kit/kit/blob/master/examples/stringsvc1/main.go
  10. 14.

    Please choose Your framework now The red pill and its

    opposite, the blue pill, are a metaphor representing the choice between: • Knowledge, freedom and the brutal truths of reality (red pill) • Happiness, beauty, and the blissful ignorance of illusion (blue pill)
  11. 15.
  12. 17.

    Dockerfile FROM golang:1.8-alpine ADD . /go/src/hello-app RUN go install hello-app

    FROM alpine:latest COPY --from=0 /go/bin/hello-app . ENV PORT 8080 CMD ["./hello-app"] Ref.: https://github.com/GoogleCloudPlatform/kubernetes-engine-samples/blob/master/hello-app/Dockerfile
  13. 21.

    1. https://console.cloud.google.com/ 2. Creation a new project "meetup-golang-marseille-03" 3. Creation

    a new Kubernetes cluster "gmm03" (take some time) 4. Use of Cloud shell 5. Cloneing of the project from the git(hub/lab/...) repository 6. Building the container & Deploying it on the cluster 7. Checking status and getting public IP 8. Testing via HTTP from the developer's PC 9. Changing cluster size 10. Testing via HTTP again from the developer's PC 11. App modification 12. Redeploying it 13. Testing via HTTP again from the developer's PC k8s demo steps
  14. 22.

    Tools - kubectl kubectl controls the Kubernetes cluster manager. Used

    Commands : run Run a particular image on the cluster expose Take a replication controller, service, deployment or pod and expose it as a new Kubernetes Service get Display one or many resources set Set specific features on objects Find more information at: https://kubernetes.io/docs/reference/kubectl/overview/
  15. 23.

    gcloud manage Google Cloud Platform resources and developer workflow. Used

    commands : container deploy and manage clusters of machines for running containers docker enable Docker CLI access to Google Container Registry Find more information at: https://cloud.google.com/sdk/gcloud/reference/container/ Tools - gcloud
  16. 24.

    Create cluster and deploy the app git clone https://github.com/GoogleCloudPlatform/kubernetes-engine-samples cd

    kubernetes-engine-samples/hello-app gcloud container clusters get-credentials gmm03 --zone europe-west1-b docker build -t gcr.io/meetup-golang-marseille-03/hello-node:v1 $PWD gcloud docker -- push gcr.io/meetup-golang-marseille-03/hello-node:v1 kubectl run hello-node --image=gcr.io/meetup-golang-marseille-03/hello-node:v1 --port=8080 kubectl expose deployment hello-node --type="LoadBalancer"
  17. 26.

    Modify and redeploy the app sed -i -e 's/1.0.0/2.0.0/g' main.go

    docker build -t gcr.io/meetup-golang-marseille-03/hello-node:v2 $PWD gcloud docker -- push gcr.io/meetup-golang-marseille-03/hello-node:v2 kubectl set image deployment/hello-node hello-node=gcr.io/meetup-golang-marseille-03/hello-node:v2 && echo 'image updated' kubectl get service hello-node
  18. 29.

    Ressources & related materials • microservices in go ◦ https://medium.com/seek-blog/microservices-in-go-2fc1570f6800

    ◦ https://mycodesmells.com/post/little-bit-about-go-kit ◦ https://www.youtube.com/watch?v=1AjaZi4QuGo ◦ https://www.youtube.com/watch?v=NX0sHF8ZZgw ◦ https://www.youtube.com/watch?v=1ScP5DyS1_g ◦ https://www.youtube.com/watch?v=OcjMi9cXItY ◦ https://dev.to/plutov/microservices-with-go-kit-part-1-13dd ◦ https://dev.to/plutov/packagemain-13-microservices-with-go-kit-part-2-4lgh ◦ https://about.sourcegraph.com/go/gophercon-2018-from-prototype-to-production-lessons-from-building-and/ ◦ https://github.com/ru-rocker/gokit-playground ◦ https://www.reddit.com/r/golang/comments/5tesl9/why_i_recommend_to_avoid_using_the_gokit_library/ ◦ http://gokit.io/examples/stringsvc.html ◦ http://www.ru-rocker.com/2017/02/17/micro-services-using-go-kit-http-endpoint/ ◦ https://github.com/micro/examples ◦ https://github.com/udacity/ud615 ◦ https://github.com/matryer/goblueprints • docker ◦ https://docs.docker.com/engine/reference/builder/ https://github.com/GoogleCloudPlatform/kubernetes-engine-samples/blob/master/hello-app/Dockerfile • google cloud & k8s ◦ https://about.sourcegraph.com/go/gophercon-2018-from-prototype-to-production-lessons-from-building-and/ ◦ https://cloud.google.com/ ◦ https://cloud.google.com/kubernetes-engine/ ◦ https://cloud.google.com/shell/ ◦ https://github.com/GoogleCloudPlatform/kubernetes-engine-samples/ ◦ https://medium.com/google-cloud/deploy-go-application-to-kubernetes-in-30-seconds-ebff0f51d67b • istio ◦ https://istio.io/ ◦ https://github.com/thesandlord/Istio101