Building a Go microservice from 0 to Hero

Slides of "Building a Go microservice from 0 to Hero" talk made at GoLang Meetup Marseille, France (October 10, 2018) #golang #microservices #gcp #k8s

Transcript

1. Building a Go microservice from 0 to Hero GoLang Marseille Meetup @FedirFr
2. Fedir RYKHTIK • Building open source web since 2007 ◦ Back-end developer ◦ Independent scientist ◦ DevOps / SA • CTO @AgenceStratis since 2015 • Gopher
3. Plan • Introduction to microservices • Go microservices frameworks & toolkits • Packaging • Hosting & Deployment • Q&A
4. Introduction to microservices
5. • Complex applications are composed of small, independent processes communicating with each other using language-agnostic APIs • Highly decoupled and focus on doing a small task Microservices
6. http://martinfowler.com/articles/microservices.html
7. “There are certain things you need to get sorted out before you can put your first microservices system into production: monitoring, provisioning, and a devops culture.” Martin Fowler http://martinfowler.com/bliki/MicroservicePrerequisites.html
8. (micro)service oriented architecture (SOA) typical problems • Service discovery • Monitoring • Logging • Load balancing • Circuit breaking • Scalability
9. Go microservices frameworks & toolkits
10. Go microsevice toolkits statistics rating • The most popular project is go-kit/kit • The newest project is NYTimes/gizmo • The project with more commits is koding/kite • The project with biggest average contribution period is rsms/gotalk • The project with more tags is micro/go-micro • The project made by most notable top contributors is koding/kite • The project with best errors resolving rate is micro/go-micro • The project with more commits by day is koding/kite • The project with the most active number of forkers is NYTimes/gizmo • The project with biggest number of returning contributors is go-kit/kit • The best project (taking in account placements in all competitions) is go-kit/kit Details: https://docs.google.com/spreadsheets/d/1d0WcI5ahmJdysyyHeZB9CVlolXF027UlY1fTPbyiFCE/edit?usp=sharing *Statistics made with github.com/fedir/ghstat comparator
11. Go Micro Great for beginners, more opionionated, than others. Code generator. Enforces gRPC https://github.com/micro/go-micro Gizmo Good for pub/sub use cases gRPC, HTTP https://github.com/NYTimes/gizmo Kite Good for beginners, difficult to integrate with other solutions Websockets + RPC https://github.com/koding/kite Go microservices toolkits Go-Kit Very much production ready. Flexible. Patterns are a bit complex, especially for beginners (SOLID, DDD, hexagonal). gRPC, HTTP, Thrift https://github.com/go-kit/kit
12. go-micro example package main import ( "context" "log" proto "github.com/micro/examples/helloworld/proto" "github.com/micro/go-micro" ) type Greeter struct{} func (g *Greeter) Hello(ctx context.Context, req *proto.HelloRequest, rsp *proto.HelloResponse) error { rsp.Greeting = "Hello " + req.Name return nil } func main() { service := micro.NewService( micro.Name("greeter"), ) service.Init() proto.RegisterGreeterHandler(service.Server(), new(Greeter)) if err := service.Run(); err != nil { log.Fatal(err) } } Ref.: https://github.com/micro/examples/blob/master/helloworld/main.go
13. go-kit example package main import ( "context" "encoding/json" "errors" "log" "net/http" "strings" "github.com/go-kit/kit/endpoint" httptransport "github.com/go-kit/kit/transport/http" ) // StringService provides operations on strings. type StringService interface { Uppercase(string) (string, error) Count(string) int } // stringService is a concrete implementation of StringService type stringService struct{} func (stringService) Uppercase(s string) (string, error) { if s == "" { return "", ErrEmpty } return strings.ToUpper(s), nil } func (stringService) Count(s string) int { return len(s) } // ErrEmpty is returned when an input string is empty. var ErrEmpty = errors.New("empty string") // For each method, we define request and response structs type uppercaseRequest struct { S string `json:"s"` } type uppercaseResponse struct { V string `json:"v"` Err string `json:"err,omitempty"` // errors don't define JSON marshaling } type countRequest struct { S string `json:"s"` } type countResponse struct { V int `json:"v"` } // Endpoints are a primary abstraction in go-kit. An endpoint represents a single RPC (method in our service interface) func makeUppercaseEndpoint(svc StringService) endpoint.Endpoint { return func(_ context.Context, request interface{}) (interface{}, error) { req := request.(uppercaseRequest) v, err := svc.Uppercase(req.S) if err != nil { return uppercaseResponse{v, err.Error()}, nil } return uppercaseResponse{v, ""}, nil } } func makeCountEndpoint(svc StringService) endpoint.Endpoint { return func(_ context.Context, request interface{}) (interface{}, error) { req := request.(countRequest) v := svc.Count(req.S) return countResponse{v}, nil } } // Transports expose the service to the network. In this first example we utilize JSON over HTTP. func main() { svc := stringService{} uppercaseHandler := httptransport.NewServer( makeUppercaseEndpoint(svc), decodeUppercaseRequest, encodeResponse, ) countHandler := httptransport.NewServer( makeCountEndpoint(svc), decodeCountRequest, encodeResponse, ) http.Handle("/uppercase", uppercaseHandler) http.Handle("/count", countHandler) log.Fatal(http.ListenAndServe(":8080", nil)) } func decodeUppercaseRequest(_ context.Context, r *http.Request) (interface{}, error) { var request uppercaseRequest if err := json.NewDecoder(r.Body).Decode(&request); err != nil { return nil, err } return request, nil } func decodeCountRequest(_ context.Context, r *http.Request) (interface{}, error) { var request countRequest if err := json.NewDecoder(r.Body).Decode(&request); err != nil { return nil, err } return request, nil } func encodeResponse(_ context.Context, w http.ResponseWriter, response interface{}) error { return json.NewEncoder(w).Encode(response) } Ref.: https://github.com/go-kit/kit/blob/master/examples/stringsvc1/main.go
14. Please choose Your framework now The red pill and its opposite, the blue pill, are a metaphor representing the choice between: • Knowledge, freedom and the brutal truths of reality (red pill) • Happiness, beauty, and the blissful ignorance of illusion (blue pill)
15. Packaging
16. packaging > github Dockerfile
17. Dockerfile FROM golang:1.8-alpine ADD . /go/src/hello-app RUN go install hello-app FROM alpine:latest COPY --from=0 /go/bin/hello-app . ENV PORT 8080 CMD ["./hello-app"] Ref.: https://github.com/GoogleCloudPlatform/kubernetes-engine-samples/blob/master/hello-app/Dockerfile
18. Hosting & Deployment
19. google cloud + k8s
20. Dockerfile deployment schema Docker image git k8s cluster
21. 1. https://console.cloud.google.com/ 2. Creation a new project "meetup-golang-marseille-03" 3. Creation a new Kubernetes cluster "gmm03" (take some time) 4. Use of Cloud shell 5. Cloneing of the project from the git(hub/lab/...) repository 6. Building the container & Deploying it on the cluster 7. Checking status and getting public IP 8. Testing via HTTP from the developer's PC 9. Changing cluster size 10. Testing via HTTP again from the developer's PC 11. App modification 12. Redeploying it 13. Testing via HTTP again from the developer's PC k8s demo steps
22. Tools - kubectl kubectl controls the Kubernetes cluster manager. Used Commands : run Run a particular image on the cluster expose Take a replication controller, service, deployment or pod and expose it as a new Kubernetes Service get Display one or many resources set Set specific features on objects Find more information at: https://kubernetes.io/docs/reference/kubectl/overview/
23. gcloud manage Google Cloud Platform resources and developer workflow. Used commands : container deploy and manage clusters of machines for running containers docker enable Docker CLI access to Google Container Registry Find more information at: https://cloud.google.com/sdk/gcloud/reference/container/ Tools - gcloud
24. Create cluster and deploy the app git clone https://github.com/GoogleCloudPlatform/kubernetes-engine-samples cd kubernetes-engine-samples/hello-app gcloud container clusters get-credentials gmm03 --zone europe-west1-b docker build -t gcr.io/meetup-golang-marseille-03/hello-node:v1 $PWD gcloud docker -- push gcr.io/meetup-golang-marseille-03/hello-node:v1 kubectl run hello-node --image=gcr.io/meetup-golang-marseille-03/hello-node:v1 --port=8080 kubectl expose deployment hello-node --type="LoadBalancer"
25. Get external IP (takes some time) kubectl get service hello-node --watch
26. Modify and redeploy the app sed -i -e 's/1.0.0/2.0.0/g' main.go docker build -t gcr.io/meetup-golang-marseille-03/hello-node:v2 $PWD gcloud docker -- push gcr.io/meetup-golang-marseille-03/hello-node:v2 kubectl set image deployment/hello-node hello-node=gcr.io/meetup-golang-marseille-03/hello-node:v2 && echo 'image updated' kubectl get service hello-node
27. Q&A Any questions ?
28. Recommended books
29. Ressources & related materials • microservices in go ◦ https://medium.com/seek-blog/microservices-in-go-2fc1570f6800 ◦ https://mycodesmells.com/post/little-bit-about-go-kit ◦ https://www.youtube.com/watch?v=1AjaZi4QuGo ◦ https://www.youtube.com/watch?v=NX0sHF8ZZgw ◦ https://www.youtube.com/watch?v=1ScP5DyS1_g ◦ https://www.youtube.com/watch?v=OcjMi9cXItY ◦ https://dev.to/plutov/microservices-with-go-kit-part-1-13dd ◦ https://dev.to/plutov/packagemain-13-microservices-with-go-kit-part-2-4lgh ◦ https://about.sourcegraph.com/go/gophercon-2018-from-prototype-to-production-lessons-from-building-and/ ◦ https://github.com/ru-rocker/gokit-playground ◦ https://www.reddit.com/r/golang/comments/5tesl9/why_i_recommend_to_avoid_using_the_gokit_library/ ◦ http://gokit.io/examples/stringsvc.html ◦ http://www.ru-rocker.com/2017/02/17/micro-services-using-go-kit-http-endpoint/ ◦ https://github.com/micro/examples ◦ https://github.com/udacity/ud615 ◦ https://github.com/matryer/goblueprints • docker ◦ https://docs.docker.com/engine/reference/builder/ https://github.com/GoogleCloudPlatform/kubernetes-engine-samples/blob/master/hello-app/Dockerfile • google cloud & k8s ◦ https://about.sourcegraph.com/go/gophercon-2018-from-prototype-to-production-lessons-from-building-and/ ◦ https://cloud.google.com/ ◦ https://cloud.google.com/kubernetes-engine/ ◦ https://cloud.google.com/shell/ ◦ https://github.com/GoogleCloudPlatform/kubernetes-engine-samples/ ◦ https://medium.com/google-cloud/deploy-go-application-to-kubernetes-in-30-seconds-ebff0f51d67b • istio ◦ https://istio.io/ ◦ https://github.com/thesandlord/Istio101
30. Merci :) @FedirFR