Slide 1

Slide 1 text

Building a Go microservice from 0 to Hero GoLang Marseille Meetup @FedirFr

Slide 2

Slide 2 text

Fedir RYKHTIK ● Building open source web since 2007 ○ Back-end developer ○ Independent scientist ○ DevOps / SA ● CTO @AgenceStratis since 2015 ● Gopher

Slide 3

Slide 3 text

Plan ● Introduction to microservices ● Go microservices frameworks & toolkits ● Packaging ● Hosting & Deployment ● Q&A

Slide 4

Slide 4 text

Introduction to microservices

Slide 5

Slide 5 text

● Complex applications are composed of small, independent processes communicating with each other using language-agnostic APIs ● Highly decoupled and focus on doing a small task Microservices

Slide 6

Slide 6 text

http://martinfowler.com/articles/microservices.html

Slide 7

Slide 7 text

“There are certain things you need to get sorted out before you can put your first microservices system into production: monitoring, provisioning, and a devops culture.” Martin Fowler http://martinfowler.com/bliki/MicroservicePrerequisites.html

Slide 8

Slide 8 text

(micro)service oriented architecture (SOA) typical problems ● Service discovery ● Monitoring ● Logging ● Load balancing ● Circuit breaking ● Scalability

Slide 9

Slide 9 text

Go microservices frameworks & toolkits

Slide 10

Slide 10 text

Go microsevice toolkits statistics rating ● The most popular project is go-kit/kit ● The newest project is NYTimes/gizmo ● The project with more commits is koding/kite ● The project with biggest average contribution period is rsms/gotalk ● The project with more tags is micro/go-micro ● The project made by most notable top contributors is koding/kite ● The project with best errors resolving rate is micro/go-micro ● The project with more commits by day is koding/kite ● The project with the most active number of forkers is NYTimes/gizmo ● The project with biggest number of returning contributors is go-kit/kit ● The best project (taking in account placements in all competitions) is go-kit/kit Details: https://docs.google.com/spreadsheets/d/1d0WcI5ahmJdysyyHeZB9CVlolXF027UlY1fTPbyiFCE/edit?usp=sharing *Statistics made with github.com/fedir/ghstat comparator

Slide 11

Slide 11 text

Go Micro Great for beginners, more opionionated, than others. Code generator. Enforces gRPC https://github.com/micro/go-micro Gizmo Good for pub/sub use cases gRPC, HTTP https://github.com/NYTimes/gizmo Kite Good for beginners, difficult to integrate with other solutions Websockets + RPC https://github.com/koding/kite Go microservices toolkits Go-Kit Very much production ready. Flexible. Patterns are a bit complex, especially for beginners (SOLID, DDD, hexagonal). gRPC, HTTP, Thrift https://github.com/go-kit/kit

Slide 12

Slide 12 text

go-micro example package main import ( "context" "log" proto "github.com/micro/examples/helloworld/proto" "github.com/micro/go-micro" ) type Greeter struct{} func (g *Greeter) Hello(ctx context.Context, req *proto.HelloRequest, rsp *proto.HelloResponse) error { rsp.Greeting = "Hello " + req.Name return nil } func main() { service := micro.NewService( micro.Name("greeter"), ) service.Init() proto.RegisterGreeterHandler(service.Server(), new(Greeter)) if err := service.Run(); err != nil { log.Fatal(err) } } Ref.: https://github.com/micro/examples/blob/master/helloworld/main.go

Slide 13

Slide 13 text

go-kit example package main import ( "context" "encoding/json" "errors" "log" "net/http" "strings" "github.com/go-kit/kit/endpoint" httptransport "github.com/go-kit/kit/transport/http" ) // StringService provides operations on strings. type StringService interface { Uppercase(string) (string, error) Count(string) int } // stringService is a concrete implementation of StringService type stringService struct{} func (stringService) Uppercase(s string) (string, error) { if s == "" { return "", ErrEmpty } return strings.ToUpper(s), nil } func (stringService) Count(s string) int { return len(s) } // ErrEmpty is returned when an input string is empty. var ErrEmpty = errors.New("empty string") // For each method, we define request and response structs type uppercaseRequest struct { S string `json:"s"` } type uppercaseResponse struct { V string `json:"v"` Err string `json:"err,omitempty"` // errors don't define JSON marshaling } type countRequest struct { S string `json:"s"` } type countResponse struct { V int `json:"v"` } // Endpoints are a primary abstraction in go-kit. An endpoint represents a single RPC (method in our service interface) func makeUppercaseEndpoint(svc StringService) endpoint.Endpoint { return func(_ context.Context, request interface{}) (interface{}, error) { req := request.(uppercaseRequest) v, err := svc.Uppercase(req.S) if err != nil { return uppercaseResponse{v, err.Error()}, nil } return uppercaseResponse{v, ""}, nil } } func makeCountEndpoint(svc StringService) endpoint.Endpoint { return func(_ context.Context, request interface{}) (interface{}, error) { req := request.(countRequest) v := svc.Count(req.S) return countResponse{v}, nil } } // Transports expose the service to the network. In this first example we utilize JSON over HTTP. func main() { svc := stringService{} uppercaseHandler := httptransport.NewServer( makeUppercaseEndpoint(svc), decodeUppercaseRequest, encodeResponse, ) countHandler := httptransport.NewServer( makeCountEndpoint(svc), decodeCountRequest, encodeResponse, ) http.Handle("/uppercase", uppercaseHandler) http.Handle("/count", countHandler) log.Fatal(http.ListenAndServe(":8080", nil)) } func decodeUppercaseRequest(_ context.Context, r *http.Request) (interface{}, error) { var request uppercaseRequest if err := json.NewDecoder(r.Body).Decode(&request); err != nil { return nil, err } return request, nil } func decodeCountRequest(_ context.Context, r *http.Request) (interface{}, error) { var request countRequest if err := json.NewDecoder(r.Body).Decode(&request); err != nil { return nil, err } return request, nil } func encodeResponse(_ context.Context, w http.ResponseWriter, response interface{}) error { return json.NewEncoder(w).Encode(response) } Ref.: https://github.com/go-kit/kit/blob/master/examples/stringsvc1/main.go

Slide 14

Slide 14 text

Please choose Your framework now The red pill and its opposite, the blue pill, are a metaphor representing the choice between: ● Knowledge, freedom and the brutal truths of reality (red pill) ● Happiness, beauty, and the blissful ignorance of illusion (blue pill)

Slide 15

Slide 15 text

Packaging

Slide 16

Slide 16 text

packaging > github Dockerfile

Slide 17

Slide 17 text

Dockerfile FROM golang:1.8-alpine ADD . /go/src/hello-app RUN go install hello-app FROM alpine:latest COPY --from=0 /go/bin/hello-app . ENV PORT 8080 CMD ["./hello-app"] Ref.: https://github.com/GoogleCloudPlatform/kubernetes-engine-samples/blob/master/hello-app/Dockerfile

Slide 18

Slide 18 text

Hosting & Deployment

Slide 19

Slide 19 text

google cloud + k8s

Slide 20

Slide 20 text

Dockerfile deployment schema Docker image git k8s cluster

Slide 21

Slide 21 text

1. https://console.cloud.google.com/ 2. Creation a new project "meetup-golang-marseille-03" 3. Creation a new Kubernetes cluster "gmm03" (take some time) 4. Use of Cloud shell 5. Cloneing of the project from the git(hub/lab/...) repository 6. Building the container & Deploying it on the cluster 7. Checking status and getting public IP 8. Testing via HTTP from the developer's PC 9. Changing cluster size 10. Testing via HTTP again from the developer's PC 11. App modification 12. Redeploying it 13. Testing via HTTP again from the developer's PC k8s demo steps

Slide 22

Slide 22 text

Tools - kubectl kubectl controls the Kubernetes cluster manager. Used Commands : run Run a particular image on the cluster expose Take a replication controller, service, deployment or pod and expose it as a new Kubernetes Service get Display one or many resources set Set specific features on objects Find more information at: https://kubernetes.io/docs/reference/kubectl/overview/

Slide 23

Slide 23 text

gcloud manage Google Cloud Platform resources and developer workflow. Used commands : container deploy and manage clusters of machines for running containers docker enable Docker CLI access to Google Container Registry Find more information at: https://cloud.google.com/sdk/gcloud/reference/container/ Tools - gcloud

Slide 24

Slide 24 text

Create cluster and deploy the app git clone https://github.com/GoogleCloudPlatform/kubernetes-engine-samples cd kubernetes-engine-samples/hello-app gcloud container clusters get-credentials gmm03 --zone europe-west1-b docker build -t gcr.io/meetup-golang-marseille-03/hello-node:v1 $PWD gcloud docker -- push gcr.io/meetup-golang-marseille-03/hello-node:v1 kubectl run hello-node --image=gcr.io/meetup-golang-marseille-03/hello-node:v1 --port=8080 kubectl expose deployment hello-node --type="LoadBalancer"

Slide 25

Slide 25 text

Get external IP (takes some time) kubectl get service hello-node --watch

Slide 26

Slide 26 text

Modify and redeploy the app sed -i -e 's/1.0.0/2.0.0/g' main.go docker build -t gcr.io/meetup-golang-marseille-03/hello-node:v2 $PWD gcloud docker -- push gcr.io/meetup-golang-marseille-03/hello-node:v2 kubectl set image deployment/hello-node hello-node=gcr.io/meetup-golang-marseille-03/hello-node:v2 && echo 'image updated' kubectl get service hello-node

Slide 27

Slide 27 text

Q&A Any questions ?

Slide 28

Slide 28 text

Recommended books

Slide 29

Slide 29 text

Ressources & related materials ● microservices in go ○ https://medium.com/seek-blog/microservices-in-go-2fc1570f6800 ○ https://mycodesmells.com/post/little-bit-about-go-kit ○ https://www.youtube.com/watch?v=1AjaZi4QuGo ○ https://www.youtube.com/watch?v=NX0sHF8ZZgw ○ https://www.youtube.com/watch?v=1ScP5DyS1_g ○ https://www.youtube.com/watch?v=OcjMi9cXItY ○ https://dev.to/plutov/microservices-with-go-kit-part-1-13dd ○ https://dev.to/plutov/packagemain-13-microservices-with-go-kit-part-2-4lgh ○ https://about.sourcegraph.com/go/gophercon-2018-from-prototype-to-production-lessons-from-building-and/ ○ https://github.com/ru-rocker/gokit-playground ○ https://www.reddit.com/r/golang/comments/5tesl9/why_i_recommend_to_avoid_using_the_gokit_library/ ○ http://gokit.io/examples/stringsvc.html ○ http://www.ru-rocker.com/2017/02/17/micro-services-using-go-kit-http-endpoint/ ○ https://github.com/micro/examples ○ https://github.com/udacity/ud615 ○ https://github.com/matryer/goblueprints ● docker ○ https://docs.docker.com/engine/reference/builder/ https://github.com/GoogleCloudPlatform/kubernetes-engine-samples/blob/master/hello-app/Dockerfile ● google cloud & k8s ○ https://about.sourcegraph.com/go/gophercon-2018-from-prototype-to-production-lessons-from-building-and/ ○ https://cloud.google.com/ ○ https://cloud.google.com/kubernetes-engine/ ○ https://cloud.google.com/shell/ ○ https://github.com/GoogleCloudPlatform/kubernetes-engine-samples/ ○ https://medium.com/google-cloud/deploy-go-application-to-kubernetes-in-30-seconds-ebff0f51d67b ● istio ○ https://istio.io/ ○ https://github.com/thesandlord/Istio101

Slide 30

Slide 30 text

Merci :) @FedirFR