Microservices @ simplesurance

Transcript

1. Microservices @ simplesurance Monika Sobkowiak Golang Meetup Berlin, May 2017
2. What we will talk about? • How an example service look like? • Closer look at some elements of a service • Shared libraries (e.g. logging, metrics, etc.) • Q&A 2
3. Service structure 3 <service-name>/ |- service.go <--- implements the GRPC interface |- cmd/ |- <service-name> |- main.go <--- main package, executable of the Microservice |- store/ |- storer.go <--- definition of the storage interface |- postgres/ |- storage.go <--- implementation of the PostgreSQL storage |- migrations/ <--- SQL files the migration tool to setup the database scheme |- memory/ | storage.go <--- in-memory implementation, can be useful for testing the service without a DB |- provider/ <--- integration with external services, e.g. sms providers, etc. |- deploy/ |- deploy.yml <--- ansible deployment script |- config.ini.dist <--- configuration file template |- Makefile <--- instructions to setup the environment for a service, compile and install it
4. main.go 4 func main() { sisuapp.SetAppInfo(ServiceName, Version) config := config.GetConfig() ctx := metadata.NewContext( context.Background(), metadata.Pairs("trace_id", generator.UUID())) sisulog.SetLogLevel(ctx, config.LogWithStack) if config.ShowVersion { sisulog.Info(ctx, Version) os.Exit(0) } rollbar.Token = config.RbToken rollbar.Environment = config.RbEnv lis, err := net.Listen("tcp", config.AddressRPCListenAddr) if err != nil { sisulog.Fatal(ctx, "Failed to start listening", "error", err) } db, err := pgdb.NewPGDB(pgdb.PGConfig{ ... }) if err != nil { sisulog.Fatal(ctx, "Cannot connect to database", "error", err) } dbCheck := sqldb.NewSQLDBChecker("Address DB", db) if err := dbCheck.SimpleCheck(); err != nil { sisulog.Fatal(ctx, "DB check failed", "error", err) } if config.TestConfig { sisulog.Info(ctx, "Database connection established") os.Exit(0) } storage := postgres.NewPgStorage(db) s := rpcutil.DefaultGRPCServer(rpcutil.MsgSize3MiB) addresspb.RegisterAddressServiceServer(s, address.NewService(storage)) rpcutil.RunServer(s, lis) }
5. Default gRPC Server 5 // DefaultGRPCServer return a new GRPC // server with pre-configured interceptors. // Set maxMsgSize to overwrite the default value in MiB, // Additional interceptors may be passed. func DefaultGRPCServer(maxMsgSize int, extra ...grpc.UnaryServerInterceptor) *grpc.Server { // Set sisu log as GRPC internal logger sdtLogger := sisulog.NewStdLogger() grpclog.SetLogger(sdtLogger) interceptors := []grpc.UnaryServerInterceptor{ NewRecoverPanicUnaryInterceptor(), NewRequestTracingUnaryInterceptor(), grpc_prometheus.UnaryServerInterceptor, } interceptors = append(interceptors, extra...) return grpc.NewServer( grpc_middleware.WithUnaryServerChain(interceptors...), grpc.MaxMsgSize(maxMsgSize), ) }
6. service.go 6 // Service is a struct for address operations type Service struct { storage store.Storer } // NewService initializes and returns a new service func NewService(s store.Storer) *Service { return &Service{ storage: s, } } // CreateAddress creates a new address func (s *Service) CreateAddress(ctx context.Context, request *addresspb.Address) (reply *addresspb.CreateAddressReply, err error) {...} // GetAddress returns an address by given ID func (s *Service) GetAddress(ctx context.Context, request *addresspb.GetAddressRequest) (reply *addresspb.Address, err error) { reply, err = s.storage.Get(ctx, request.Id) if err != nil { sisulog.Error(ctx, "AddressService GetAddress error", "error", err) return nil, sisuerr.UnWrapErr(err) } return reply, err } // DeleteAddress removes an address by given ID func (s *Service) DeleteAddress(ctx context.Context, request *addresspb.DeleteAddressRequest) (reply *addresspb.DeleteAddressReply, err error) {...}
7. Storer interface 7 // Storer is an interface for addresses storage type Storer interface { // Get retrieves an address from a storage Get(ctx context.Context, id string) (*addresspb.Address, error) // Create adds an address to a storage Create(ctx context.Context, addr *addresspb.Address) error // Get removes an address from a storage Delete(ctx context.Context, id string) error }
8. PostgreSQL storer • xo package - for generating go code based on database schema ◦ easy to use command line tool ◦ custom templates for additional functionalities (e.g. pagination) ◦ xo pgsql://user:pass@host/dbname -o models • more complex queries created manually 8
9. Shared libraries 9
10. Logging - sisulog • wrapper around log15 package • uses context to pass by trace_id and auth_user_id • outputs the messages in easy to parse and search format • logs errors to rollbar Example: url := "http://google.com" err := errors.New("terrible error") sisulog.Info(ctx, "Test log message", "url", url, "error", err) Test log message url=http://google.de error="terrible error" trace_id=123 auth_user_id=123 10
11. Error handling - sisuerr • wrapper around Dave Cheney’s errors package • context aware • WrapErr() - ensures no error of the lower layers is lost and a call trace is created for the error path • UnwrapErr() - returns unwrapped error, used when leaving the application 11
12. Metric collection - metrics • helpers for service instrumentation • prometheus for collecting basic metrics • grafana for visualisation Example: func SomeFunc(ctx context.Context, in request) (reply, error) { defer metrics.ElapsedTime("some_func_elapsed", time.Now()) ... } 12
13. Other libs • config - parses configuration files, provides helper functions to setup commonly used parameters • pgdb - used to establish a connection to a PostgreSQL DB and manage the PostgreSQL configuration • sisuio - used to close I/O streams that can return an error; logs an error if Close() fails • … 13
14. References • xo package • log15 package • errors package 14
15. Thank you!