Micro services architectures using Gilmour

Micro services architectures using Gilmour

Ee5407f7a79eb620c4fd54c136847b33?s=128

Piyush Verma

February 23, 2016
Tweet

Transcript

  1. GILMOUR MICRO-SERVICE ARCHITECTURES USING

  2. SOA (VS SERVERS) SERVICES ▸ Logical Entities ▸ Independent ▸

    Comprised of more services ▸ Multiple servers for load and tolerance ▸ Scalability ▸ Flexibility
  3. TO ERR IS HUMAN, TO BLAME IT ON SOMEONE IS

    MANAGEMENT POTENTIAL. - Someone on Internet WHY?
  4. WHO DUN IT? WHY? ▸ Software Problem ▸ Function Point

    Scalability ▸ Management Problem ▸ Authority ▸ Responsibility.
  5. DESIGN PATTERNS

  6. ASYNC (SIGNALS & SLOTS)

  7. ASYNC: SIGNALS AND SLOTS Slots Signal item.buy.500 item.buy.* item.buy.* item.buy.*

    group: push_msg FP scaling Push Message Server Push Message Server Web GILMOUR Email notifier SMS notifier Push Message server Item purchased VIP Item purchased item.buy.420
  8. ASYNC: SLOTS func main() { redis := backends.MakeRedis("127.0.0.1:6379", "pass") g

    := G.Get(redis) g.Slot("example.log", func(req *G.Request, _ *G.Message) { var msg string req.Data(&msg) log.Println(req.Sender(), "->", msg) }, nil) g.Start() } line := "Filtering out stop words" g.Signal("example.log", G.NewMessage().SetData(line)) ASYNC: SIGNALS
  9. REQUEST RESPONSE

  10. REQUEST RESPONSE Caller Service Hello? Aloha! Wrong Number type Message

    struct { Data interface{} `json:"data"` //Actual Payload Code int `json:"code"` //Response code if this is a response. This uses HTTP error codes Sender string `json:"sender"` //Origin of the request (unique for each request) } SENDER_ID CODE
  11. REQUEST data := G.NewMessage().SetData("hello?") opts := G.NewRequestOpts().SetTimeout(200) g.Request("echo", data, func(req

    *G.Request, resp *G.Message) { var msg string req.Data(&msg); fmt.Println("Echoclient: received", msg) }, opts) opts := G.NewHandlerOpts().SetGroup("exclusive") g.ReplyTo("echo", func(req *G.Request, resp *G.Message) { var msg string req.Data(&msg) resp.SetData(fmt.Sprintf("Pong %v", msg)) }, opts) RESPONSE
  12. SIGNAL-SLOT VS. REQUEST-RESPONSE

  13. ERRORS: DETECTION AND HANDLING

  14. GILMOUR ERRORS ERROR DETECTION ▸ timeouts ▸ server side -

    execution timed out. ▸ client side - response too late ▸ confirm_subscribers
  15. ERRORS: SIGNALS & SLOTS Web GILMOUR Email notifier SMS notifier

    Push message notifier Slots Error reporter Signal
  16. GILMOUR ERRORS Caller Service Hello? Wrong Number SENDER_ID CODE Error

    reporter type gilmourError struct { topic string //topic to which the request was sent requestData string //request payload userData string //implementation dependent information sender string //sender from the request timestamp string //timestamp of error backtrace interface{} //backtrace of the error code int //response code (non-200) }
  17. GILMOUR ERRORS ERROR HANDLING ▸ Ignore ▸ Publish ▸ Queue

    func main() { redis := backends.MakeRedis("127.0.0.1:6379", "pass") g := G.Get(redis) //g.SetErrorPolicy(protocol.ErrorPolicyQueue) //g.SetErrorPolicy(protocol.ErrorPolicyIgnore) g.SetErrorPolicy(protocol.ErrorPolicyPublish) }
  18. GILMOUR ERRORS ERROR SAMPLE

  19. OPS: MONITORING

  20. OPS: MONITORING SERVERS ▸ Log forwarding & aggregation ▸ Error

    Reporting ▸ Server and Health Monitoring
  21. OPS: MONITORING SERVICES ▸ Are the services servicing? ▸ Do

    services have spare capacity? ▸ Do services respond in time?
  22. OPS: MONITORING HEALTH BULLETIN Feb 23 12:25:14 qa-next-backend-manager0.datascale.io gilmour_health: D,

    [2016-02-23T06:55:12.930435 #22111] DEBUG -- : Ensuring topics have at least one subscriber Feb 23 12:25:14 qa-next-backend-manager0.datascale.io gilmour_health: D, [2016-02-23T06:55:12.931160 #22111] DEBUG -- : Ensuring that gilmour servers respond to health ping Feb 23 12:26:14 qa-next-backend-manager0.datascale.io backend_manager: D, [2016-02-23T06:56:12.932775 #9605] DEBUG -- : Executing handler for gilmour.health.ip-10-0-0-146-pid-9605- uuid-9d0484dd-08e5-4ec3-a3e4-9a109634712b from 2881867d41f24228b462b9eff0b7e88e
  23. OPS: MONITORING HEALTH BULLETIN

  24. OPS: MONITORING HEALTH BULLETIN

  25. 
 COMPOSITION

  26. COMPOSITION ▸ compose - service1 | service2 | composition3 ▸

    andand - service1 && service2 && composition3 ▸ oror - service1 || composition2 || service3 ▸ batch ▸ (service1; service2; service3) > out ▸ service1; composition2; service3 > out ▸ parallel
  27. COMPOSITION popular := g.NewParallel( g.NewRequestComposition("example.popular3"), g.NewRequestComposition("example.popular4"), ) batch := g.NewPipe(

    g.NewRequestComposition("example.fetch"), g.NewRequestComposition("example.words"), g.NewRequestComposition("example.count"), popular, ) data := G.NewMessage() data.SetData("https://s3.amazonaws.com/ds-data-sample/test.txt") msg := <-batch.Execute(data)
  28. COMPOSITION popular := g.NewParallel( g.NewRequestComposition("example.popular3"), g.NewRequestComposition(“example.popular4"), g.NewRequestComposition("example.popular5"), ) batch :=

    g.NewPipe( g.NewRequestComposition("example.fetch"), g.NewRequestComposition("example.words"), g.NewRequestComposition("example.stopfilter"), g.NewRequestComposition("example.count"), popular, &MyComposer{}, ) 
 data := G.NewMessage() data.SetData("https://s3.amazonaws.com/ds-data-sample/test.txt") msg := <-batch.Execute(data)
  29. 
 EXAMPLES

  30. EXAMPLES: ECHO SERVERS & CLIENT

  31. SAMPLE TEXT

  32. EXAMPLES: COMPOSITION

  33. GILMOUR RECAP ▸ Library only; no external process. ▸ No

    Load Balancer ▸ No Service Discovery ▸ Async - Signal, Slots ▸ Sync - Request, Reply ▸ Scaling - Exclusion groups. ▸ Error and Health Monitoring - HealthBulletin. ▸ Failure Detection ▸ Composition - Real micro-services. ▸ No message persistence.
  34. GILMOUR CREDITS & LINKS ▸ Aditya Godbole ▸ aa@gdbl.me ▸

    Datascale.IO ▸ datascale.io ▸ Repository ▸ github.com/gilmour-libs