Lock in $30 Savings on PRO—Offer Ends Soon! ⏳

Consul @ PagerDuty

Ranjib Dey
November 17, 2016

Consul @ PagerDuty

How we use consul at PagerDuty across containerized microservices, legacy applications and some custom workflows

Ranjib Dey

November 17, 2016
Tweet

More Decks by Ranjib Dey

Other Decks in Programming

Transcript

  1. • Scope • Deployment Architecture • Use cases • Q

    & A Agenda Hashicorp User Group-SF, November 16th 2016
  2. • SRE@PagerDuty • All things Automation • Chef, Containers (OpenVZ/LXC/docker/rkt/ns

    pawn..) • FOSS Enthusiast • @RanjibDey About me Hashicorp User Group-SF, November 16th 2016
  3. • Introduced early October, 2015 • Agent is present in

    entire infrastructure • Different features used in different parts of the system • Almost 12-15% of the current infra is using consul in one or other way (mostly around containerized workload) • Incremental deployment across mission critical services Hashicorp User Group-SF, November 16th 2016
  4. dc-1 dc-2 dc-3 consul-masters consul-masters consul-masters fleet fleet fleet Hashicorp

    User Group-SF, November 16th 2016 consul-replicate consul-replicate
  5. producer service host consul-agent consul-master Hashicorp User Group-SF, November 16th

    2016 health check consumer service host consul-agent service process DNS lookup
  6. producer service host consul-agent consul-master Hashicorp User Group-SF, November 16th

    2016 consumer service host consul-agent service process
  7. • Less frequent or one-off communication • Example: Obtaining configuration

    endpoint or vault end point on application startup • Bad for load balancing Hashicorp User Group-SF, November 16th 2016
  8. producer service host consul-agent consul-master Hashicorp User Group-SF, November 16th

    2016 health check consumer service host consul-agent service process HTTP API Helper library
  9. producer service host consul-agent consul-master Hashicorp User Group-SF, November 16th

    2016 consumer service host consul-agent service process Helper library
  10. • Communication in between backends • Can be of any

    protocol (binary or plain text alike) • Actual performance and user experience will vary among the client libraries • Most dynamic language based libraries are not very good in this. C, Java, go has some decent options, but limited to certain tech stacks Hashicorp User Group-SF, November 16th 2016
  11. producer service host consul-agent consul-master Hashicorp User Group-SF, November 16th

    2016 health check consumer service host consul-template haproxy
  12. producer service host consul-agent consul-master Hashicorp User Group-SF, November 16th

    2016 consumer service host consul-template haproxy producer service host consul-agent producer service host consul-agent
  13. • Communication from app servers to DB • Analytics stack

    (Haddop, Tez) • Almost all low latency, binary protocol based stuff • TCP based communications Hashicorp User Group-SF, November 16th 2016
  14. Hashicorp User Group-SF, November 16th 2016 Nomad cluster Nginx+ &

    consul-template Nginx+ & consul-template Container-1 Container-1 Container-1 Container-1 Container-1 Container-1 Container-1 Container-1 consul-master Nomad-master
  15. • Front end load balancing • Clients, customers, partners, channels

    to your software end points • HTTP, websocket, spdy • Fancier endpoint or host header based routing Hashicorp User Group-SF, November 16th 2016
  16. Configuration repository consul-master Hashicorp User Group-SF, November 16th 2016 consumer

    service host consul-template config file Developers PR workflow GoCD git2consul
  17. dc-1 dc-2 dc-3 consul-master Hashicorp User Group-SF, November 16th 2016

    consul-replicate consul-replicate Entire infra register consul-master consul-master host1 host1 host1 host1 host1 host1 host1 host1 host1 host1 runner runner runner runner runner runner runner runner runner runner