Upgrade to Pro — share decks privately, control downloads, hide ads and more …

System Integration: OpenStack's success story

System Integration: OpenStack's success story

OpenStack is a huge, open-source cloud provider. One of the main tenets of OpenStack is the (Shared Nothing Architecture) to which all modules stick very closely. In order to do that, services within OpenStack have adopted different strategies to integrate themselves and share data without sacrificing performance nor moving away from SNA.

This strategies are not applicable just to OpenStack but to any distributed system. Sharing data, regardless what that data is, is a must-have requirement of any successful cloud service.

This talk will present some of the existing integration strategies that are applicable to cloud infrastructures and enterprise services. The talk will be based on the strategies that have helped OpenStack to be successful and most importantly, scalable.



May 23, 2014

More Decks by flaper87

Other Decks in Programming


  1. @flaper87 System Integration OpenStack's success story

  2. @flaper87 @flaper87

  3. @flaper87 member of

  4. @flaper87 • GsoC / OPW mentor • Rust language contributor

    • Member of MongoDB Masters • Physics and Philosophy Other things ...
  5. @flaper87 What does integrating a system mean? Theory

  6. @flaper87 Vertical Integration Star Integration Horizontal Integration Theory

  7. @flaper87 Horizontal Integration from an application's perspective Theory

  8. @flaper87 Methods FilesDatabases RPC Messaging

  9. @flaper87 Speaking of files Probably one of the oldest method

    Good for few and very specific cases Try not to use it
  10. @flaper87 Methods Databases RPC Messaging Files

  11. @flaper87 Speaking of databases Asynchronous data-wise Not a message broker

    Probably the most common Great for storing states
  12. @flaper87 Methods RPCMessaging Files Databases

  13. @flaper87 Speaking of RPC Remote Procedure Calls Most used throughout

    OpenStack Message's channels may vary (database, broker, etc) Tightly coupled
  14. @flaper87 Methods Messaging Files Databases RPC

  15. @flaper87 Speaking of Messaging Loosely coupled Add more complexity Commonly

    used for notifications May depend on message routers, transformation, etc.
  16. @flaper87 OpenStack's case Shared Nothing Architecture Databases (Inter-service) RPC (Inter-service)

    Messaging (Cross-service)
  17. @flaper87 Brokers can be are a PITA ← Scaling brokers

    is hard →
  18. @flaper87 Brokers can be are a PITA Brokers need lot

    of memory depending on your use-case
  19. @flaper87 Brokers can be are a PITA Brokers need storage

    if you want messages to be durable
  20. @flaper87 Brokers can be are a PITA Prefer federation over

    centralization AMQP 1.0 Message Router (qpid-dispatch)
  21. @flaper87 Tips / Tricks Transmission protocol matters

  22. @flaper87 Tips / Tricks Use versions for your wire protocol

  23. @flaper87 Tips / Tricks Keep everything explicit

  24. @flaper87 Tips / Tricks Design by contract

  25. @flaper87 Tips / Tricks Keep services isolated... As much as

  26. @flaper87 Q&A Thanks we're hiring