Taking Django Distributed

Transcript

1. Taking Django Distributed Andrew Godwin @andrewgodwin Taking Django Distributed

2. Hi, I’m Andrew Godwin • Django core developer • Senior

   Software Engineer at • Needs to stop running towards code on fire

3. Computers hate you.

4. This makes distributed hard.

5. 2001: A Space Odyssey Copyright Warner Brothers

6. It’s time to split things up a bit. But how?

   And why?

7. Code Databases Team

8. There is no one solution

9. Read-heavy? Write-heavy? Spiky? Predictable? Chatty?

10. Code

11. Use apps! They’re a good start! Ignore the way I

    wrote code for the first 5 years of Django.

12. Formalise interfaces between apps Preferably in an RPC style

13. Split along those interfaces Into separate processes, or machines

14. Inventory Payments Presentation

15. How do you communicate? HTTP? Channels? Smoke signals?

16. None
17. None
18. None
19. None

20. Databases

21. Users Vertically Partitioned Database Images Comments

22. Main DB Replica Replica Replica Single main database with replication

23. Partition Tolerant Available Consistent

24. Non-consistency is everywhere It’s sneaky like that

25. National Museum of American History

26. Load Balancing

27. Equally balanced servers Consistent load times Similar users

28. Split logic Different processor loads Wildly varying users

29. Reqs Time

30. Reqs Time

31. W E B S O C K E T S

32. W E B S O C K E T S

    • They can last for hours • There’s not many tools that handle them • They have 4 different kinds of failure

33. Design for failure, and then use it! Kill off sockets

    early and often.

34. Team

35. Developers are people too! They need time and interesting things

36. Technical debt can be poisonous But you need a little

    bit to compete

37. Single repo? Multiple repos? Each has distinct advantages.

38. Teams per service? Split responsibility? Do you split ops/QA across

    teams too?

39. Ownership gaps They’re very hard to see.

40. Strategies

41. Don’t go too micro on those services It’s easier in

    the short term, but will confuse you in the long term.

42. Communicate over a service bus Preferably Channels, but you get

    to choose.

43. Work out where to allow old data Build in deliberate

    caching or read only modes

44. Design for future sharding Route everything through one model or

    set of functions

45. Expect long-polls/sockets to die Design for load every time, and

    treat as a happy optimisation

46. Independent, full-stack teams From ops to frontend, per major service

47. Architect as a part-time position You need some, but not

    in an ivory tower

48. 2001: A Space Odyssey Copyright Warner Brothers

49. Maybe, just maybe, keep that monolith A well maintained and

    separated one beats bad distributed

50. Thanks. Andrew Godwin @andrewgodwin aeracode.org