On Distributed Systems and Distributed Teams at Elastic

A wall clock that ran backward and a skull and crossbones flag. Fictional parliamentary systems of ancient civilizations. @all threads that run into the several hundred replies. All are interesting stories and illustrate concepts that make Elastic...elastic?

The distributed systems movement, and open source more broadly, is fueled by solving a series of complex problems: consensus, leader election, failure semantics, among others. Our reasoning about distributed concepts influences system design and implementation. Our understanding impacts that which we build...particularly when we build distributed companies. How does a distributed team building distributed systems, at Elastic, function?

Tyler Hannan l Director, Product Marketing l Elastic

Transcript

1. Elastic
   07/03/2017
   @tylerhannan
   On Distributed Systems and Distributed Teams
   Tyler Hannan, Director Product & Technical Marketing
   

   View Slide

2. whoami?
   

   View Slide

3. View Slide

4. View Slide

5. View Slide

6. View Slide

7. distributed systems
   and
   distributed teams
   

   View Slide

8. View Slide

9. and
   

   View Slide

10. View Slide

11. an expression whose
    meaning refers to a
    figurative meaning
    

    View Slide

12. known through common
    use
    (i.e. kick the bucket)
    

    View Slide

13. View Slide

14. ‘We compare apples and
    oranges all the time!
    

    View Slide

15. We compare them by
    price,
    

    View Slide

16. by how much we like
    the taste,
    

    View Slide

17. by the likely sweetness
    and ripeness,
    

    View Slide

18. by how well they'll go in
    a tasty fruit cocktail and
    so on.
    

    View Slide

19. In fact, every time we go
    to the store and buy
    

    View Slide

20. apples rather than
    oranges
    -- or vice versa --
    

    View Slide

21. we are necessarily
    

    View Slide

22. (if implicitly)
    

    View Slide

23. comparing apples and
    oranges.'
    

    View Slide

24. ‘apples and democracy’
    

    View Slide

25. ‘oranges and the
    multiplication table’
    

    View Slide

26. distributed
    by
    design
    

    View Slide

27. 'A distributed system is a software
    system in which components located
    on network computers communicate
    and coordinate their actions by
    passing messages.
    

    View Slide

28. The components interact with each
    other in order to achieve a common
    goal.'
    Wikipedia
    

    View Slide

29. 'A distributed system is one
    which the failure of a computer
    you didn't even know existed
    can render your own computer
    unusable'
    Leslie Lamport
    

    View Slide

30. 'In pioneer days, they used
    oxen for heavy pulling, and
    when one ox couldn't budge a
    log, they didn't try to grow a
    bigger ox.'
    Grace Hopper
    

    View Slide

31. 299,792,458
    

    View Slide

32. View Slide

33. View Slide

34. 225,308,160
    

    View Slide

35. latency
    

    View Slide

36. ~10ms
    

    View Slide

37. 1%
    

    View Slide

38. 1906 London
    

    View Slide

39. View Slide

40. villagers,
    

    View Slide

41. farmers,
    

    View Slide

42. ranchers,
    

    View Slide

43. doctors,
    

    View Slide

44. men,
    

    View Slide

45. women,
    

    View Slide

46. children,
    

    View Slide

47. professionals,
    

    View Slide

48. laborers,
    

    View Slide

49. 787 guesses
    

    View Slide

50. 1,197 pounds
    

    View Slide

51. weight for it…
    

    View Slide

52. 1,198 pounds.
    

    View Slide

53. 'The results seem more
    creditable to the
    trustworthiness of a
    democratic judgment...'
    

    View Slide

54. collective intelligence
    

    View Slide

55. ‘group think’
    

    View Slide

56. Client Proposer Acceptor Learner
    | | | | | | | --- First Request ---
    X-------->| | | | | | Request
    | X--------->|->|->| | | Prepare(N)
    | |<---------X--X--X | | Promise(N,I,{Va,Vb,Vc})
    | X--------->|->|->| | | Accept!(N,I,Vm)
    | |<---------X--X--X------>|->| Accepted(N,I,Vm)
    |<---------------------------------X--X Response
    | | | | | | |
    

    View Slide

57. the part time parliament
    

    View Slide

58. View Slide

59. Paxos
    

    View Slide

60. 'Writing about a lost civilization
    allowed me to eliminate
    uninteresting details and indicate
    generalizations by saying that
    some details of the parliamentary
    protocol had been lost.
    

    View Slide

61. To carry the image further, I
    gave a few lectures in the
    persona of an Indiana-Jones-
    style archaeologist, replete
    with Stetson hat and hip flask.
    

    View Slide

62. My attempt at inserting
    some humor into the
    subject was a dismal
    failure.’
    

    View Slide

63. Client Proposer Acceptor Learner
    | | | | | | | --- First Request ---
    X-------->| | | | | | Request
    | X--------->|->|->| | | Prepare(N)
    | |<---------X--X--X | | Promise(N,I,{Va,Vb,Vc})
    | X--------->|->|->| | | Accept!(N,I,Vm)
    | |<---------X--X--X------>|->| Accepted(N,I,Vm)
    |<---------------------------------X--X Response
    | | | | | | |
    

    View Slide

64. consensus
    

    View Slide

65. message was lost
    in the medium
    

    View Slide

66. the delivery clouded the
    applicability
    

    View Slide

67. distributed
    by
    design
    

    View Slide

68. partial failure
    

    View Slide

69. distributed teams
    by
    design
    

    View Slide

70. View Slide

71. View Slide

72. View Slide

73. View Slide

74. or, perhaps,
    

    View Slide

75. 463 employees
    

    View Slide

76. 32 countries
    

    View Slide

77. at least 19 timezones
    

    View Slide

78. 30+ spoken languages
    

    View Slide

79. we aren’t remote.
    

    View Slide

80. we are distributed.
    

    View Slide

81. it requires intent
    

    View Slide

82. electronic
    communications
    lose
    nonverbal cues
    

    View Slide

83. active,
    focused,
    and truly invested
    

    View Slide

84. some all@ emails
    

    View Slide

85. become t-shirts
    

    View Slide

86. tiij ji je
    

    View Slide

87. UzzrzOu))6,)6.32!&)8&&&
    

    View Slide

88. View Slide

89. View Slide

90. View Slide

91. some all@ emails
    

    View Slide

92. become public
    statements
    

    View Slide

93. View Slide

94. ‘a more
    diverse company
    will make for
    

    View Slide

95. a better company,’
    

    View Slide

96. and we should
    continuously strive for
    it.
    

    View Slide

97. distributed availability
    

    View Slide

98. View Slide

99. View Slide

100. ‘your location is
     

     View Slide

101. a descriptor
     

     View Slide

102. not a defining attribute.’
     

     View Slide

103. elastic heart
     

     View Slide

104. View Slide

105. View Slide

106. View Slide

107. software is all about
     

     View Slide

108. ‘what does your
     company do?’
     

     View Slide

109. ‘why do you live in
     amsterdam?’
     

     View Slide

110. ‘what tools do you use?’
     

     View Slide

111. wrong question.
     

     View Slide

112. in my life
     

     View Slide

113. there are humans.
     

     View Slide

114. humans
     i work for.
     

     View Slide

115. humans
     i work with.
     

     View Slide

116. humans
     who work for me.
     

     View Slide

117. humans
     i raise.
     

     View Slide

118. View Slide

119. View Slide

120. there is
     a
     reason
     

     View Slide

121. there
     is
     a
     reason
     

     View Slide

122. there is a
     reason
     

     View Slide

123. software is all about
     humans
     

     View Slide

124. ‘The most important
     thing I've accomplished,
     other than building the
     compiler, is training
     young people.
     

     View Slide

125. They come to me, you
     know, and say, 'Do you
     think we can do this?' I
     say, "Try it."
     

     View Slide

126. And I back 'em up. They
     need that.
     

     View Slide

127. I keep track of them as
     they get older and I stir
     'em up at intervals so
     they don't forget to take
     chances.’
     

     View Slide

128. Grace Hopper
     

     View Slide

129. written while listening to:
     22, a million - Bon Iver
     Christine and Queens
     Here Their Dreams - Bombazine Black
     

     View Slide

130. this presentation
     

     View Slide

131. View Slide

132. View Slide

133. #133
     

     View Slide

134. www.elastic.co
     

     View Slide

135. Except where otherwise noted, this work is licensed under
     http://creativecommons.org/licenses/by-nd/4.0/
     Creative Commons and the double C in a circle are
     registered trademarks of Creative Commons in the United States and other countries.
     Third party marks and brands are the property of their respective holders.
     135
     Please attribute Elastic with a link to elastic.co
     

     View Slide