Let's talk concurrency

Transcript

1. A tale by @josevalim from @plataformatec CONCURRENCY LET’S TALK Friday,

   September 21, 2012

2. Friday, September 21, 2012

3. Core Team Member Friday, September 21, 2012

4. Friday, September 21, 2012

5. CONCURRENCY LET’S TALK Friday, September 21, 2012

6. “Execute several tasks simultaneously” Friday, September 21, 2012

7. Server Server Server Server Friday, September 21, 2012

8. Server Friday, September 21, 2012

9. MULTI CORE SINGLE PROCESS Friday, September 21, 2012

10. state concurrency off off Friday, September 21, 2012

11. The Declarative Model Because mathematics is awesome! Friday, September 21,

    2012

12. •no mutation •no concurrency •just functions Friday, September 21, 2012

13. factorial = lambda do |x| case x when 0 1

    else x * factorial.(x-1) end end print factorial.(10) # => 3628800 Friday, September 21, 2012

14. Determinism Same input = > same output Friday, September 21,

    2012

15. •no i/o •no random •no side-effects •always returns the same

    result Friday, September 21, 2012

16. lambda do |a, b| c = expensive_function.(a) d = also_expensive.(b)

    c + d end Friday, September 21, 2012

17. lambda do |a, b| c = expensive_function.(a) d = also_expensive.(b)

    c + d end a = 1 Friday, September 21, 2012

18. lambda do |1, b| c = 42 d = also_expensive.(b)

    c + d end Friday, September 21, 2012

19. lambda do |a, b| c = expensive_function.(a) d = also_expensive.(b)

    c + d end Friday, September 21, 2012

20. lambda do |a, b| d = also_expensive.(b) c = expensive_function.(a)

    c + d end Friday, September 21, 2012

21. Haskell Using determinism for performance and expressiveness Friday, September 21,

    2012

22. l = lambda { |a,b,c| a + b + c

    } l.(1, 2, 3) #=> 6 l.curry #=> #<Proc> l.curry.(1) #=> #<Proc> l.curry.(1).(2).(3) #=> 6 Currying Friday, September 21, 2012

23. l = lambda { |a,b| a * b } double

    = l.curry.(2) triple = l.curry.(3) Currying Friday, September 21, 2012

24. sum a b c = a + b + c

    (sum 1 2 3) (((sum 1) 2) 3) Currying Friday, September 21, 2012

25. x = expensive(1) y = expensive(3) z = (sum x)

    ((z 2) y) Friday, September 21, 2012

26. x = expensive(1) y = expensive(3) z = (sum x)

    ((z 2) y) Compiler Friday, September 21, 2012

27. x = expensive(1) y = expensive(3) z = (sum x)

    ((z 2) y) Compiler (sum expensive(1) 2 expensive(3)) Friday, September 21, 2012

28. CONCURRENCY ON LET’S TURN Friday, September 21, 2012

29. state concurrency off on Friday, September 21, 2012

30. Dataflow variables Painless concurrency! Friday, September 21, 2012

31. lambda do |a, b| c = expensive_function.(a) d = also_expensive.(b)

    c + d end Friday, September 21, 2012

32. lambda do |a, b| thread { c = expensive_function.(a) }

    thread { d = also_expensive.(b) } c + d end Friday, September 21, 2012

33. main Friday, September 21, 2012

34. main spawn thread 1 Friday, September 21, 2012

35. main spawn thread 1 spawn thread 2 Friday, September 21,

    2012

36. main spawn thread 1 spawn thread 2 unbound c Friday,

    September 21, 2012

37. main spawn thread 1 spawn thread 2 unbound c defines

    c Friday, September 21, 2012

38. main spawn thread 1 spawn thread 2 unbound c defines

    c unbound d Friday, September 21, 2012

39. main spawn thread 1 spawn thread 2 unbound c defines

    c unbound d defines d Friday, September 21, 2012

40. main spawn thread 1 spawn thread 2 unbound c defines

    c unbound d defines d c + d Friday, September 21, 2012

41. STATE ON LET’S TURN Friday, September 21, 2012

42. state concurrency on on Friday, September 21, 2012

43. Friday, September 21, 2012

44. PROBLEM THE Friday, September 21, 2012

45. class Counter mattr_accessor :i self.i = 0 end thread {

    Counter.i = Counter.i + 1 } Friday, September 21, 2012

46. thread 1 Counter.i thread 2 Friday, September 21, 2012

47. thread 1 Counter.i thread 2 0 Friday, September 21, 2012

48. thread 1 Counter.i thread 2 0 0 Friday, September 21,

    2012

49. thread 1 Counter.i thread 2 0 0 1 Friday, September

    21, 2012

50. thread 1 Counter.i thread 2 0 0 1 1 Friday,

    September 21, 2012

51. thread 1 Counter.i thread 2 0 1 0 1 1

    Friday, September 21, 2012

52. thread 1 Counter.i thread 2 0 1 0 1 1

    1 Friday, September 21, 2012

53. thread 1 Counter.i thread 2 0 1 0 1 1

    1 1 Friday, September 21, 2012

54. thread 1 Counter.i thread 2 0 1 0 1 1

    1 1 1 Friday, September 21, 2012

55. thread 1 Counter.i thread 2 0 1 0 1 1

    1 1 1 2 Friday, September 21, 2012

56. thread 1 Counter.i thread 2 0 1 0 1 1

    1 1 1 2 2 Friday, September 21, 2012

57. thread 1 Counter.i thread 2 0 1 0 1 1

    1 1 1 2 2 2 Friday, September 21, 2012

58. thread 1 Counter.i thread 2 0 1 0 1 1

    1 1 1 2 2 2 2 Friday, September 21, 2012

59. •shared-memory concurrent model •locks •transactional memory •message-passing concurrent model Friday,

    September 21, 2012

60. •shared-memory concurrent model •locks •transactional memory •message-passing concurrent model Friday,

    September 21, 2012

61. class Counter mattr_accessor :i self.i = 0 end thread {

    Counter.i = Counter.i + 1 } Friday, September 21, 2012

62. class Counter mattr_accessor :i self.i = 0 end thread {

    synchronize { Counter.i = Counter.i + 1 } } Friday, September 21, 2012

63. thread 1 Counter.i thread 2 Friday, September 21, 2012

64. thread 1 Counter.i thread 2 0 Friday, September 21, 2012

65. thread 1 Counter.i thread 2 0 0 Friday, September 21,

    2012

66. thread 1 Counter.i thread 2 0 0 1 Friday, September

    21, 2012

67. thread 1 Counter.i thread 2 0 0 1 1 Friday,

    September 21, 2012

68. thread 1 Counter.i thread 2 0 1 0 1 1

    Friday, September 21, 2012

69. thread 1 Counter.i thread 2 0 1 0 1 1

    1 synchronize { Friday, September 21, 2012

70. thread 1 Counter.i thread 2 0 1 0 1 1

    2 1 synchronize { Friday, September 21, 2012

71. thread 1 Counter.i thread 2 0 1 0 1 1

    2 1 synchronize { 2 } Friday, September 21, 2012

72. thread 1 Counter.i thread 2 0 1 0 1 1

    2 2 1 synchronize { 2 } Friday, September 21, 2012

73. thread 1 Counter.i thread 2 0 1 0 1 1

    2 2 2 1 synchronize { 2 } Friday, September 21, 2012

74. thread 1 Counter.i thread 2 0 1 0 1 1

    2 2 3 2 1 synchronize { 2 } Friday, September 21, 2012

75. thread 1 Counter.i thread 2 0 1 0 1 1

    2 2 3 3 2 1 synchronize { 2 } Friday, September 21, 2012

76. +most popular approach +explicit control over the lock Friday, September

    21, 2012

77. - explicit control over the lock - pessimistic approach Friday,

    September 21, 2012

78. •shared-memory concurrent model •locks •transactional memory •message-passing concurrent model Friday,

    September 21, 2012

79. class Counter mattr_accessor :i self.i = ref { 0 }

    end thread { atomic { Counter.i = Counter.i + 1 } } Friday, September 21, 2012

80. thread 1 Counter.i thread 2 Friday, September 21, 2012

81. thread 1 Counter.i thread 2 0 Friday, September 21, 2012

82. thread 1 Counter.i thread 2 0 0 Friday, September 21,

    2012

83. thread 1 Counter.i thread 2 0 0 1 Friday, September

    21, 2012

84. thread 1 Counter.i thread 2 0 0 1 1 Friday,

    September 21, 2012

85. thread 1 Counter.i thread 2 0 1 0 1 1

    Friday, September 21, 2012

86. thread 1 Counter.i thread 2 0 1 0 1 1

    1 atomic { Friday, September 21, 2012

87. thread 1 Counter.i thread 2 0 1 0 1 1

    1 1 atomic { Friday, September 21, 2012

88. thread 1 Counter.i thread 2 0 1 0 1 1

    1 1 1 atomic { Friday, September 21, 2012

89. thread 1 Counter.i thread 2 0 1 0 1 1

    1 1 2 1 atomic { Friday, September 21, 2012

90. thread 1 Counter.i thread 2 0 1 0 1 1

    1 1 2 2 1 atomic { Friday, September 21, 2012

91. thread 1 Counter.i thread 2 0 1 0 1 1

    1 1 2 2 2 1 atomic { Friday, September 21, 2012

92. thread 1 Counter.i thread 2 0 1 0 1 1

    1 1 2 2 2 1 atomic { } Friday, September 21, 2012

93. thread 1 Counter.i thread 2 0 1 0 1 1

    1 1 2 2 2 1 atomic { } Friday, September 21, 2012

94. thread 1 Counter.i thread 2 0 1 0 1 1

    1 1 2 2 2 1 atomic { } Friday, September 21, 2012

95. thread 1 Counter.i thread 2 0 1 0 1 1

    1 1 2 2 2 2 1 atomic { } Friday, September 21, 2012

96. thread 1 Counter.i thread 2 0 1 0 1 1

    1 1 2 2 2 2 2 1 atomic { } Friday, September 21, 2012

97. thread 1 Counter.i thread 2 0 1 0 1 1

    1 1 2 2 2 2 2 3 1 atomic { } Friday, September 21, 2012

98. thread 1 Counter.i thread 2 0 1 0 1 1

    1 1 2 2 2 2 2 3 3 1 atomic { } Friday, September 21, 2012

99. + optimistic approach + no deadlock or race conditions Friday,

    September 21, 2012

100. - potential unnecessary retries - transaction overhead Friday, September 21,

     2012

101. •shared-memory concurrent model •locks •transactional memory •message-passing concurrent model Friday,

     September 21, 2012

102. server = lambda do |i| receive when :increment server.(i+1) when

     :check client <- i server.(i) else warn "unknown message" server.(i) end end server.(0) Friday, September 21, 2012

103. thread { server <- :increment } Friday, September 21, 2012

104. thread 1 server thread 2 Friday, September 21, 2012

105. thread 1 server thread 2 0 Friday, September 21, 2012

106. thread 1 server thread 2 0 :increment Friday, September 21,

     2012

107. thread 1 server thread 2 0 1 :increment Friday, September

     21, 2012

108. thread 1 server thread 2 0 1 :increment :increment Friday,

     September 21, 2012

109. thread 1 server thread 2 0 2 1 :increment :increment

     Friday, September 21, 2012

110. thread 1 server thread 2 0 2 1 :increment :increment

     :increment Friday, September 21, 2012

111. thread 1 server thread 2 0 2 1 3 :increment

     :increment :increment Friday, September 21, 2012

112. thread 1 server thread 2 0 2 1 3 :increment

     :increment :increment :check Friday, September 21, 2012

113. thread 1 server thread 2 0 2 1 3 3

     :increment :increment :increment :check Friday, September 21, 2012

114. thread 1 server thread 2 0 2 1 3 3

     :increment :increment :increment :check Friday, September 21, 2012

115. thread 1 server thread 2 0 2 1 3 3

     3 :increment :increment :increment :check Friday, September 21, 2012

116. “Do not communicate by sharing memory; instead, share memory by

     communicating;” Effective Go Friday, September 21, 2012

117. +no need for synchronization +easy to distribute (location transparency) Friday,

     September 21, 2012

118. - coordination may be tricky - non-conventional approach Friday, September

     21, 2012

119. UP SUMMING Friday, September 21, 2012

120. message-passing locks stm go, erlang ruby clojure dataflow oz Friday,

     September 21, 2012

121. github.com/celluloid Friday, September 21, 2012

122. @elixirlang / elixir-lang.org Friday, September 21, 2012

123. MATTER DEFAULTS Friday, September 21, 2012

124. Erlang and Clojure defaults to immutable Friday, September 21, 2012

125. MAGIC BULLET THERE IS NO Friday, September 21, 2012

126. REFERENCES Seven languages in seven weeks Friday, September 21, 2012

127. REFERENCES Concepts, Techniques and Models of Computer Programming Friday, September

     21, 2012

128. REFERENCES Software Transactional Memory http://java.ociweb.com/mark/stm/article.html Persistent Data Structures http://www.infoq.com/presentations/Value-Identity-State-Rich-Hickey Friday,

     September 21, 2012

129. ? QUESTIONS @josevalim Friday, September 21, 2012