Parallel/Concurrent

is Parallel/Concurrent programming hard, if so, what can we do ?

Transcript

1. if so, is Parallel/Concurrent Programming HARD What Can We Do About IT ? @Yuan
2. Concurrent or Parallel
3. Related but Different
4. Concurrency is about dealing with lots of things at once. Parallelism is about doing lots of things at once. “ - Rob Pike
5. Parallel Programming Goals
6. Performance Productivity Generality
10. What Makes Parallel Programming Hard? • Work Partitioning • Parallel Access Control • Resource Partitioning and Replication • Interacting with Hardware
11. Hardware and Its Habits
12. $16
    1FSGPSNBODF
    BU
    JUT
    #FTU
13. $16
    .FFUT
    B
    1JQFMJOF
    'MVTI
14. $16
    .FFUT
    B
    .FNPSZ
    3FGFSFODF
15. $16
    .FFUT
    B
    .FNPSZ
    #BSSJFS
    
    
16. $16
    .FFUT
    B
    $BDIF
    .JTT
17. $16
    .FFUT
    B
    *0
    $PNQMFUJPO
18. Partitioning and Synchronization
20. Dining Philosophers Problem
21. Five philosophers are sitting at a table.
22. /PX
    FBDI
    QIJMPTPQIFS
    IBT
    UXP
    GPSLT
    
    MFGU
    GPSL
    BOE
    SJHIU
    GPSL
    *G
    B
    3PO
    HFUT
    UXP
    GPSLT
    IF
    DBO
    FBU
23. *G
    IF
    POMZ
    IBT
    POF
    GPSL
    IF
    DBO
    OPU
    FBU
    
    4P
    UIF
    3PO
    OFFE
    UP
    MFBSO
    UP
    TIBSF
    GPSLT
24. 2
    IPX
    EP
    ZPV
    NBLF
    TVSF
    FWFSZ
    3PO
    HFUT
    UP
    FBU
25. %FBEMPDL
28. -JWFMPDL
31. Resource Hierarchy
32. 36-&
    JG
    3PO
    VTJOH
    UXP
    GPSLT
    ZPV
    OFFE
    UP
    QJDL
    VQ
    UIF
    MPXFS
    OVNCFSFE
    GPSL
    GJSTU
35. Waiter
36. Solution: if there are 5 forks, only 4 Rons should be allowed at the table. We’ll have a waiter control access to the table.
38. Chandy/Misra ,
    .BOJ
    $IBOEZ
    BOE
    +
    .JTSB
39. "MM
    GPSLT
    DBO
    CF
    DMFBO
    PS
    EJSUZ
40. *OJUJBMMZ
    BMM
    GPSLT
    BSF
    EJSUZ
41. 
    /VNCFS
    BMM
    UIF
    3POT
42. 
    'PS
    FWFSZ
    QBJS
    PG
    3POT
    HJWF
    UIF
    GPSL
    UP
    UIF
    HVZ
    XJUI
    UIF
    TNBMMFS
    JE
43. 4. When a Ron needs a fork, he asks his neighbor for his fork. When his neighbor gets the request: • if his fork is clean, he keep it • if his fork is dirty, he cleans it and sends it over
45. All programs with concurrency have the same problem.
46. :PVS
    QSPHSBN
    VTFT
    TPNF
    NFNPSZ
47. 8IFO
    ZPVS
    DPEF
    JT
    TJOHMFUISFBEFE
    UIFSF
    JT
    KVTU
    POF
    UISFBE
    XSJUJOH
    UP
    NFNPSZ
    :PV
    BSF
    "0,
48. #VU
    JG
    ZPV
    IBWF
    NPSF
    UIBO
    POF
    UISFBE
    UIFZ
    DPVME
    PWFSXSJUF
    FBDI
    PUIFST
    DIBOHFT
49. Locks / Actors / STM
50. Locks
52. Single-thread is good and safe, so why don’t use it anyway.
53. Actors
54. &WFSZ
    BDUPS
    NBOBHFT
    JUT
    PXO
    TUBUF
55. "DUPST
    BTL
    FBDI
    PUIFS
    UP
    EP
    UIJOHT
    CZ
    QBTTJOH
    NFTTBHFT
56. • Actors never share state so they never need to compete for locks for access to shared data • Actors are never shared between threads, so only one thread ever accesses the actor’s state • When you pass a message to an actor, it goes in his mailbox. The actor reads messages from his mailbox and does those tasks one at a time
58. Celluloid.io
59. Celluloid is a concurrent object oriented programming framework for Ruby which lets you build multithreaded programs out of concurrent objects just as easily as you build sequential programs out of regular objects “
60. Code
61. • Actors are computational entities that can receive messages • Each actor has a unique address • If you know an actor's address, you can send it messages • Actors can create new actors
63. Software Transactional Memory (STM)
64. • Vars - global • Atoms - synchronous • Agents - asynchronous • Refs - transaction
65. Code
66. ruby-concurrency/concurrent-ruby
67. Hardware Transactional Memory(HTM)
68. Can i use now ? NO
69. MacBook Pro (Mid 2014)’s CPU Support HTM via TSX
70. `sysctl -n hw.cpufamily`.to_i.to_s(16)
71. Conclusion
72. Locks • Available in most languages • Give you fine-grained control over your code • Complicated to use. Your code will have subtle deadlock / starvation issues
73. Actors • No shared state, so writing thread-safe is a breeze • No locks, so no deadlock unless your actors block • All your code needs to use actors and message passing, so you may need to restructure your code
74. STM • Very easy to use, don’t need to restructure code • No locks, so no deadlock • Good performance (threads spend less time idling)
75. Copy-on-Write
77. *OTFSU
    
78. *OTFSU
    
79. Immutable
80. $PQZ
    PO
    8SJUF
81. Ruby 2.0 faster than 1.9…
82. Read-Copy-Update
83. RCU Fundamentals • Publish-Subscribe Mechanism (for insertion) • Wait For Pre-Existing RCU Readers to Complete (for deletion) • Maintain Multiple Versions of Recently Updated Objects (for readers)
84. • https://lwn.net/Articles/262464/ • https://lwn.net/Articles/263130/
85. Thank You!