A banker, a baker, a distributed system maker

Slide 1

Slide 1 text

A banker, a baker, a distributed system maker Mark Allen [email protected] @bytemeorg

Slide 2

Slide 2 text

Agenda • Background • Multiprogramming • Intergalactic networking • The mutual exclusion problem • Sequential processes (cooperating and communicating) • ARPANet • Proving properties of distributed systems • Bibliography / Image Credits

Slide 3

Slide 3 text

BACKGROUND

Slide 4

Slide 4 text

Vacuum Tubes [a]

Slide 5

Slide 5 text

Transistors [b]

Slide 6

Slide 6 text

Solid State Advancements Eliminating vacuum tubes in favor of transitors in the late 1950s resulted in a massive speed increase.

Slide 7

Slide 7 text

The problem with concurrency • We have all this ridiculously expensive hardware that is mostly idle. Shouldn’t we be doing something with it to get maximum utility from it? • What could possibly go wrong?

Slide 8

Slide 8 text

MULTIPROGRAMMING

Slide 9

Slide 9 text

What is it? From [3] (but see also [7]): 1. Computations are concurrent for more than one user. 2. Computations share pools of resources. 3. Computations vary in demand for resources needed to complete a specific computation. 4. Reference to shared information between computations is common.

Slide 10

Slide 10 text

In Machine Language “Every programmer knows the importance of avoiding self destruction of programs. With multiprogramming, mutual destruction becomes a serious problem.” Multiprogramming: The Programmer’s View,1959. [2]

Slide 11

Slide 11 text

You think concurrency is hard, bro? • Eventually lead to the “software crisis” conference in 1968. • In response, structured programming because a major research and educational focus in the early 70s: Wirth, Nygaard and Dahl, Knuth. • However, this is not a talk about programming languages. (See [1] for more.)

Slide 12

Slide 12 text

E.F. Codd • Best known for the relational data model, 1970. • RAF fighter pilot in WWII. (Became US citizen in the mid 1960s.) • Led IBM team implementing multiprogramming using solid state hardware, 1957. • ACM Turing Winner, 1981

Slide 13

Slide 13 text

STRETCH • Feasibility paper [4] laid out six basic properties for “profitable” multiprogramming. • Later papers discussed work scheduling across space and time abstractly [5] and concretely [6].

Slide 14

Slide 14 text

Intergalactic Networking

Slide 15

Slide 15 text

J.C.R. Licklider • Helped establish MIT’s Lincoln Laboratory, early 1950s • Led IPTO office at DARPA, 1962-64 • Advocated “man-computer symbiosis” and “intergalactic networking” • Funded and later led Project MAC, to timeshare large computer systems.

Slide 16

Slide 16 text

Intergalactic Networking “Is the network control language the same thing as the time-sharing control language? … Is the network control language different from the time-sharing control language, and is the network-control language common to the several netted facilities? Is there no such thing as a network-control language?” JCR Licklider [8]

Slide 17

Slide 17 text

Imagining Networking • Licklider discusses: • browsing remote files, • running programs remotely and • retriveing results at a local workstation • Doesn’t that sound familiar to you?

Slide 18

Slide 18 text

Mutual Exclusion

Slide 19

Slide 19 text

Edsger W. Dijkstra • Graph traversal algorithm • "GO TO considered harmful" • 1st implementation of ALGOL 1960 • Dijkstra Prize in Distributed Computing • ACM Turing Winner, 1972

Slide 20

Slide 20 text

[14]

Slide 21

Slide 21 text

[18], see also [22]

Slide 22

Slide 22 text

[17]

Slide 23

Slide 23 text

“[T]he intellectual level needed for system design is in general grossly underestimated. I am more than ever convinced that this type of work is just difficult and that every effort to do it with other than the best people is doomed to either failure or moderate success at enormous expenses.” – E.W. Dijkstra [17]

Slide 24

Slide 24 text

[20]

Slide 25

Slide 25 text

No content

Slide 26

Slide 26 text

[15]

Slide 27

Slide 27 text

0 If L = S, then S := (S+1) mod K If L ≠ S, then S := L 0 ≤ S < 6 K > N; K = 6 N=5 1 2 3 4 5

Slide 28

Slide 28 text

0 If L = S, then S := (S+1) mod 6 If L ≠ S, then S := L 0 ≤ S < 6 K > N; K = 6 N=5 1 2 3 4 5

Slide 29

Slide 29 text

0 If L = S, then S := (S+1) mod 6 If L ≠ S, then S := L 0 ≤ S < 6 K > N; K = 6 N=5 2 2 3 4 5

Slide 30

Slide 30 text

0 If L = S, then S := (S+1) mod 6 If L ≠ S, then S := L 0 ≤ S < 6 K > N; K = 6 N=5 2 2 3 4 5

Slide 31

Slide 31 text

0 If L = S, then S := (S+1) mod 6 If L ≠ S, then S := L 0 ≤ S < 6 K > N; K = 6 N=5 2 3 3 4 5

Slide 32

Slide 32 text

0 If L = S, then S := (S+1) mod 6 If L ≠ S, then S := L 0 ≤ S < 6 K > N; K = 6 N=5 2 3 4 5 5

Slide 33

Slide 33 text

0 If L = S, then S := (S+1) mod 6 If L ≠ S, then S := L 0 ≤ S < 6 K > N; K = 6 N=5 2 3 4 5 0

Slide 34

Slide 34 text

0 If L = S, then S := (S+1) mod 6 If L ≠ S, then S := L 0 ≤ S < 6 K > N; K = 6 N=5 2 3 4 5 0

Slide 35

Slide 35 text

0 If L = S, then S := (S+1) mod 6 If L ≠ S, then S := L 0 ≤ S < 6 K > N; K = 6 N=5 3 4 5 0 0

Slide 36

Slide 36 text

0 If L = S, then S := (S+1) mod 6 If L ≠ S, then S := L 0 ≤ S < 6 K > N; K = 6 N=5 4 5 0 0 0

Slide 37

Slide 37 text

0 If L = S, then S := (S+1) mod 6 If L ≠ S, then S := L 0 ≤ S < 6 K > N; K = 6 N=5 5 0 0 0 0

Slide 38

Slide 38 text

0 If L = S, then S := (S+1) mod 6 If L ≠ S, then S := L 0 ≤ S < 6 K > N; K = 6 N=5 0 0 0 0 0

Slide 39

Slide 39 text

1 If L = S, then S := (S+1) mod 6 If L ≠ S, then S := L 0 ≤ S < 6 K > N; K = 6 N=5 0 0 0 0 0

Slide 40

Slide 40 text

Sequential Processes

Slide 41

Slide 41 text

[16]

Slide 42

Slide 42 text

C.A.R. Hoare • Invented the QuickSort algorithm • Major foundational work on formal methods of program correctness • Structured programming advocate. • ACM Turing Winner, 1980

Slide 43

Slide 43 text

[19]

Slide 44

Slide 44 text

ARPANet

Slide 45

Slide 45 text

Vinton Cerf • Best known for work on TCP and IP. • Cowrote major portions of precursor NCP “Network control program” (see RFC 33) • ACM Turing Winner, 2004

Slide 46

Slide 46 text

[21]

Slide 47

Slide 47 text

[c]

Slide 48

Slide 48 text

[d]

Slide 49

Slide 49 text

Proving Properties of Distributed Systems

Slide 50

Slide 50 text

Robert W. Floyd • Proof-reader and critic of Knuth’s Art of Computering Programming. • Algorithms for finding the shortest path in a directed graph, graphic dithering, and calculating the median value of a set. • Passionate about verifying the correctness of a program using logical predicates. • ACM Turing Winner, 1978

Slide 51

Slide 51 text

Programming Paradigms “It remains as true now as when I entered the computer field in 1956 that everyone wants to design a new programming language. In the words written on the wall of a Stanford University graduate student office, "I would rather write programs to help me write programs than write programs."” - R.W. Floyd, [13]

Slide 52

Slide 52 text

[11]

Slide 53

Slide 53 text

[10]

Slide 54

Slide 54 text

[12]

Slide 55

Slide 55 text

Conclusions

Slide 56

Slide 56 text

Out of the Tar Pit “Complexity is the single major difficulty in the successful development of large-scale software systems… [but] this is the unfortunate truth: Simplicity is HARD.” – Ben Moseley and Peter Marks, [9]

Slide 57

Slide 57 text

“For most of the 1970s, one ‘solved’ the mutual exclusion problem by using semaphores or monitors or conditional critical regions or some other language construct. This is like solving the sorting problem by using a programming language with a sort command.” - Leslie Lamport [24]

Slide 58

Slide 58 text

A Way Forward? 1. Programming disciplines that are simple(r) to reason about than the existing state of the art. 2. Formal methods that make building an accurate model of the problem feasible for non-experts (perhaps baked into a programming language?) 3. Reading old papers is dope. We often take them for granted and over time forget important details and caveats.

Slide 59

Slide 59 text

THANK YOU! Mark Allen [email protected] @bytemeorg

Slide 60

Slide 60 text

Bibliography • [1] Allen, Mark, “This has all happened before and it will all happen again”, Strange Loop, 2014. https://youtu.be/jmRE5pXFi04 • [2] Opler, A, Baird, N, Multiprogramming: The Programmer’s View, Proceedings of the 14th annual meeting of the ACM, ACM,1959. http://dx.doi.org/10.1145/612201.612215

Slide 61

Slide 61 text

Bibliography • [3] Dennis, J. and Van Horn, E., Programming semantics for multiprogrammed computations, CACM, March 1966. http://dx.doi.org/10.1145/365230.365252 • [4] Codd, E.F., et al, Multiprogramming STRETCH: Feasibility Considerations, CACM, Nov, 1959. http://dx.doi.org/10.1145/368481.368502 • [5] Codd, E.F., Multiprogram scheduling, parts 1 and 2. introduction and theory, CACM, June 1960. http://dx.doi.org/10.1145/367297.367317

Slide 62

Slide 62 text

Bibliography • [6] Codd, E.F., Multiprogram scheduling, parts 3 and 4. scheduling algorithm and external constraints, CACM, July1960. http://dx.doi.org/10.1145/367349.367356 • [7] Ryle, B.L., Multiple programming data processing, CACM, February 1961. http://dx.doi.org/10.1145/366105.366132 • [8] Licklider, JCR, MEMORANDUM FOR: Members and Affiliates of the Intergalactic Computer Network, April 1963. http://www.packet.cc/files/memo.html

Slide 63

Slide 63 text

Bibliography • [9] Moseley, B and Marks, P, Out of the Tar Pit, 2006, http://shaffner.us/cs/papers/tarpit.pdf • [10] Lamport, L., Proving the Correctness of Multiprocess Programs, IEEE Transactions on Software Engineering, March 1977. http://research.microsoft.com/en- us/um/people/lamport/pubs/proving.pdf • [11] Floyd, R.W., Assigning Meanings to Programs, 1966. http://www.cs.virginia.edu/~weimer/2007- 615/reading/FloydMeaning.pdf

Slide 64

Slide 64 text

Bibliography • [12] Alpern, B and Schneider, F., Defining liveness, TR- 85-650, Cornell, 1984. https://ecommons.cornell.edu/bitstream/handle/1813/ 6495/85-650.pdf • [13] Floyd, R.W., The Paradigms of Programming, 1978 Turing Award Lecture, ACM, http://dl.acm.org/ft_gateway.cfm?id=1283934&type=p df

Slide 65

Slide 65 text

Bibliography • [14] Dijsktra, E.W., Solution of a problem in concurrent program control., CACM, Sept 1965. http://dx.doi.org/10.1145/365559.365617 • [15] --, Self-stabilizing system in spite of distributed control, CACM, 1974. http://dx.doi.org/10.1145/361179.361202 • [16] – Cooperating Sequential Processes, 1965. http://www.cs.utexas.edu/users/EWD/ewd01xx/EWD123.PDF • [17] – The structure of the “THE” multiprogramming system, Proceedings of SOSP, ACM, 1967. http://dx.doi.org/10.1145/800001.811672 •

Slide 66

Slide 66 text

Bibliography • [18] Lamport, L., A New Solution of Dijkstra’s Concurrent Programming Problem, CACM, Aug 1974. • [19] Hoare, C.A.R., Communicating Sequential Processes, CACM, Aug 1978. http://dx.doi.org/10.1145/359576.359585 • [20] Dijkstra, E.W., Hierarchical Ordering of Sequential Processes, http://www.cs.utexas.edu/users/EWD/ewd03xx/EWD 310.PDF

Slide 67

Slide 67 text

Bibliography • [21] RFC 33, http://tools.ietf.org/html/rfc33 • [22] Lamport, L., The computer science of concurrency: the early years, CACM, June 2015, http://dx.doi.org/10.1145/2771951 • [23] Hansen, P.B., The Origin of Concurrent Programming, Springer, 2002. • [24] Lamport, L., Teaching concurrency, http://dx.doi.org/10.1145/1515698.1515713

Slide 68

Slide 68 text

Image Credits [a] Vacuum tubes, threethan, https://flic.kr/p/bBCBAD [b] Transistors, Vahid alpha, https://en.wikipedia.org/wiki /File:Transistors_110.jpg [c, d] http://www.computerhistory.org/revolution/networ king/19/407/2086