A banker, a baker, a distributed system maker

This talk covers the early history of concurrency from about 1957ish forward until around 1985. Topics discussed include multiprogramming, JCR Licklider, mutual exclusion, sequential processes, ARPANet, proving properties of distributed systems, and a bibliography.

Transcript

1. A banker, a baker, a distributed system maker Mark Allen mallen@basho.com @bytemeorg
2. Agenda • Background • Multiprogramming • Intergalactic networking • The mutual exclusion problem • Sequential processes (cooperating and communicating) • ARPANet • Proving properties of distributed systems • Bibliography / Image Credits
3. BACKGROUND
4. Vacuum Tubes [a]
5. Transistors [b]
6. Solid State Advancements Eliminating vacuum tubes in favor of transitors in the late 1950s resulted in a massive speed increase.
7. The problem with concurrency • We have all this ridiculously expensive hardware that is mostly idle. Shouldn’t we be doing something with it to get maximum utility from it? • What could possibly go wrong?
8. MULTIPROGRAMMING
9. What is it? From [3] (but see also [7]): 1. Computations are concurrent for more than one user. 2. Computations share pools of resources. 3. Computations vary in demand for resources needed to complete a specific computation. 4. Reference to shared information between computations is common.
10. In Machine Language “Every programmer knows the importance of avoiding self destruction of programs. With multiprogramming, mutual destruction becomes a serious problem.” Multiprogramming: The Programmer’s View,1959. [2]
11. You think concurrency is hard, bro? • Eventually lead to the “software crisis” conference in 1968. • In response, structured programming because a major research and educational focus in the early 70s: Wirth, Nygaard and Dahl, Knuth. • However, this is not a talk about programming languages. (See [1] for more.)
12. E.F. Codd • Best known for the relational data model, 1970. • RAF fighter pilot in WWII. (Became US citizen in the mid 1960s.) • Led IBM team implementing multiprogramming using solid state hardware, 1957. • ACM Turing Winner, 1981
13. STRETCH • Feasibility paper [4] laid out six basic properties for “profitable” multiprogramming. • Later papers discussed work scheduling across space and time abstractly [5] and concretely [6].
14. Intergalactic Networking
15. J.C.R. Licklider • Helped establish MIT’s Lincoln Laboratory, early 1950s • Led IPTO office at DARPA, 1962-64 • Advocated “man-computer symbiosis” and “intergalactic networking” • Funded and later led Project MAC, to timeshare large computer systems.
16. Intergalactic Networking “Is the network control language the same thing as the time-sharing control language? … Is the network control language different from the time-sharing control language, and is the network-control language common to the several netted facilities? Is there no such thing as a network-control language?” JCR Licklider [8]
17. Imagining Networking • Licklider discusses: • browsing remote files, • running programs remotely and • retriveing results at a local workstation • Doesn’t that sound familiar to you?
18. Mutual Exclusion
19. Edsger W. Dijkstra • Graph traversal algorithm • "GO TO considered harmful" • 1st implementation of ALGOL 1960 • Dijkstra Prize in Distributed Computing • ACM Turing Winner, 1972
20. [14]
21. [18], see also [22]
22. [17]
23. “[T]he intellectual level needed for system design is in general grossly underestimated. I am more than ever convinced that this type of work is just difficult and that every effort to do it with other than the best people is doomed to either failure or moderate success at enormous expenses.” – E.W. Dijkstra [17]
24. [20]
26. [15]
27. 0 If L = S, then S := (S+1) mod K If L ≠ S, then S := L 0 ≤ S < 6 K > N; K = 6 N=5 1 2 3 4 5
28. 0 If L = S, then S := (S+1) mod 6 If L ≠ S, then S := L 0 ≤ S < 6 K > N; K = 6 N=5 1 2 3 4 5
29. 0 If L = S, then S := (S+1) mod 6 If L ≠ S, then S := L 0 ≤ S < 6 K > N; K = 6 N=5 2 2 3 4 5
30. 0 If L = S, then S := (S+1) mod 6 If L ≠ S, then S := L 0 ≤ S < 6 K > N; K = 6 N=5 2 2 3 4 5
31. 0 If L = S, then S := (S+1) mod 6 If L ≠ S, then S := L 0 ≤ S < 6 K > N; K = 6 N=5 2 3 3 4 5
32. 0 If L = S, then S := (S+1) mod 6 If L ≠ S, then S := L 0 ≤ S < 6 K > N; K = 6 N=5 2 3 4 5 5
33. 0 If L = S, then S := (S+1) mod 6 If L ≠ S, then S := L 0 ≤ S < 6 K > N; K = 6 N=5 2 3 4 5 0
34. 0 If L = S, then S := (S+1) mod 6 If L ≠ S, then S := L 0 ≤ S < 6 K > N; K = 6 N=5 2 3 4 5 0
35. 0 If L = S, then S := (S+1) mod 6 If L ≠ S, then S := L 0 ≤ S < 6 K > N; K = 6 N=5 3 4 5 0 0
36. 0 If L = S, then S := (S+1) mod 6 If L ≠ S, then S := L 0 ≤ S < 6 K > N; K = 6 N=5 4 5 0 0 0
37. 0 If L = S, then S := (S+1) mod 6 If L ≠ S, then S := L 0 ≤ S < 6 K > N; K = 6 N=5 5 0 0 0 0
38. 0 If L = S, then S := (S+1) mod 6 If L ≠ S, then S := L 0 ≤ S < 6 K > N; K = 6 N=5 0 0 0 0 0
39. 1 If L = S, then S := (S+1) mod 6 If L ≠ S, then S := L 0 ≤ S < 6 K > N; K = 6 N=5 0 0 0 0 0
40. Sequential Processes
41. [16]
42. C.A.R. Hoare • Invented the QuickSort algorithm • Major foundational work on formal methods of program correctness • Structured programming advocate. • ACM Turing Winner, 1980
43. [19]
44. ARPANet
45. Vinton Cerf • Best known for work on TCP and IP. • Cowrote major portions of precursor NCP “Network control program” (see RFC 33) • ACM Turing Winner, 2004
46. [21]
47. [c]
48. [d]
49. Proving Properties of Distributed Systems
50. Robert W. Floyd • Proof-reader and critic of Knuth’s Art of Computering Programming. • Algorithms for finding the shortest path in a directed graph, graphic dithering, and calculating the median value of a set. • Passionate about verifying the correctness of a program using logical predicates. • ACM Turing Winner, 1978
51. Programming Paradigms “It remains as true now as when I entered the computer field in 1956 that everyone wants to design a new programming language. In the words written on the wall of a Stanford University graduate student office, "I would rather write programs to help me write programs than write programs."” - R.W. Floyd, [13]
52. [11]
53. [10]
54. [12]
55. Conclusions
56. Out of the Tar Pit “Complexity is the single major difficulty in the successful development of large-scale software systems… [but] this is the unfortunate truth: Simplicity is HARD.” – Ben Moseley and Peter Marks, [9]
57. “For most of the 1970s, one ‘solved’ the mutual exclusion problem by using semaphores or monitors or conditional critical regions or some other language construct. This is like solving the sorting problem by using a programming language with a sort command.” - Leslie Lamport [24]
58. A Way Forward? 1. Programming disciplines that are simple(r) to reason about than the existing state of the art. 2. Formal methods that make building an accurate model of the problem feasible for non-experts (perhaps baked into a programming language?) 3. Reading old papers is dope. We often take them for granted and over time forget important details and caveats.
59. THANK YOU! Mark Allen mallen@basho.com @bytemeorg
60. Bibliography • [1] Allen, Mark, “This has all happened before and it will all happen again”, Strange Loop, 2014. https://youtu.be/jmRE5pXFi04 • [2] Opler, A, Baird, N, Multiprogramming: The Programmer’s View, Proceedings of the 14th annual meeting of the ACM, ACM,1959. http://dx.doi.org/10.1145/612201.612215
61. Bibliography • [3] Dennis, J. and Van Horn, E., Programming semantics for multiprogrammed computations, CACM, March 1966. http://dx.doi.org/10.1145/365230.365252 • [4] Codd, E.F., et al, Multiprogramming STRETCH: Feasibility Considerations, CACM, Nov, 1959. http://dx.doi.org/10.1145/368481.368502 • [5] Codd, E.F., Multiprogram scheduling, parts 1 and 2. introduction and theory, CACM, June 1960. http://dx.doi.org/10.1145/367297.367317
62. Bibliography • [6] Codd, E.F., Multiprogram scheduling, parts 3 and 4. scheduling algorithm and external constraints, CACM, July1960. http://dx.doi.org/10.1145/367349.367356 • [7] Ryle, B.L., Multiple programming data processing, CACM, February 1961. http://dx.doi.org/10.1145/366105.366132 • [8] Licklider, JCR, MEMORANDUM FOR: Members and Affiliates of the Intergalactic Computer Network, April 1963. http://www.packet.cc/files/memo.html
63. Bibliography • [9] Moseley, B and Marks, P, Out of the Tar Pit, 2006, http://shaffner.us/cs/papers/tarpit.pdf • [10] Lamport, L., Proving the Correctness of Multiprocess Programs, IEEE Transactions on Software Engineering, March 1977. http://research.microsoft.com/en- us/um/people/lamport/pubs/proving.pdf • [11] Floyd, R.W., Assigning Meanings to Programs, 1966. http://www.cs.virginia.edu/~weimer/2007- 615/reading/FloydMeaning.pdf
64. Bibliography • [12] Alpern, B and Schneider, F., Defining liveness, TR- 85-650, Cornell, 1984. https://ecommons.cornell.edu/bitstream/handle/1813/ 6495/85-650.pdf • [13] Floyd, R.W., The Paradigms of Programming, 1978 Turing Award Lecture, ACM, http://dl.acm.org/ft_gateway.cfm?id=1283934&type=p df
65. Bibliography • [14] Dijsktra, E.W., Solution of a problem in concurrent program control., CACM, Sept 1965. http://dx.doi.org/10.1145/365559.365617 • [15] --, Self-stabilizing system in spite of distributed control, CACM, 1974. http://dx.doi.org/10.1145/361179.361202 • [16] – Cooperating Sequential Processes, 1965. http://www.cs.utexas.edu/users/EWD/ewd01xx/EWD123.PDF • [17] – The structure of the “THE” multiprogramming system, Proceedings of SOSP, ACM, 1967. http://dx.doi.org/10.1145/800001.811672 •
66. Bibliography • [18] Lamport, L., A New Solution of Dijkstra’s Concurrent Programming Problem, CACM, Aug 1974. • [19] Hoare, C.A.R., Communicating Sequential Processes, CACM, Aug 1978. http://dx.doi.org/10.1145/359576.359585 • [20] Dijkstra, E.W., Hierarchical Ordering of Sequential Processes, http://www.cs.utexas.edu/users/EWD/ewd03xx/EWD 310.PDF
67. Bibliography • [21] RFC 33, http://tools.ietf.org/html/rfc33 • [22] Lamport, L., The computer science of concurrency: the early years, CACM, June 2015, http://dx.doi.org/10.1145/2771951 • [23] Hansen, P.B., The Origin of Concurrent Programming, Springer, 2002. • [24] Lamport, L., Teaching concurrency, http://dx.doi.org/10.1145/1515698.1515713
68. Image Credits [a] Vacuum tubes, threethan, https://flic.kr/p/bBCBAD [b] Transistors, Vahid alpha, https://en.wikipedia.org/wiki /File:Transistors_110.jpg [c, d] http://www.computerhistory.org/revolution/networ king/19/407/2086