Managing Externalities in a Functional World

The modern software developer is spoilt for choice. With the options available today, it has never been easier to build solutions, run experiments, and recover from mistakes. Cheap leverage however encourages incidental complexity, reduces the inherent need for cooperation, and obviates common standards.

Functional programming offers unprecedented leverage, which amplifies these externalities. Knowingly or not, in this way the modern software developer wields tremendous power, but at the same time walks on the knife's edge.

In this talk, Evadne will attempt to provide an overview of these externalities, the historical context, near- and long-term implications, and a few projections as to how things will play out. She will also share some ideas as to how we can mitigate these problems together as a community.

OBJECTIVES

The main objective of this talk is to provide a historic and humanistic perspective that can be used by software developers to interpret functional programming, and its meteoric re-emergence in the web development arena.

The secondary objective is to actually induct people into the world of functional programming by making its premises accessible, but at the same time ensuring that they start with a long-term positive mindset.

Evadne believes that a large number of Elixir developers initially adopted Elixir with a continuous improvement mindset. This talk aims to appropriately echo the mentality, examine the problems currently faced by the community and share potential approaches to be used in our own daily work.

Transcript

1. Managing Externalities in a Functional World Evadne Wu Head of Exam Systems Faria Education Group date 16 August 2018
2. Goal To promote the application of functional languages,
 in general, and Elixir in particular,
 from a tool of last resort, used in anger,
 to the tool of choice. In order to navigate the future, one must understand the past (context),
 so in we go…
3. 1 Past Literature on Adoption
4. Hudak’s Experiment [1994] Haskell vs. Ada vs. C++ vs. Awk vs. ... An Experiment in Software Prototyping Productivity Paul Hudak Mark P. Jones Yale University Department of Computer Science New Haven, CT 06518 hudak-paul,jones-mark @cs.yale.edu July 4, 1994
5. Slave Doctrine Weapon Doctrine Engageability Zone Hostile Aircraft Commercial Aircraft Tight Zone Carrier Aegis Ship
6. Outcome: Dramatic FP Edge The Navy secretly had the Haskell experiment re-done by a beginner,
 because the original numbers were too dramatic. Language Lines of code Lines of documentation Development time (hours) (1) Haskell 85 465 10 (2) Ada 767 714 23 (3) Ada9X 800 200 28 (4) C++ 1105 130 – (5) Awk/Nawk 250 150 – (6) Rapide 157 0 54 (7) Griffin 251 0 34 (8) Proteus 293 79 26 (9) Relational Lisp 274 12 3 (10) Haskell 156 112 8 Figure 3: Summary of Prototype Software Development Metrics 1. Rapide, a language developed at Stanford, uses a partial-ordering-on-events semantics and a high-level-module structure, and is targeted primarily for simulation and software archi-
7. Confusion > They were convinced it was incomplete… > “Clever,” “Cute but not Extensible,” … > “Too cute for its own good.” W.E. Carlson, P. Hudak, and M.P. Jones. An experiment using Haskell to prototype “geometric region servers” for navy command and control. Research Report 1031, Department of Computer Science, Yale University, November 1993.
8. Hudak’s Conclusion: Barriers > If functional languages are to become more widely used, various sociological and psychological barriers must be overcome. P. Hudak, and M.P. Jones. An Experiment in Software Prototyping Productivity. 1994.
9. Wadler’s Reasons [2004] How enterprises use functional languages, and why they don’t Philip Wadler Bell Labs, Lucent Technologies wadler@research.bell-labs.com Logic programming and functional programming row in the same boat. Meth- ods used to achieve success with one often transpose to the other, and both face similar obstacles. Here I o↵er a compendium of success stories for functional programs, followed by a list of obstacles to more widespread use of functional programming, in the belief that much of this experience is relevant to logic pro- grammers. This material first appeared as columns in ACM SIGPLAN Notices [29, 30]. The final section contains a few remarks specific to the relations between functional and logic programming.
10. 7 Adoption Inhibitors Technical Non-Technical Compatibility, especially with C/C++ libraries (i.e. Bindings) Availability of Libraries Portability and Ease of Installation Small Footprint, for embedding in standalone applications Availability of Commercial Support Training for developers used to an imperative style Popularity and Social Proof
11. Erlang’s “Right Steps Taken” A⁄b > Erlang succeeded not just because it was a good language design, but because its designers took the right steps to promote its growth. P. Wadler. How enterprises use functional languages, and why they don’t. 2004.
12. Erlang’s “Right Steps Taken” B⁄b > They evolved the language in tandem with its applications, worked closely with developers, and provided documentation, courses, hot-lines, and consultants. P. Wadler. How enterprises use functional languages, and why they don’t. 2004.
13. Discussion: “Killer Apps” Some hoped that existence of killer apps will help spread the gospel. We have seen many already. BEAM killer apps, old and new:
 AXD 301, WhatsApp, old Facebook Chat, Bleacher Report, Discord, … n.b. AXD 301 is mostly C, according to Mats Cronqvist, Erlang Factory (SF Bay) 2010: “there was much more C than Erlang… no restarts and no upgrades… functionality was very well defined… nevertheless… the system was very reliable.”
14. Summary Existing literature focused on how to persuade established organisations to adopt FP. They identify the main problem of lack of FP adoption in these contexts as mainly psychological and periphery, related to training, integration, and deployment.
15. 2 What has changed in
 today’s world?
16. Discussion All of this is irrelevant if you have technical control. Then the problem becomes one of contingency, training, and general good technical management. Good technical management is more likely to beget good technology, and good product.
17. Technical Inhibitors vs Elixir Technical Inhibitor Elixir Today Evaluation Bindings NIF, Port Driver, C Node,
 Fork/Exec + Ports, etc Still Not Trivial Libraries 7k Hex Packages Great Portability Homebrew, Docker Hub, Nerves Great Footprint Not a Concern in 2018? Great
19. How To Sell Elixir [2017] Perception Reality Antidote Prerequisite Technology is
 New or Unproven Implementation Risk Early End-to-End Prototype Freedom to Explore
 & Develop Technology is
 Not ‘Standard’ IT Governance Complexity Bottom-Up
 Cultural Change Explicit KPIs
 & Past Successes Technology is
 Obscure Recruiting &
 Maintenance Risk Professional Development Community Engagement
20. Summary Essentially we have democratised control by more readily allowing and properly managing failures. Cultural acceptance of agile development has gone up; i.e. “Build to Throw Away”, which then enables us to actually more efficiently “Build to Last”.
21. 3 The Future
22. Elixir Today A well-built tool with hyper-specific application in an increasingly popular domain, and valid answers to all of Hudak’s FP Adoption Inhibitors. Lower adoption barrier than its big brother, possibility due to visual familiarity. What does the future have in store for us?
23. Possible Future 1 of N Complete normalisation after a decade of use as part of the default toolchain for Web apps. Some people attend conferences, but most expense their tickets. Industrial user groups remain, but everybody wears suits. Tinkerers have flocked to lesser-known languages.
24. Possible Future 2 of N Altruistic domination, having been at the heart of Web applications for a decade and the core contributors do not seem to have stopped at all. People still organise events and hack-nights, and new Pull Requests are regularly raised and reviewed.
25. Possible Future 3 of N Remaining a niche language that is very good at what it does, with its number of core contributors remaining somewhat consistent with earlier levels. Some people lamented its disuse, while others quietly shipped more products with it.
26. Conclusions Can’t drive adoption with technical excellence alone; the parts considered non-technical are as important. The quality of people & community dictates outcome of an open-source project, barring significant and deliberate redirection of resources.
27. Things to Consider Deliberately take actions to ensure long-term viability and sustainability. Seek Technical, Commercial, Community, and Personal alignment, full if possible, partially otherwise.
28. * Thank You
29. Resources: Papers Hudak, Paul and Mark P. Jones.
 “An Experiment in Software Prototyping Productivity.” (1994). Wiger, Ulf T..
 “Four-fold Increase in Productivity and Quality.” (2001). Wadler, Philip.
 “How enterprises use functional languages , and why they don’t.” (2004).
30. Resources: Talks & Books Cronqvist, Mats.
 “Ruminations on Tools and Strategies” (2010). Wu, Evadne.
 “How to Sell Elixir” (2017). Douthwaite, Boru. Enabling Innovation: A Practical Guide to Understanding and Fostering Technological Change. London: Zed Books, 2002. Print. DeGrace, Paul and Stahl, Leslie Hulet. Wicked Problems, Righteous Solutions: A Catalogue of Modern Software Engineering Paradigms. Yourdon Press, 1990. Print.
31. Contact Evadne Wu <ev@radi.ws>