Managing Externalities in a Functional World

The modern software developer is spoilt for choice. With the options available today, it has never been easier to build solutions, run experiments, and recover from mistakes. Cheap leverage however encourages incidental complexity, reduces the inherent need for cooperation, and obviates common standards.

Functional programming offers unprecedented leverage, which amplifies these externalities. Knowingly or not, in this way the modern software developer wields tremendous power, but at the same time walks on the knife's edge.

In this talk, Evadne will attempt to provide an overview of these externalities, the historical context, near- and long-term implications, and a few projections as to how things will play out. She will also share some ideas as to how we can mitigate these problems together as a community.

OBJECTIVES

The main objective of this talk is to provide a historic and humanistic perspective that can be used by software developers to interpret functional programming, and its meteoric re-emergence in the web development arena.

The secondary objective is to actually induct people into the world of functional programming by making its premises accessible, but at the same time ensuring that they start with a long-term positive mindset.

Evadne believes that a large number of Elixir developers initially adopted Elixir with a continuous improvement mindset. This talk aims to appropriately echo the mentality, examine the problems currently faced by the community and share potential approaches to be used in our own daily work.

Transcript

1. Managing Externalities in a
   Functional World
   Evadne Wu
   Head of Exam Systems
   Faria Education Group
   date
   
   16 August 2018
   

   View Slide

2. Goal
   To promote the application of functional languages,

   in general, and Elixir in particular,

   from a tool of last resort, used in anger,

   to the tool of choice.
   In order to navigate the future, one must understand the past (context),

   so in we go…
   

   View Slide

3. 1
   Past Literature on Adoption
   

   View Slide

4. Hudak’s Experiment [1994]
   Haskell vs. Ada vs. C++ vs. Awk vs. ...
   An Experiment in Software Prototyping Productivity
   Paul Hudak
   Mark P. Jones
   Yale University
   Department of Computer Science
   New Haven, CT 06518
   hudak-paul,jones-mark @cs.yale.edu
   July 4, 1994
   

   View Slide

5. Slave Doctrine
   Weapon Doctrine
   Engageability
   Zone
   Hostile
   Aircraft
   Commercial
   Aircraft
   Tight Zone
   Carrier
   Aegis Ship
   

   View Slide

6. Outcome: Dramatic FP Edge
   The Navy secretly had the Haskell experiment re-done by a beginner,

   because the original numbers were too dramatic.
   Language Lines of code Lines of documentation Development time (hours)
   (1) Haskell 85 465 10
   (2) Ada 767 714 23
   (3) Ada9X 800 200 28
   (4) C++ 1105 130 –
   (5) Awk/Nawk 250 150 –
   (6) Rapide 157 0 54
   (7) Griffin 251 0 34
   (8) Proteus 293 79 26
   (9) Relational Lisp 274 12 3
   (10) Haskell 156 112 8
   Figure 3: Summary of Prototype Software Development Metrics
   1. Rapide, a language developed at Stanford, uses a partial-ordering-on-events semantics and
   a high-level-module structure, and is targeted primarily for simulation and software archi-
   

   View Slide

7. Confusion
   > They were convinced it was incomplete…
   > “Clever,” “Cute but not Extensible,” …
   > “Too cute for its own good.”
   W.E. Carlson, P. Hudak, and M.P. Jones. An experiment using Haskell to prototype
   “geometric region servers” for navy command and control. Research Report 1031,
   Department of Computer Science, Yale University, November 1993.
   

   View Slide

8. Hudak’s Conclusion: Barriers
   > If functional languages are to become more widely
   used, various sociological and psychological barriers
   must be overcome.
   P. Hudak, and M.P. Jones. An Experiment in Software Prototyping Productivity. 1994.
   

   View Slide

9. Wadler’s Reasons [2004]
   How enterprises use functional languages, and
   why they don’t
   Philip Wadler
   Bell Labs, Lucent Technologies
   wadler@research.bell-labs.com
   Logic programming and functional programming row in the same boat. Meth-
   ods used to achieve success with one often transpose to the other, and both face
   similar obstacles. Here I o↵er a compendium of success stories for functional
   programs, followed by a list of obstacles to more widespread use of functional
   programming, in the belief that much of this experience is relevant to logic pro-
   grammers. This material first appeared as columns in ACM SIGPLAN Notices
   [29, 30]. The final section contains a few remarks specific to the relations between
   functional and logic programming.
   

   View Slide

10. 7 Adoption Inhibitors
    Technical
    Non-Technical
    Compatibility, especially with C/C++ libraries (i.e. Bindings)
    Availability of Libraries
    Portability and Ease of Installation
    Small Footprint, for embedding in standalone applications
    Availability of Commercial Support
    Training for developers used to an imperative style
    Popularity and Social Proof
    

    View Slide

11. Erlang’s “Right Steps Taken” A⁄b
    > Erlang succeeded not just because it was a good
    language design, but because its designers took the
    right steps to promote its growth.
    P. Wadler. How enterprises use functional languages, and why they don’t. 2004.
    

    View Slide

12. Erlang’s “Right Steps Taken” B⁄b
    > They evolved the language in tandem with its
    applications, worked closely with developers, and
    provided documentation, courses, hot-lines, and
    consultants.
    P. Wadler. How enterprises use functional languages, and why they don’t. 2004.
    

    View Slide

13. Discussion: “Killer Apps”
    Some hoped that existence of killer apps will help
    spread the gospel. We have seen many already.
    BEAM killer apps, old and new:

    AXD 301, WhatsApp, old Facebook Chat, Bleacher Report, Discord, …
    n.b. AXD 301 is mostly C, according to Mats Cronqvist, Erlang Factory (SF Bay) 2010:
    “there was much more C than Erlang… no restarts and no upgrades… functionality
    was very well defined… nevertheless… the system was very reliable.”
    

    View Slide

14. Summary
    Existing literature focused on how to persuade
    established organisations to adopt FP.
    They identify the main problem of lack of FP adoption in
    these contexts as mainly psychological and periphery,
    related to training, integration, and deployment.
    

    View Slide

15. 2
    What has changed in

    today’s world?
    

    View Slide

16. Discussion
    All of this is irrelevant if you have technical control.
    Then the problem becomes one of contingency, training,
    and general good technical management.
    Good technical management is more likely to beget good technology, and good
    product.
    

    View Slide

17. Technical Inhibitors vs Elixir
    Technical Inhibitor Elixir Today Evaluation
    Bindings NIF, Port Driver, C Node,

    Fork/Exec + Ports, etc
    Still Not Trivial
    Libraries 7k Hex Packages Great
    Portability Homebrew, Docker Hub,
    Nerves
    Great
    Footprint Not a Concern in 2018? Great
    

    View Slide

18. View Slide

19. How To Sell Elixir [2017]
    Perception Reality Antidote Prerequisite
    Technology is

    New or Unproven
    Implementation Risk
    Early End-to-End
    Prototype
    Freedom to Explore

    & Develop
    Technology is

    Not ‘Standard’
    IT Governance
    Complexity
    Bottom-Up

    Cultural Change
    Explicit KPIs

    & Past Successes
    Technology is

    Obscure
    Recruiting &

    Maintenance Risk
    Professional
    Development
    Community
    Engagement
    

    View Slide

20. Summary
    Essentially we have democratised control by more
    readily allowing and properly managing failures.
    Cultural acceptance of agile development has gone up;
    i.e. “Build to Throw Away”, which then enables us to
    actually more efficiently “Build to Last”.
    

    View Slide

21. 3
    The Future
    

    View Slide

22. Elixir Today
    A well-built tool with hyper-specific application in an
    increasingly popular domain, and valid answers to all of
    Hudak’s FP Adoption Inhibitors.
    Lower adoption barrier than its big brother, possibility
    due to visual familiarity.
    What does the future have in store for us?
    

    View Slide

23. Possible Future 1 of N
    Complete normalisation after a decade of use as part
    of the default toolchain for Web apps. Some people
    attend conferences, but most expense their tickets.
    Industrial user groups remain, but everybody wears
    suits. Tinkerers have flocked to lesser-known languages.
    

    View Slide

24. Possible Future 2 of N
    Altruistic domination, having been at the heart of Web
    applications for a decade and the core contributors do
    not seem to have stopped at all. People still organise
    events and hack-nights, and new Pull Requests are
    regularly raised and reviewed.
    

    View Slide

25. Possible Future 3 of N
    Remaining a niche language that is very good at what
    it does, with its number of core contributors remaining
    somewhat consistent with earlier levels. Some people
    lamented its disuse, while others quietly shipped more
    products with it.
    

    View Slide

26. Conclusions
    Can’t drive adoption with technical excellence alone; the
    parts considered non-technical are as important.
    The quality of people & community dictates outcome of
    an open-source project, barring significant and
    deliberate redirection of resources.
    

    View Slide

27. Things to Consider
    Deliberately take actions to ensure long-term viability
    and sustainability.
    Seek Technical, Commercial, Community, and Personal
    alignment, full if possible, partially otherwise.
    

    View Slide

28. *
    Thank You
    

    View Slide

29. Resources: Papers
    Hudak, Paul and Mark P. Jones.

    “An Experiment in Software Prototyping Productivity.” (1994).
    Wiger, Ulf T..

    “Four-fold Increase in Productivity and Quality.” (2001).
    Wadler, Philip.

    “How enterprises use functional languages , and why they don’t.” (2004).
    

    View Slide

30. Resources: Talks & Books
    Cronqvist, Mats.

    “Ruminations on Tools and Strategies” (2010).
    Wu, Evadne.

    “How to Sell Elixir” (2017).
    Douthwaite, Boru. Enabling Innovation: A Practical Guide to Understanding and
    Fostering Technological Change. London: Zed Books, 2002. Print.
    DeGrace, Paul and Stahl, Leslie Hulet. Wicked Problems, Righteous Solutions: A
    Catalogue of Modern Software Engineering Paradigms. Yourdon Press, 1990. Print.
    

    View Slide

31. Contact
    Evadne Wu
    

    View Slide

32. View Slide