How to Sell Elixir

http://www.elixir.london/Elixir-LDN-2017/evadne-wu

* * *

Technical complexity is now a major cost of doing business and the Elixir/Erlang ecosystems offer appropriate tools to efficiently manage complexity. In its realisation we have built a product entirely based on this technology, and now have stories to share.

In this talk, Evadne will attempt to cover:

1. Their economic case for adopting this particular ecosystem, from both a development velocity point of view and a maintenance/cost point of view.
2. Problems they have faced and solutions they have come up with, both on an organisational level and a technical level, in realising the project.
3. Some ideas for technical leaders wishing to adopt Elixir for their next greenfield or brownfield project.

Talk objectives:
To provide the audience with an appropriate evaluation framework for candidate technologies (i.e. boss persuasion)
To provide a collection of anecdotes on our adoption of the Erlang/OTP and Elixir ecosystems for the benefit of the community (i.e. war stories)
To identify areas for further enhancement both within the technology stack, and around how the stack is enhanced. (i.e. work suggestions)
Target audience:
This talk is intended for technical team leaders and senior software engineers wishing to transition into a role with some more management responsibilities; it is also best suited for relatively small teams (5-25 developers) that have prior experience in developing web applications, since that is the audience who shall find the views most relatable.
Evadne is a London-based technologist with a passion for data visualisation and experience in technical management. She has experience in software architecture, application prototyping, and system operations management. Evadne presently serves as the Head of Exam Systems for Faria Education Group, which is an international education SaaS company with staff in UK, EU and USA.

Transcript

1. How to Sell Elixir Evadne Wu Head of Exam Systems, Faria Education Group ev@radi.ws / github.com/evadne last updated 16 August 2017
2. Outline ➤ Takeaway (Precepts) ➤ Team & Organisation Background ➤ Our First & Second Contacts with OTP ➤ Lessons Learned from Transitioning to OTP ➤ Closing Thoughts
3. Precepts
4. Boehm [1983]
5. Risk / Value When choosing technologies, you do not always have the luxury of hindsight. Therefore it is always a matter of managing risk. Optimise for different goals in different phases: ➤ Raise Potential Value by proper inception/design ➤ Reduce Value at Risk by proper execution ➤ Reduce impact to Value by designing for operations / maintenance
6. Complexity SaaS Adage: More Features, More Customers ➤ Not exactly true However, ➤ Complexity ≠ Profitability ➤ Resource Requirements ∝ Complexity
7. Leverage Resource Requirements ∝ (Effort / Leverage). ➤ With higher Leverage, you can do more with less i.e. ship earlier. ➤ Shorter Time Horizon can reduce Risk. NB: Good technology can’t save you from bad specification ➤ But finding out about it earlier also reduces Risk
8. Background
9. The Organisation SaaS Provider for the International Education sector ➤ Established in 2007 ➤ 39–67% market share in our niche segments ➤ 2,300 customers in 120 countries ➤ Goal: Support the Transition From Paper ➤ We offer Software with a Human Touch; no IVRs ever
10. The Team UK branch for Managed Services ➤ We build and operate applications for strategically important clients ➤ And we help the rest of the organisation with intersectional work ➤ Business Analysis and Specification-Writing ➤ Service Development ➤ Not Maintenance
11. First & Second Contacts
12. Shifting Business Requirements We can capture more of the market by becoming an integrator The downside of being an integrator is having to integrate So, one day we wrote a message broker in Ruby… ➤ Classic Message Bus architecture ➤ Messages, Queues hosted by RabbitMQ ➤ Processing, Fanout and Reconciliation in CRuby
13. Some of The Many Problems Spawning one OS process for each Customer is highly inefficient ➤ Threading in CRuby is still a mystery Process monitoring seems easy, but not when you need to be dynamic Work distribution among nodes is also not very easy ➤ SELECT * FROM schools WHERE host = ‘x’ The system dies completely on un-caught exceptions ➤ Error handling logic everywhere
14. Solutions? We went and made two more prototypes ➤ JRuby + custom supervision logic + true threads via Concurrent Ruby ➤ Elixir + OTP magic + dynamic supervision tree Ultimately we did not have enough time to do what we wanted ➤ We signed a contract with another vendor to deal with this for us
15. Postmortem We would not have been able to make the original solution work ➤ Its performance characteristics are not in the bracket we require However, we learned a lot from the two other prototypes ➤ The JVM ecosystem is a very good alternative when you need things done ➤ The OTP ecosystem has better tooling for a few specific things ➤ Especially for orchestrating highly concurrent, interactive workloads
16. Second Contact Vendor B, which we use, gets bought by Vendor C ➤ Hello, here is the new version with a totally different API… ➤ …you must store all the files with us now… ➤ …yes, there is a migration tool but it does not move metadata… ➤ …the cost estimate is $x,000 per month, we hope you’d like it…
17. Second Contact Our VPE is not having any of it ➤ Start sending out tenders and getting replies from other Vendors ➤ Big “Buy vs Build” decision starting to brew in VPE’s head
18. Vendors Start to Reply… ➤ “We do not have this feature — it will cost $100,000 or so extra, and we will try to build this in for you specifically, if you sign the dotted line.” ➤ “We do not offer trials but we can send you a copy once you sign.” ➤ “Download a test version here!” (The Download site is broken.)
19. Adventures in “Buy vs Build” World It was difficult to get even a stable (predictable) quote or even a testable artefact without committing sizeable chunks of money. ➤ Complete antithesis of how we operate ➤ We maintain a full-featured demonstration environment for anybody who visits the link — no information required. ➤ Unnecessarily drains our Research & Professional Development budget
20. Looking at Options We are at the crossroads looking at one of three options ➤ New Custom-Built component ➤ New COTS component ➤ Another SaaS component
21. Mutual Realisation (×2) Custom integration work is completely inevitable. A team that is in-house is much easier to work with, than one that needs to serve many customers with potentially conflicting requirements. ➤ As an ISV ourselves, we have our own Decision Framework shared with customers explaining how we approach new feature requests. ➤ Roughly based on popularity, priority, and future applicability.
22. Our Proposal Take what we have learned from dealing with >1m documents. Make a proper service which serves our own use cases. Demonstrated lower TCO — vs. COTS/SaaS ➤ Also considering Communication/Coordination Cost Demonstrated Feasibility with end-to-end prototype
23. “Just Get It Done” To ensure prompt decision, our Proposal comes with a Prototype ➤ Fully working for the high impact use cases ➤ Fully deployed via AWS CFN ➤ We were then told to “get it done”. Sold!
24. Transitioning to OTP
25. Components From To Results Rails Phoenix Assets Pipeline Brunch ⏳→→ ActionCable/Faye Phoenix Channels ActiveRecord Ecto
26. Tasks Task From To Results AWS Interop Fog ExAWS Authentication Devise Guardian Thumbnail Generation CarrierWave Custom C Program ⏳→→
27. Closing Thoughts
28. Precepts Most of the problems can be looked through the lenses of… ➤ Risk Assessment and Management: Long-Term vs One-Shot ➤ Nature of the Problem: Wicked vs Righteous ➤ Team Organisation: Mechanical vs Organic ➤ Modes of Control: Free Market, Contractural vs Cultural
29. Grove [1983]
30. Objections / Antidotes Perception Reality Antidote Prerequisite “New (Unproven)” Implementation Risk Early End-to-End Prototype Freedom to Explore & Develop “Non-Standard” IT Governance Complexity Bottom-Up
 Cultural Change Explicit KPIs and Past Successes “Obscure” Recruiting &
 Maintenance Risk Professional Development Community Engagement