How to Sell Elixir Evadne Wu Head of Exam Systems, Faria Education Group [email protected] / github.com/evadne last updated 16 August 2017 

Outline ➤ Takeaway (Precepts) ➤ Team & Organisation Background ➤ Our First & Second Contacts with OTP ➤ Lessons Learned from Transitioning to OTP ➤ Closing Thoughts 

Precepts 

Boehm [1983] 

Risk / Value When choosing technologies, you do not always have the luxury of hindsight. Therefore it is always a matter of managing risk. Optimise for different goals in different phases: ➤ Raise Potential Value by proper inception/design ➤ Reduce Value at Risk by proper execution ➤ Reduce impact to Value by designing for operations / maintenance 

Complexity SaaS Adage: More Features, More Customers ➤ Not exactly true However, ➤ Complexity ≠ Profitability ➤ Resource Requirements ∝ Complexity 

Leverage Resource Requirements ∝ (Effort / Leverage). ➤ With higher Leverage, you can do more with less i.e. ship earlier. ➤ Shorter Time Horizon can reduce Risk. NB: Good technology can’t save you from bad specification ➤ But finding out about it earlier also reduces Risk 

Background 

The Organisation SaaS Provider for the International Education sector ➤ Established in 2007 ➤ 39–67% market share in our niche segments ➤ 2,300 customers in 120 countries ➤ Goal: Support the Transition From Paper ➤ We offer Software with a Human Touch; no IVRs ever 

The Team UK branch for Managed Services ➤ We build and operate applications for strategically important clients ➤ And we help the rest of the organisation with intersectional work ➤ Business Analysis and Specification-Writing ➤ Service Development ➤ Not Maintenance 

First & Second Contacts 

Shifting Business Requirements We can capture more of the market by becoming an integrator The downside of being an integrator is having to integrate So, one day we wrote a message broker in Ruby… ➤ Classic Message Bus architecture ➤ Messages, Queues hosted by RabbitMQ ➤ Processing, Fanout and Reconciliation in CRuby 

Some of The Many Problems Spawning one OS process for each Customer is highly inefficient ➤ Threading in CRuby is still a mystery Process monitoring seems easy, but not when you need to be dynamic Work distribution among nodes is also not very easy ➤ SELECT * FROM schools WHERE host = ‘x’ The system dies completely on un-caught exceptions ➤ Error handling logic everywhere 

Solutions? We went and made two more prototypes ➤ JRuby + custom supervision logic + true threads via Concurrent Ruby ➤ Elixir + OTP magic + dynamic supervision tree Ultimately we did not have enough time to do what we wanted ➤ We signed a contract with another vendor to deal with this for us 

Postmortem We would not have been able to make the original solution work ➤ Its performance characteristics are not in the bracket we require However, we learned a lot from the two other prototypes ➤ The JVM ecosystem is a very good alternative when you need things done ➤ The OTP ecosystem has better tooling for a few specific things ➤ Especially for orchestrating highly concurrent, interactive workloads 

Second Contact Vendor B, which we use, gets bought by Vendor C ➤ Hello, here is the new version with a totally different API… ➤ …you must store all the files with us now… ➤ …yes, there is a migration tool but it does not move metadata… ➤ …the cost estimate is $x,000 per month, we hope you’d like it… 

Second Contact Our VPE is not having any of it ➤ Start sending out tenders and getting replies from other Vendors ➤ Big “Buy vs Build” decision starting to brew in VPE’s head 

Vendors Start to Reply… ➤ “We do not have this feature — it will cost $100,000 or so extra, and we will try to build this in for you specifically, if you sign the dotted line.” ➤ “We do not offer trials but we can send you a copy once you sign.” ➤ “Download a test version here!” (The Download site is broken.) 

Adventures in “Buy vs Build” World It was difficult to get even a stable (predictable) quote or even a testable artefact without committing sizeable chunks of money. ➤ Complete antithesis of how we operate ➤ We maintain a full-featured demonstration environment for anybody who visits the link — no information required. ➤ Unnecessarily drains our Research & Professional Development budget 

Looking at Options We are at the crossroads looking at one of three options ➤ New Custom-Built component ➤ New COTS component ➤ Another SaaS component 

Mutual Realisation (×2) Custom integration work is completely inevitable. A team that is in-house is much easier to work with, than one that needs to serve many customers with potentially conflicting requirements. ➤ As an ISV ourselves, we have our own Decision Framework shared with customers explaining how we approach new feature requests. ➤ Roughly based on popularity, priority, and future applicability. 

Our Proposal Take what we have learned from dealing with >1m documents. Make a proper service which serves our own use cases. Demonstrated lower TCO — vs. COTS/SaaS ➤ Also considering Communication/Coordination Cost Demonstrated Feasibility with end-to-end prototype 

“Just Get It Done” To ensure prompt decision, our Proposal comes with a Prototype ➤ Fully working for the high impact use cases ➤ Fully deployed via AWS CFN ➤ We were then told to “get it done”. Sold! 

Transitioning to OTP 

Components From To Results Rails Phoenix Assets Pipeline Brunch ⏳→→ ActionCable/Faye Phoenix Channels ActiveRecord Ecto 

Tasks Task From To Results AWS Interop Fog ExAWS Authentication Devise Guardian Thumbnail Generation CarrierWave Custom C Program ⏳→→ 

Closing Thoughts 

Precepts Most of the problems can be looked through the lenses of… ➤ Risk Assessment and Management: Long-Term vs One-Shot ➤ Nature of the Problem: Wicked vs Righteous ➤ Team Organisation: Mechanical vs Organic ➤ Modes of Control: Free Market, Contractural vs Cultural 

Grove [1983] 

Objections / Antidotes Perception Reality Antidote Prerequisite “New (Unproven)” Implementation Risk Early End-to-End Prototype Freedom to Explore & Develop “Non-Standard” IT Governance Complexity Bottom-Up
 Cultural Change Explicit KPIs and Past Successes “Obscure” Recruiting &
 Maintenance Risk Professional Development Community Engagement