Back to the future with Erlang (and Elixir)

Lightning talk presented at the Sydney DevOps meetup

Transcript

1. Back to the future with Erlang (and Elixir) Peter Ericson (pericson@odecee.com.au) @pdericson Thursday, 15 August 13
2. What is Erlang? • Programming language that compiles to bytecode - BEAM runtime (a.k.a “EVM”) • It’s a functional language that supports concurrency, is soft realtime and has fault tolerance built in Thursday, 15 August 13
3. What is Erlang? cont... • Supports hotswapping of code - think Continuous Delivery with zero downtime • Extremely mature and battle tested • Erlang the movie Thursday, 15 August 13
4. What is Cowboy? • Cowboy is a HTTP framework written in Erlang • support for upcoming protocols like SPDY • Also used by Elixir’s web framework Dynamo Thursday, 15 August 13
5. What is Elixir? • Elixir is language with Ruby and Erlang semantics • It compiles directly to Erlang (BEAM) bytecode • Erlang and Elixir modules are interchangable with zero overhead and translation cost Thursday, 15 August 13
6. What is Elixir? cont... • Most of Elixir is written in Elixir due to excellent macro support • More approachable than Erlang? You decide :) Thursday, 15 August 13
7. Why Erlang, why now? • Erlang “processes” are scheduled across CPU cores automatically • Erlang “processes” can be scheduled across multiple nodes • Erlang is Web scale Thursday, 15 August 13
8. Why Erlang, why now? cont... • A single Erlang node can replace dozens of separate nodes running Python or Ruby - it’s more efficient (both CPU and memory) • As we move more computing to the cloud we need to account more for the dollar spend on resources Thursday, 15 August 13
9. What now? • Create a dev environment with Vagrant and Puppet • Deploy two example apps (one Erlang, one Elixir) • Look at some code, do some hotswapping and demonstrate inherent concurrency Thursday, 15 August 13
10. Vagrant.configure("2") do |config| config.vm.define :web do |web| ... end config.vm.define :cache do |cache| ... end config.vm.define :db do |db| ... end end Vagrantfile Thursday, 15 August 13
11. config.vm.define :web do |web| web.vm.box = "precise64_vmware" web.vm.box_url = "http://files.vagrantup.com/precise64_vmware.box" web.vm.hostname = "web" web.vm.network :private_network, ip: "192.168.100.100" web.vm.provision :shell, :inline => $script web.vm.provision :puppet do |puppet| puppet.manifest_file = "web.pp" puppet.module_path = "modules" puppet.options = "--confdir etc/puppet --verbose" end end Vagrantfile, cont... Thursday, 15 August 13
12. include elixir include erlang include git include postgresql::client include rebar include rsync include tmux Puppet manifest (web.pp) Thursday, 15 August 13
13. $ vagrant up --provider vmware_fusion vagrant up... Thursday, 15 August 13
14. all: hello_world elixir_hello_world .PHONY: hello_world hello_world: vagrant ssh web -c "sudo rsync -av --delete --exclude deps --exclude ebin /vagrant/ hello_world /usr/local/" vagrant ssh web -c "sudo chown -R root:root /usr/local/hello_world" vagrant ssh web -c "cd /usr/local/hello_world && sudo rebar get-deps compile" vagrant ssh web -c "cd /usr/local/hello_world && { sudo tmux ls 2> /dev/null | grep hello_world > /dev/null || sudo tmux new -d -s hello_world ./start.sh; }" .PHONY: elixir_hello_world elixir_hello_world: vagrant ssh web -c "sudo rsync -av --delete --exclude deps --exclude ebin /vagrant/ elixir_hello_world /usr/local/" vagrant ssh web -c "sudo chown -R root:root /usr/local/elixir_hello_world" vagrant ssh web -c "cd /usr/local/elixir_hello_world && sudo env PATH=$$PATH:/usr/local/ elixir/bin mix deps.get compile" vagrant ssh web -c "cd /usr/local/elixir_hello_world && { sudo tmux ls 2> /dev/null | grep elixir_hello_world > /dev/null || sudo tmux new -d -s elixir_hello_world 'env PATH=$$PATH:/ usr/local/elixir/bin mix run --no-halt';\ }" Makefile I’m sure you can do better than this... Thursday, 15 August 13
15. hello_world/ README.md rebar.config src src/hello_world.app.src src/hello_world.erl src/hello_world_app.erl src/hello_world_sup.erl src/toppage_handler.erl start.sh Example app (Erlang) Thursday, 15 August 13
16. start(_Type, _Args) -> Dispatch = cowboy_router:compile([ ! {'_', [ {"/", toppage_handler, []} ! ! ]} ! ]), ! {ok, _} = cowboy:start_http(http, 100, [{port, 8080}], [ {env, [{dispatch, Dispatch}]} ! ]), ! hello_world_sup:start_link(). Example app (Erlang), cont... URL to module map Thursday, 15 August 13
17. hello_world/ README.md rebar.config src src/hello_world.app.src src/hello_world.erl src/hello_world_app.erl src/hello_world_sup.erl src/toppage_handler.erl start.sh Example app (Erlang), cont... Thursday, 15 August 13
18. % @doc Hello world handler. -module(toppage_handler). -export([init/3]). -export([handle/2]). -export([terminate/3]). init(_Transport, Req, []) -> {ok, Req, undefined}. handle(Req, State) -> ! {ok, Req2} = cowboy_req:reply(200, [], <<"Hello world!">>, Req), ! {ok, Req2, State}. terminate(_Reason, _Req, _State) -> ! ok. Example app (Erlang), cont... Thursday, 15 August 13
19. elixir_hello_world/ lib lib/elixir_hello_world lib/elixir_hello_world/supervisor.ex lib/elixir_hello_world/top_page_handler.ex lib/elixir_hello_world.ex mix.exs README.md test test/elixir_hello_world_test.exs test/test_helper.exs Example app (Elixir) Thursday, 15 August 13
20. defmodule ElixirHelloWorld.TopPageHandler do def init(_transport, req, []) do {:ok, req, nil} end def handle(req, state) do {:ok, req} = :cowboy_req.reply(200, [], "Hello world!", req) {:ok, req, state} end def terminate(_reason, _req, _state), do: :ok end Example app (Elixir), cont... Thursday, 15 August 13
21. $ mix dynamo ~/git/hello_dynamo * creating README.md * creating .gitignore * creating mix.lock * creating mix.exs * creating web * creating web/routers * creating web/routers/application_router.ex * creating web/templates * creating web/templates/index.html.eex * creating lib * creating lib/hello_dynamo.ex * creating lib/hello_dynamo * creating lib/hello_dynamo/dynamo.ex * creating lib/hello_dynamo/environments * creating lib/hello_dynamo/environments/dev.exs * creating lib/hello_dynamo/environments/test.exs * creating lib/hello_dynamo/environments/prod.exs * creating priv * creating priv/static * creating priv/static/favicon.ico * creating test * creating test/test_helper.exs * creating test/features * creating test/features/home_test.exs * creating test/routers * creating test/routers/application_router_test.exs Aside: Elixir Dynamo Create a skeleton app with one command Tests included! Thursday, 15 August 13
22. defmodule HomeTest do use HelloDynamo.TestCase use Dynamo.HTTP.Case test "returns OK" do conn = get("/") assert conn.status == 200 end end Aside: Elixir Dynamo, cont... $ mix test ... Thursday, 15 August 13
23. $ siege -b -c 100 -t 60S http://192.168.100.100:8080/ ... HTTP/1.1 200 0.02 secs: 14 bytes ==> GET / HTTP/1.1 200 0.02 secs: 14 bytes ==> GET / HTTP/1.1 200 0.02 secs: 14 bytes ==> GET / Lifting the server siege... done. Transactions:! ! 325851 hits Availability:! ! 100.00 % Elapsed time:! ! 59.03 secs Data transferred:! 4.21 MB Response time:!! 0.02 secs Transaction rate:! 5520.09 trans/sec Throughput:! ! 0.07 MB/sec Concurrency:! ! 99.63 Successful transactions: 325851 Failed transactions:! 0 Longest transaction:! 0.07 Shortest transaction:! 0.00 Benchmark ... Thursday, 15 August 13
24. Erlang R16B01 (erts-5.10.2) [source-bdf5300] [64-bit] [async-threads:10] [hipe] [kernel-poll:false] Eshell V5.10.2 (abort with ^G) 1> Point your browser at http://localhost:8080 Reloading toppage_handler ... ok. Benchmark, cont... ...no dropped connections while reloading! Thursday, 15 August 13
25. config.vm.define :web do |web| web.vm.box = "precise64_vmware" web.vm.box_url = "http://files.vagrantup.com/precise64_vmware.box" web.vm.hostname = "web" web.vm.network :private_network, ip: "192.168.100.100" web.vm.provision :shell, :inline => $script web.vm.provision :puppet do |puppet| puppet.manifest_file = "web.pp" puppet.module_path = "modules" puppet.options = "--confdir etc/puppet --verbose" end web.vm.provider :vmware_fusion do |v| v.vmx["numvcpus"] = "8" end end Concurrency... Thursday, 15 August 13
26. All your cores are belong to us Thursday, 15 August 13
27. TODO • Demonstrate a basic REST API resource • Add a PostgreSQL query, e.g. “SELECT now()” and see what breaks (hint: queries to Postgres will start timing out...) • Benchmark comparison with Varnish (~30% faster?) Thursday, 15 August 13
28. Who’s using Erlang? • Amazon - SimpleDB • Basho - Riak • Facebook - ejabberd • Chef - server version 11 • RabbitMQ • GitHub - RPC proxying Ruby • Goldman Sachs - HF trading • Ericcson - GPRS and 3G networking • Yahoo! - Delicious • WhatsApp - messenging • Zotonic (CMS) • RackSpace? Thursday, 15 August 13
29. Questions? https://bitbucket.org/pdericson/erlang_future The Cowboy cowgirl Thursday, 15 August 13