Keep Phoenix App Productivity ᇙ໌ल XFLAG ελδΦ

Who am I • Akihide Kan(@kanmo_ak) • XFLAG(mixi, Inc.) • ໿൒೥લʹݱࡏͷϓϩδΣΫτʢElixirͷήʔϜαʔό ։ൃʣʹࢀՃ

Elixir on XFLAG • Elixir͸ήʔϜͷଞʹID؅ཧɺܾࡁɺίϛϡχέʔγϣ ϯαʔϏεɺECαΠτͷҰ෦Ͱར༻͍ͯ͠Δ • ৽نࣄۀͰ΋ଟ͘࠾༻͞Ε͍ͯΔ • Elixir࢖͍ͬͯΔϝϯόʔ͸େମ40ਓ͘Β͍

Objectives • ։ൃ1೥ɺӡ༻1೥ଓ͚͖ͯͨதͰPhoenix Application ͷੜ࢈ੑΛԼ͛Δ໰୊ʹͲ͏ରॲ͖͔ͯͨ͠ڞ༗ • Umbrella • Compile • Monitoring

Our Project

Our project • ϞόΠϧήʔϜͷόοΫΤϯυ • Elixir1.5.1ɺ Phoenix1.3.0 • 2016/06։ൃελʔτɺ2017/06ӡ༻։࢝ • ։ൃ౰࣌͸Elixir1.1ɺPhoenix1.1

Architecture

։ൃ౰࣌ͷ໨తɺؔ৺ • ڠྗ/ରઓήʔϜɺଞϢʔβʔ΁ͷ௨஌ • αʔόʹෳࡶͳϩδοΫΛ࣋ͯΔ͔ • ΫϥΠΞϯτͱͷ઀ଓΛ੾ΒͣʹσϓϩΠͰ͖Δ͔ • ϥΠϒϥϦͷ਺ɺߋ৽ස౓

Elixir Conf 2017Ͱൃද

αʔϏε։࢝ ͦͷޙແࣄϦϦʔε ໿̍೥ؒɺػೳ։ൃΛଓ͚ͨ

Running for 1 year $ grep ‘defmodule’ apps/**/*.ex{,s} | wc -l 4083 $ wc -l apps/**/*.ex{,s} | tail -n1 326131 total

Phoenix Productivity http://phoenixframework.org/

Umbrella Project

Many Services • ࠷ॳ͸ҰͭͷPhoenixΞϓϦέʔγϣϯͱͯ͠ελʔτ • ΞϓϦέʔγϣϯ͕੒௕͢ΔʹͭΕͯଟ͘ͷαʔϏεΛ ։ൃ͢Δඞཁ͕ग़͖ͯͨ • ίϯςϯπ؅ཧɺCSɺӡ༻πʔϧɺetc • Deploy΋͕͔͔࣌ؒΔ

Umbrella Project

Umbrella Project

Umbrella Project Feature • ෳ਺ͷαʔϏεΛҰͭͷϦϙδτϦͰผʑʹ։ൃ͢Δ • αʔϏεؒͰίʔυΛڞ༗Ͱ͖Δ • αʔϏεຖʹσϓϩΠɺςετ • ࠷ॳ͸ϞϊϦεͱͯ͠։ൃΛਐΊɺେ͖͘ͳͬͨΒΞϓ ϦέʔγϣϯΛ෼཭͢Δ

Umbrella Deploy

Distillery Release # rel/config.exs release :test_app_all do set version: “0.0.1" set applications: [:test_app_api, :test_app_socket, :test_app_cs, :test_app_cms, :test_app_to ols] … end # release separately release :test_app_api do set version: current_version(:test_app_api) set applications: [:test_app_api] … end

Separate logic

Separate logic def deps do # api_app, mix.exs {:interface_lib_app, inumbrella: true} … end

Umbrella or Microservices Repository Interface Deploy Umbrella Single Via subapp Both(one app or separately) Microservices Separate Repositories Defined Interface Separately

Elixir Compile

Elixir Compile • Elixir͸macroʹΑͬͯmeta programmingͷػೳΛ࣮ ݱ • macro͸ίϯύΠϧ࣌ʹͷΈධՁ • Compile time dependencies͕ੜ·ΕΔͱͦͷϞδϡʔ ϧʹؔ࿈ͨ͠Ϟδϡʔϧ΋ίϯύΠϧର৅ʹͳΔ

Compile Time Dependencies defmodule MacroTest do require MacroA def test, do: MacroA.use_macro("hello") end defmodule MacroA do defmacro use_macro(x) do if ModuleA.func() do quote(do: unquote("#{x} macro")) else quote(do: unquote("#{x} dependency")) end end end

Compile Time Dependencies defmodule MacroTest do require MacroA def test, do: MacroA.use_macro("hello") end defmodule MacroA do defmacro use_macro(x) do if ModuleA.func() do quote(do: unquote("#{x} macro")) else quote(do: unquote("#{x} dependency")) end end end

Compile Time Dependencies defmodule StructTest do def test do %StructA{field: “hello”} end end defmodule StructA do defstruct [:field] end

Compile Time Dependencies defmodule StructTest do def test do %StructA{field: “hello”} end end defmodule StructA do defstruct [:field] end

Compile Time Dependencies defmodule Test do import ModuleA end defmodule BehaviourTest do @behaviour BehaviourA def test(x), do: x end

Compile Time Dependencies defprotocol ProtocolA do def test(x) end # protocol_impl.ex defimpl ProtocolA, for: StructA do def test(_), do: “test” end # protocol_caller.ex defmodule ProtocolCaller do def use_protocol(x), do ProtocolA.test(x) end end

Compile Time Dependencies defprotocol ProtocolA do def test(x) end # protocol_impl.ex defimpl ProtocolA, for: StructA do def test(_), do: “test” end # protocol_caller.ex defmodule ProtocolCaller do def use_protocol(x), do ProtocolA.test(x) end end

Mix xref $ mix xref graph —format dot

Mix xref $ mix xref graph —format dot digraph "xref graph" { "lib/macro_a.ex" "lib/macro_a.ex" -> “lib/module_a.ex" "lib/macro_test.ex" "lib/macro_test.ex" -> "lib/macro_a.ex" [label="(compile)"] }

Mix xref $ mix xref graph —format dot digraph "xref graph" { "lib/macro_a.ex" "lib/macro_a.ex" -> “lib/module_a.ex" "lib/macro_test.ex" "lib/macro_test.ex" -> "lib/macro_a.ex" [label="(compile)"] }

Dirty Trick defmodule MacroTest do def test() do func_map = %{key1: Test.Module.Function1, key2: Test.Module.Function2} end end

Dirty Trick defmodule MacroTest do def test() do func_map = %{key1: Module.concat([Test, Module, Function1], key2: Module.concat([Test, Module, Function2]} end end defmodule MacroTest do def test() do func_map = %{key1: Test.Module.Function1, key2: Test.Module.Function2} end end

Slow Test

Slow Test • Test࣮ߦ࣌ؒΑΓϩʔυɺίϯύΠϧ͕࣌ؒ஗͍ • 1ϑΝΠϧमਖ਼͚ͨͩ͠Ͱ΋଴ͨ͞ΕΔ • ຊདྷ͸ద੾ͳαΠζͰϓϩδΣΫτΛ෼ׂ͢Δͷ͕ྑ͍ ͕ɺͦ͏Ͱ͖ͳ͍ͱ͖͸IExͷϔϧύʔؔ਺Λ࡞ͬͨ

IEx Helper # .iex.exs defmodule IExHelper do def rerun_spec(file_name) do Mix.Task.reenable “espec” … Code.load_file(file_name) |> get_in([Access.at(0), Access.elem(0)]) |> ESpec.SuiteRunner.run(ESpec.Configuration.all) |> ESpec.Output.final_result() end end

Monitoring and Operation

Prometheus and Grafana ओʹ؂ࢹ͍ͯ͠Δ߲໨ ETS table, Output&Received Bytes, Context Switches, Run Queues Length, Processes

API Metrics

Example

Example

Prometheus and Grafana Prometheus ClientͷϥΠϒϥϦ΍Erlang VM༻ͷGrafana μογϡϘʔυΛར༻͍ͯ͠Δ

$=proc:<0.13622.1> State: Garbing Name: ‘Elixir.Timex_meck' Spawned as: proc_lib:init_p/5 Spawned by: <0.4265.0> Started: Wed May 17 05:15:18 2017 Message queue length: 4 Number of heap fragments: 0 Heap fragment data: 14666 Link list: [<0.4265.0>, {from,<0.16290.3>,#Ref<0.0.5505025.4620>}] Reductions: 79249321 Stack+heap: 164783480 … Remsh remshͰىಈதͷphoenixϓϩηεʹ઀ଓͯ͠ௐࠪͰ͖Δ

$=proc:<0.13622.1> State: Garbing Name: ‘Elixir.Timex_meck' Spawned as: proc_lib:init_p/5 Spawned by: <0.4265.0> Started: Wed May 17 05:15:18 2017 Message queue length: 4 Number of heap fragments: 0 Heap fragment data: 14666 Link list: [<0.4265.0>, {from,<0.16290.3>,#Ref<0.0.5505025.4620>}] Reductions: 79249321 Stack+heap: 164783480 … $ iex —name “app_name” —cookie “mycookie” —remsh “app_name-11111@hostname iex()1> Process.whereis(Elixir.Timex_meck) #PID<0.19553.2> iex()2>:erlang.process_info(pid(“0.19553.2”), :total_heap_s ize) {:total_heap_size, 224366925} Remsh remshͰىಈதͷphoenixϓϩηεʹ઀ଓͯ͠ௐࠪͰ͖Δ

Library

Library • total: 6647 (2018/06/13) • ඞཁͳ΋ͷ͸Ұ௨Γἧ͍ͬͯΔ

Library • ͳ͍΋ͷ͸࡞ͬͨɺ΋͘͠͸Ұ෦มߋͯ͠࢖ͬͯΔ • nats pubsub • db sharding • cache utility • redis cluster

Conclusion

·ͱΊ • ΞϓϦέʔγϣϯ͕େ͖͘ͳΔʹͭΕͯੜ࢈ੑ͕Լ͕Δ໰୊͕ग़ͯ ͘Δʢcompile, test, deploy…ʣ • Umbrella͸ϓϩδΣΫτ෼ׂΛ؆୯ʹͰ͖ΔΑ͏ʹαϙʔτ͠ ͯ͘ΕΔ • Compileର৅͕ଟ͍৔߹͸ɺແବͳCompile Time DependenciesΛ࡟আ͢Δ • Ϣʔβʔͷ઀ଓ؅ཧͱϩδοΫΛॲཧ͢ΔαʔόΛ෼͚ͨ

·ͱΊ • LibraryɺFWͳͲͷΤίγεςϜʹ͍ͭͯ • ։ൃɺӡ༻Λͯ͠ΈͯLibrary͕গͳ͗͢Δ໰୊͸ಛʹͳ͔ͬͨ • ඞཁͳ࣌ʹ࡞Ε͹ྑ͍(ύοέʔδϚωʔδϟʔɺπʔϧ͕༏ल) • ElixirɺPhoenixͷόʔδϣϯΞοϓ͸׆ൃ →ɹؤுͬͯ௥ै • (Type specΛॻ͍͓ͯ͘ͱυΩϡϝϯτʹͳΔͷͰίʔυ͕ಡΈ΍͢ ͍)

Thank you