Keep Phoenix App Productivity

Erlang & Elixir Fest 2018 発表資料

Transcript

1. Keep Phoenix App Productivity ᇙ໌ल XFLAG ελδΦ
2. Who am I • Akihide Kan(@kanmo_ak) • XFLAG(mixi, Inc.) • ໿൒೥લʹݱࡏͷϓϩδΣΫτʢElixirͷήʔϜαʔό ։ൃʣʹࢀՃ
3. Elixir on XFLAG • Elixir͸ήʔϜͷଞʹID؅ཧɺܾࡁɺίϛϡχέʔγϣ ϯαʔϏεɺECαΠτͷҰ෦Ͱར༻͍ͯ͠Δ • ৽نࣄۀͰ΋ଟ͘࠾༻͞Ε͍ͯΔ • Elixir࢖͍ͬͯΔϝϯόʔ͸େମ40ਓ͘Β͍
4. Objectives • ։ൃ1೥ɺӡ༻1೥ଓ͚͖ͯͨதͰPhoenix Application ͷੜ࢈ੑΛԼ͛Δ໰୊ʹͲ͏ରॲ͖͔ͯͨ͠ڞ༗ • Umbrella • Compile • Monitoring
5. Our Project
6. Our project • ϞόΠϧήʔϜͷόοΫΤϯυ • Elixir1.5.1ɺ Phoenix1.3.0 • 2016/06։ൃελʔτɺ2017/06ӡ༻։࢝ • ։ൃ౰࣌͸Elixir1.1ɺPhoenix1.1
7. Architecture
8. ։ൃ౰࣌ͷ໨తɺؔ৺ • ڠྗ/ରઓήʔϜɺଞϢʔβʔ΁ͷ௨஌ • αʔόʹෳࡶͳϩδοΫΛ࣋ͯΔ͔ • ΫϥΠΞϯτͱͷ઀ଓΛ੾ΒͣʹσϓϩΠͰ͖Δ͔ • ϥΠϒϥϦͷ਺ɺߋ৽ස౓
9. Elixir Conf 2017Ͱൃද
10. αʔϏε։࢝ ͦͷޙແࣄϦϦʔε ໿̍೥ؒɺػೳ։ൃΛଓ͚ͨ
11. Running for 1 year $ grep ‘defmodule’ apps/**/*.ex{,s} | wc -l 4083 $ wc -l apps/**/*.ex{,s} | tail -n1 326131 total
12. Phoenix Productivity http://phoenixframework.org/
13. Umbrella Project
14. Many Services • ࠷ॳ͸ҰͭͷPhoenixΞϓϦέʔγϣϯͱͯ͠ελʔτ • ΞϓϦέʔγϣϯ͕੒௕͢ΔʹͭΕͯଟ͘ͷαʔϏεΛ ։ൃ͢Δඞཁ͕ग़͖ͯͨ • ίϯςϯπ؅ཧɺCSɺӡ༻πʔϧɺetc • Deploy΋͕͔͔࣌ؒΔ
15. Umbrella Project
16. Umbrella Project
17. Umbrella Project Feature • ෳ਺ͷαʔϏεΛҰͭͷϦϙδτϦͰผʑʹ։ൃ͢Δ • αʔϏεؒͰίʔυΛڞ༗Ͱ͖Δ • αʔϏεຖʹσϓϩΠɺςετ • ࠷ॳ͸ϞϊϦεͱͯ͠։ൃΛਐΊɺେ͖͘ͳͬͨΒΞϓ ϦέʔγϣϯΛ෼཭͢Δ
18. Umbrella Deploy
19. Distillery Release # rel/config.exs release :test_app_all do set version: “0.0.1" set applications: [:test_app_api, :test_app_socket, :test_app_cs, :test_app_cms, :test_app_to ols] … end # release separately release :test_app_api do set version: current_version(:test_app_api) set applications: [:test_app_api] … end
20. Separate logic
21. Separate logic def deps do # api_app, mix.exs {:interface_lib_app, inumbrella: true} … end
22. Umbrella or Microservices Repository Interface Deploy Umbrella Single Via subapp Both(one app or separately) Microservices Separate Repositories Defined Interface Separately
23. Elixir Compile
24. Elixir Compile • Elixir͸macroʹΑͬͯmeta programmingͷػೳΛ࣮ ݱ • macro͸ίϯύΠϧ࣌ʹͷΈධՁ • Compile time dependencies͕ੜ·ΕΔͱͦͷϞδϡʔ ϧʹؔ࿈ͨ͠Ϟδϡʔϧ΋ίϯύΠϧର৅ʹͳΔ
25. Compile Time Dependencies defmodule MacroTest do require MacroA def test, do: MacroA.use_macro("hello") end defmodule MacroA do defmacro use_macro(x) do if ModuleA.func() do quote(do: unquote("#{x} macro")) else quote(do: unquote("#{x} dependency")) end end end
26. Compile Time Dependencies defmodule MacroTest do require MacroA def test, do: MacroA.use_macro("hello") end defmodule MacroA do defmacro use_macro(x) do if ModuleA.func() do quote(do: unquote("#{x} macro")) else quote(do: unquote("#{x} dependency")) end end end
27. Compile Time Dependencies defmodule StructTest do def test do %StructA{field: “hello”} end end defmodule StructA do defstruct [:field] end
28. Compile Time Dependencies defmodule StructTest do def test do %StructA{field: “hello”} end end defmodule StructA do defstruct [:field] end
29. Compile Time Dependencies defmodule Test do import ModuleA end defmodule BehaviourTest do @behaviour BehaviourA def test(x), do: x end
30. Compile Time Dependencies defprotocol ProtocolA do def test(x) end # protocol_impl.ex defimpl ProtocolA, for: StructA do def test(_), do: “test” end # protocol_caller.ex defmodule ProtocolCaller do def use_protocol(x), do ProtocolA.test(x) end end
31. Compile Time Dependencies defprotocol ProtocolA do def test(x) end # protocol_impl.ex defimpl ProtocolA, for: StructA do def test(_), do: “test” end # protocol_caller.ex defmodule ProtocolCaller do def use_protocol(x), do ProtocolA.test(x) end end
32. Mix xref $ mix xref graph —format dot
33. Mix xref $ mix xref graph —format dot digraph "xref graph" { "lib/macro_a.ex" "lib/macro_a.ex" -> “lib/module_a.ex" "lib/macro_test.ex" "lib/macro_test.ex" -> "lib/macro_a.ex" [label="(compile)"] }
34. Mix xref $ mix xref graph —format dot digraph "xref graph" { "lib/macro_a.ex" "lib/macro_a.ex" -> “lib/module_a.ex" "lib/macro_test.ex" "lib/macro_test.ex" -> "lib/macro_a.ex" [label="(compile)"] }
35. Dirty Trick defmodule MacroTest do def test() do func_map = %{key1: Test.Module.Function1, key2: Test.Module.Function2} end end
36. Dirty Trick defmodule MacroTest do def test() do func_map = %{key1: Module.concat([Test, Module, Function1], key2: Module.concat([Test, Module, Function2]} end end defmodule MacroTest do def test() do func_map = %{key1: Test.Module.Function1, key2: Test.Module.Function2} end end
37. Slow Test
38. Slow Test • Test࣮ߦ࣌ؒΑΓϩʔυɺίϯύΠϧ͕࣌ؒ஗͍ • 1ϑΝΠϧमਖ਼͚ͨͩ͠Ͱ΋଴ͨ͞ΕΔ • ຊདྷ͸ద੾ͳαΠζͰϓϩδΣΫτΛ෼ׂ͢Δͷ͕ྑ͍ ͕ɺͦ͏Ͱ͖ͳ͍ͱ͖͸IExͷϔϧύʔؔ਺Λ࡞ͬͨ
39. IEx Helper # .iex.exs defmodule IExHelper do def rerun_spec(file_name) do Mix.Task.reenable “espec” … Code.load_file(file_name) |> get_in([Access.at(0), Access.elem(0)]) |> ESpec.SuiteRunner.run(ESpec.Configuration.all) |> ESpec.Output.final_result() end end
40. Monitoring and Operation
41. Prometheus and Grafana ओʹ؂ࢹ͍ͯ͠Δ߲໨ ETS table, Output&Received Bytes, Context Switches, Run Queues Length, Processes
42. API Metrics
43. Example
44. Example
45. Prometheus and Grafana Prometheus ClientͷϥΠϒϥϦ΍Erlang VM༻ͷGrafana μογϡϘʔυΛར༻͍ͯ͠Δ
46. $=proc:<0.13622.1> State: Garbing Name: ‘Elixir.Timex_meck' Spawned as: proc_lib:init_p/5 Spawned by: <0.4265.0> Started: Wed May 17 05:15:18 2017 Message queue length: 4 Number of heap fragments: 0 Heap fragment data: 14666 Link list: [<0.4265.0>, {from,<0.16290.3>,#Ref<0.0.5505025.4620>}] Reductions: 79249321 Stack+heap: 164783480 … Remsh remshͰىಈதͷphoenixϓϩηεʹ઀ଓͯ͠ௐࠪͰ͖Δ
47. $=proc:<0.13622.1> State: Garbing Name: ‘Elixir.Timex_meck' Spawned as: proc_lib:init_p/5 Spawned by: <0.4265.0> Started: Wed May 17 05:15:18 2017 Message queue length: 4 Number of heap fragments: 0 Heap fragment data: 14666 Link list: [<0.4265.0>, {from,<0.16290.3>,#Ref<0.0.5505025.4620>}] Reductions: 79249321 Stack+heap: 164783480 … $ iex —name “app_name” —cookie “mycookie” —remsh “app_name-11111@hostname iex()1> Process.whereis(Elixir.Timex_meck) #PID<0.19553.2> iex()2>:erlang.process_info(pid(“0.19553.2”), :total_heap_s ize) {:total_heap_size, 224366925} Remsh remshͰىಈதͷphoenixϓϩηεʹ઀ଓͯ͠ௐࠪͰ͖Δ
48. Library
49. Library • total: 6647 (2018/06/13) • ඞཁͳ΋ͷ͸Ұ௨Γἧ͍ͬͯΔ
50. Library • ͳ͍΋ͷ͸࡞ͬͨɺ΋͘͠͸Ұ෦มߋͯ͠࢖ͬͯΔ • nats pubsub • db sharding • cache utility • redis cluster
51. Conclusion
52. ·ͱΊ • ΞϓϦέʔγϣϯ͕େ͖͘ͳΔʹͭΕͯੜ࢈ੑ͕Լ͕Δ໰୊͕ग़ͯ ͘Δʢcompile, test, deploy…ʣ • Umbrella͸ϓϩδΣΫτ෼ׂΛ؆୯ʹͰ͖ΔΑ͏ʹαϙʔτ͠ ͯ͘ΕΔ • Compileର৅͕ଟ͍৔߹͸ɺແବͳCompile Time DependenciesΛ࡟আ͢Δ • Ϣʔβʔͷ઀ଓ؅ཧͱϩδοΫΛॲཧ͢ΔαʔόΛ෼͚ͨ
53. ·ͱΊ • LibraryɺFWͳͲͷΤίγεςϜʹ͍ͭͯ • ։ൃɺӡ༻Λͯ͠ΈͯLibrary͕গͳ͗͢Δ໰୊͸ಛʹͳ͔ͬͨ • ඞཁͳ࣌ʹ࡞Ε͹ྑ͍(ύοέʔδϚωʔδϟʔɺπʔϧ͕༏ल) • ElixirɺPhoenixͷόʔδϣϯΞοϓ͸׆ൃ →ɹؤுͬͯ௥ै • (Type specΛॻ͍͓ͯ͘ͱυΩϡϝϯτʹͳΔͷͰίʔυ͕ಡΈ΍͢ ͍)
54. Thank you