Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Keep Phoenix App Productivity

86f6cc36138b8f2706c28d9598e7e36c?s=47 kanmo
June 16, 2018

Keep Phoenix App Productivity

Erlang & Elixir Fest 2018 発表資料

86f6cc36138b8f2706c28d9598e7e36c?s=128

kanmo

June 16, 2018
Tweet

More Decks by kanmo

Other Decks in Programming

Transcript

  1. Keep Phoenix App Productivity ᇙ໌ल XFLAG ελδΦ

  2. Who am I • Akihide Kan(@kanmo_ak) • XFLAG(mixi, Inc.) •

    ໿൒೥લʹݱࡏͷϓϩδΣΫτʢElixirͷήʔϜαʔό ։ൃʣʹࢀՃ
  3. Elixir on XFLAG • Elixir͸ήʔϜͷଞʹID؅ཧɺܾࡁɺίϛϡχέʔγϣ ϯαʔϏεɺECαΠτͷҰ෦Ͱར༻͍ͯ͠Δ • ৽نࣄۀͰ΋ଟ͘࠾༻͞Ε͍ͯΔ • Elixir࢖͍ͬͯΔϝϯόʔ͸େମ40ਓ͘Β͍

  4. Objectives • ։ൃ1೥ɺӡ༻1೥ଓ͚͖ͯͨதͰPhoenix Application ͷੜ࢈ੑΛԼ͛Δ໰୊ʹͲ͏ରॲ͖͔ͯͨ͠ڞ༗ • Umbrella • Compile •

    Monitoring
  5. Our Project

  6. Our project • ϞόΠϧήʔϜͷόοΫΤϯυ • Elixir1.5.1ɺ Phoenix1.3.0 • 2016/06։ൃελʔτɺ2017/06ӡ༻։࢝ •

    ։ൃ౰࣌͸Elixir1.1ɺPhoenix1.1
  7. Architecture

  8. ։ൃ౰࣌ͷ໨తɺؔ৺ • ڠྗ/ରઓήʔϜɺଞϢʔβʔ΁ͷ௨஌ • αʔόʹෳࡶͳϩδοΫΛ࣋ͯΔ͔ • ΫϥΠΞϯτͱͷ઀ଓΛ੾ΒͣʹσϓϩΠͰ͖Δ͔ • ϥΠϒϥϦͷ਺ɺߋ৽ස౓

  9. Elixir Conf 2017Ͱൃද

  10. αʔϏε։࢝ ͦͷޙແࣄϦϦʔε ໿̍೥ؒɺػೳ։ൃΛଓ͚ͨ

  11. Running for 1 year $ grep ‘defmodule’ apps/**/*.ex{,s} | wc

    -l 4083 $ wc -l apps/**/*.ex{,s} | tail -n1 326131 total
  12. Phoenix Productivity http://phoenixframework.org/

  13. Umbrella Project

  14. Many Services • ࠷ॳ͸ҰͭͷPhoenixΞϓϦέʔγϣϯͱͯ͠ελʔτ • ΞϓϦέʔγϣϯ͕੒௕͢ΔʹͭΕͯଟ͘ͷαʔϏεΛ ։ൃ͢Δඞཁ͕ग़͖ͯͨ • ίϯςϯπ؅ཧɺCSɺӡ༻πʔϧɺetc •

    Deploy΋͕͔͔࣌ؒΔ
  15. Umbrella Project

  16. Umbrella Project

  17. Umbrella Project Feature • ෳ਺ͷαʔϏεΛҰͭͷϦϙδτϦͰผʑʹ։ൃ͢Δ • αʔϏεؒͰίʔυΛڞ༗Ͱ͖Δ • αʔϏεຖʹσϓϩΠɺςετ •

    ࠷ॳ͸ϞϊϦεͱͯ͠։ൃΛਐΊɺେ͖͘ͳͬͨΒΞϓ ϦέʔγϣϯΛ෼཭͢Δ
  18. Umbrella Deploy

  19. Distillery Release # rel/config.exs release :test_app_all do set version: “0.0.1"

    set applications: [:test_app_api, :test_app_socket, :test_app_cs, :test_app_cms, :test_app_to ols] … end # release separately release :test_app_api do set version: current_version(:test_app_api) set applications: [:test_app_api] … end
  20. Separate logic

  21. Separate logic def deps do # api_app, mix.exs {:interface_lib_app, inumbrella:

    true} … end
  22. Umbrella or Microservices Repository Interface Deploy Umbrella Single Via subapp

    Both(one app or separately) Microservices Separate Repositories Defined Interface Separately
  23. Elixir Compile

  24. Elixir Compile • Elixir͸macroʹΑͬͯmeta programmingͷػೳΛ࣮ ݱ • macro͸ίϯύΠϧ࣌ʹͷΈධՁ • Compile

    time dependencies͕ੜ·ΕΔͱͦͷϞδϡʔ ϧʹؔ࿈ͨ͠Ϟδϡʔϧ΋ίϯύΠϧର৅ʹͳΔ
  25. Compile Time Dependencies defmodule MacroTest do require MacroA def test,

    do: MacroA.use_macro("hello") end defmodule MacroA do defmacro use_macro(x) do if ModuleA.func() do quote(do: unquote("#{x} macro")) else quote(do: unquote("#{x} dependency")) end end end
  26. Compile Time Dependencies defmodule MacroTest do require MacroA def test,

    do: MacroA.use_macro("hello") end defmodule MacroA do defmacro use_macro(x) do if ModuleA.func() do quote(do: unquote("#{x} macro")) else quote(do: unquote("#{x} dependency")) end end end
  27. Compile Time Dependencies defmodule StructTest do def test do %StructA{field:

    “hello”} end end defmodule StructA do defstruct [:field] end
  28. Compile Time Dependencies defmodule StructTest do def test do %StructA{field:

    “hello”} end end defmodule StructA do defstruct [:field] end
  29. Compile Time Dependencies defmodule Test do import ModuleA end defmodule

    BehaviourTest do @behaviour BehaviourA def test(x), do: x end
  30. Compile Time Dependencies defprotocol ProtocolA do def test(x) end #

    protocol_impl.ex defimpl ProtocolA, for: StructA do def test(_), do: “test” end # protocol_caller.ex defmodule ProtocolCaller do def use_protocol(x), do ProtocolA.test(x) end end
  31. Compile Time Dependencies defprotocol ProtocolA do def test(x) end #

    protocol_impl.ex defimpl ProtocolA, for: StructA do def test(_), do: “test” end # protocol_caller.ex defmodule ProtocolCaller do def use_protocol(x), do ProtocolA.test(x) end end
  32. Mix xref $ mix xref graph —format dot

  33. Mix xref $ mix xref graph —format dot digraph "xref

    graph" { "lib/macro_a.ex" "lib/macro_a.ex" -> “lib/module_a.ex" "lib/macro_test.ex" "lib/macro_test.ex" -> "lib/macro_a.ex" [label="(compile)"] }
  34. Mix xref $ mix xref graph —format dot digraph "xref

    graph" { "lib/macro_a.ex" "lib/macro_a.ex" -> “lib/module_a.ex" "lib/macro_test.ex" "lib/macro_test.ex" -> "lib/macro_a.ex" [label="(compile)"] }
  35. Dirty Trick defmodule MacroTest do def test() do func_map =

    %{key1: Test.Module.Function1, key2: Test.Module.Function2} end end
  36. Dirty Trick defmodule MacroTest do def test() do func_map =

    %{key1: Module.concat([Test, Module, Function1], key2: Module.concat([Test, Module, Function2]} end end defmodule MacroTest do def test() do func_map = %{key1: Test.Module.Function1, key2: Test.Module.Function2} end end
  37. Slow Test

  38. Slow Test • Test࣮ߦ࣌ؒΑΓϩʔυɺίϯύΠϧ͕࣌ؒ஗͍ • 1ϑΝΠϧमਖ਼͚ͨͩ͠Ͱ΋଴ͨ͞ΕΔ • ຊདྷ͸ద੾ͳαΠζͰϓϩδΣΫτΛ෼ׂ͢Δͷ͕ྑ͍ ͕ɺͦ͏Ͱ͖ͳ͍ͱ͖͸IExͷϔϧύʔؔ਺Λ࡞ͬͨ

  39. IEx Helper # .iex.exs defmodule IExHelper do def rerun_spec(file_name) do

    Mix.Task.reenable “espec” … Code.load_file(file_name) |> get_in([Access.at(0), Access.elem(0)]) |> ESpec.SuiteRunner.run(ESpec.Configuration.all) |> ESpec.Output.final_result() end end
  40. Monitoring and Operation

  41. Prometheus and Grafana ओʹ؂ࢹ͍ͯ͠Δ߲໨ ETS table, Output&Received Bytes, Context Switches,

    Run Queues Length, Processes
  42. API Metrics

  43. Example

  44. Example

  45. Prometheus and Grafana Prometheus ClientͷϥΠϒϥϦ΍Erlang VM༻ͷGrafana μογϡϘʔυΛར༻͍ͯ͠Δ

  46. $=proc:<0.13622.1> State: Garbing Name: ‘Elixir.Timex_meck' Spawned as: proc_lib:init_p/5 Spawned by:

    <0.4265.0> Started: Wed May 17 05:15:18 2017 Message queue length: 4 Number of heap fragments: 0 Heap fragment data: 14666 Link list: [<0.4265.0>, {from,<0.16290.3>,#Ref<0.0.5505025.4620>}] Reductions: 79249321 Stack+heap: 164783480 … Remsh remshͰىಈதͷphoenixϓϩηεʹ઀ଓͯ͠ௐࠪͰ͖Δ
  47. $=proc:<0.13622.1> State: Garbing Name: ‘Elixir.Timex_meck' Spawned as: proc_lib:init_p/5 Spawned by:

    <0.4265.0> Started: Wed May 17 05:15:18 2017 Message queue length: 4 Number of heap fragments: 0 Heap fragment data: 14666 Link list: [<0.4265.0>, {from,<0.16290.3>,#Ref<0.0.5505025.4620>}] Reductions: 79249321 Stack+heap: 164783480 … $ iex —name “app_name” —cookie “mycookie” —remsh “app_name-11111@hostname iex()1> Process.whereis(Elixir.Timex_meck) #PID<0.19553.2> iex()2>:erlang.process_info(pid(“0.19553.2”), :total_heap_s ize) {:total_heap_size, 224366925} Remsh remshͰىಈதͷphoenixϓϩηεʹ઀ଓͯ͠ௐࠪͰ͖Δ
  48. Library

  49. Library • total: 6647 (2018/06/13) • ඞཁͳ΋ͷ͸Ұ௨Γἧ͍ͬͯΔ

  50. Library • ͳ͍΋ͷ͸࡞ͬͨɺ΋͘͠͸Ұ෦มߋͯ͠࢖ͬͯΔ • nats pubsub • db sharding •

    cache utility • redis cluster
  51. Conclusion

  52. ·ͱΊ • ΞϓϦέʔγϣϯ͕େ͖͘ͳΔʹͭΕͯੜ࢈ੑ͕Լ͕Δ໰୊͕ग़ͯ ͘Δʢcompile, test, deploy…ʣ • Umbrella͸ϓϩδΣΫτ෼ׂΛ؆୯ʹͰ͖ΔΑ͏ʹαϙʔτ͠ ͯ͘ΕΔ •

    Compileର৅͕ଟ͍৔߹͸ɺແବͳCompile Time DependenciesΛ࡟আ͢Δ • Ϣʔβʔͷ઀ଓ؅ཧͱϩδοΫΛॲཧ͢ΔαʔόΛ෼͚ͨ
  53. ·ͱΊ • LibraryɺFWͳͲͷΤίγεςϜʹ͍ͭͯ • ։ൃɺӡ༻Λͯ͠ΈͯLibrary͕গͳ͗͢Δ໰୊͸ಛʹͳ͔ͬͨ • ඞཁͳ࣌ʹ࡞Ε͹ྑ͍(ύοέʔδϚωʔδϟʔɺπʔϧ͕༏ल) • ElixirɺPhoenixͷόʔδϣϯΞοϓ͸׆ൃ →ɹؤுͬͯ௥ै

    • (Type specΛॻ͍͓ͯ͘ͱυΩϡϝϯτʹͳΔͷͰίʔυ͕ಡΈ΍͢ ͍)
  54. Thank you