Connection Proxying with Phoenix

In this presentation, I explain the need for a Service Mesh type solution to go with Elixir Umbrella projects, analyse existing approaches and present a new, consolidated way to achieve the same goals with much less cognitive overhead.

Presented at Code Elixir LDN 2019 as a Lightning Talk, with further revisions.

Transcript

1. Connection Proxying with Phoenix Evadne Wu Faria Education Group revised 18 July 2019
2. About Me Application Developer Inquiries: ev@radi.ws Arguments: twitter.com/evadne
3. 1 Background
4. Motive We have created various Elixir applications and have found it useful to organise code in Umbrella Projects.
5. Umbrellas: The Good Parts Clear and concise path to deployment. Easy management of inter-dependent contexts. Reduced cognitive load on the developer. Reduced inter-app communication load on the service.
6. Umbrellas: The Not-So-Good Parts Default Umbrella layout encourages proliferation of listeners and ports, which beget configuration churn. Various workarounds exist, but are either tedious to use, or they subtly break Phoenix.
7. What We Want One Umbrella Application One Release One Container Image One Exposed Port No Compromises
8. 2 Existing Approaches
9. Default Umbrella Layout App 1 App 2 App 3 Umbrella Nginx Backend 3 Backend 2 Backend 1
10. Desired Umbrella Layout Proxy App 1 App 2 App 3 Umbrella
11. Approach 1: Custom Plug Check conn.request_path and call the appropriate Endpoint? 1. Does not work with Phoenix Sockets 2. Forwarded path not part of conn.script_name 3. General chaos with URL helpers
12. Approach 2: Custom Cowboy Handler Instead of calling Endpoints in a Plug, call them in a Cowboy Handler? 1. Works with Sockets (lots of custom code, though) 2. Forwarded path not part of conn.script_name 3. General chaos with URL helpers
13. Our Conclusion The correct solution should embrace existing primitives as much as possible, without slowing us down or compromising vital functionalities. Phoenix Problems require Phoenix Solutions.
14. 3 Solution Design
15. Endpoint Supervision Tree Config PubSub Watcher Endpoint Socket Server
16. Socket Socket Proxy App 1 App 2 App 3 Endpoint Router Endpoint Endpoint __sockets__ Endpoint __sockets__ Router Router Router Socket __sockets__
17. Socket Socket Proxy App 1 App 2 App 3 Endpoint Router Endpoint Endpoint Mount Point __sockets__ __sockets__ Endpoint __sockets__ Mount Point Router Router Router Socket Mount Point /admin /auth /
18. Socket Socket Proxy App 1 App 2 App 3 Endpoint Router Endpoint Endpoint __sockets__ Endpoint __sockets__ Router Router Router Socket Phoenix.Router.forward/4 (macro) __sockets__ /admin /auth / Mount Point Mount Point Mount Point
19. Socket Socket Proxy App 1 App 2 App 3 Endpoint Router Endpoint Endpoint __sockets__ Endpoint __sockets__ Router Router Router Socket Phoenix.Router.forward/4 (macro) Phoenix.Endpoint.socket/3 (macro) __sockets__ /admin /auth / Mount Point Mount Point Mount Point
20. Socket Socket Proxy App 1 App 2 App 3 Endpoint Router Endpoint Endpoint __sockets__ Endpoint __sockets__ Router Router Router Socket Phoenix.Router.forward/4 (macro) Phoenix.Endpoint.socket/3 (macro) __sockets__ /admin /auth / Mount Point Mount Point Mount Point
21. 4 Solution Implementation
22. Callee: Endpoint Configuration Run server (server: true), but don’t specify the port bit for dev or prod configurations. This ensures that Endpoints are started, but the Cowboy listeners are not. Keep the port specified for test configuration. ExUnit tests are run sequentially for each app in the Umbrella.
23. Callee: Endpoint Each Callee Endpoint should also know its Mount Point. It is best to set this via :url configuration (with :path) which simplifies downstream code.
24. Callee: config/config.exs config :my_api, MyAPI.Endpoint, url: [path: "/api"]
25. Callee: config/test.exs config :my_api, MyAPI.Endpoint, url: [path: "/"]
26. Proxy Construction The Proxy application should contain: 1. Root Module: To hold helper functions. 2. Within Proxy Endpoint: Walk the list of Endpoints and Mount Points; build sockets with socket. 3. Within Proxy Router: Walk the list of Endpoints and Mount Points; build routes with forward.
27. Proxy Config config :my_proxy, MyProxy, endpoints: [ my_admin: MyAdmin.Endpoint, my_api: MyAPI.Endpoint, my_web: MyWeb.Endpoint ] # port can be specified here or dynamically specified in # init/2 callback for Proxy Endpoint. Enclosing OTP app
28. Proxy Root def get_endpoints do Application.get_env(@otp_app, __MODULE__) |> Keyword.get(:endpoints, []) |> Enum.map(&{elem(&1, 1), get_mount(&1)}) end defp get_mount({app, module}) do Application.get_env(app, module) |> get_in(~w(url path)a) || "/" end
29. Proxy Endpoint @external_resource "config/config.exs" for {endpoint, mount} <- MyProxy.get_endpoints() do sockets = endpoint.__sockets__() for {path, socket_module, options} <- sockets do socket_path = Path.join([mount, path]) socket(socket_path, socket_module, options) end end
30. Proxy Router @external_resource "config/config.exs" for {endpoint, mount} <- MyProxy.get_endpoints() do forward mount, endpoint end
31. 5 Conclusion
32. Assets 1. Each application retains its own assets and Webpack pipeline. No need to have all applications use the same version of a particular NPM package (which, being JS, could be problematic in some cases). 2. All digested assets work (since Endpoints continue to run). 3. Watchers work (see next section).
33. Code Reloading: Further Changes 1. The two plugs — Phoenix.LiveReloader and Phoenix.CodeReloader — need to be re-hooked in the Proxy Endpoint. 2. The socket — Phoenix.LiveReloader.Socket — also needs re-hooking.
34. Code Reloading: Proxy Endpoint if code_reloading? do plug Phoenix.LiveReloader plug Phoenix.CodeReloader socket_path = "/phoenix/live_reload/socket" socket_module = Phoenix.LiveReloader.Socket case List.keyfind(@phoenix_sockets, socket_path, 0) do {_, ^socket_module, _} -> nil nil -> socket(socket_path, socket_module) end end
35. Code Reloading: Proxy Endpoint Explanation: If there is a “Web” application that acts as the fallback / catch-all target for all traffic, and it has already defined Phoenix.LiveReloader.Socket, then it would already have been defined in the Proxy Endpoint and does not need to be defined again.
36. Deployment 1. Deployment process is unchanged. 2. Only one port (which is held by the Proxy Endpoint) needs to be exposed. 3. No complex topology required to run multiple containers.
37. Adding Another App 1. Add said Application in the Umbrella. 2. Configure said Application with a unique Mount Point. 3. Change Proxy configuration; add one entry to refer to the newly added Application. 4. No further infrastructure change required.
38. Results One Umbrella Application One Release One Container Image One Exposed Port Few Compromises† † __sockets__ is private
39. Thank You