Connection Proxying with Phoenix

In this presentation, I explain the need for a Service Mesh type solution to go with Elixir Umbrella projects, analyse existing approaches and present a new, consolidated way to achieve the same goals with much less cognitive overhead.

Presented at Code Elixir LDN 2019 as a Lightning Talk, with further revisions.

Transcript

1. Connection Proxying with Phoenix
   Evadne Wu
   Faria Education Group
   revised
   
   18 July 2019
   

   View Slide

2. About Me
   Application Developer
   Inquiries: ev@radi.ws
   Arguments: twitter.com/evadne
   

   View Slide

3. 1
   Background
   

   View Slide

4. Motive
   We have created various Elixir applications and have
   found it useful to organise code in Umbrella Projects.
   

   View Slide

5. Umbrellas: The Good Parts
   Clear and concise path to deployment.
   Easy management of inter-dependent contexts.
   Reduced cognitive load on the developer.
   Reduced inter-app communication load on the service.
   

   View Slide

6. Umbrellas: The Not-So-Good Parts
   Default Umbrella layout encourages proliferation of
   listeners and ports, which beget configuration churn.
   Various workarounds exist, but are either tedious to use,
   or they subtly break Phoenix.
   

   View Slide

7. What We Want
   One Umbrella Application
   One Release
   One Container Image
   One Exposed Port
   No Compromises
   

   View Slide

8. 2
   Existing Approaches
   

   View Slide

9. Default Umbrella Layout
   App 1 App 2 App 3
   Umbrella
   Nginx
   Backend 3
   Backend 2
   Backend 1
   

   View Slide

10. Desired Umbrella Layout
    Proxy App 1 App 2 App 3
    Umbrella
    

    View Slide

11. Approach 1: Custom Plug
    Check conn.request_path and call the appropriate
    Endpoint?
    1. Does not work with Phoenix Sockets
    2. Forwarded path not part of conn.script_name
    3. General chaos with URL helpers
    

    View Slide

12. Approach 2: Custom Cowboy Handler
    Instead of calling Endpoints in a Plug, call them in a
    Cowboy Handler?
    1. Works with Sockets (lots of custom code, though)
    2. Forwarded path not part of conn.script_name
    3. General chaos with URL helpers
    

    View Slide

13. Our Conclusion
    The correct solution should embrace existing primitives
    as much as possible, without slowing us down or
    compromising vital functionalities.
    Phoenix Problems require Phoenix Solutions.
    

    View Slide

14. 3
    Solution Design
    

    View Slide

15. Endpoint Supervision Tree
    Config
    PubSub
    Watcher
    Endpoint
    Socket
    Server
    

    View Slide

16. Socket Socket
    Proxy App 1 App 2 App 3
    Endpoint
    Router
    Endpoint Endpoint
    __sockets__
    Endpoint
    __sockets__
    Router Router Router
    Socket
    __sockets__
    

    View Slide

17. Socket Socket
    Proxy App 1 App 2 App 3
    Endpoint
    Router
    Endpoint Endpoint
    Mount Point
    __sockets__ __sockets__
    Endpoint
    __sockets__
    Mount Point
    Router Router Router
    Socket
    Mount Point
    /admin /auth /
    

    View Slide

18. Socket Socket
    Proxy App 1 App 2 App 3
    Endpoint
    Router
    Endpoint Endpoint
    __sockets__
    Endpoint
    __sockets__
    Router Router Router
    Socket
    Phoenix.Router.forward/4 (macro)
    __sockets__
    /admin /auth /
    Mount Point Mount Point
    Mount Point
    

    View Slide

19. Socket Socket
    Proxy App 1 App 2 App 3
    Endpoint
    Router
    Endpoint Endpoint
    __sockets__
    Endpoint
    __sockets__
    Router Router Router
    Socket
    Phoenix.Router.forward/4 (macro)
    Phoenix.Endpoint.socket/3 (macro)
    __sockets__
    /admin /auth /
    Mount Point Mount Point
    Mount Point
    

    View Slide

20. Socket Socket
    Proxy App 1 App 2 App 3
    Endpoint
    Router
    Endpoint Endpoint
    __sockets__
    Endpoint
    __sockets__
    Router Router Router
    Socket
    Phoenix.Router.forward/4 (macro)
    Phoenix.Endpoint.socket/3 (macro)
    __sockets__
    /admin /auth /
    Mount Point Mount Point
    Mount Point
    

    View Slide

21. 4
    Solution Implementation
    

    View Slide

22. Callee: Endpoint Configuration
    Run server (server: true), but don’t specify the port
    bit for dev or prod configurations.
    This ensures that Endpoints are started, but the Cowboy
    listeners are not.
    Keep the port specified for test configuration. ExUnit
    tests are run sequentially for each app in the Umbrella.
    

    View Slide

23. Callee: Endpoint
    Each Callee Endpoint should also know its Mount Point.
    It is best to set this via :url configuration
    (with :path) which simplifies downstream code.
    

    View Slide

24. Callee: config/config.exs
    config :my_api, MyAPI.Endpoint,
    url: [path: "/api"]
    

    View Slide

25. Callee: config/test.exs
    config :my_api, MyAPI.Endpoint,
    url: [path: "/"]
    

    View Slide

26. Proxy Construction
    The Proxy application should contain:
    1. Root Module: To hold helper functions.
    2. Within Proxy Endpoint: Walk the list of Endpoints and
    Mount Points; build sockets with socket.
    3. Within Proxy Router: Walk the list of Endpoints and
    Mount Points; build routes with forward.
    

    View Slide

27. Proxy Config
    config :my_proxy, MyProxy,
    endpoints: [
    my_admin: MyAdmin.Endpoint,
    my_api: MyAPI.Endpoint,
    my_web: MyWeb.Endpoint
    ]
    # port can be specified here or dynamically specified in
    # init/2 callback for Proxy Endpoint.
    Enclosing
    OTP app
    

    View Slide

28. Proxy Root
    def get_endpoints do
    Application.get_env(@otp_app, __MODULE__)
    |> Keyword.get(:endpoints, [])
    |> Enum.map(&{elem(&1, 1), get_mount(&1)})
    end
    defp get_mount({app, module}) do
    Application.get_env(app, module)
    |> get_in(~w(url path)a) || "/"
    end
    

    View Slide

29. Proxy Endpoint
    @external_resource "config/config.exs"
    for {endpoint, mount} <- MyProxy.get_endpoints() do
    sockets = endpoint.__sockets__()
    for {path, socket_module, options} <- sockets do
    socket_path = Path.join([mount, path])
    socket(socket_path, socket_module, options)
    end
    end
    

    View Slide

30. Proxy Router
    @external_resource "config/config.exs"
    for {endpoint, mount} <- MyProxy.get_endpoints() do
    forward mount, endpoint
    end
    

    View Slide

31. 5
    Conclusion
    

    View Slide

32. Assets
    1. Each application retains its own assets and Webpack
    pipeline. No need to have all applications use the same
    version of a particular NPM package (which, being JS,
    could be problematic in some cases).
    2. All digested assets work (since Endpoints continue to run).
    3. Watchers work (see next section).
    

    View Slide

33. Code Reloading: Further Changes
    1. The two plugs — Phoenix.LiveReloader and
    Phoenix.CodeReloader — need to be re-hooked
    in the Proxy Endpoint.
    2. The socket — Phoenix.LiveReloader.Socket
    — also needs re-hooking.
    

    View Slide

34. Code Reloading: Proxy Endpoint
    if code_reloading? do
    plug Phoenix.LiveReloader
    plug Phoenix.CodeReloader
    socket_path = "/phoenix/live_reload/socket"
    socket_module = Phoenix.LiveReloader.Socket
    case List.keyfind(@phoenix_sockets, socket_path, 0) do
    {_, ^socket_module, _} -> nil
    nil -> socket(socket_path, socket_module)
    end
    end
    

    View Slide

35. Code Reloading: Proxy Endpoint
    Explanation: If there is a “Web” application that acts as
    the fallback / catch-all target for all traffic, and it has
    already defined Phoenix.LiveReloader.Socket,
    then it would already have been defined in the Proxy
    Endpoint and does not need to be defined again.
    

    View Slide

36. Deployment
    1. Deployment process is unchanged.
    2. Only one port (which is held by the Proxy Endpoint)
    needs to be exposed.
    3. No complex topology required to run multiple
    containers.
    

    View Slide

37. Adding Another App
    1. Add said Application in the Umbrella.
    2. Configure said Application with a unique Mount Point.
    3. Change Proxy configuration; add one entry to refer to
    the newly added Application.
    4. No further infrastructure change required.
    

    View Slide

38. Results
    One Umbrella Application
    One Release
    One Container Image
    One Exposed Port
    Few Compromises†
    † __sockets__ is private
    

    View Slide

39. Thank You
    

    View Slide