What our system needed to do • Process and send orders to a Point of Sale system in a store. • Throttle the volume of orders to a store. • Have resilience against stores being down & having inconsistent internet access.
1. Phoenix is not your application. 2. Embrace state outside of the database. 3. If it’s concurrent, extract it into an OTP application. 4. (Don’t just) Let it crash. Elixir Design principles
Order Scheduler Store Availability Web Order Tracker Store Store Store Supervisor Store Store Store Supervisor Stores Supervisor Store Sup Store Sup API Web worker Web worker Web worker
Queues orders to send to store, delivering orders 15 minutes before the timeslot. Must handle stores being down / delayed without impacting other stores. PURPOSE
Store #1 Store #2 Order Scheduler Store Supervisor Store Manager Delivery Supervisor Delivery Worker Delivery Worker Delivery Worker Store Supervisor Store Manager Delivery Supervisor Delivery Worker Delivery Worker Delivery Worker
def handle_info(:process, state) do # Do work state = do_process_orders(state) # Call ourselves again Process.send_after(self, :process, @send_time_ms) {:noreply, state} end
Order Scheduler Store Availability Web Order Tracker Store Store Store Supervisor Store Store Store Supervisor Stores Supervisor Store Sup Store Sup API Web worker Web worker Web worker
Store Supervisor Store Manager Order Tracker Task Supervisor Store Supervisor Store Manager Task Supervisor Delivery Worker Delivery Worker Order Update Delivery Worker Delivery Worker Order Update
def process_feed_items(store_id, events) do events |> Enum.map(&start_tracking_worker(store_id, &1)) end def start_tracking_worker(store_id, event) do Task.Supervisor.start_child(worker_sup_ref(store_id), fn -> # Process the XML, update db state etc OrderEventProcessor.process(store_id, event) end) end
Order Scheduler Store Availability Web Order Tracker Store Store Store Supervisor Store Store Store Supervisor Stores Supervisor Store Sup Store Sup API Web worker Web worker Web worker
Store Availability Store Supervisor ETS Table Store Manager ETS Table Manager Store Supervisor ETS Table Store Manager ETS Table Manager Store Supervisor ETS Table Store Manager ETS Table Manager
def handle_call({:confirm, datetime, id}, table) do # Update the counter for the timeslot in ETS :ets.update_counter( table, datetime, [{2,0},{3,0},{4,1}] ) {:reply, :ok, table} end
defp start_availability_manager(%{id: store_id} = store) do # Setup the manager with the initial state from database availability = StoreAvailabilityGenerator.get_matrix(store) # Start the store manager with the state # This is really just a call to `Supervisor.start_child` StoreAvailability.start_store_manager(store_id, availability) end
# Number of concurrent sessions in pool :ibrowse.set_max_sessions(host, port, @max_sessions) # Queue size per session :ibrowse.set_max_pipeline_size(host, port, @max_pipeline)