The Architecture Behind Deploying Livebook Apps with Livebook Teams

Slide 1

Slide 1 text

The Architecture Behind Deploying Livebook Apps with Livebook Teams By Hugo Baraúna (@hugobarauna)

Slide 2

Slide 2 text

the story of a button click

Slide 3

Slide 3 text

No content

Slide 4

Slide 4 text

Apps

Slide 5

Slide 5 text

A Livebook app is a way to run a Livebook notebook as an application

Slide 6

Slide 6 text

Demo deploying a Livebok app

Slide 7

Slide 7 text

Hugo Baraúna Dev advocate @ Livebook 🥑 Elixir Radar newsletter 📩 Elixir Patterns book with @akoutmos 📕 From Brazil 🇧🇷

Slide 8

Slide 8 text

Architecture Overview

Slide 9

Slide 9 text

Livebook (e.g. localhost) Livebook app server (your own infrastructure ) Livebook Teams (SaaS)

Slide 10

Slide 10 text

Livebook (e.g. localhost) Livebook app server (your own infrastructure ) Livebook Teams (SaaS)

Slide 11

Slide 11 text

Livebook (e.g. localhost) Livebook app server (your own infrastructure ) Livebook Teams (SaaS)

Slide 12

Slide 12 text

Communication architecture

Slide 13

Slide 13 text

Livebook Teams Livebook HTTP REST API (for sync comms) protobuf messages WebSocket (for async comms) internet

Slide 14

Slide 14 text

Application Teams.Connection TeamClient connect_hub start_link :gen_statem.start_link UserSocket WebSocket.connect connection established protobuf message (Connected) start Livebook (app server) Livebook Teams

Slide 15

Slide 15 text

Show me the code establishing the websocket connection

Slide 16

Slide 16 text

Real-time updates in a geo-distributed system

Slide 17

Slide 17 text

Demo realtime updates

Slide 18

Slide 18 text

Livebook Livebook Teams Teams.Connection UserSocket Phoenix.PubSub LiveView Phoenix.PubSub LiveView subscribe(“org:1”) WebSocket.connect(“org:1”) connection established subscribe(“org:1”) broadcast( “org:1”, %LivebookProto.Event{} ) save env var send({ %LivebookProto.Event{} }) push(<<10, 17, 10, …>>) send broadcast handle_info

Slide 19

Slide 19 text

Show me the code realtime updates

Slide 20

Slide 20 text

Deployment with Livebook Teams

Slide 21

Slide 21 text

Livebook App Server Livebook Livebook Teams deploy app HTTP POST start app deployment upload encrypted zipped app websocket noti fi cation: app deployment started download app presigned URL of the app GET app object app object run the app record app deployment zip and encrypt app deployment Decrypt app deployment

Slide 22

Slide 22 text

Slide 23

Slide 23 text

TeamClient Apps.Manager Apps.Deployer Apps.AppSpec Sessions PubSub deploy(app_spec) sync_apps list_deployers deployers_pids deploy(pid, app_spec) load(app_spec) notebook App create_session send(:deploy_resullt) start(notebook) broadcast(:apps_manager_status) send(:apps_manager_status) send_message(%AppDeploymentStatusReport{})

Slide 24

Slide 24 text

Apps.Manager Apps.Deployer list_deployers deployers_pids deploy(pid, app_spec) TeamClient Apps.AppSpec Sessions PubSub deploy(app_spec) sync_apps load(app_spec) notebook App create_session send(:deploy_resullt) start(notebook) broadcast(:apps_manager_status) send(:apps_manager_status) send_message(%AppDeploymentStatusReport{})

Slide 25

Slide 25 text

Automatic redeployment when app server node goes down

Slide 26

Slide 26 text

Livebook app server (your own infrastructure ) Livebook Teams (SaaS) internet internet internet local network WebSocket Livebook app server (your own infrastructure ) Erlang distributed WebSocket

Slide 27

Slide 27 text

Demo Automatic redeploy when node goes down

Slide 28

Slide 28 text

Next steps

Slide 29

Slide 29 text

Using Livebook to build and deploy internal tools https://youtu.be/VIbVKO9CW5E

Slide 30

Slide 30 text

livebook-dev/livebook Code diving @ WebSocket client using gen_statem lib/livebook/teams/connection.ex Protobuf message de fi nitions proto/* Livebook app deployment orchestration lib/livebook/apps/manager.ex lib/livebook/apps/deployer.ex

Slide 31

Slide 31 text

demos source code https://github.com/hugobarauna/livebook-notebooks/tree/main/talks/alchemyconf

Slide 32

Slide 32 text

thank you / obrigado livebook-dev/livebook livebook.dev @hugobarauna.com @hugobarauna