Zenohex - an eloquent, scalable and fast communication library for Elixir

Code BEAM America 2024
2024/03/07
https://codebeamamerica.com/
https://codebeamamerica.com/talks/zenohex/

Speakers:
Shintaro Hosoai, Hideki Takase and Mitsuhiro Osaki

Abstract:
How do you handle communication in complex networks or when communicating with other languages and IoT devices? Elixir offers various Pub/Sub communication libraries, but network configurations may limit their usability, and some require a broker server.
Zenohex is a high-performance Pub/Sub communication library based on Zenoh, implemented in Rust. Zenoh has broker-based and broker-less Pub/Sub communications, akin to MQTT and DDS, respectively, but with unique features that enable communication across languages, platforms, and devices. Since Zenohex is realized by using Rustler, this talk will also cover how to integrate Rust modules in your Elixir library. Note that our library also provides Rustler-Precompiled binaries, making it easy to use without a Rust.
Github: https://github.com/b5g-ex/zenohex, Hex.pm: https://hex.pm/packages/zenohex

Transcript

1. Zenohex an eloquent, scalable and fast communication library for Elixir 2024/03/07 Speakers: Shintaro Hosoai (Univ. of Tokyo) Hideki Takase (Univ. of Tokyo) Mitsuhiro Osaki (CityNet Inc.)
2. 2 Who we are?? @ShintaroHosoai Project Researcher The University of Tokyo @takasehideki Associate Professor The University of Tokyo OHSAKISAN a.k.a @rigutter Technical Supervisor CityNet Inc. aims to establish a cutting-edge development methodology for IoT systems with 5G and Beyond 5G network supported by National Institute of Information and Communications Technology (NICT), Japan
3. 3 Agenda • Whatʼs the issues? —Introduction of Giocci —Wide-area Distributed? • Introduction of Zenoh —Basic Features and Protocol —Communication Architecture • Zenohex —How to Realize? Integration of Rust —Demonstration!!
4. 4 Agenda • Whatʼs the issues? —Introduction of Giocci —Wide-area Distributed? • Introduction of Zenoh —Basic Features and Protocol —Communication Architecture • Zenohex —How to Realize? Integration of Rust —Demonstration!!
5. 5 Whatʼs the issues? • More and more system configurations Local Edge Client Develop Edge-Client, Server, MEC, Cloud ・Spec ・OS ・Language ・Cloud-Service Configuration Connected Network Local, Mobile, WAN, Leased circuit Communication Protocol Serial, http(s), MQTT, ROS, WebSocket REST API, gRPC, FTP, SMTP ・・and more Cloud Server Leased circuit Edge Client WAN Cloud Mobile Cloud MEC MEC Server
6. 6 Whatʼs the issues? A certain IoT configuration 1. Device information (such as images) is acquired at the Edge → Transmitting and receiving information between Edges as needed 2. Image analysis processing (modules) is executed on any Edge, MEC, or in the Cloud 3. Execution results are returned from the any Cloud or MEC to the Edge Cloud MEC Edge Edge Server Devices ② mod ule mod ule mod ule mod ule mod ule mod ule Cloud Server mod ule mod ule ① ③ Local5G WAN What we want ①Simplify the placement of modules ②Transparently select and use the processing environment based on specific conditions. ③Send the execution results to any Edge
7. 7 Introduction of Giocci • Resource Transparent Distributed Processing Framework Edge Node p-contact module Fa-al/device name server Engine (persists data) Engine Store Client Relay module module module v-contact on Cloud on exMec Relay Server Relay Server
8. 8 Wide-area Distributed? • Erlang cluster Node.connect specific issues Node Node Node Node Node Node Node Node Node Router Router Node Node NAPT Node.connect() true Node.connect() false 1.Full-Mesh Network 2.Cannot go beyond NAPT
9. 9 Agenda • Whatʼs the issues? —Introduction of Giocci —Wide-area Distributed? • Introduction of Zenoh —Basic Features and Protocol —Communication Architecture • Zenohex —How to Realize? Integration of Rust —Demonstration!!
10. 10 Publish/Subscribe Messaging • Brokered (e.g., MQTT) — need to know where is broker • Peer-to-Peer (e.g., DDS) — autonomous search for partners — typically limited on the same NW • Benefits — Easy to construct asynchronous and loosely coupled architecture — nodes can be added/deleted/ restarted independently publisher1 publisher2 topicA topicB subscriber1 sub2 sub3 broker
11. • Zero Overhead Pub/Sub, Store/Query and Compute. — Zero network overhead protocol — DDS-like communication within a network and MQTT-like communication between networks • Dev leader: ZettaScale Technology Ltd. — GitHub: https://github.com/eclipse-zenoh/ üOne of the Eclipse Project üEclipse Public License 2.0 and/or Apache 2.0 — Initially implemented in OCaml, and then migrated to Rust in Oct 2020 11 What is??
12. 12 What is?? Scalable decentralized and routing Eloquent pub/sub, storages, query, and queryables Fast lightweight and easy to use
13. 13 • Pub / Sub (Push) — basic pub/sub method • Pub / Sub (Pull) — Sub receives in its own timing • Pub / Store / Get — KVS based computation • Get / Reply — RPC-like communication node node node node node node node node Pub Sub(callback) Sub(Pull) Pub Pub Get Reply Pull Eloquent
14. 14 Peer to Peer Clique Mesh Brokered Routed Scalable Peer Peer Peer Peer Peer Peer Peer Peer Peer client client Router Router Router client client client client client client • Zenoh router behaves as a broker between networks • We can construct a wide area network structure easily! Router Router
15. • Low latency and High throughput — 10 us latency in the single machine, 16 us in multiple machines (P2P config.) — ~70 Gbps at 8 KB payload ü35x higher than MQTT, 23x than Kafka, 3.3x than DDS • Why?: minimum wire overhead — only 5 bytes for delivering messages 15 https://zenoh.io/blog/2023-03- 21-zenoh-vs-mqtt-kafka-dds/ arxiv:2303.09419 Fast
16. 16 QUIC, TLS, TCP, UDP Unicast, UDP Multicast IPv4, IPv6 6LoWPAN WiFi, Ethernet, Bluetooth, Serial APIs for various languages • zenoh-python • zenoh-c • zenoh-cpp • zenoh-java https://zenoh.io/docs/ getting-started/first-app/ Runs Everywhere! • zenoh-kotlin • zenoh-csharp • zenoh-go Getting Started with Python Native
17. Why isn’t there Elixir??
18. 18 Agenda • Whatʼs the issues? —Introduction of Giocci —Wide-area Distributed? • Introduction of Zenoh —Basic Features and Protocol —Communication Architecture • Zenohex —How to Realize? Integration of Rust —Demonstration!!
19. 19 Zenohex • Why Zenohex? —We find network library that over the network. —I like Zenoh and Elixir • How to Implement︖ —Zenoh written by Rust —Use Rustler
20. 20 Rustler • Easy to binding Rust and Elixir • Generate boilerplate project • Integrated cargo and mix build • Rustler-precompiled
21. 21 Zenohex Software Structure Zenoh library Zenohex_nif Zenohex API （Similar to Zenoh API） Zenohex.Nif Rustler Zenohex advanced API （Elixir like library）
22. 22 How to use Zenohex • add {:zenohex, "~> 0.2.0"} to mix.exs • defmodule ZenohElixir.Pub do def main do {:ok, session} = Zenohex.open() {:ok, publisher} = Zenohex.Session.declare_publisher( session, "key/expression") spawn(ZenohElixir.Pub, :publish, [publisher, 0]) end def publish(publisher, num) do msg = "Hello from Elixir!! " <> to_string(num) IO.puts "[pub.ex] " <> msg Zenohex.Publisher.put(publisher, msg) Process.sleep(1000) publish(publisher, num + 1) end end defmodule ZenohElixir.Sub do def main do {:ok, session} = Zenohex.open() {:ok, subscriber} = Zenohex.Session.declare_subscriber( session, "key/expression") spawn(ZenohElixir.Sub, :subscribe, [subscriber]) end def subscribe(subscriber) do case Zenohex.Subscriber.recv_timeout(subscriber) do {:error, :timeout} -> nil {:ok, msg} -> IO.puts "[sub.ex] " <> msg.value end subscribe(subscriber) end end Publisher Subscriber
23. Python Zenoh Subscri ber 23 論よりRUN!! "ron yori run" The RUN is mightier than the word DEMO1: over the Language ①Zenoh example (Rust → Rust) ②Zenoh to Zenohex (Rust → Elixir) ③Zenohex to Zenoh (Elixir → Elixir/Rust/python) Elixir Zenohex Subscriber Elixir Zenohex Publisher Rust Zenoh Subscriber Rust Zenoh Publisher ① ② ③ ③
24. 24 論よりRUN!! "ron yori run" The RUN is mightier than the word DEMO2: over the network Elixir Zenohex Subscriber Elixir Zenohex Publisher Zenoh router (zenohd) Phoenix + Zenohex application Zenoh router (zenohd) :7447/tcp :4000/tcp ①Local ← Global ②Local → Global Amazon EC2 this PC
25. 25 Thank you! merci!! Arigato!!! A part of this work is supported by the commissioned research (04001) by National Institute of Information and Communications Technology (NICT), Japan.