Zenoh Tutorial

This presentation introduces the key ideas behind zenoh -- an Internet scale data-centric protocol that unifies data-sharing between any kind of device including those constrained with respect to the node resources, such as computational resources and power, as well as the network.

Transcript

1. CEO/CTO ZettaScale Angelo Corsaro, PhD Zenoh Tutorial Part I
2. The Problem
3. Cloud to Microcontroller An increasing number of systems today span from the cloud down to the micro-controller This poses new challenges as existing technologies were not designed to work in this scenario And the result is… Cloud Edge Device
4. Simplicity does not precede complexity, but follows it — A. Perils The Digital Frankenstein Today, building systems that span from the micro-controller to the data-centre feels like assembling a digital Frankenstein Multiple technologies have to be stitched together only to make data fl ow end-to- end Few more have to be packed-up to deal with data storage… Not to talk about computations
5. A Cross Industry Problem Robotics Automotive IoT / Edge
6. The Solution
7. Dragons teach us that if we want to climb high we have to do it against the wind. Pub/Sub/Query protocol that Uni fi es data in motion, data at rest and computations from embedded microcontrollers up the data centre Provides location-transparent abstractions for high performance pub/sub and distributed queries across heterogeneous systems Built-in support for zero-copy and shared memory
8. Runs Everywhere Written in Rust for security, safety and performance Native libraries and API bindings for many programming languages, e.g., Rust, C/C++, Python, JS, REST, C# and Kotlin Built-in support Shared Memory and Zero Copy Supports network technologies from transport layer down- to the data link. Currently runs on, TCP/IP, UDP/IP, QUIC, Serial, Bluetooth, OpenThreadX, Unix Sockets, Shared Memory Available on embedded and extremely constrained devices and networks — 5 bytes minimal overhead Data Link Network Transport Physical …
9. Runs Everywhere OS Linux, MacOS, Windows, QNX (alpha) Embedded Targets Arduino, ESP32, mbed, Zephyr Automotive Targets AUTOSAR Classic (microSAR)
10. Router Router Any Topology Peer-to-peer Clique and mesh topologies Brokered Routed Router Router Router Client Client Routed Routers forward data to and from peers and clients Brokered Clients communicate through a router or a peer Mesh Peer Peer Peer Peer Peer Router Client Client Client Client Router Router Client Clique Peer Peer Peer Peer Client
11. Topology in Perspective
12. Router Storage Plugins Protocol Plugins MAIN MEMORY FILE SYSTEM Runtime Plugins Zenoh Flow RocksDB Router Plug-Ins
13. Adoption
14. Robotics
15. Robotics Zenoh has witnessed a strong adoption in Robotics, especially in supporting R2X over wireless networks Along with adoption has grown the number of commercially deployed solutions that build Zenoh
16. Zenoh: The RMW Alternate Intrinsic is already implementing Zenoh RMW as the major contribution for next ROS2 release The full report is available here Zenoh has been selected as the fi rst non-DDS protocol to be natively supported in ROS2
17. ROS User’s Preferred Choice A poll on the ROS2 community revealed that Zenoh was the user’s favourite choice as next generation ROS2 protocol
18. Automotive
19. Zenoh identi fi ed as the most appropriate protocol for V2X applications for autonomous and assisted driving (Final report attached to the email) ITU
20. At the Forefront of Innovation ZettaScale has been identi fi ed as one of the 10 startups in vehicle communication to be watched in 2023
21. At the Forefront of Innovation ZettaScale has been identi fi ed as one of the 20 startups in vehicle communication to be watched in 2024
22. Eclipse SDV Zenoh selected as one of the communication middleware in addition to MQTT
23. uProtocol Eclipse uProtocol selected Zenoh as the fi rst protocol to be integrated
24. GM GM selected Zenoh as the protocol for uProtocol
25. Other Verticals
26. Running on Zenoh Internet scale Analytics platforms Blockchain Industrial Automation Platforms …
27. In a recent article that evaluates mainstream protocols, including MQTT, DDS, AMQP an CoAP Amazon describes Zenoh as: “Perhaps the most viable protocol that will help us mature to this ideal model is Zenoh which is an Eclipse Incubation project” Amazon IoT
28. Programming Model
29. Abstractions Resource. A named data, in other terms a (key, value) (e.g. home/kitchen/sensor/temp, 21.5 home/kitchen/sensor/hum, 0.67) (e.g. home/*/sensor/air?_where=co2>12&_project=humidity) Selector. An expression identifying a set of resources (e.g. home/kitchen/sensor/* home/**/temp Key expression. An expression identifying a set of keys
30. Abstractions Publisher. A spring of values for a key expression Subscriber. A sink of values for a key expression Queryable. A well of values for a key expression (e.g. home/kitchen/sensor/temp home/kitchen/sensor/* ) (e.g. home/kitchen/sensor/temp home/kitchen/sensor/*) (e.g. home/**)
31. Primitives open/close — Open/Close a zenoh session. declare_keyexpr — Declares a key-expressions and optimises its representation on the wire declare_subscriber — Declares a subscriber with a user provided call-back that will be triggered when data is available. declare_publisher — Declares a publisher and optimise the communication stack for repetitive publications. Notice that Zenoh does not require a publisher in order to perform publications, this is just an optimisation. declare_queryable — Declares a queryable with a user provided call-back that will be triggered whenever a query needs to be answered.
32. Primitives put — puts a value for a key expression. delete — deletes a resource pull — Pulls data for a pull subscriber. get — Issues a distributed query and returns a stream of results. The query target, coverage and consolidation depends on policies.
33. Zenoh’s Abstractions Universality Zenoh’s abstraction are universal since they allow to express the key patterns in distributed computing, namely: Publish/Subscribe. Trivially supported by Zenoh’s Publisher and Subscriber Remote Computation. A Queryable represents a generalised computation, since it can transparently deal with replication and partitioning Storage. Represented by the combination of a Queryable and a Subscriber Additionally all these primitives enjoy location transparency by the virtue of being data-centric.
34. Storage fleet/1/** Subscriber fleet/*/car/*/position Storage fleet/2/** Publisher fleet/2/car/1/position Publisher fleet/1/car/1/position ZZ AWS Azure Private Cloud Edge Server 5G MEC 5G MEC Pull Subscriber fleet/2/car/*/position Storage fleet/1/car/1/** Storage fleet/1/car/2/** Storage fleet/1/car/3/** Subscriber fleet/2/car/*/position Subscriber fleet/2/car/1/position Subscriber fleet/2/car/*/position Pub/Sub Queryable fleet/*/adas/navigation/** Queryable fleet/2/adas/navigation/** Queryable fleet/1/adas/navigation/**
35. Storage fleet/1/** Subscriber fleet/*/car/*/position Storage fleet/2/** Publisher fleet/2/car/1/position Publisher fleet/1/car/1/position ZZ AWS Azure Private Cloud Edge Server 5G MEC 5G MEC Pull Subscriber fleet/2/car/*/position Storage fleet/1/car/1/** Storage fleet/1/car/2/** Storage fleet/1/car/3/** Get fleet/1/car/*/something Distributed Query Queryable fleet/*/adas/navigation/** Queryable fleet/2/adas/navigation/** Queryable fleet/1/adas/navigation/** Subscriber fleet/2/car/*/position Subscriber fleet/2/car/1/position Subscriber fleet/2/car/*/position
36. Storage fleet/1/** Distributed Query Subscriber fleet/*/car/*/position Storage fleet/2/** Publisher fleet/2/car/1/position Publisher fleet/1/car/1/position ZZ AWS Azure Private Cloud Edge Server 5G MEC 5G MEC Pull Subscriber fleet/2/car/*/position Storage fleet/1/car/1/** Storage fleet/1/car/2/** Storage fleet/1/car/3/** Queryable fleet/*/adas/navigation/** Queryable fleet/2/adas/navigation/** Get fleet/car/*/something Queryable fleet/1/adas/navigation/** Subscriber fleet/2/car/*/position Subscriber fleet/2/car/1/position Subscriber fleet/2/car/*/position
37. Storage fleet/1/** Subscriber fleet/*/car/*/position Storage fleet/2/** Publisher fleet/2/car/1/position Publisher fleet/1/car/1/position ZZ AWS Azure Private Cloud Edge Server 5G MEC 5G MEC Pull Subscriber fleet/2/car/*/position Storage fleet/1/car/1/** Storage fleet/1/car/2/** Storage fleet/1/car/3/** Queryable fleet/*/adas/navigation/** Queryable fleet/1/adas/navigation/** Queryable fleet/2/adas/navigation/** Computation get fleet/1/adas/navigation/car/1 Subscriber fleet/2/car/*/position Subscriber fleet/2/car/1/position Subscriber fleet/2/car/*/position
38. get fleet/1/adas/navigation/car/1 Storage fleet/1/** Subscriber fleet/*/car/*/position Storage fleet/2/** Publisher fleet/2/car/1/position Publisher fleet/1/car/1/position ZZ AWS Azure Private Cloud Edge Server 5G MEC 5G MEC Pull Subscriber fleet/2/car/*/position Storage fleet/1/car/1/** Storage fleet/1/car/2/** Storage fleet/1/car/3/** Queryable fleet/*/adas/navigation/** Queryable fleet/1/adas/navigation/** Queryable fleet/2/adas/navigation/** Computation Subscriber fleet/2/car/*/position Subscriber fleet/2/car/1/position Subscriber fleet/2/car/*/position
39. Programming API
40. Zenoh Ecosystem There are two implementations of the Zenoh protocol Eclipse Zenoh written in Rust and Eclipse Zenoh Pico written in C and targeting micro-controllers When using C/C++ APIs, applications can target either of these versions — it is a compile- time decision C C CPP Zenoh Pico Zenoh Rust C CPP Python Kotlin Java … Rust
41. Rust API
42. API Design Support for both async and sync programming model Builders everywhere to ensure evolvability Needs to “resolve” by calling res() in order to craft the request
43. Synch or Async: Pick Your Choice Zenoh supports both sync and async APIs Depending on your application and the integration requirement with other libraries and native code it may be preferable to use one or the other To decide which API to use you need to import the right “prelude”
44. Opening a Session use zenoh::prelude::sync::*; fn main() { let z = zenoh::open(config::default()).res().unwrap(); // use the session… } use zenoh::prelude::r#async::*; fn main() { let z = zenoh::open(config::default()).res().await.unwrap(); // use the session… } Sync API Async API
45. API Design Leverage modern C++ as much as possible Make API ergonomic for a C++ Programmer Avoid the use of exceptions and use variants to propagate errors
46. Opening a Session #include <zenoh.hxx> int main() { zenoh::Config c; auto z = std::get<zenoh::Session>(zenoh::open(std::move(c))); // use the session return 0; } std::get will fail if the variant is returning an error… But what you’d want is to propagate the error explaining why the session failed. A more idiomatic way of opening a session is the following: #include <zenoh.hxx> int main() { zenoh::Config c; auto z = zenoh::expect<zenoh::Session>(zenoh::open(std::move(c))); // use the session return 0; }
47. Publishing Data
48. Producing Data w/o a Publisher Zenoh makes it possible to “put” data without needing to declare a publisher The declaration of a publisher is just an optimisation that should be done for resources that are written regularly let _ = z .put(“zetta/webinar/zenoh/tutorial/time", "15:00") .res() .await;
49. Producing Data w/o a Publisher Zenoh makes it possible to “put” data without needing to declare a publisher The declaration of a publisher is just an optimisation that should be done for resources that are written regularly z.put(“zetta/webinar/zenoh/tutorial/time", "15:00");
50. Producing Data with a Publisher The declaration of a publisher is just an optimisation It is a way to indicating to Zenoh that a given resource is going to be written often and thus implement a series of runtime and wire optimisations — in other terms make the common case fast! let z_pub = z.declare_publisher("zetta/webinar/zenoh/unleashed/video") .res() .await .unwrap(); // grab frame z_pub.put(frame.to_vec()).res().await.unwrap(); Publications can be made for any valid key expression, such as, zetta/webinar/zenoh/unleashed/* and zetta/webinar/zenoh/**
51. Producing Data with a Publisher The declaration of a publisher is just an optimisation It is a way to indicating to Zenoh that a given resource is going to be written often and thus implement a series of runtime and wire optimisations — in other terms make the common case fast! auto p = zenoh::expect(z.declare_publisher("demo/bulletin/ascii-image")); // generate ascii-image p.put(s_img); Publications can be made for any valid key expression, such as, zetta/demo/bulletin/* and demo/**, etc.
52. What Kind of Data can Zenoh Grok? Zenoh provides support for a series of encoding an makes it simpler to deal with data these types For instance, you can natively writes, Strings, Integers, Floats, JSON, etc. Applications are able to provide their own encoding impl Encoding { pub const EMPTY: Encoding = ...; pub const APP_OCTET_STREAM: Encoding = ...; pub const APP_CUSTOM: Encoding = ...; pub const TEXT_PLAIN: Encoding = ...; pub const APP_PROPERTIES: Encoding = ...; pub const APP_JSON: Encoding = ...; pub const APP_SQL: Encoding = ...; pub const APP_INTEGER: Encoding = ...; pub const APP_FLOAT: Encoding = ...; pub const APP_XML: Encoding = ...; pub const APP_XHTML_XML: Encoding = ...; pub const APP_XWWW_FORM_URLENCODED: Encoding = ...; pub const TEXT_JSON: Encoding = ...; pub const TEXT_HTML: Encoding = ...; pub const TEXT_XML: Encoding = ...; pub const TEXT_CSS: Encoding = ...; pub const TEXT_CSV: Encoding = ...; pub const TEXT_JAVASCRIPT: Encoding = ...; pub const IMAGE_JPEG: Encoding = ...; pub const IMAGE_PNG: Encoding = ...; pub const IMAGE_GIF: Encoding = ...; }
53. Few Examples let _ = z .put(“zetta/webinar/zenoh/tutorial/time", "15:00") .res().await; let _ = z .put(“zetta/webinar/zenoh/tutorial/stars”, 5.0) .res().await; let _ = z .put(“zetta/webinar/zenoh/tutorial/episodes", 1) .res().await; let mut vec = Vec::<u8>::new(); vec.push(1); vec.push(2); vec.push(3); let _ = z .put(“zetta/webinar/zenoh/tutorial/attachment", vec) .res() .await; text/plain application/ fl oat application/integer application/octet-stream
54. What Kind of Data can Zenoh Grok? let webinar = r#" { "title": "Understanding Zenoh", "speaker": "Angelo Corsaro" "date": "26/11/2023", "time": "15.00", "timezone": "CET" "abstract": "some cute abstract here" "stars": 5.0, "episodes": 1 "live-audio": "zetta/webinar/zenoh/unleashed/audio" "live-video": “zetta/webinar/zenoh/unleashed/video" }"#; let _ = z .put("zetta/webinar/zenoh/unleashed", webinar) .encoding(KnownEncoding::AppJson) .res() .await; application/json JSON is also natively supported This makes it easy to produce and consume JSON
55. Setting Encoding in C++ auto p_opts = zenoh::PutOptions(); p_opts.set_encoding(zenoh::EncodingPrefix::Z_ENCODING_PREFIX_APP_CUSTOM); z.put("demo/bulletin/ascii-image", slbd, p_opts); In C++ options are provided by means of a dedicated type associated to each operation. In the case of put, this is called PutOptions
56. Subscribing
57. Subscriber Declaration A subscriber is declared by providing a key expression, such as, zetta/webinar/zenoh/ unleashed/* and zetta/webinar/** Zenoh guarantees that a subscriber will receive the put operation performed for keys- expressions that match with that of the subscriber Example: A Subscriber for zetta/webinar/zenoh/** matches zetta/webinar/zenoh/unleashed, zetta/webinar/zenoh/unleashed/title, zetta/webinar/zenoh/ROS2Connector etc. let sub = z .declare_subscriber("zetta/webinar/**") .res() .await .unwrap();
58. Subscriber Declaration A subscriber is declared by providing a key expression, such as, zetta/webinar/zenoh/ unleashed/* and zetta/webinar/** Zenoh guarantees that a subscriber will receive the put operation performed for keys- expressions that match with that of the subscriber Example: A Subscriber for zetta/webinar/zenoh/** matches zetta/webinar/zenoh/ unleashed, zetta/webinar/zenoh/unleashed/title, zetta/webinar/zenoh/ ROS2Connector etc. auto s = zenoh::expect<zenoh::Subscriber>( z.declare_subscriber(“zetta/webinar/**”, callback)); The subscription will be closed once the subscriber goes out of scope.
59. Samples received by a subscriber can be consumed asynchronously using a stream and as such avoiding any inversion of control This keeps the code highly readable and can leverage all the stream combinators Subscribers as Streams while let Ok(s) = sub.recv_async().await { println!(" ({}, {}) -- Encoding: {}", s.key_expr.as_str(), s.value, s.encoding); } let rstream = sub.recv_async().fuse() .map(|rs| { if let Ok(s) = rs { Some((s.key_expr, s.value)) } else { None }});
60. QoS
61. Reliability Reliability in Zenoh is controlled by the Subscriber let sub = z .declare_subscriber("zetta/webinar/**") .reliability(reliability) .res() .await .unwrap(); pub enum Reliability { #[default] BestEffort, Reliable, }
62. Congestion Control The producer has control over the congestion control policy pub enum CongestionControl { #[default] Drop = 0, Block = 1, } let _ = z .put("zetta/webinar/zenoh/unleashed/attachment", vec) .congestion_control(CongestionControl::Block) .res() .await; let z_pub = z.declare_publisher("zetta/webinar/zenoh/unleashed/video") .priority(Priority::RealTime) .congestion_control(CongestionControl::Drop) .res().unwrap(); The default congestion control can be provided when creating a publisher
63. Controlling Priority pub enum Priority { RealTime = 1, InteractiveHigh = 2, InteractiveLow = 3, DataHigh = 4, #[default] Data = 5, DataLow = 6, Background = 7, } let _ = z .put("zetta/webinar/zenoh/unleashed/time", “15:00") .priority(Priority::DataHigh) .res() .await; Zenoh supports tra ff i c scheduling based on 7 priority levels The priority can be individually controlled for each put operation Higher priority samples over-takes lower priority ones
64. Setting QoS in C++ auto p_opts = zenoh::PutOptions(); p_opts .set_encoding(zenoh::EncodingPrefix::Z_ENCODING_PREFIX_APP_CUSTOM) .set_congestion_control(zenoh::CongestionControl ::Z_CONGESTION_CONTROL_DROP) .set_priority(zenoh::Priority::Z_PRIORITY_INTERACTIVE_HIGH); z.put("demo/bulletin/ascii-image", slbd, p_opts);
65. Building
66. Cargo [package] name = “your-app" version = "0.1.0" edition = "2021" [[bin]] name = "pub" path = "src/pub.rs" [[bin]] name = "sub" path = "src/sub.rs" [dependencies] zenoh = "0.10.1-rc" async-std = { version = "=1.12.0", features = ["attributes", "unstable"] }
67. CMake cmake_minimum_required(VERSION 3.27) project(Your_Project) set(CMAKE_CXX_STANDARD 17) include(FetchContent) FetchContent_declare(c_backend GIT_REPOSITORY "https://github.com/eclipse-zenoh/zenoh-c" GIT_TAG release-0.10.1-rc) FetchContent_MakeAvailable(c_backend) FetchContent_declare(cpp_wrapper GIT_REPOSITORY "https://github.com/eclipse-zenoh/zenoh-cpp" GIT_TAG release-0.10.1-rc) FetchContent_MakeAvailable(cpp_wrapper) add_executable(your_app your_src-1.cpp your_src-2.cpp your_src-n.cpp) target_link_libraries(your_app PRIVATE zenohcxx::zenohc::lib)
68. CMake — Using Zenoh-Pico cmake_minimum_required(VERSION 3.27) project(Your_Project) set(CMAKE_CXX_STANDARD 17) include(FetchContent) FetchContent_declare(zenohpico_backend GIT_REPOSITORY “https://github.com/eclipse-zenoh/zenoh-pico" GIT_TAG release-0.10.1-rc) FetchContent_MakeAvailable(zenohpico_backend) FetchContent_declare(cpp_wrapper GIT_REPOSITORY "https://github.com/eclipse-zenoh/zenoh-cpp" GIT_TAG release-0.10.1-rc) FetchContent_MakeAvailable(cpp_wrapper) add_executable(your_app your_src-1.cpp your_src-2.cpp your_src-n.cpp) target_link_libraries(your_app PRIVATE zenohcxx::zenohpico)
69. User Types Serialisation
70. Serialisation We are often asked how you can use Protobuf or other similar technologies with Zenoh It is actually quite straightforward — Let’s take a look!
71. Concluding Remarks
72. Final Remarks The Zenoh team is innovating at fast pace The community is growing swiftly and expanding across Robotics, Automotive, IoT & More Growing awareness around the advantages provided by Zenoh Help us spreading the word!
73. Patience, persistence and perspiration make an unbeatable combination for success. Thank You
74. CEO/CTO ZettaScale Angelo Corsaro, PhD Zenoh Tutorial Part II
75. Building Zenoh To build all stable Zenoh features, you should do: $ cargo build --release --all-targets Let’s see what comes out of the box with this..
76. Features List [features] auth_pubkey = ["zenoh-transport/auth_pubkey"] auth_usrpwd = ["zenoh-transport/auth_usrpwd"] complete_n = ["zenoh-codec/complete_n"] shared-memory = [ "zenoh-shm", "zenoh-protocol/shared-memory", "zenoh-transport/shared-memory", ] stats = ["zenoh-transport/stats", "zenoh-protocol/stats"] transport_multilink = ["zenoh-transport/transport_multilink"] transport_compression = ["zenoh-transport/transport_compression"] transport_quic = ["zenoh-transport/transport_quic"] transport_serial = ["zenoh-transport/transport_serial"] transport_unixpipe = ["zenoh-transport/transport_unixpipe"] transport_tcp = ["zenoh-transport/transport_tcp"] transport_tls = ["zenoh-transport/transport_tls"] transport_udp = ["zenoh-transport/transport_udp"] transport_unixsock-stream = ["zenoh-transport/transport_unixsock-stream"] transport_ws = ["zenoh-transport/transport_ws"]
77. Default Features These are the features that are built by default For anything else you need to expliticely opt-in default = [ "auth_pubkey", "auth_usrpwd", "transport_multilink", "transport_compression", "transport_quic", "transport_tcp", "transport_tls", "transport_udp", "transport_unixsock-stream", "transport_ws", ]
78. Minimal Build Suppose you wanted to build only one transport, say TCP/IP. This is how you can achieve it: cargo build --release --no-default-features;--features=zenoh-transport/transport_tcp Notice that in this case you will still have scouting support via multicast, but you won’t be able to open a session over UDP/IP
79. Minimal Build Cargo Dependency In Cargo.toml you can use the following dependency to control the features you want to opt-in for zenoh = { version = "0.10.1-rc", default-features = false, features = ["transport_tcp"] }
80. Minimising Runtime Threads To minimise the number of runtime threads you should fi rst reduce the number of features to the bare minimum You should reduce the number of threads used for the transport (see con fi g) Finally you should control the number of threads used by the Async-Std runtime using the ASYNC_STD_THREAD_COUNT environment variable { transport: { link: { tx: { threads: 1 } } } } con fi g.json5 minimal git:(main) ✗ ASYNC_STD_THREAD_COUNT=1 ./target/debug/m_sub con fi g.json5 Example
81. Unstable Features Liveliness. Introduces the concept of liveliness token to very scalable “presence” and group management Attachment. Allows for custom “attachments” to be added to put operations. Source Info. Provide additional information on the source into the Sample Matching Listeners. Allows to register listeners to detect matching events, such as a subscriber matching a publication. Publication Locality. Controls the distribution scope of a publication.
82. Enabling Unstable Features Notice that unstable features need to be enabled either when building or when using Zenoh as a crate Building unstable features: $ cargo build --release --all-targets -F=unstable
83. Liveliness
84. Liveliness Zenoh liveliness provides a scalable way to assert and monitor liveliness at a granularity controlled by the application The basic mechanism is based on liveliness tokens: More info at: https://github.com/eclipse-zenoh/roadmap/discussions/72 let mut token = z.liveliness() .declare_token(“x-men/wolverine”) .res() .await .unwrap(); $ cargo build --examples -F=unstable
85. Liveliness Subscriptions Liveliness changes can be observed by subscribing to the proper key expression let subscriber = z .liveliness() .declare_subscriber(“x-men/*”) .res() .await .unwrap(); Sample::Put will indicate new tokens (join) while Sample::Delete will indicate disposed tokens (leave)
86. Liveliness Query A snapshot of the current liveliness situation for a given key expression can be retrieved with a query: let subscriber = z .get(“@/liveliness/x-men/*”) .res() .await .unwrap(); Sample::Put will indicate new tokens (join) while Sample::Delete will indicate disposed tokens (leave)
87. Attachment
88. Attachment Zenoh provides now a mechanism to associate an attachment with a publication — think of it as an “email attachment” An attachment is represented as a list of key/values This feature was always part of the zenoh protocol, but only recently we have provided APIs to exploit it
89. Use Cases End-to-End Protection. Attachments can be used to piggy-back checksums to verify the integrity of data on the other side Meta-Information. Attachments can be used to associate meta- information with user-data w/o having to pollute the user type with this information Security. Attachments can be used to carry identities, and other information useful in security.
90. Code Example pub const AUTOSAR: Crc<u8> = Crc::<u8>::new(&CRC_8_AUTOSAR); let mut checksum = vec!(0u8, 1); checksum[0] = AUTOSAR.checksum(value.as_bytes()); let mut atch = Attachment::new(); atch.insert("crc", checksum.as_slice()); let _ = z .put("demo/attachment", value) .with_attachment(atch.clone()) .res().await; }
91. Code Example let sub = z.declare_subscriber("demo/attachment").res().await.unwrap(); pub const AUTOSAR: Crc<u8> = Crc::<u8>::new(&CRC_8_AUTOSAR); while let Ok(s) = sub.recv_async().await { let payload = s.payload.contiguous(); let cs = AUTOSAR.checksum(payload.as_ref()); let checks = match &s.attachment { Some(a) => { let r = "crc".as_bytes(); if let Some(checksum) = a.get(&r) { if checksum.contiguous()[0] == cs { true } else { false } } else { false } }, None => { false } }; if checks { println!("Received valid data: {:?} -- {}", String::from_utf8_lossy(&payload), cs); } else { println!("Received corrupted data"); } } }
92. Discovery
93. Scouting Technically, Scouting is not part of the Zenoh protocol Scouting is one way of bootstrapping the Zenoh protocol, by discovering other Zenoh runtimes and their locators Scouting has to be considered as a plug-in, with the default implementation working on multicast
94. Scouting An API is available to “scout around”… let receiver = zenoh::scout(WhatAmI::Peer | WhatAmI::Router, Config::default()) .res() .await .unwrap();
95. Discovery Zenoh discovery works by default by closure In other terms, Zenoh runtimes exchange neighbours views in order to discover neighbour’s neighbours This feature can be disabled by con fi guration
96. Routing Zenoh supports two routing algorithms, one that assumes you have a clique, the other that supports arbitrary connected graphs By default peers are con fi gured to use clique routing, while routers use link-state that supports arbitrary graphs These can be changed by con fi guration
97. Interest Propagation Zenoh has a “interest” propagation based on set theory. The function declare_subscriber/declare_queryable “may” causes the emission of an interest packet. That said depending on previous declarations nothing may happen or a generalisation may take place Example I: declare_subscriber(“a/b/**”) declare_subscriber(a/b/c/*) — this won’t cause any emission
98. Query & Queryable
99. Queryable A queryable is a network endpoint that answers to queries matching its key expression The answer to a query is a fi nite stream of (key, values) Queryable are quite powerful and can be used to represent both the reading-half of a datastore/database as well as trigger computations
100. Declaring a Queryable The declaration of a queryable requires minimally the key expression associated with it let queryable = z.declare_queryable("zetta/webinar/zenoh/**") .complete(is_complete) .res().await.unwrap(); You can also specify the completeness of this queryable w.r.t. the set of keys represented by its key expression. If the queryable “has answers” for any key in the set expressed by its key expression then it ice complete, otherwise it is not.
101. Visualising Completeness Another way of visualising complete queryable, is like a set w/o any “holes” This property is especially important if you want to use queryable to do quorums Complete Non Complete
102. Query In Zenoh queries are issued by using the get operation z.get(k) .target(query_target) .consolidation(consolidation) .res().await.unwrap(); The result of a query is a fi nite stream of Reply pub struct Reply { pub sample: Result<Sample, Value>, pub replier_id: ZenohId, } Controls who, among matching, is eligible to be targeted Controls if and how replies are consolidated
103. Query Target The query target controls, among the queryable whose key expression matches that of the query, the set that should be used to execute the query pub enum QueryTarget { #[default] BestMatching, All, AllComplete, #[cfg(feature = "complete_n")] Complete(u64), } BestMatching. Bigger intersection and closer in routing terms All. All the queryable that intersect the query AllComplete. All queryable that fully cover the query. In other terms the query key expression is superset of the query expression Complete(N). Among the queryable that are complete pick N — routing considerations will be used to decide. (This is useful for read quorums, N is your read quorum :-)
104. Understanding Query Target zetta/webinar/** zetta/webinar/** zetta/webinar/** Router Router Router Router Router Peer Peer Peer Peer Peer Router Client Client Client Client Router Router Client Peer Peer Peer Peer Client zetta/webinar/** zetta/webinar/** zetta/news/** zetta/news/**
105. Understanding Query Target zetta/webinar/** zetta/webinar/** zetta/webinar/** Router Router Router Router Router Peer Peer Peer Peer Peer Router Client Client Client Client Router Router Client Peer Peer Peer Peer Client zetta/webinar/** zetta/webinar/** z.get("zetta/webinar/zenoh/*") .target(BestMatching) .res().await.unwrap(); zetta/news/** zetta/news/**
106. Understanding Query Target zetta/webinar/** zetta/webinar/** zetta/webinar/** Router Router Router Router Router Peer Peer Peer Peer Peer Router Client Client Client Client Router Router Client Peer Peer Peer Peer Client zetta/webinar/** zetta/webinar/** zetta/news/** zetta/news/** z.get("zetta/webinar/zenoh/*") .target(BestMatching) .res().await.unwrap();
107. Understanding Query Target zetta/webinar/** zetta/webinar/** zetta/webinar/** Router Router Router Router Router Peer Peer Peer Peer Peer Router Client Client Client Client Router Router Client Peer Peer Peer Peer Client zetta/webinar/** zetta/webinar/** zetta/news/** zetta/news/** z.get("zetta/webinar/zenoh/*") .target(BestMatching) .res().await.unwrap();
108. Understanding Query Target zetta/webinar/** zetta/webinar/** Router Router Router Router Router Peer Peer Peer Peer Peer Router Client Client Client Client Router Router Client Peer Peer Peer Peer Client zetta/webinar/** zetta/webinar/** zetta/news/** zetta/news/** zetta/webinar/** z.get("zetta/webinar/zenoh/*") .target(BestMatching) .res().await.unwrap();
109. Understanding Query Target zetta/webinar/** zetta/webinar/** zetta/webinar/** Router Router Router Router Router Peer Peer Peer Peer Peer Router Client Client Client Client Router Router Client Peer Peer Peer Peer Client zetta/webinar/** zetta/webinar/** zetta/news/** zetta/news/** z.get("zetta/webinar/zenoh/*") .target(All) .res().await.unwrap();
110. Understanding Query Target zetta/webinar/** zetta/webinar/** zetta/webinar/** Router Router Router Router Router Peer Peer Peer Peer Peer Router Client Client Client Client Router Router Client Peer Peer Peer Peer Client zetta/webinar/** zetta/webinar/** zetta/news/** zetta/news/** z.get("zetta/webinar/zenoh/*") .target(All) .res().await.unwrap();
111. Understanding Query Target zetta/webinar/** zetta/webinar/** zetta/webinar/** Router Router Router Router Router Peer Peer Peer Peer Peer Router Client Client Client Client Router Router Client Peer Peer Peer Peer Client zetta/webinar/** zetta/webinar/** zetta/news/** zetta/news/** z.get("zetta/webinar/zenoh/*") .target(AllComplete) .res().await.unwrap();
112. Understanding Query Target zetta/webinar/** zetta/webinar/** zetta/webinar/** Router Router Router Router Router Peer Peer Peer Peer Peer Router Client Client Client Client Router Router Client Peer Peer Peer Peer Client zetta/webinar/** zetta/webinar/** zetta/news/** zetta/news/** z.get("zetta/webinar/zenoh/*") .target(AllComplete) .res().await.unwrap();
113. Understanding Query Target zetta/webinar/** zetta/webinar/** zetta/webinar/** Router Router Router Router Router Peer Peer Peer Peer Peer Router Client Client Client Client Router Router Client Peer Peer Peer Peer Client zetta/webinar/** zetta/webinar/** zetta/news/** zetta/news/** z.get("zetta/webinar/zenoh/*") .target(Complete(2)) .res().await.unwrap();
114. Understanding Query Target zetta/webinar/** zetta/webinar/** zetta/webinar/** Router Router Router Router Router Peer Peer Peer Peer Peer Router Client Client Client Client Router Router Client Peer Peer Peer Peer Client zetta/webinar/** zetta/webinar/** zetta/news/** zetta/news/** z.get("zetta/webinar/zenoh/*") .target(Complete(2)) .res().await.unwrap();
115. Consolidation The query consolidation allow to control how to deal with multiple answers for the same key coming from di ff erent queryable pub enum ConsolidationMode { None, Monotonic, Latest, } None. The issuer of the query gets all replies Monotonic. Along the routing paths, samples for the same keys with older timestamp are dropped. Notice that as there may be di ff erent routing path more samples for the same key may still be received Latest. Ensures that only the latest value for each sample is receive
116. Latest News
117. Zenoh Moves to Tokio Zenoh has moved to the Tokio Async Framework This was a sizeable PR which was merged on main branch the 12/3/2024 All tests are passing and performance are stable… Some gains on latency.
118. Zenoh vs DDS, MQTT & Kafka Zenoh can deliver at peak performance of ~70Gbps at 8Kb payload: - 3.3x higher than DDS 
 - 23x higher than Kafka 
 - 35x than MQTT (higher for larger payload) 
 Zenoh’s latency 10us, 6us with ultra-low- latency support 
 - 25us for MQTT 
 - 75us for Kafka 
 - 8us DDS
119. Wrapping Up
120. Final Remarks The Zenoh team is innovating at fast pace The community is growing swiftly and expanding across Robotics, Automotive, IoT & More Growing awareness around the advantages provided by Zenoh Help us spreading the word!
121. Patience, persistence and perspiration make an unbeatable combination for success. Thank You
122. Latest News
123. Top 10 Automotive Startups ZettaScale has been identi fi ed as one of the 10 startups in vehicle communication to be watched in 2023 The core technology considered in the report was Zenoh
124. In a recent article that evaluates mainstream protocols, including MQTT, DDS, AMQP an CoAP Amazon describes Zenoh as: “Perhaps the most viable protocol that will help us mature to this ideal model is Zenoh which is an Eclipse Incubation project” Amazon IoT
125. 2023-09 ROS 2 RMW alternate Abstract........................................................................................................................................ 1 User Challenges with DDS..........................................................................................................2 DDS has a fully-connected graph of participants.....................................................................2 DDS uses UDP multicast for discovery................................................................................... 3 DDS can have difficulty transferring large data....................................................................... 3 DDS can struggle on some WiFi networks.............................................................................. 4 DDS tends to have complex tuning parameters...................................................................... 4 Vendor specific non-standard DDS extensions....................................................................... 4 Next Steps............................................................................................................................... 5 Requirements gathering............................................................................................................. 5 User Survey............................................................................................................................. 5 Demographics..............................................................................................................6 Technical Data............................................................................................................. 6 Alternative middleware options.................................................................................... 8 Requirements.......................................................................................................................... 9 Comparative analysis of currently available middlewares.................................................... 11 Complete list of investigated middlewares.......................................................................11 Performance.................................................................................................................... 13 Middlewares X Requirements................................................................................................13 Conclusion................................................................................................................................. 14 Appendix A.................................................................................................................................15 Abstract The ROS MiddleWare interface (RMW) is an abstraction layer that allows ROS 2 to swap out its underlying communication mechanism (middleware) at both compile time and runtime. All current Tier 1 implementations of RMW are based on DDS. At the time that this solution was chosen, it met many of the initial requirements. Over the last 8 years of use, based on user feedback a number of problems have been identified, including: Intrinsics / Open Robotics in 2023 Zenoh selected as the fi rst non DDS protocol to be natively supported Intrinsic implementing Zenoh RMW as the major contribution for next ROS2 release Very insightful report, worth reading