Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Asynchronous I/O and Coroutines for Smooth Data...

Asynchronous I/O and Coroutines for Smooth Data Streaming

Linux kernel 5.1 introduced io_uring, which is a mechanism to do asynchronous I/O, primarily for network and disk operations. With asynchronous I/O, the responsiveness of your program is enhanced, but it can easily lead to "callback hell", where you register callbacks that processes arrived data, which feeds information to other callbacks, and so on. C++20 brings us language level coroutines. Coroutines are a generalization of functions, that can be suspended in the middle to allow other computations, and then resumed again, all in the same thread. One such suspension point can be to wait for the arrival of data. In this presentation I will bring a brief introduction to both topics, and then show how to use io_uring and coroutines to write code that reads asynchronous data in several short loops, seemingly running in parallel, without having to worry about threading issues.

Björn Fahller

April 08, 2022
Tweet

More Decks by Björn Fahller

Other Decks in Programming

Transcript

  1. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 1/170 Asynchronous I/O and coroutines for smooth data streaming Björn Fahller #include <coroutine> ... x = co_await source; co_yield computation(x); io_uring
  2. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 2/170 Asynchronous I/O and coroutines for smooth data streaming
  3. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 3/170 Asynchronous I/O and coroutines for smooth data streaming
  4. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 4/170 Asynchronous I/O and coroutines for smooth data streaming
  5. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 5/170 Asynchronous I/O and coroutines for smooth data streaming
  6. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 6/170 Asynchronous I/O and coroutines for smooth data streaming Björn Fahller #include <coroutine> ... x = co_await source; co_yield computation(x); io_uring
  7. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 7/170 Linux networking • Traditionally we use select/poll/epoll to register file descriptors we want to react to • And read/recv/recvmsg/recvmmsg to read the data (and corresponding to send).
  8. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 8/170 class poller { public: using worker = std::function<void(std::span<char> data)>; void add(int fd, worker w) { fds_.push_back({fd, POLLIN, 0}); cbs_.emplace(fd, std::move(w)); } void wait() { auto r = poll(fds_.data(), fds_.size(), -1); for (auto& e : fds_) { if (e.revents & POLLIN) { char buffer[1500]; auto len = ::read(e.fd, buffer, sizeof(buffer)); cbs_[e.fd](std::span(buffer).first(len)); } } } private: std::vector<pollfd> fds_; std::map<int, worker> cbs_; };
  9. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 9/170 Synchronous I/O with poll/read
  10. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 10/170 Synchronous I/O with poll/read poll()
  11. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 11/170 Synchronous I/O with poll/read poll() fill from kernel
  12. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 12/170 Synchronous I/O with poll/read fill from kernel read()
  13. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 13/170 Synchronous I/O with poll/read fill from kernel read()
  14. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 14/170 Synchronous I/O with poll/read fill from kernel read() fill from kernel
  15. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 15/170 Synchronous I/O with poll/read fill from kernel read() process fill from kernel
  16. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 16/170 Synchronous I/O with poll/read fill from kernel read() process poll() fill from kernel
  17. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 18/170 io_uring userspace kernel submission queue (SQ) IP stack Filesystem completion queue (CQ)
  18. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 19/170 io_uring userspace kernel submission queue (SQ) IP stack Filesystem completion queue (CQ) #include <liburing.h> io_uring uring; io_uring_queue_init(8, &uring, 0); ... auto entry = io_uring_get_sqe(&uring); io_uring_prep_read(entry, fd, ptr, size, 0); io_uring_sqe_set_data(entry, work); ... io_uring_cqe* entry; auto e = io_uring_wait_cqe(&uring, &entry);
  19. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 20/170 io_uring userspace kernel submission queue (SQ) IP stack Filesystem completion queue (CQ) #include <liburing.h> io_uring uring; io_uring_queue_init(8, &uring, 0); ... auto entry = io_uring_get_sqe(&uring); io_uring_prep_read(entry, fd, ptr, size, 0); io_uring_sqe_set_data(entry, work); ... io_uring_cqe* entry; auto e = io_uring_wait_cqe(&uring, &entry); Size of queue, i.e. max number of pending entries
  20. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 21/170 io_uring userspace kernel submission queue (SQ) IP stack Filesystem completion queue (CQ) #include <liburing.h> io_uring uring; io_uring_queue_init(8, &uring, 0); ... auto entry = io_uring_get_sqe(&uring); io_uring_prep_read(entry, fd, ptr, size, 0); io_uring_sqe_set_data(entry, work); ... io_uring_cqe* entry; auto e = io_uring_wait_cqe(&uring, &entry); Annoyingly only one word, 64-bits, to express the operation and the data, so an indirection is almost always needed.
  21. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 22/170 io_uring userspace kernel submission queue (SQ) IP stack Filesystem completion queue (CQ) #include <liburing.h> io_uring uring; io_uring_queue_init(8, &uring, 0); ... auto entry = io_uring_get_sqe(&uring); io_uring_prep_read(entry, fd, ptr, size, 0); io_uring_sqe_set_data(entry, work); ... io_uring_cqe* entry; auto e = io_uring_wait_cqe(&uring, &entry);
  22. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 23/170 uring #include <liburing.h> class ring { public: using work = std::function<bool(std::span<char>)>; ring(); ring& operator=(ring&&) = delete; ~ring(); void add(int fd, work); void wait(); private: struct read_work; std::list<read_work> pending_; io_uring uring_; };
  23. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 24/170 uring #include <liburing.h> class ring { public: using work = std::function<bool(std::span<char>)>; ring(); ring& operator=(ring&&) = delete; ~ring(); void add(int fd, work); void wait(); private: struct read_work; std::list<read_work> pending_; io_uring uring_; }; struct ring::read_work {     work cb_;     int fd_;     std::array<char, 1500> buffer_; }; void ring::add(int fd, work w) { auto& work = pending_.emplace_back(); work.cb_ = std::move(w); work.fd_ = fd; auto entry = io_uring_get_sqe(&uring_); io_uring_prep_read(entry, fd, work.buffer_.data(), work.buffer_.size(), 0); io_uring_sqe_set_data(entry, &work); }
  24. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 25/170 uring #include <liburing.h> class ring { public: using work = std::function<bool(std::span<char>)>; ring(); ring& operator=(ring&&) = delete; ~ring(); void add(int fd, work); void wait(); private: struct read_work; std::list<read_work> pending_; io_uring uring_; }; struct ring::read_work {     work cb_;     int fd_;     std::array<char, 1500> buffer_; }; void ring::add(int fd, work w) { auto& work = pending_.emplace_back(); work.cb_ = std::move(w); work.fd_ = fd; auto entry = io_uring_get_sqe(&uring_); io_uring_prep_read(entry, fd, work.buffer_.data(), work.buffer_.size(), 0); io_uring_sqe_set_data(entry, &work); } The data area to read into needs to be available when preparing the read operation
  25. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 26/170 uring #include <liburing.h> class ring { public: using work = std::function<bool(std::span<char>)>; ring(); ring& operator=(ring&&) = delete; ~ring(); void add(int fd, work); void wait(); private: struct read_work; std::list<read_work> pending_; io_uring uring_; }; struct ring::read_work {     work cb_;     int fd_;     std::array<char, 1500> buffer_; }; void ring::add(int fd, work w) { auto& work = pending_.emplace_back(); work.cb_ = std::move(w); work.fd_ = fd; auto entry = io_uring_get_sqe(&uring_); io_uring_prep_read(entry, fd, work.buffer_.data(), work.buffer_.size(), 0); io_uring_sqe_set_data(entry, &work); } Get a submission queue entry and prepare a read to the buffer from the file descriptor
  26. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 27/170 uring #include <liburing.h> class ring { public: using work = std::function<bool(std::span<char>)>; ring(); ring& operator=(ring&&) = delete; ~ring(); void add(int fd, work); void wait(); private: struct read_work; std::list<read_work> pending_; io_uring uring_; }; struct ring::read_work {     work cb_;     int fd_;     std::array<char, 1500> buffer_; }; void ring::add(int fd, work w) { auto& work = pending_.emplace_back(); work.cb_ = std::move(w); work.fd_ = fd; auto entry = io_uring_get_sqe(&uring_); io_uring_prep_read(entry, fd, work.buffer_.data(), work.buffer_.size(), 0); io_uring_sqe_set_data(entry, &work); } And associate the work struct with the data area and callback with the submission queue entry
  27. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 28/170 data buffers ready
  28. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 29/170 data buffers ready fill from kernel
  29. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 30/170 data buffers ready fill from kernel wait
  30. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 31/170 data buffers ready fill from kernel wait fill from kernel
  31. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 32/170 data buffers ready fill from kernel wait fill from kernel process
  32. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 33/170 data buffers ready fill from kernel wait fill from kernel process fill from kernel
  33. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 34/170 data buffers ready fill from kernel wait fill from kernel process process fill from kernel
  34. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 35/170 data buffers ready fill from kernel wait fill from kernel process process fill from kernel
  35. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 36/170 data buffers ready fill from kernel wait fill from kernel process process fill from kernel Fewer system calls too!
  36. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 39/170 coroutines Offers a way for you to write asynchronous code as if they were continuous loops
  37. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 40/170 coroutines Offers a way for you to write asynchronous code as if they were continuous loops Language support from C++20
  38. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 41/170 coroutines Offers a way for you to write asynchronous code as if they were continuous loops Language support from C++20 Compiler magic converts it to something else, via types that you must write
  39. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 42/170 coroutines Offers a way for you to write asynchronous code as if they were continuous loops Language support from C++20 Compiler magic converts it to something else, via types that you must write – And they’re mindbogglingly hard to understand
  40. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 43/170 coroutines Offers a way for you to write asynchronous code as if they were continuous loops Language support from C++20 Compiler magic converts it to something else, via types that you must write – And they’re mindbogglingly hard to understand – And the standard library doesn’t help
  41. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 44/170 coroutines for (;;) { … } for (;;) { … }
  42. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 45/170 coroutines for (;;) { … } for (;;) { … } Suspend execution
  43. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 46/170 coroutines for (;;) { … } for (;;) { … } Compute x Suspend execution
  44. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 47/170 coroutines for (;;) { … } for (;;) { … } Compute x Suspend execution Resume execution with x Suspend execution
  45. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 48/170 coroutines for (;;) { … } for (;;) { … } Compute x Suspend execution Resume execution with x Work with x Suspend execution
  46. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 49/170 coroutines for (;;) { … } for (;;) { … } Compute x Suspend execution Resume execution with x Work with x Suspend execution Suspend execution Resume execution
  47. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 50/170 coroutines for (;;) { … } for (;;) { … } Compute x Suspend execution Resume execution with x Work with x Suspend execution Suspend execution Resume execution Compute x
  48. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 51/170 coroutines for (;;) { … } for (;;) { … } Compute x Suspend execution Resume execution with x Work with x Suspend execution Suspend execution Resume execution Compute x Suspend execution Resume execution with x
  49. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 52/170 coroutines for (;;) { … } for (;;) { … } Compute x Suspend execution Resume execution with x Work with x Suspend execution Suspend execution Resume execution Compute x Suspend execution Resume execution with x Work with x
  50. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 53/170 coroutines for (;;) { … } for (;;) { … } Compute x Suspend execution Resume execution with x Work with x Suspend execution Suspend execution Resume execution Compute x Suspend execution Resume execution with x Work with x Suspend execution Resume execution
  51. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 54/170 coroutines coroutine_type my_coro(int x, int y, coro_src& src) {     int result = 0;     while (int a = co_await src)     {         result += work(a,x,y);     }     co_return result; } auto coro_obj = my_coro(3, 8, source);
  52. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 55/170 coroutines coroutine_type my_coro(int x, int y, coro_src& src) {     int result = 0;     while (int a = co_await src)     {         result += work(a,x,y);     }     co_return result; } auto coro_obj = my_coro(3, 8, source); class my_coro { public:     my_coro(int x_, int y_) : x(x_), y(y_) {}     int get_result() const { return result; }     void operator()(int a)     {     result += work(a, x, y);     } private:     int x;     int y;     int result = 0; }; auto coro_obj = impl(new my_coro(3, 8), source); Compiler rewrites like
  53. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 56/170 coroutines coroutine_type my_coro(int x, int y, coro_src& src) {     int result = 0;     while (int a = co_await src)     {         result += work(a,x,y);     }     co_return result; } auto coro_obj = my_coro(3, 8, source); class my_coro { public:     my_coro(int x_, int y_) : x(x_), y(y_) {}     int get_result() const { return result; }     void operator()(int a)     {     result += work(a, x, y);     } private:     int x;     int y;     int result = 0; }; auto coro_obj = impl(new my_coro(3, 8), source); It’s not this simple! Compiler rewrites like
  54. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 57/170 coroutines coroutine_type my_coro(int x, int y, coro_src& src) {     int result = 0;     while (int a = co_await src)     {         result += work(a,x,y);     }     co_return result; } auto coro_obj = my_coro(3, 8, source);
  55. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 58/170 coroutines coroutine_type my_coro(int x, int y, coro_src& src) {     int result = 0;     while (int a = co_await src)     {         result += work(a,x,y);     }     co_return result; } auto coro_obj = my_coro(3, 8, source); We must write this type
  56. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 59/170 coroutines coroutine_type my_coro(int x, int y, coro_src& src) {     int result = 0;     while (int a = co_await src)     {         result += work(a,x,y);     }     co_return result; } auto coro_obj = my_coro(3, 8, source); We must write this type and this type
  57. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 60/170 coroutine return object (task) template <typename T> struct task { using promise_type = promise<T>; auto operator co_await() const noexcept; private: task(promise<T>* p) : m_promise(p) { } friend class promise<T>; promise_ptr<T> m_promise; };
  58. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 61/170 coroutine return object (task) template <typename T> struct task { using promise_type = promise<T>; auto operator co_await() const noexcept; private: task(promise<T>* p) : m_promise(p) { } friend class promise<T>; promise_ptr<T> m_promise; }; NOT std::promise<T>!
  59. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 62/170 coroutine return object (task) template <typename T> struct task { using promise_type = promise<T>; auto operator co_await() const noexcept; private: task(promise<T>* p) : m_promise(p) { } friend class promise<T>; promise_ptr<T> m_promise; }; NOT std::promise<T>! And we have to write it ourselves
  60. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 63/170 coroutine return object (task) template <typename T> struct task { using promise_type = promise<T>; auto operator co_await() const noexcept; private: task(promise<T>* p) : m_promise(p) { } friend class promise<T>; promise_ptr<T> m_promise; }; We will be given a promise<T>, allocated by compiler magic.
  61. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 64/170 coroutine return object (task) template <typename T> struct task { using promise_type = promise<T>; auto operator co_await() const noexcept; private: task(promise<T>* p) : m_promise(p) { } friend class promise<T>; promise_ptr<T> m_promise; }; And we must destroy it the right way. A smart pointer makes this easy.
  62. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 65/170 coroutine return object (task) template <typename T> struct task { using promise_type = promise<T>; auto operator co_await() const noexcept; private: task(promise<T>* p) : m_promise(p) { } friend class promise<T>; promise_ptr<T> m_promise; }; struct coro_deleter { template <typename promise> void operator()(promise* p) const noexcept { using handle = std::coroutine_handle<promise>; handle::from_promise(*p).destroy(); } }; template <typename T> using promise_ptr = std::unique_ptr<promise<T>, coro_deleter>;
  63. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 66/170 template <typename T> struct promise { task<T> get_return_object() noexcept; std::suspend_never initial_suspend() noexcept; std::suspend_always final_suspend() noexcept;   bool is_ready() const noexcept; T get(); void unhandled_exception(); template <typename U> std::suspend_always yield_value(U&& u); void return_void(); std::coroutine_handle<> m_continuation; std::optional<T> m_value; }; coroutines promise
  64. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 67/170 template <typename T> struct promise { task<T> get_return_object() noexcept; std::suspend_never initial_suspend() noexcept; std::suspend_always final_suspend() noexcept;   bool is_ready() const noexcept; T get(); void unhandled_exception(); template <typename U> std::suspend_always yield_value(U&& u); void return_void(); std::coroutine_handle<> m_continuation; std::optional<T> m_value; }; coroutines promise The function that creates the task<T> object
  65. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 68/170 template <typename T> struct promise { task<T> get_return_object() noexcept; std::suspend_never initial_suspend() noexcept; std::suspend_always final_suspend() noexcept;   bool is_ready() const noexcept; T get(); void unhandled_exception(); template <typename U> std::suspend_always yield_value(U&& u); void return_void(); std::coroutine_handle<> m_continuation; std::optional<T> m_value; }; coroutines promise Behaviour at the beginning and end of the life of the coroutine
  66. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 69/170 template <typename T> struct promise { task<T> get_return_object() noexcept; std::suspend_never initial_suspend() noexcept; std::suspend_always final_suspend() noexcept;   bool is_ready() const noexcept; T get(); void unhandled_exception(); template <typename U> std::suspend_always yield_value(U&& u); void return_void(); std::coroutine_handle<> m_continuation; std::optional<T> m_value; }; coroutines promise Utility functions for our own implementation
  67. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 70/170 template <typename T> struct promise { task<T> get_return_object() noexcept; std::suspend_never initial_suspend() noexcept; std::suspend_always final_suspend() noexcept;   bool is_ready() const noexcept; T get(); void unhandled_exception(); template <typename U> std::suspend_always yield_value(U&& u); void return_void(); std::coroutine_handle<> m_continuation; std::optional<T> m_value; }; coroutines promise Handle to suspended coroutine
  68. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 71/170 template <typename T> struct promise { task<T> get_return_object() noexcept; std::suspend_never initial_suspend() noexcept; std::suspend_always final_suspend() noexcept;   bool is_ready() const noexcept; T get(); void unhandled_exception(); template <typename U> std::suspend_always yield_value(U&& u); void return_void(); std::coroutine_handle<> m_continuation; std::optional<T> m_value; }; template <typename T> struct promise { task<T> get_return_object() noexcept; std::suspend_never initial_suspend() noexcept; std::suspend_always final_suspend() noexcept;   bool is_ready() const noexcept; T get(); void unhandled_exception(); template <typename U> std::suspend_always yield_value(U&& u){ m_value.emplace(std::forward<U>(u)); m_continuation.resume(); return {}; } }; coroutines promise Resume execution of the coroutine that is suspended waiting for a value.
  69. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 72/170 coroutine return object (task) cont. template <typename T> struct task {     auto operator co_await() const noexcept {         struct awaitable {             bool await_ready() const noexcept {                 return m_promise.is_ready();             }             void await_suspend(std::coroutine_handle<> next) const noexcept {                 m_promise.m_continuation = next;             }             T await_resume() const {                 return m_promise.get();             }             promise<T>& m_promise;         };         return awaitable{ *m_promise };     } };
  70. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 73/170 coroutine return object (task) cont. template <typename T> struct task {     auto operator co_await() const noexcept {         struct awaitable {             bool await_ready() const noexcept {                 return m_promise.is_ready();             }             void await_suspend(std::coroutine_handle<> next) const noexcept {                 m_promise.m_continuation = next;             }             T await_resume() const {                 return m_promise.get();             }             promise<T>& m_promise;         };         return awaitable{ *m_promise };     } }; Operator is called when code calls: co_await task;
  71. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 74/170 coroutine return object (task) cont. template <typename T> struct task {     auto operator co_await() const noexcept {         struct awaitable {             bool await_ready() const noexcept {                 return m_promise.is_ready();             }             void await_suspend(std::coroutine_handle<> next) const noexcept {                 m_promise.m_continuation = next;             }             T await_resume() const {                 return m_promise.get();             }             promise<T>& m_promise;         };         return awaitable{ *m_promise };     } }; It returns an awaitable object that communicates with the promise<T>
  72. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 75/170 coroutine return object (task) cont. template <typename T> struct task {     auto operator co_await() const noexcept {         struct awaitable {             bool await_ready() const noexcept {                 return m_promise.is_ready();             }             void await_suspend(std::coroutine_handle<> next) const noexcept {                 m_promise.m_continuation = next;             }             T await_resume() const {                 return m_promise.get();             }             promise<T>& m_promise;         };         return awaitable{ *m_promise };     } }; Check if the promise<T> holds a value
  73. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 76/170 coroutine return object (task) cont. template <typename T> struct task {     auto operator co_await() const noexcept {         struct awaitable {             bool await_ready() const noexcept {                 return m_promise.is_ready();             }             void await_suspend(std::coroutine_handle<> next) const noexcept {                 m_promise.m_continuation = next;             }             T await_resume() const {                 return m_promise.get();             }             promise<T>& m_promise;         };         return awaitable{ *m_promise };     } }; Store the calling coroutine as the one to continue when the promise<T> gets a value
  74. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 77/170 coroutine return object (task) cont. template <typename T> struct task {     auto operator co_await() const noexcept {         struct awaitable {             bool await_ready() const noexcept {                 return m_promise.is_ready();             }             void await_suspend(std::coroutine_handle<> next) const noexcept {                 m_promise.m_continuation = next;             }             T await_resume() const {                 return m_promise.get();             }             promise<T>& m_promise;         };         return awaitable{ *m_promise };     } }; And finally, on resume, get the value from the promise<T>, making it empty again.
  75. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 78/170 coroutine return object (task) cont. template <typename T> struct task {     auto operator co_await() const noexcept {         struct awaitable {             bool await_ready() const noexcept {                 return m_promise.is_ready();             }             void await_suspend(std::coroutine_handle<> next) const noexcept {                 m_promise.m_continuation = next;             }             T await_resume() const {                 return m_promise.get();             }             promise<T>& m_promise;         };         return awaitable{ *m_promise };     } };
  76. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 80/170 Making a computation pipeline template <typename T, typename P> task<T> filter_in(P predicate, task<T>& in) { for (;;) { auto v = co_await in; if (predicate(v)) { co_yield v; } } }
  77. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 81/170 Making a computation pipeline template <typename T, typename P> task<T> filter_in(P predicate, task<T>& in) { for (;;) { auto v = co_await in; if (predicate(v)) { co_yield v; } } } Calls the yield_value() member function on the promise of the return type for the coroutine.
  78. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 82/170 Making a computation pipeline template <typename T, typename P> task<T> filter_in(P predicate, task<T>& in) { for (;;) { auto v = co_await in; if (predicate(v)) { co_yield v; } } } int main() { auto is_odd = [](auto v) { return v & 1;}; auto incoming = task<int>::make(); auto odd_values = filter_in(is_odd, incoming); auto printer = print_all(odd_values); for (int i = 0; i < 10; ++i) { incoming.get_promise().yield_value(i); } }
  79. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 83/170 Making a computation pipeline template <typename T, typename P> task<T> filter_in(P predicate, task<T>& in) { for (;;) { auto v = co_await in; if (predicate(v)) { co_yield v; } } } int main() { auto is_odd = [](auto v) { return v & 1;}; auto incoming = task<int>::make(); auto odd_values = filter_in(is_odd, incoming); auto printer = print_all(odd_values); for (int i = 0; i < 10; ++i) { incoming.get_promise().yield_value(i); } }
  80. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 84/170 Making a computation pipeline template <typename T, typename P> task<T> filter_in(P predicate, task<T>& in) { for (;;) { auto v = co_await in; if (predicate(v)) { co_yield v; } } } int main() { auto is_odd = [](auto v) { return v & 1;}; auto incoming = task<int>::make(); auto odd_values = filter_in(is_odd, incoming); auto printer = print_all(odd_values); for (int i = 0; i < 10; ++i) { incoming.get_promise().yield_value(i); } }
  81. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 85/170 Making a computation pipeline template <typename T, typename P> task<T> filter_in(P predicate, task<T>& in) { for (;;) { auto v = co_await in; if (predicate(v)) { co_yield v; } } } int main() { auto is_odd = [](auto v) { return v & 1;}; auto incoming = task<int>::make(); auto odd_values = filter_in(is_odd, incoming); auto printer = print_all(odd_values); for (int i = 0; i < 10; ++i) { incoming.get_promise().yield_value(i); } }
  82. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 86/170 Making a computation pipeline template <typename T, typename P> task<T> filter_in(P predicate, task<T>& in) { for (;;) { auto v = co_await in; if (predicate(v)) { co_yield v; } } } int main() { auto is_odd = [](auto v) { return v & 1;}; auto incoming = task<int>::make(); auto odd_values = filter_in(is_odd, incoming); auto printer = print_all(odd_values); for (int i = 0; i < 10; ++i) { incoming.get_promise().yield_value(i); } }
  83. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 88/170 io_uring + coroutines = 💖
  84. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 89/170 io_uring + coroutines = 💖 We’ve seen how:
  85. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 90/170 io_uring + coroutines = 💖 We’ve seen how: • io_uring offers asynchronous data
  86. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 91/170 io_uring + coroutines = 💖 We’ve seen how: • io_uring offers asynchronous data • Calling yield_value() on a coroutine promise pushes data through the coroutine pipeline
  87. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 92/170 io_uring + coroutines = 💖 We’ve seen how: • io_uring offers asynchronous data • Calling yield_value() on a coroutine promise pushes data through the coroutine pipeline • How to read values from an upstream coroutine with co_await
  88. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 93/170 io_uring + coroutines = 💖 We’ve seen how: • io_uring offers asynchronous data • Calling yield_value() on a coroutine promise pushes data through the coroutine pipeline • How to read values from an upstream coroutine with co_await • How to forward values downstream with co_yield
  89. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 94/170 io_uring + coroutines = 💖 We’ve seen how: • io_uring offers asynchronous data • Calling yield_value() on a coroutine promise pushes data through the coroutine pipeline • How to read values from an upstream coroutine with co_await • How to forward values downstream with co_yield Let’s put the pieces together
  90. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 95/170 uring + coroutines auto to_promise = [](auto& promise) { return [&](auto packet) { promise.yield_value(packet); return true; }; }; Convenient tool to generate a callback that writes to a promise, and thus drives a coroutine pipeline.
  91. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 96/170 uring + coroutines int main() { uint64_t num_bytes = 0; uint64_t num_packets = 0; bool done = false; auto in_4000 = task<std::span<char>>::make(); auto counted_packets = count_packet_data(num_packets, num_bytes, in_4000); auto strings = to_string(counted_packets); auto stripped_strings = strip_trailing_newline(strings); auto lines = concatenate_to<40>(stripped_strings); auto print = print_lines(std::cout, lines); …
  92. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 97/170 uring + coroutines int main() { uint64_t num_bytes = 0; uint64_t num_packets = 0; bool done = false; auto in_4000 = task<std::span<char>>::make(); auto counted_packets = count_packet_data(num_packets, num_bytes, in_4000); auto strings = to_string(counted_packets); auto stripped_strings = strip_trailing_newline(strings); auto lines = concatenate_to<40>(stripped_strings); auto print = print_lines(std::cout, lines); …
  93. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 98/170 uring + coroutines int main() { uint64_t num_bytes = 0; uint64_t num_packets = 0; bool done = false; auto in_4000 = task<std::span<char>>::make(); auto counted_packets = count_packet_data(num_packets, num_bytes, in_4000); auto strings = to_string(counted_packets); auto stripped_strings = strip_trailing_newline(strings); auto lines = concatenate_to<40>(stripped_strings); auto print = print_lines(std::cout, lines); … task<std::span<char>> count_packet_data(uint64_t& packets, uint64_t& bytes, task<std::span<char>>& in) { for (;;) { auto packet = co_await in; ++packets; bytes += packet.size(); co_yield packet; } }
  94. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 99/170 uring + coroutines int main() { uint64_t num_bytes = 0; uint64_t num_packets = 0; bool done = false; auto in_4000 = task<std::span<char>>::make(); auto counted_packets = count_packet_data(num_packets, num_bytes, in_4000); auto strings = to_string(counted_packets); auto stripped_strings = strip_trailing_newline(strings); auto lines = concatenate_to<40>(stripped_strings); auto print = print_lines(std::cout, lines); …
  95. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 100/170 uring + coroutines int main() { uint64_t num_bytes = 0; uint64_t num_packets = 0; bool done = false; auto in_4000 = task<std::span<char>>::make(); auto counted_packets = count_packet_data(num_packets, num_bytes, in_4000); auto strings = to_string(counted_packets); auto stripped_strings = strip_trailing_newline(strings); auto lines = concatenate_to<40>(stripped_strings); auto print = print_lines(std::cout, lines); … task<std::string> to_string(task<std::span<char>>& in) { for (;;) { auto packet = co_await in; co_yield std::string{packet.begin(), packet.end()}; } }
  96. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 101/170 uring + coroutines int main() { uint64_t num_bytes = 0; uint64_t num_packets = 0; bool done = false; auto in_4000 = task<std::span<char>>::make(); auto counted_packets = count_packet_data(num_packets, num_bytes, in_4000); auto strings = to_string(counted_packets); auto stripped_strings = strip_trailing_newline(strings); auto lines = concatenate_to<40>(stripped_strings); auto print = print_lines(std::cout, lines); …
  97. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 102/170 uring + coroutines int main() { uint64_t num_bytes = 0; uint64_t num_packets = 0; bool done = false; auto in_4000 = task<std::span<char>>::make(); auto counted_packets = count_packet_data(num_packets, num_bytes, in_4000); auto strings = to_string(counted_packets); auto stripped_strings = strip_trailing_newline(strings); auto lines = concatenate_to<40>(stripped_strings); auto print = print_lines(std::cout, lines); … task<std::string> strip_trailing_newline(task<std::string>& in) { for (;;) { auto s = co_await in; while (s.ends_with("\n")) { s.resize(s.length() - 1); } co_yield s; } }
  98. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 103/170 uring + coroutines int main() { uint64_t num_bytes = 0; uint64_t num_packets = 0; bool done = false; auto in_4000 = task<std::span<char>>::make(); auto counted_packets = count_packet_data(num_packets, num_bytes, in_4000); auto strings = to_string(counted_packets); auto stripped_strings = strip_trailing_newline(strings); auto lines = concatenate_to<40>(stripped_strings); auto print = print_lines(std::cout, lines); …
  99. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 104/170 uring + coroutines int main() { uint64_t num_bytes = 0; uint64_t num_packets = 0; bool done = false; auto in_4000 = task<std::span<char>>::make(); auto counted_packets = count_packet_data(num_packets, num_bytes, in_4000); auto strings = to_string(counted_packets); auto stripped_strings = strip_trailing_newline(strings); auto lines = concatenate_to<40>(stripped_strings); auto print = print_lines(std::cout, lines); … template <size_t line_length> task<std::string> concatenate_to(task<std::string>& in) { std::string current_line; for (;;) { auto next_piece = co_await in; if (current_line.length() + next_piece.length() + 1 > line_length) { co_yield std::exchange(current_line, next_piece); } else if (current_line.empty()) { current_line = next_piece; } else { current_line += " " + next_piece; } } }
  100. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 105/170 uring + coroutines int main() { uint64_t num_bytes = 0; uint64_t num_packets = 0; bool done = false; auto in_4000 = task<std::span<char>>::make(); auto counted_packets = count_packet_data(num_packets, num_bytes, in_4000); auto strings = to_string(counted_packets); auto stripped_strings = strip_trailing_newline(strings); auto lines = concatenate_to<40>(stripped_strings); auto print = print_lines(std::cout, lines); …
  101. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 106/170 uring + coroutines int main() { uint64_t num_bytes = 0; uint64_t num_packets = 0; bool done = false; auto in_4000 = task<std::span<char>>::make(); auto counted_packets = count_packet_data(num_packets, num_bytes, in_4000); auto strings = to_string(counted_packets); auto stripped_strings = strip_trailing_newline(strings); auto lines = concatenate_to<40>(stripped_strings); auto print = print_lines(std::cout, lines); … task<void> print_lines(std::ostream& os, task<std::string>& in) { for (;;) { auto line = co_await in; os << ':' << line << ":\n"; } }
  102. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 107/170 uring + coroutines ... auto print_and_exit = [&](auto&&) { std::cout << "packets=" << num_packets << " bytes=" << num_bytes << '\n'; done = true; return false; }; ring r; auto port4000 = udp_socket("127.0.0.1", 4000); auto port4001 = udp_socket("127.0.0.1", 4001); r.add(port4000.fd(), to_promise(in_4000.get_promise())); r.add(port4001.fd(), print_and_exit); while (!done) { r.wait(); } }
  103. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 109/170 uring + coroutines int main() { uint64_t num_bytes = 0; uint64_t num_packets = 0; bool done = false; auto in_4000 = task<std::span<char>>::make(); auto counted_packets = count_packet_data(num_packets, num_bytes, in_4000); auto strings = to_string(counted_packets); auto stripped_strings = strip_trailing_newline(strings); auto lines = concatenate_to<40>(stripped_strings); auto print = print_lines(std::cout, lines); …
  104. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 110/170 uring + coroutines int main() { uint64_t num_bytes = 0; uint64_t num_packets = 0; bool done = false; auto in_4000 = task<std::span<char>>::make(); auto counted_packets = count_packet_data(num_packets, num_bytes, in_4000); auto strings = to_string(counted_packets); auto stripped_strings = strip_trailing_newline(strings); auto lines = concatenate_to<40>(stripped_strings); auto print = print_lines(std::cout, lines); … Keeping these coroutine objects alive as explicit objects is ugly, annoying and error prone.
  105. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 111/170 uring + coroutines int main() { uint64_t num_bytes = 0; uint64_t num_packets = 0; bool done = false; auto in_4000 = task<std::span<char>>::make(); auto counted_packets = count_packet_data(num_packets, num_bytes, in_4000); auto strings = to_string(counted_packets); auto stripped_strings = strip_trailing_newline(strings); auto lines = concatenate_to<40>(stripped_strings); auto print = print_lines(std::cout, lines); … Keeping these coroutine objects alive as explicit objects is ugly, annoying and error prone. Can we do better?
  106. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 112/170 pipeline as a value type template <typename T, typename P> task<T> filter_in(P predicate, task<T> in) { for (;;) { auto v = co_await in; if (predicate(v)) { co_yield v; } } }
  107. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 113/170 pipeline as a value type template <typename T, typename P> task<T> filter_in(P predicate, task<T> in) { for (;;) { auto v = co_await in; if (predicate(v)) { co_yield v; } } } By value
  108. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 114/170 pipeline as a value type template <typename T, typename P> task<T> filter_in(P predicate, task<T> in) { for (;;) { auto v = co_await in; if (predicate(v)) { co_yield v; } } } template <typename T> task<void> print_all(task<T> in) { for (;;) { auto v = co_await in; std::cout << v << '\n'; } }
  109. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 115/170 pipeline as a value type template <typename T, typename P> task<T> filter_in(P predicate, task<T> in) { for (;;) { auto v = co_await in; if (predicate(v)) { co_yield v; } } } template <typename T> task<void> print_all(task<T> in) { for (;;) { auto v = co_await in; std::cout << v << '\n'; } } By value
  110. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 116/170 pipeline as a value type template <typename T, typename P> task<T> filter_in(P predicate, task<T> in) { for (;;) { auto v = co_await in; if (predicate(v)) { co_yield v; } } } template <typename T> task<void> print_all(task<T> in) { for (;;) { auto v = co_await in; std::cout << v << '\n'; } } int main() { auto is_odd = [](auto v) { return v & 1;}; auto incoming = task<int>::make(); auto &promise = incoming.get_promise(); auto pipeline = print_all(filter_in(is_odd, std::move(incoming)); for (int i = 0; i < 10; ++i) { promise.yield_value(i); } }
  111. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 117/170 pipeline as a value type template <typename T, typename P> task<T> filter_in(P predicate, task<T> in) { for (;;) { auto v = co_await in; if (predicate(v)) { co_yield v; } } } template <typename T> task<void> print_all(task<T> in) { for (;;) { auto v = co_await in; std::cout << v << '\n'; } } int main() { auto is_odd = [](auto v) { return v & 1;}; auto incoming = task<int>::make(); auto &promise = incoming.get_promise(); auto pipeline = print_all(filter_in(is_odd, std::move(incoming)); for (int i = 0; i < 10; ++i) { promise.yield_value(i); } }
  112. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 118/170 pipeline as a value type template <typename T, typename P> task<T> filter_in(P predicate, task<T> in) { for (;;) { auto v = co_await in; if (predicate(v)) { co_yield v; } } } template <typename T> task<void> print_all(task<T> in) { for (;;) { auto v = co_await in; std::cout << v << '\n'; } } int main() { auto is_odd = [](auto v) { return v & 1;}; auto incoming = task<int>::make(); auto &promise = incoming.get_promise(); auto pipeline = print_all(filter_in(is_odd, std::move(incoming)); for (int i = 0; i < 10; ++i) { promise.yield_value(i); } }
  113. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 119/170 pipeline as a value type template <typename T, typename P> task<T> filter_in(P predicate, task<T> in) { for (;;) { auto v = co_await in; if (predicate(v)) { co_yield v; } } } template <typename T> task<void> print_all(task<T> in) { for (;;) { auto v = co_await in; std::cout << v << '\n'; } } int main() { auto is_odd = [](auto v) { return v & 1;}; auto incoming = task<int>::make(); auto &promise = incoming.get_promise(); auto pipeline = print_all(filter_in(is_odd, std::move(incoming)); for (int i = 0; i < 10; ++i) { promise.yield_value(i); } }
  114. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 120/170 pipeline as a value type template <typename T, typename P> task<T> filter_in(P predicate, task<T> in) { for (;;) { auto v = co_await in; if (predicate(v)) { co_yield v; } } } template <typename T> task<void> print_all(task<T> in) { for (;;) { auto v = co_await in; std::cout << v << '\n'; } } int main() { auto is_odd = [](auto v) { return v & 1;}; auto incoming = task<int>::make(); auto &promise = incoming.get_promise(); auto pipeline = print_all(filter_in(is_odd, std::move(incoming)); for (int i = 0; i < 10; ++i) { promise.yield_value(i); } } But this is not very nice either, is it?
  115. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 121/170 Another level of indirection auto filter_in = [](auto predicate) {     return [predicate]<typename T>(task<T> in) -> task<T>     {         for (;;) {             auto v = co_await in;             if (predicate(v)) {                 co_yield v;             }         }     }; };
  116. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 122/170 Another level of indirection auto filter_in = [](auto predicate) {     return [predicate]<typename T>(task<T> in) -> task<T>     {         for (;;) {             auto v = co_await in;             if (predicate(v)) {                 co_yield v;             }         }     }; }; By value
  117. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 123/170 Another level of indirection auto filter_in = [](auto predicate) {     return [predicate]<typename T>(task<T> in) -> task<T>     {         for (;;) {             auto v = co_await in;             if (predicate(v)) {                 co_yield v;             }         }     }; }; By value
  118. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 124/170 Another level of indirection auto filter_in = [](auto predicate) {     return [predicate]<typename T>(task<T> in) -> task<T>     {         for (;;) {             auto v = co_await in;             if (predicate(v)) {                 co_yield v;             }         }     }; }; Explicit return type
  119. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 125/170 Another level of indirection auto filter_in = [](auto predicate) {     return [predicate]<typename T>(task<T> in) -> task<T>     {         for (;;) {             auto v = co_await in;             if (predicate(v)) {                 co_yield v;             }         }     }; }; Explicit return type auto print_all = [](std::ostream& dest) {     return [&dest]<typename T>(task<T> in) -> task<void>     {         for (;;)         {             auto v = co_await in;             dest << v << '\n' << std::flush;         }     }; };
  120. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 126/170 Another level of indirection auto filter_in = [](auto predicate) {     return [predicate]<typename T>(task<T> in) -> task<T>     {         for (;;) {             auto v = co_await in;             if (predicate(v)) {                 co_yield v;             }         }     }; }; auto print_all = [](std::ostream& dest) {     return [&dest]<typename T>(task<T> in) -> task<void>     {         for (;;)         {             auto v = co_await in;             dest << v << '\n' << std::flush;         }     }; }; Capture the stream
  121. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 127/170 Another level of indirection auto filter_in = [](auto predicate) {     return [predicate]<typename T>(task<T> in) -> task<T>     {         for (;;) {             auto v = co_await in;             if (predicate(v)) {                 co_yield v;             }         }     }; }; auto print_all = [](std::ostream& dest) {     return [&dest]<typename T>(task<T> in) -> task<void>     {         for (;;)         {             auto v = co_await in;             dest << v << '\n' << std::flush;         }     }; }; int main() { auto is_odd = [](auto v) { return v & 1;}; auto incoming = task<int>::make(); auto &promise = incoming.get_promise(); auto to_cout = print_all(std::cout); auto pipeline = to_cout(filter_in(is_odd)); auto coro = pipeline(std::move(incoming)); for (int i = 0; i < 10; ++i) { promise.yield_value(i); } }
  122. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 128/170 Another level of indirection auto filter_in = [](auto predicate) {     return [predicate]<typename T>(task<T> in) -> task<T>     {         for (;;) {             auto v = co_await in;             if (predicate(v)) {                 co_yield v;             }         }     }; }; auto print_all = [](std::ostream& dest) {     return [&dest]<typename T>(task<T> in) -> task<void>     {         for (;;)         {             auto v = co_await in;             dest << v << '\n' << std::flush;         }     }; }; int main() { auto is_odd = [](auto v) { return v & 1;}; auto incoming = task<int>::make(); auto &promise = incoming.get_promise(); auto to_cout = print_all(std::cout); auto pipeline = to_cout(filter_in(is_odd)); auto coro = pipeline(std::move(incoming)); for (int i = 0; i < 10; ++i) { promise.yield_value(i); } } But this is still not very nice, is it?
  123. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 129/170 A little helper template <typename T> struct coro_stage : T {         coro_stage(T t) : T(std::move(t)) {} }; template <typename T> coro_stage(T) -> coro_stage<T>; template <typename T, typename U> auto operator|(coro_stage<T> lh, coro_stage<U> rh) {     return coro_stage{         [lh = std::move(lh), rh=std::move(rh)]         <typename T>(task<T> in)         {             return rh(lh(std::move(in)));         }     }; }
  124. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 130/170 A little helper template <typename T> struct coro_stage : T {         coro_stage(T t) : T(std::move(t)) {} }; template <typename T> coro_stage(T) -> coro_stage<T>; template <typename T, typename U> auto operator|(coro_stage<T> lh, coro_stage<U> rh) {     return coro_stage{         [lh = std::move(lh), rh=std::move(rh)]         <typename T>(task<T> in)         {             return rh(lh(std::move(in)));         }     }; } An indirection wrapper that can be created from any class type.
  125. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 131/170 A little helper template <typename T> struct coro_stage : T {         coro_stage(T t) : T(std::move(t)) {} }; template <typename T> coro_stage(T) -> coro_stage<T>; template <typename T, typename U> auto operator|(coro_stage<T> lh, coro_stage<U> rh) {     return coro_stage{         [lh = std::move(lh), rh=std::move(rh)]         <typename T>(task<T> in)         {             return rh(lh(std::move(in)));         }     }; } Pipe operators are cool
  126. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 132/170 A little helper template <typename T> struct coro_stage : T {         coro_stage(T t) : T(std::move(t)) {} }; template <typename T> coro_stage(T) -> coro_stage<T>; template <typename T, typename U> auto operator|(coro_stage<T> lh, coro_stage<U> rh) {     return coro_stage{         [lh = std::move(lh), rh=std::move(rh)]         <typename T>(task<T> in)         {             return rh(lh(std::move(in)));         }     }; } By value
  127. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 133/170 A little helper template <typename T> struct coro_stage : T {         coro_stage(T t) : T(std::move(t)) {} }; template <typename T> coro_stage(T) -> coro_stage<T>; template <typename T, typename U> auto operator|(coro_stage<T> lh, coro_stage<U> rh) {     return coro_stage{         [lh = std::move(lh), rh=std::move(rh)]         <typename T>(task<T> in)         {             return rh(lh(std::move(in)));         }     }; } Call the right side with the result from calling the left side
  128. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 134/170 A little helper template <typename T> struct coro_stage : T {         coro_stage(T t) : T(std::move(t)) {} }; template <typename T> coro_stage(T) -> coro_stage<T>; template <typename T, typename U> auto operator|(coro_stage<T> lh, coro_stage<U> rh) {     return coro_stage{         [lh = std::move(lh), rh=std::move(rh)]         <typename T>(task<T> in)         {             return rh(lh(std::move(in)));         }     }; } Move these, in case they hold move-only data
  129. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 135/170 A little helper template <typename T> struct coro_stage : T {         coro_stage(T t) : T(std::move(t)) {} }; template <typename T> coro_stage(T) -> coro_stage<T>; template <typename T, typename U> auto operator|(coro_stage<T> lh, coro_stage<U> rh) {     return coro_stage{         [lh = std::move(lh), rh=std::move(rh)]         <typename T>(task<T> in)         {             return rh(lh(std::move(in)));         }     }; } And wrap this new stage for more pipes
  130. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 136/170 Another level of indirection auto filter_in = [](auto predicate) {   return coro_stage{ [predicate]<typename T>(task<T> in) -> task<T> { for (;;) { auto v = co_await in; if (predicate(v)) { co_yield v; } } } }; };
  131. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 137/170 Another level of indirection auto filter_in = [](auto predicate) {   return coro_stage{ [predicate]<typename T>(task<T> in) -> task<T> { for (;;) { auto v = co_await in; if (predicate(v)) { co_yield v; } } } }; }; Use the wrapper
  132. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 138/170 Another level of indirection auto filter_in = [](auto predicate) {   return coro_stage{ [predicate]<typename T>(task<T> in) -> task<T> { for (;;) { auto v = co_await in; if (predicate(v)) { co_yield v; } } } }; }; auto print_all = [](std::ostream& dest) {     return coro_stage{ [&dest]<typename T>(task<T> in) -> task<void>     {         for (;;)         {             auto v = co_await in;             dest << v << '\n' << std::flush;         }     } }; }; Use the wrapper
  133. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 139/170 Another level of indirection auto filter_in = [](auto predicate) {   return coro_stage{ [predicate]<typename T>(task<T> in) -> task<T> { for (;;) { auto v = co_await in; if (predicate(v)) { co_yield v; } } } }; }; auto print_all = [](std::ostream& dest) {     return coro_stage{ [&dest]<typename T>(task<T> in) -> task<void>     {         for (;;)         {             auto v = co_await in;             dest << v << '\n' << std::flush;         }     } }; }; int main() { auto is_odd = [](auto v) { return v & 1;}; auto incoming = task<int>::make(); auto &promise = incoming.get_promise(); auto pipeline = filter_in(is_odd) | print_all(std::cout); auto coro = pipeline(std::move(incoming)); for (int i = 0; i < 10; ++i) { promise.yield_value(i); } }
  134. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 140/170 Another level of indirection auto filter_in = [](auto predicate) {   return coro_stage{ [predicate]<typename T>(task<T> in) -> task<T> { for (;;) { auto v = co_await in; if (predicate(v)) { co_yield v; } } } }; }; auto print_all = [](std::ostream& dest) {     return coro_stage{ [&dest]<typename T>(task<T> in) -> task<void>     {         for (;;)         {             auto v = co_await in;             dest << v << '\n' << std::flush;         }     } }; }; int main() { auto is_odd = [](auto v) { return v & 1;}; auto incoming = task<int>::make(); auto &promise = incoming.get_promise(); auto pipeline = filter_in(is_odd) | print_all(std::cout); auto coro = pipeline(std::move(incoming)); for (int i = 0; i < 10; ++i) { promise.yield_value(i); } } Now we’re getting somewhere!
  135. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 142/170 class poller { public: using worker = std::function<void(std::span<char> data)>; void add(int fd, worker w) { fds_.push_back({fd, POLLIN, 0}); cbs_.emplace(fd, std::move(w)); } ... }; Feeding the pipeline from io_uring
  136. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 143/170 class poller { public: using worker = std::function<void(std::span<char> data)>; void add(int fd, worker w) { fds_.push_back({fd, POLLIN, 0}); cbs_.emplace(fd, std::move(w)); } ... }; Feeding the pipeline from io_uring std::function<F> requires F to be copyable
  137. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 144/170 class poller { public: using worker = std::function<void(std::span<char> data)>; void add(int fd, worker w) { fds_.push_back({fd, POLLIN, 0}); cbs_.emplace(fd, std::move(w)); } ... }; Feeding the pipeline from io_uring
  138. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 145/170 class poller { public: using worker = std::function<void(std::span<char> data)>; void add(int fd, worker w) { fds_.push_back({fd, POLLIN, 0}); cbs_.emplace(fd, std::move(w)); } ... }; Feeding the pipeline from io_uring Ouch
  139. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 146/170 class poller { public: using worker = std::function<void(std::span<char> data)>; void add(int fd, worker w) { fds_.push_back({fd, POLLIN, 0}); cbs_.emplace(fd, std::move(w)); } ... }; Feeding the pipeline from io_uring template <typename> class move_only_function; template <typename R, typename ... Args> class move_only_function<R(Args...)> { public:         move_only_function() = default;         move_only_function(move_only_function&& r)         ... }; Rolled my own naïve version
  140. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 147/170 class poller { public: using worker = std::function<void(std::span<char> data)>; void add(int fd, worker w) { fds_.push_back({fd, POLLIN, 0}); cbs_.emplace(fd, std::move(w)); } ... }; Feeding the pipeline from io_uring template <typename> class move_only_function; template <typename R, typename ... Args> class move_only_function<R(Args...)> { public:         move_only_function() = default;         move_only_function(move_only_function&& r)         ... }; Rolled my own naïve version class poller { public: using worker = move_only_function<void(std::span<char> data)>; void add(int fd, worker w) { fds_.push_back({fd, POLLIN, 0}); cbs_.emplace(fd, std::move(w)); } ... };
  141. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 148/170 Feeding the pipeline from io_uring How create something that is callable with span<char> that feeds into a pipeline via a task<>?
  142. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 149/170 Feeding the pipeline from io_uring template <typename task_type, typename pipeline> auto init_task(pipeline pipe) {     auto input = task<task_type>::make();     auto& promise = input.get_promise();     using coro_type = decltype(pipe(std::move(input)));     return [&promise, pipe = std::move(pipe), input = std::move(input), coro = std::unique_ptr<coro_type>{}](task_type value) mutable     {         if (!coro) coro = std::make_unique<coro_type>(pipe(std::move(input)));         promise.yield_value(value);         return true;     }; }
  143. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 150/170 Feeding the pipeline from io_uring template <typename task_type, typename pipeline> auto init_task(pipeline pipe) {     auto input = task<task_type>::make();     auto& promise = input.get_promise();     using coro_type = decltype(pipe(std::move(input)));     return [&promise, pipe = std::move(pipe), input = std::move(input), coro = std::unique_ptr<coro_type>{}](task_type value) mutable     {         if (!coro) coro = std::make_unique<coro_type>(pipe(std::move(input)));         promise.yield_value(value);         return true;     }; } Create the task
  144. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 151/170 Feeding the pipeline from io_uring template <typename task_type, typename pipeline> auto init_task(pipeline pipe) {     auto input = task<task_type>::make();     auto& promise = input.get_promise();     using coro_type = decltype(pipe(std::move(input)));     return [&promise, pipe = std::move(pipe), input = std::move(input), coro = std::unique_ptr<coro_type>{}](task_type value) mutable     {         if (!coro) coro = std::make_unique<coro_type>(pipe(std::move(input)));         promise.yield_value(value);         return true;     }; } Get the promise
  145. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 152/170 Feeding the pipeline from io_uring template <typename task_type, typename pipeline> auto init_task(pipeline pipe) {     auto input = task<task_type>::make();     auto& promise = input.get_promise();     using coro_type = decltype(pipe(std::move(input)));     return [&promise, pipe = std::move(pipe), input = std::move(input), coro = std::unique_ptr<coro_type>{}](task_type value) mutable     {         if (!coro) coro = std::make_unique<coro_type>(pipe(std::move(input)));         promise.yield_value(value);         return true;     }; } Get the coroutine type
  146. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 153/170 Feeding the pipeline from io_uring template <typename task_type, typename pipeline> auto init_task(pipeline pipe) {     auto input = task<task_type>::make();     auto& promise = input.get_promise();     using coro_type = decltype(pipe(std::move(input)));     return [&promise, pipe = std::move(pipe), input = std::move(input), coro = std::unique_ptr<coro_type>{}](task_type value) mutable     {         if (!coro) coro = std::make_unique<coro_type>(pipe(std::move(input)));         promise.yield_value(value);         return true;     }; } Return something callable that initializes the coroutine return object on the first call.
  147. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 154/170 Feeding the pipeline from io_uring template <typename task_type, typename pipeline> auto init_task(pipeline pipe) {     auto input = task<task_type>::make();     auto& promise = input.get_promise();     using coro_type = decltype(pipe(std::move(input)));     return [&promise, pipe = std::move(pipe), input = std::move(input), coro = std::unique_ptr<coro_type>{}](task_type value) mutable     {         if (!coro) coro = std::make_unique<coro_type>(pipe(std::move(input)));         promise.yield_value(value);         return true;     }; } And yields the value to the promise on each call
  148. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 155/170 Feeding the pipeline from io_uring template <typename task_type, typename pipeline> auto init_task(pipeline pipe) {     auto input = task<task_type>::make();     auto& promise = input.get_promise();     using coro_type = decltype(pipe(std::move(input)));     return [&promise, pipe = std::move(pipe), input = std::move(input), coro = std::unique_ptr<coro_type>{}](task_type value) mutable     {         if (!coro) coro = std::make_unique<coro_type>(pipe(std::move(input)));         promise.yield_value(value);         return true;     }; } Not proud of this code
  149. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 156/170 Feeding the pipeline from the uring int main() { uint64_t num_bytes = 0; uint64_t num_packets = 0; bool done = false; ring r; r.add(port4000.fd(), init_task<std::span<char>>(count_packet_data(num_packets, num_bytes) | to_string | strip_trailing_newline | concatenate_to(40) | print_lines(std::cout) )); …
  150. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 157/170 Feeding the pipeline from the uring int main() { uint64_t num_bytes = 0; uint64_t num_packets = 0; bool done = false; ring r; r.add(port4000.fd(), init_task<std::span<char>>(count_packet_data(num_packets, num_bytes) | to_string | strip_trailing_newline | concatenate_to(40) | print_lines(std::cout) )); … But this is quite nice, isn’t it?
  151. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 159/170 Takeaways • Coroutines can be cool and powerful • Lack of a good library is a major pain • cancelling both coroutines and operations from io_uring can be tricky • There are so many ways you can tweak coroutine behaviour, this was but one simplistic example • We need a good library and we don’t have one • and writing one is really hard
  152. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 161/170 Takeaways • Testing and debugging is terrible, especially if you get the future<>/task<>/awaitable<> types wrong
  153. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 162/170 Takeaways • Testing and debugging is terrible, especially if you get the future<>/task<>/awaitable<> types wrong template <size_t N> struct str { constexpr str(const char* p) noexcept { std::copy_n(p, N, cstr); } friend std::ostream& operator<<(std::ostream& os, const str& s) { return os << s.cstr; } char cstr[N]; }; template <size_t N> str(const char (&)[N]) -> str<N>;
  154. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 163/170 Takeaways • Testing and debugging is terrible, especially if you get the future<>/task<>/awaitable<> types wrong template <size_t N> struct str { constexpr str(const char* p) noexcept { std::copy_n(p, N, cstr); } friend std::ostream& operator<<(std::ostream& os, const str& s) { return os << s.cstr; } char cstr[N]; }; template <size_t N> str(const char (&)[N]) -> str<N>; template <typename T, str name> struct promise { task<T,name> get_return_object() noexcept { std::cerr << "task<T, " << name << ">::get_return_object()\n"; return {this}; } … }; template <str name> task<void, "print_lines"> print_lines(std::ostream& os, task<std::string, name>& in) { …
  155. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 164/170 Takeaways • Testing and debugging is terrible, especially if you get the future<>/task<>/awaitable<> types wrong template <size_t N> struct str { constexpr str(const char* p) noexcept { std::copy_n(p, N, cstr); } friend std::ostream& operator<<(std::ostream& os, const str& s) { return os << s.cstr; } char cstr[N]; }; template <size_t N> str(const char (&)[N]) -> str<N>; template <typename T, str name> struct promise { task<T,name> get_return_object() noexcept { std::cerr << "task<T, " << name << ">::get_return_object()\n"; return {this}; } … }; template <str name> task<void, "print_lines"> print_lines(std::ostream& os, task<std::string, name>& in) { …
  156. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 165/170 Takeaways • Testing and debugging is terrible, especially if you get the future<>/task<>/awaitable<> types wrong template <size_t N> struct str { constexpr str(const char* p) noexcept { std::copy_n(p, N, cstr); } friend std::ostream& operator<<(std::ostream& os, const str& s) { return os << s.cstr; } char cstr[N]; }; template <size_t N> str(const char (&)[N]) -> str<N>; template <typename T, str name> struct promise { task<T,name> get_return_object() noexcept { std::cerr << "task<T, " << name << ">::get_return_object()\n"; return {this}; } … }; template <str name> task<void, "print_lines"> print_lines(std::ostream& os, task<std::string, name>& in) { …
  157. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 166/170 Takeaways • Testing and debugging is terrible, especially if you get the future<>/task<>/awaitable<> types wrong template <size_t N> struct str { constexpr str(const char* p) noexcept { std::copy_n(p, N, cstr); } friend std::ostream& operator<<(std::ostream& os, const str& s) { return os << s.cstr; } char cstr[N]; }; template <size_t N> str(const char (&)[N]) -> str<N>; template <typename T, str name> struct promise { task<T,name> get_return_object() noexcept { std::cerr << "task<T, " << name << ">::get_return_object()\n"; return {this}; } … }; template <str name> task<void, "print_lines"> print_lines(std::ostream& os, task<std::string, name>& in) { …
  158. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 167/170 Takeaways • Testing and debugging is terrible, especially if you get the future<>/task<>/awaitable<> types wrong • Writing asynchronous code as if they were local loops is very convenient
  159. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 168/170 Takeaways • Testing and debugging is terrible, especially if you get the future<>/task<>/awaitable<> types wrong • Writing asynchronous code as if they were local loops is very convenient • Connecting “pipelines” offers great support for generic utilities
  160. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 169/170 Resources Shuveb Hussain – “Lord of the io_uring” https://unixism.net/loti/ Pavel Novikov – “Understanding coroutines by example”, C++London, Feb 2021 https://www.youtube.com/watch?v=7sKUAyWXNHA Lewis Baker – “CppCoro” https://github.com/lewissbaker/cppcoro Facebook experimental – “libunifex” https://github.com/facebookexperimental/libunifex
  161. Asynchronous I/O and coroutines – ACCU 2022 © Björn Fahller

    @bjorn_fahller 170/170 [email protected] @bjorn_fahller @rollbear Björn Fahller Asynchronous I/O and coroutines for smooth data streaming