Is V4L ready?

For most of its existence, the video4linux framework felt short behind in upstream support from vendors. However, over the last few years, more and more vendors have started to contribute to the upstream kernel community, including video4linux. New features are happening in video4linux to enable it to support the features that new hardware requires.

Transcript

1. Is Video4Linux ready for all cutting-edge hardware? Ezequiel Garcia ezequiel@collabora.com
2. 2 Is Video4Linux ready for all cutting-edge hardware?
3. 3 Is Video4Linux ready for all cutting-edge hardware? tl;dr: no
4. 4 Agenda • Traditional V4L APIs • New APIs • The future
5. 5 Current Video4Linux
6. 6 V4L API • VIDIOC_{ENUM_TRY,G,S}_FMT • VIDIOC_{G,S}_STD, VIDIOC_QUERYSTD • VIDIOC_REQBUFS • VIDIOC_QBUF, VIDIOC_DQBUF • VIDIOC_STREAMON, VIDIOC_STREAMOFF and more...
7. 7 Stream API ctrl QBUF buffer S_EXT_CTRL S_FMT DQBUF buffer t buffer buffer ctrl buffer buffer (metadata) (data)
8. 8 Codecs
9. 9 Stateful codecs • Device handle full bitstream, so drivers shouldn’t do any parsing. Performing software stream processing, header generation etc. in the driver is strongly discouraged. • Uses the traditional V4L (stream-based) API. • Specification in progress: [PATCH 0/2] Document memory-to-memory video codec interfaces
10. 10 Stateful codecs Headers Compressed frame
11. 11 Stateless codecs • Device accelerates the encoding/decoding job. • Device handles raw compressed bitstreams, but needs software to do the extra parsing. • Uses the Request (slice-based) API. • Specification also in progress: [RFC PATCH] media: docs-rst: Document m2m stateless video decoder interface
12. 12 Stateless codecs Metadata Compressed frame
13. 13 Stream API ctrl QBUF buffer S_EXT_CTRL S_FMT DQBUF buffer t buffer buffer ctrl buffer buffer (metadata) (data)
14. 14 Request API request ctrl QBUF S_EXT_CTRL REQUEST_ALLOC REQUEST_QUEUE
15. 15 Request API request ctrl QBUF S_EXT_CTRL REQUEST_ALLOC REQUEST_QUEUE
16. 16 Request API request ctrl QBUF S_EXT_CTRL REQUEST_ALLOC REQUEST_QUEUE
17. 17 What does the future look?
18. 18 Fences • Attaching in-fences and out-fences to buffers can reduce latency and improve efficiency. • Work in-progress by Gustavo Padovan and Ezequiel Garcia: [PATCH v10 00/16] V4L2 Explicit Synchronization
19. 19 Fences
20. 20 Fences
21. 21 Async UVC • High-quality devices require more bandwidth from USB controllers and drivers. • Multi-core SoCs capable of processing USB packets in parallel. • Work by Kieran Bingham from Ideas on Board: [RFC/RFT PATCH 0/6] Asynchronous UVC
22. 22 Async UVC (before) static void uvc_video_complete(struct urb *urb) { [..] stream->decode(urb, stream, buf, buf_meta); if ((ret = usb_submit_urb(urb, GFP_ATOMIC)) < 0) { /* ouch */ } }
23. 23 Async UVC (after) static void uvc_video_complete(struct urb *urb) { [..] stream->decode(uvc_urb, buf, buf_meta); if (!uvc_urb->async_operations) { ret = usb_submit_urb(uvc_urb->urb, GFP_ATOMIC); if (ret < 0) return; } INIT_WORK(&uvc_urb->work, uvc_video_copy_data_work); queue_work(stream->async_wq, &uvc_urb->work); }
24. 24 Thank you!