Extending Kubernetes* with Intel® accelerator devices Alexander Kanevskiy Intel Open Source Technology Center * Other names and brands may be claimed as the property of others. 

Agenda • Hardware accelerators: “What?” and “Why?” • How do they work in Kubernetes*? • Intel accelerators • GPU • Intel® QuickAssist Technology (Intel® QAT) • FPGA • Challenges of using hardware accelerators in Kubernetes* * Other names and brands may be claimed as the property of others. 

Accelerators: “What?” and “Why?” • Variety of accelerator devices • SmartNIC • GPU • FPGA • Other specialized co-processors • Highly optimized hardware for specific tasks • Can provide significant performance gain • Saving CPU cycles for other workloads • Require application support 

Hardware Accelerators in Kubernetes* Kubelet Device Plugin Device Plugin gRPC Server Device Plugin Manager gRPC Server Register ListAndWatch Allocate PreStartContainer GetDevicePluginOptions • Alpha in 1.8 • Currently: beta * Other names and brands may be claimed as the property of others. 

Intel® Graphics Technologies • Products • Intel® Core™ processor • Intel® Xeon® processor • Intel® Visual Compute Accelerator • Variants • Intel® UHD Graphics, Intel® Iris® Graphics, Intel® Iris® Plus Graphics • GPU Acceleration use cases • Intel® Media SDK • OpenCL* containers: ... - name: demo-container-1 image: k8s.gcr.io/pause:2.0 resources: limits: gpu.intel.com/i915: 1 * Other names and brands may be claimed as the property of others. Watch Demo! 

Intel® QuickAssist Technology • Accelerator for DPDK* applications • Security • SSL/IPSec • Symmetric Cryptography: AES, KASUMI,… • Asymmetric cryptography: DH, RSA, DSA, ECDSA,… • Digest/hash: MD5, SHA1, SHA2, SHA3,… • Compression • Deflate: LZ77 with gzip or zlib header • Stateless compression and decompression * Other names and brands may be claimed as the property of others. containers: ... - name: demo-container-2 image: k8s.gcr.io/pause:2.0 resources: limits: qat.intel.com/generic: 1 Watch Demo! 

Field Programmable Gate Array (FPGA) • Products • PCIe* programmable acceleration cards • Intel® Arria® 10 GX FPGA • Intel® Stratix® 10 SX FPGA • Intel® Xeon® Scalable processors with integrated FPGA • Workload types • Open Programmable Acceleration Engine • OpenCL* * Other names and brands may be claimed as the property of others. 

Using FPGAs in the Cloud • Usability aspects • Not user friendly IDs • Region (interface) ID • Accelerator Function ID • CRDs for user friendly IDs • Security aspects • User vs. Infra Programming • Bitstream access apiVersion: fpga.intel.com/v1 kind: AcceleratorFunction metadata: name: arria10-compress spec: afuId: 18b79ffa2ee54aa096ef4230dafacb5f apiVersion: fpga.intel.com/v1 kind: FpgaRegion metadata: name: arria10 spec: interfaceId: 9926ab6d6c925a68aabca7d84c545738 resources: limits: fpga.intel.com/arria10-compress: 1 

Using FPGAs in the Cloud Different modes of operation • Pre-programmed • Orchestration programmed containers: ... - name: fpga-container-1 image: k8s.gcr.io/pause:2.0 resources: limits: fpga.intel.com/af-d8424dc4a4a383f9040b: 1 containers: ... - name: fpga-container-2 image: k8s.gcr.io/pause:2.0 resources: limits: fpga.intel.com/arria10-nlb3: 1 Watch Demo! 

FPGA and Kubernetes*: how it works Control Plane Node Kubelet CRI-O PreStart CRI-O Hook FPGA OPAE Kernel driver FPGA Device Plugin etcd Scheduler API Server Controller Manager FPGA Admission Controller Webhook Workload Discovery Device Plugin API Programming CRI * Other names and brands may be claimed as the property of others. 

Challenges of using accelerators in Kubernetes* … and what we should discuss as a community * Other names and brands may be claimed as the property of others. 

Accelerator’s resources challenges Devices with internal resources • We have gaps on all levels • Kubernetes* to CRI • CRI to OCI Runtimes • Runtimes to CGroups • CGroups for accelerator drivers • “Infinite” amount of resources • Multifunctional engine units • “Compatible” resources Accelerator Engine Unit Engine Unit Engine RAM Engine RAM Kubernetes* Workload Workload Workload Workload Workload CRI Runtime CGroups Kernel Driver * Other names and brands may be claimed as the property of others. 

Challenges with GPUs & Intel® QAT • GPUs • Several generations of compatible GPUs • Specific combinations of kernel and user space libraries • Specific to workload ABIs for user space libraries • Intel® QAT • UIO vs. kernel driver APIs • DPDK vs. “normal” applications • OpenSSL* vs. BoringSSL* • Topology aware allocation * Other names and brands may be claimed as the property of others. 

Challenges with FPGAs • Accelerator parameters • Inability in generic way to pass parameters for programming • Allocate() • PreStartContainer() • CRI Hooks • Different parameters for several instances of accelerators • Power management and security • Deallocate() • Complex topology between accelerators, CPUs and memory 

Device topology challenges, simplified Socket 0 Core 0 Core 1 Core 6 Core 7 Core 2 Core 3 Core 8 Core 9 Core 4 Core 5 Core 10 Core 11 PCIe UPI Socket 1 Core 0 Core 1 Core 6 Core 7 Core 2 Core 3 Core 8 Core 9 Core 4 Core 5 Core 10 Core 11 UPI PCIe Memory Controller Memory Controller Memory Controller Memory Controller $ $ $ $ $ $ $ 

Node 0 Node 2 Node 1 Node 3 Device topology challenges in real world Package 0 Core 0 Core 1 Core 6 Core 7 Memory Controller Core 3 Core 8 Memory Controller Core 4 Core 5 Core 10 Core 11 PCIe UPI Package 1 Core 0 Core 1 Core 6 Core 7 Core 3 Core 8 Core 4 Core 5 Core 10 Core 11 UPI PCIe UPI UPI Memory Controller Memory Controller PCIe PCIe UPI UPI DMI DMI Chipset QAT x16 QAT x16 QAT x16 I/O Hub 4x10G NIC 

References Intel Device Plugins for Kubernetes* GitHub* * Other names and brands may be claimed as the property of others. GPU Demo Intel® QAT Demo FPGA Demo Intel Device Plugins for Kubernetes* Demos 

Disclaimer Intel technologies’ features and benefits depend on system configuration and may require enabled hardware, software or service activation. Performance varies depending on system configuration. No computer system can be absolutely secure. Check with your system manufacturer or retailer or learn more at www.intel.com. Intel, the Intel logo, Intel Core, Iris, and Xeon are trademarks of Intel Corporation in the U.S. and/or other countries. OpenCL and the OpenCL logo are trademarks of Apple Inc. used by permission by Khronos. * Other names and brands may be claimed as the property of others. © 2018 Intel Corporation. 

Thank you