Integrating Service Mesh with Kubernetes-based connected vehicle platform

Slide 1

Slide 1 text

June 2020 / Digital Innovation, Engineering Research & Development ŏ DENSO CORPORATION All Rights Reserved. Integrating Service Mesh with Kubernetes-based connected vehicle platform DENSO CORPORATION Digital Innovation Div JunKai Yong  Tomoya Amachi / @tomoyamachi

Slide 2

Slide 2 text

May 2020 / Digital Innovation, Engineering Research & Development ŏ DENSO CORPORATION All Rights Reserved. Agenda 2 Introduction of DENSO and our team What we are building Technical details and demonstration

Slide 3

Slide 3 text

May 2020 / Digital Innovation, Engineering Research & Development ŏ DENSO CORPORATION All Rights Reserved. Introduction JunKai Yong/@yjk_miloaddict 3 Software Engineer@DENSO CORPORATION Digital Innovation Div. Graduated from Nanyang Technological University (NTU) in May 2018 Joined DENSO in Oct 2018. Working on IoT and Service-Mesh related topics

Slide 4

Slide 4 text

May 2020 / Digital Innovation, Engineering Research & Development ŏ DENSO CORPORATION All Rights Reserved. Introduction Tomoya Amachi/@tomoyamachi 4 Software Engineer Joined misaki project as part of CREATIONLINE team

Slide 5

Slide 5 text

Slide 6

Slide 6 text

May 2020 / Digital Innovation, Engineering Research & Development ŏ DENSO CORPORATION All Rights Reserved. 6 As a global manufacturer of automotive parts, DENSO is expanding its business operations around the world. DENSO supplies a wide range of products and systems as a Tier 1* supplier trusted by the world’s automobile manufacturers. World’s No.2 Consolidated net sales of 5.2 trillion yen. About DENSO

Slide 7

Slide 7 text

Slide 8

Slide 8 text

Slide 9

Slide 9 text

May 2020 / Digital Innovation, Engineering Research & Development ŏ DENSO CORPORATION All Rights Reserved. 9 Why it is difficult to develop vehicle applications? Applications on vehicle is becoming more and more important! Develop application on vehicles

Slide 10

Slide 10 text

May 2020 / Digital Innovation, Engineering Research & Development ŏ DENSO CORPORATION All Rights Reserved. Develop application on vehicles 10 There are many ECU(Electric control unit)s in a car. rade r accessmete r rain sensor camera battery sensor ECU Difficult to Develop - Need to learn about embedded systems.

Slide 11

Slide 11 text

May 2020 / Digital Innovation, Engineering Research & Development ŏ DENSO CORPORATION All Rights Reserved. Difficult to Develop - Need to learn about embedded systems. 11 There are many ECU(Electric control unit)s in a car. Not using linux in most cases. CPU/Memory resources are limited and distributed. Develop application on vehicles

Slide 12

Slide 12 text

May 2020 / Digital Innovation, Engineering Research & Development ŏ DENSO CORPORATION All Rights Reserved. No easy way to update applications on many cars simultaneously. 12 It's not always possible to run the application on your PC as it is. Develop application on vehicles Difficult to Develop - Need to learn about embedded systems.

Slide 13

Slide 13 text

May 2020 / Digital Innovation, Engineering Research & Development ŏ DENSO CORPORATION All Rights Reserved. Network connection can be unstable Because cars are moving around! 13 If connection breaks.. Cars are like highly distributed system. Developers have to deal with network disconnection for their apps. (retries/store data locally…) Develop application on vehicles Difficult to Develop - Unstable Network.

Slide 14

Slide 14 text

May 2020 / Digital Innovation, Engineering Research & Development ŏ DENSO CORPORATION All Rights Reserved. But application on vehicle is becoming more and more important! 14 It’s important to handle data on edge and and keep updating the application to minimize communications expenses. e.g: Picture / Video / Traffic Data / Metadata The Global Connected Car Market size is projected to reach USD 212.7 billion by 2027, from an estimated value of USD 42.6 billion in 2019, at a CAGR of 22.3%. Current vehicles are like feature phone at 2000s. Smart "vehicle" era is coming soon. "Applications" will be a selling point of smart vehicles same as smart phone. Develop application on vehicles https://www.marketsandmarkets.com/Market-Reports/connected-car-market-102580117.html

Slide 15

Slide 15 text

May 2020 / Digital Innovation, Engineering Research & Development ŏ DENSO CORPORATION All Rights Reserved. What we are building? Kubernetes based connected vehicle platform! 15 Minimize the need to learn about embedded systems. Easy to deploy/update application to vehicles. Remove the need for application developers to handle network issues. It's still a prototype.

Slide 16

Slide 16 text

Slide 17

Slide 17 text

May 2020 / Digital Innovation, Engineering Research & Development ŏ DENSO CORPORATION All Rights Reserved. Architecture Overview 17 App Edge Cloud App K8s Agent VPN VPN Applying Manifest Choose HELM Chart Generate K8s Manifest List Share Ideal Manifest b/w Edge and Cloud DigitalTwin API UI K8s Agent

Slide 18

Slide 18 text

May 2020 / Digital Innovation, Engineering Research & Development ŏ DENSO CORPORATION All Rights Reserved. Why Kubernetes? Flexibility of resources 18 radar accessmete r rain sensor battery sensor ECU ECUs will unified into several High Performance ECU in the future. We can use it as k8s worker node.

Slide 19

Slide 19 text

May 2020 / Digital Innovation, Engineering Research & Development ŏ DENSO CORPORATION All Rights Reserved. Flexibility of resources 19 radar accessmete r rain sensor battery sensor High Performance ECU Easy to orchestrate and manage ECU resources. These ECUs will be separated from the basic operation of the car (running, stopping, turning) Why Kubernetes?

Slide 20

Slide 20 text

May 2020 / Digital Innovation, Engineering Research & Development ŏ DENSO CORPORATION All Rights Reserved. High performance ECU: NVIDIA Jetson AGX Xavier Developer Kit 20 https://www.nvidia.com/en-us/autonomous-machines/embedded-systems/jetson-xavier-nx/ Why Kubernetes?

Slide 21

Slide 21 text

May 2020 / Digital Innovation, Engineering Research & Development ŏ DENSO CORPORATION All Rights Reserved. 21 Edge A Cloud High Spec Car We can use resources on cloud. We can run same applications even if edge resources are not enough We can run container image in different vehicles and cloud. Flexibility of resources B C VPN A Low Spec Car B C VPN Why Kubernetes?

Slide 22

Slide 22 text

Slide 23

Slide 23 text

Slide 24

Slide 24 text

Slide 25

Slide 25 text

May 2020 / Digital Innovation, Engineering Research & Development ŏ DENSO CORPORATION All Rights Reserved. Technical details about Misaki orchestrator 25 App Edge Cloud App K8s Agent VPN Apply Manifest To Cluster Choose HELM Chart Generate K8s Manifest List Share Ideal Manifest b/w Edge and Cloud DigitalTwin API UI 4 3 2 1 4 components

Slide 26

Slide 26 text

May 2020 / Digital Innovation, Engineering Research & Development ŏ DENSO CORPORATION All Rights Reserved. 26 Misaki orchestrator Applications Repository Data Processcor Data Uploader Sensory data APP1 APP2 APP3 APP4 APP1 APP2 APP3 APP4 https://www.flaticon.com/authors/vitaly- gorbachev https://www.flaticon.com/authors/ xnimrodx https://www.flaticon.com/authors/good- ware How it works:

Slide 27

Slide 27 text

May 2020 / Digital Innovation, Engineering Research & Development ŏ DENSO CORPORATION All Rights Reserved. 27 Demo: Misaki orchestrator Video scenario GPS manager Can manager Deleted Service: CAN-uploader-for-prius-phv Pod 3 Pod 4 Pod 1 Pod 2 Local Broker Redis to Ambass GPS manager Can manager New service deployed: via-containers-for-prius Pod 3 Pod 4 Pod 1 Pod 2 Local Broker Redis to Ambass Pod 5 File uploader Pod 6 Ring Buffer Pod 7 Steady video Pod 8 Video segment

Slide 28

Slide 28 text

Slide 29

Slide 29 text

May 2020 / Digital Innovation, Engineering Research & Development ŏ DENSO CORPORATION All Rights Reserved. What is Service Mesh? 29 Traffic control (load balancing, routing rules, failovers etc...) Policy control Secure communications Service monitoring (metrics, logs, traces) https://www.abhishek-tiwari.com/a-sidecar-for-your-service-mesh/ A dedicated infrastructure layer mainly for microservices

Slide 30

Slide 30 text

Slide 31

Slide 31 text

May 2020 / Digital Innovation, Engineering Research & Development ŏ DENSO CORPORATION All Rights Reserved. Why Service Mesh? 31 App Proxy Service-mesh decouples network concerns from applications. https://philcalcado.com/img/service-mesh/mesh1.png https://qiita.com/Ladicle/items/4ba57078128d6affadd5 Application developers can focus on their business values/logics Applications no longer needs to implement distributed system practices (i.e timeouts, service discoveries etc.)

Slide 32

Slide 32 text

Slide 33

Slide 33 text

May 2020 / Digital Innovation, Engineering Research & Development ŏ DENSO CORPORATION All Rights Reserved. Technical Detail about Misaki Service Mesh Architecture 33 • Provides policies and configurations to each Envoy • Connects individual envoys into a distributed system. Routing the application to appropriate service(s) Envoy: Cloud node Edge node Queue Filter Queue Filter Misaki Control Plane App A App B App C Proxy

Slide 34

Slide 34 text

Slide 35

Slide 35 text

May 2020 / Digital Innovation, Engineering Research & Development ŏ DENSO CORPORATION All Rights Reserved. Technical Detail about Misaki Control Plane What is needed to deploy an app? 35 K8s deployment manifest Envoy Custom Resource Definitions (CRD)

Slide 36

Slide 36 text

May 2020 / Digital Innovation, Engineering Research & Development ŏ DENSO CORPORATION All Rights Reserved. Technical Detail about Misaki Control Plane Deployment Manifest 36 Create, modify or delete K8s resources (i.e pods, deployment) Commonly written in .yaml - App - Envoy side-car - Control plane configmap - Initialization settings

Slide 37

Slide 37 text

May 2020 / Digital Innovation, Engineering Research & Development ŏ DENSO CORPORATION All Rights Reserved. Technical Detail about Misaki Control Plane Custom Resource Definition (CRD) 37 Custom k8s API endpoint that stores a collection of API objects I.E route CRD contains route CRD contains all the route information values

Slide 38

Slide 38 text

May 2020 / Digital Innovation, Engineering Research & Development ŏ DENSO CORPORATION All Rights Reserved. Technical Detail about Misaki Control Plane Misaki Control Plane 38 Custom k8s controller Assign appropriate settings to envoy side-car of newly deployed application, or when changes are made to an existing application

Slide 39

Slide 39 text

May 2020 / Digital Innovation, Engineering Research & Development ŏ DENSO CORPORATION All Rights Reserved. Technical Detail about Misaki Control Plane Misaki Control Plane 39 App C Proxy New k8s deployment Misaki Control Plane App C CRD Obtain Pod.Spec information 1 Retrieve App C’s CRD Generates envoy setting snapshot and sets it to the envoy side-car of the new app 3 2 App B Proxy App A Proxy

Slide 40

Slide 40 text

Slide 41

Slide 41 text

May 2020 / Digital Innovation, Engineering Research & Development ŏ DENSO CORPORATION All Rights Reserved. What’s going on normal service-mesh Data Plane? 41 https://www.slideshare.net/ThomasGraf5/accelerating-envoy-and-istio-with-cilium-and-the-linux-kernel

Slide 42

Slide 42 text

May 2020 / Digital Innovation, Engineering Research & Development ŏ DENSO CORPORATION All Rights Reserved. What’s going on Misaki Data Plane? 42 Buffering HTTP Request from application when network disconnect or not applicable network interface(3g, 4g, wifi etc). Re-sends the HTTP Request/streaming-data to server when network available again. Store HTTP Request from application if network did not recover during buffering duration.

Slide 43

Slide 43 text

May 2020 / Digital Innovation, Engineering Research & Development ŏ DENSO CORPORATION All Rights Reserved. Prototype Outbounds on Misaki Data Plane 43 volume App Proxy Proxy Server Online 1 Offline 2 volume App Proxy Proxy Server Name resolve in /etc/hosts Recover 3 volume App Proxy Proxy Server

Slide 44

Slide 44 text

May 2020 / Digital Innovation, Engineering Research & Development ŏ DENSO CORPORATION All Rights Reserved. Problems 44 Server footprint is not small Not supported Dynamic DNS resolve Network security problems

Slide 45

Slide 45 text

May 2020 / Digital Innovation, Engineering Research & Development ŏ DENSO CORPORATION All Rights Reserved. Current Outbounds on Misaki Data Plane 45 volume App Proxy volume App Proxy Local DNS Local DNS Online 1 Offline 2 Recover during buffering time 3 App Proxy Local DNS

Slide 46

Slide 46 text

May 2020 / Digital Innovation, Engineering Research & Development ŏ DENSO CORPORATION All Rights Reserved. Current Outbounds on Misaki Data Plane 46 Recover after buffering time 4 volume App Proxy Local DNS Replay Async response volume App Proxy Local DNS Replay Next request 5

Slide 47

Slide 47 text

May 2020 / Digital Innovation, Engineering Research & Development ŏ DENSO CORPORATION All Rights Reserved. Troubles 47 Envoy does not suppose offline communications. We need to customize Envoy code (written by C++11). Network security problems Create an original DNS Resolver (based on tomoyamachi/go-dnsmasq) All component images need to build for cross platforms(ARM, AMD etc).

Slide 48

Slide 48 text

Slide 49

Slide 49 text

May 2020 / Digital Innovation, Engineering Research & Development ŏ DENSO CORPORATION All Rights Reserved. Summary 49 We developed a k8s-based connected vehicle platform prototype Misaki service-mesh removes the need for developers to handle network problems But there is still many challenges to be solved.

Slide 50

Slide 50 text

May 2020 / Digital Innovation, Engineering Research & Development ŏ DENSO CORPORATION All Rights Reserved. Challenges 50 Misaki service-mesh currently only support simple HTTP/1.1 and HTTP/2.0. We want it to support more protocols (WebSocket, gRPC etc) Misaki control plane needs to be more flexible Our journey is just beginning! Minimizing cpu/memory usage on edge devices.

Slide 51

Slide 51 text

May 2020 / Digital Innovation, Engineering Research & Development ŏ DENSO CORPORATION All Rights Reserved. Visit us 51 misaki-io currently empty, but we have a plan to make it open source this year Thank You