Pushing Envoy Beyond the Edge

Slide 1

Slide 1 text

No content

Slide 2

Slide 2 text

JP Simard, Lyft Pushing Envoy Beyond the Edge

Slide 3

Slide 3 text

No content

Slide 4

Slide 4 text

What’s Envoy Mobile? • Mobile networking client • Built on Envoy • Designed for iOS & Android • Runs in-process • Polyglot Library • APIs for C++, Swift, Kotlin,   Objective-C, Java & Python • Open Source, Apache 2.0 • envoymobile.io

Slide 5

Slide 5 text

Why build this? • Bring what makes Envoy great on the server to mobile apps • Rich Observability • Support for latest networking technologies • Consistency: Same networking engine on iOS, Android & Server • Control: Lots of knobs! Open source so we can fix/change things. • Debuggability: Stepping through your networking code and seeing exactly how it works • On iOS, the networking library mostly ships with the OS and is a closed-source black box

Slide 6

Slide 6 text

Timeline • 2016: Envoy is announced • 2019: Work begins on Envoy Mobile • 2019-2021: Envoy Mobile gains mostly feature-parity with the standard networking libraries on iOS & Android • 2021-2022: Production experiments with Lyft apps begin • Mid-2022: Envoy Mobile is serving 98% of all network requests made by Lyft’s mobile apps

Slide 7

Slide 7 text

eng.lyft.com Now Powering Lyft Apps

Slide 8

Slide 8 text

Rich Observability

Slide 9

Slide 9 text

Server vs Mobile Environments Server • Stable • Fast • Single Provider • Control

Slide 10

Slide 10 text

Server vs Mobile Environments Mobile • Unreliable • Inconsistent • Many Carriers • User Configured

Slide 11

Slide 11 text

Carrier Variability

Slide 12

Slide 12 text

Mobile Constraints • Low available memory • Unreliable connectivity • Many carriers • User configurations • Frequently moving between networks and interfaces • Can get killed/backgrounded by the OS at any moment • Can’t SSH into a problematic instance to troubleshoot • New IDEs & distribution processes: Xcode & Android Studio • Binary size is important

Slide 13

Slide 13 text

Mobile Constraints (cont’d) • iOS and Android sometimes behave in surprising ways • Android can block some DNS resolutions when backgrounded and in low power mode • Android uses dual stack sockets, and for some carrier configurations this means IPv4 addresses don’t work • We always force IPv6 on Android for that reason • On iOS, Envoy Mobile can crash when the app is shutting down • Crossing the JNI (Java Native Interface) is cumbersome and costly

Slide 14

Slide 14 text

Binary Size Reduction

Slide 15

Slide 15 text

IDE Integration

Slide 16

Slide 16 text

Recent Developments • Support for VPNs on Android • Required moving from c-ares to getaddrinfo as a DNS resolver • Support for proxies on Android • Mostly complete, running production experiments now • Android platform certificate chain delegation • Next up: Support for proxies on iOS • This is the last big piece of core functionality we’re missing • Next up: QUIC/HTTP3 production experiments • Next up: Improving xDS for mobile use cases

Slide 17

Slide 17 text

Architecture Overview

Slide 18

Slide 18 text

Architecture Overview

Slide 19

Slide 19 text

Architecture Overview

Slide 20

Slide 20 text

Architecture Overview

Slide 21

Slide 21 text

Architecture Overview

Slide 22

Slide 22 text

Architecture Overview

Slide 23

Slide 23 text

Architecture Overview

Slide 24

Slide 24 text

Architecture Overview

Slide 25

Slide 25 text

Architecture Overview

Slide 26

Slide 26 text

Architecture Overview

Slide 27

Slide 27 text

Architecture Overview

Slide 28

Slide 28 text

Contributors Mike José Michael Keith Rafał and dozens more on Envoy Mobile directly, not to mention over a hundred contributors to the main Envoy project Alyssa Alan Ryan Ali

Slide 29

Slide 29 text

Questions? JP Simard • [email protected] • @simjp