No content

Agenda - Problems of LINE app’s network stack - What we did to revamp the network stack - What we achieved with this project

େੴ ক๜ Masakuni Oishi LINE Communication Platform Engineering

Background - SPDY protocol + our own push mechanism - It was introduced in the early days of LINE app for efficient API calls. - But, the successor protocols, HTTP/2 and HTTP/3, have been standardized. SPDY should be replaced by them. - The code had become difficult to maintain. - Because, we had made our own customizations to Netty library. The network stack of LINE Android app had several problems

Project Goals - Migrate SPDY to HTTP/2 - In addition, consider upgrading to HTTP/3 in the future. - Comply with standard specifications - Reduce our own customizations as much as possible. - Safely complete the migration of the network stack - A failure in the network stack may affect a huge number of LINE users.

Difficulties - Migrate all API calls in the app “at once”. - Implement bidirectional communication with the server over HTTP/2. - Release incrementally in a way that allows for rollback.

Why we need to migrate AT ONCE SPDY protocol stack has a connection pool. Server Features SPDY stack Connection pool API X API X over SPDY Client app

Why we need to migrate AT ONCE HTTP/2 protocol stack also has its own connection pool. Server Features HTTP/2 stack Connection pool API Y Client app API Y over HTTP/2

Why we need to migrate AT ONCE If both SPDY and HTTP/2 are used at the same time, the number of TCP connections to the server will be doubled. Server Features SPDY stack Connection pool API X API X over SPDY HTTP/2 stack Connection pool API Y API Y over HTTP/2 Client app

Solution Introduced a common network layer so that the protocol to be used can be switched by a feature flag. Features SPDY stack Connection pool API X,Y HTTP/2 stack Connection pool Common network layer Switch by a feature flag

Bidirectional communication with server - For real-time chatting and other purposes, LINE client app uses bidirectional communication with the server. - Previously, LINE app used a mechanism called “SPDY Push”, which utilizes server-initiated streams of the SPDY protocol.

SPDY Push When the server pushes a message to the client, it is sent as a server-initiated stream (SYN_STREAM + DATA frames) on the SPDY connection. Server Client SPDY stack SPDY connection SYN_STREAM + DATA

Problems of SPDY Push - Most network libraries do not support server-initiated streams. - For LINE Android app, we forked Netty library and made our own customizations. - These customizations were the main cause of making the code difficult to maintain. - Therefore, for HTTP/2, we have developed a new mechanism called “Streaming Push”.

Streaming Push Client makes a call of HTTP POST method, and uses its request body and response body as streams, respectively. Server Client HTTP/2 stack HTTP POST method Request body → ← Response body

Implementation of Streaming Push - The mechanism of Streaming Push is a proven one that is also used in gRPC. - It fully follows the semantics of HTTP/2. - In LINE Android app, Streaming Push is implemented on top of the bare, un-customized OkHttp library.

Prepare for rollout - SPDY and HTTP/2 protocol stack can be switched by a feature flag. - SPDY Push and Streaming Push are also switched by this flag. - We added a mechanism that allows this flag to be changed for each user by server-side settings.

Test and Rollout We spent more than a month conducting in-house tests of HTTP/2 protocol, gradually increasing the number of test targets. Finally, we rolled out the HTTP/2 protocol to all users and completed it successfully without any trouble. LINE client developers All LINE employees 5% of users 100% of users

What we achieved with this project - LINE Android app has completely migrated from SPDY to HTTP/2. - A new push mechanism that complies with HTTP/2 semantics has been introduced. - Successfully completed the rollout without any trouble. - Client-side code is now in a modern architecture using OkHttp and Kotlin Coroutines.

QUIC and HTTP/3… - The semantics of HTTP/3 have not changed from HTTP/2. - Streaming Push mechanism should also work on HTTP/3. - Currently, OkHttp does not support QUIC and HTTP/3. However, when OkHttp supports them, LINE Android app should only need to upgrade the version of the OkHttp library.

OSS library created for this project - We released an OSS library created for this project. - Lich OkHttp & Lich Thrift : https://github.com/line/lich - Coroutine-aware extensions for OkHttp & Apache Thrift - e.g. Perform a resumable download using Range requests.

Thank you