Static Hermes Tzvetan Mikov, Meta

The Fable of Sad Joe’s Slow Code Joe the RN developer

Joe tried everything he could think of, but was thwarted by fundamental limits on interpreter performance

He wondered, what if I could leverage type information to generate efficient native code ahead of time?

No content

As part of RN’s cross- platform vision, we want to empower developers like Joe to not just create harmless JavaScript crashes, but REAL fast native crashes too!

Introducing Static Hermes ● Optional ahead of time compilation to native code ● C performance with JS usability ● Mix and match bytecode and native code based on your needs ● Requires sound types* for performance

Status/Disclaimer ● Static Hermes is not ready for general usage yet, but we have positive indications of performance ● This is a work in progress, and we are sharing it early to show you what we’ve been working on and gather community feedback ● We want to thank Amazon for adding TypeScript support to Static Hermes.

Table of Contents Challenges of AOT Native JS Compilation Sound Typing Reliable Perf via AOT Native Compilation Zero-cost FFI example: reading SQLite DB Zero-cost detailed FFI example: getenv()

Why Static Hermes? ● Hermes improves startup through bytecode, but there are limits on interpreter performance after startup ● Writing C++ to improve performance is difficult ● RN developers are already accustomed to writing typed JS code ● We want to take advantage of this to bridge predictable C performance with JS usability via native AOT compilation.

Why is it difficult to compile JS ahead-of-time? function addXY(x, y) { return x + y; } • We don’t know the type of x and y ahead of time. • All possible types (string, array, number, etc) and combinations need to be supported. • The result is either a size explosion or interpreter-like performance. • Solutions in academia, specifically HopC, by Manuel Serrano, do not meet our goal (not ruling it out in the future) And why is no one doing it?

Can’t type annotations help? function addXY(x: number, y: number) { return x + y; } • No! Because type annotations in TypeScript and Flow are unsound. • The type annotations do not guarantee that types are correct at runtime. • The compiler cannot rely on type annotations for better code.

Example of type unsoundness function addXY(x: number, y: number) { return x + y; } let a: number[] = [10, 20, 30]; // Uh-oh! Adding undefined + undefined! addXY(a[10], a[11]); Despite the type annotations, both parameters can be non- numbers at runtime.

Hermes + TypeScript/Flow Unsound Types function addXY(x: number, y: number) { return x + y; } let a: number[] = [10, 20, 30]; // Uh-oh! Adding undefined + undefined! addXY(a[10], a[11]); function addXY(x: number, y: number) { return x + y; } let a: number[] = [10, 20, 30]; // RangeError thrown at runtime! addXY(a[10], a[11]); Static Hermes + TypeScript/Flow Sound Types

Sound Types ● Static Hermes modifies some JS semantics to allow efficient sound typing ● The new semantics to improve performance are opt-in on a granular level ● New code can coexist and interop with unmodified code ● Both TypeScript and Flow are supported

Are we breaking JavaScript? ● We are enforcing the semantics the user wanted ● When they wrote x: number, they didn’t intend that x could also be undefined. ● Technically that was a bug which cannot be caught at compile time ● We are strengthening the existing TypeScript and Flow type systems by enforcing them at runtime. ● It should not make a difference for correct code. ● It is all opt-in. The standard behavior is available, if you want it.

Compiling Typed JS to Native

Interpreter vs typed native nbody.js Nbody is a well-known benchmark used in The Computer Language Benchmarks Game, simulating movement of celestial bodies. This is a math heavy benchmark with many property accesses. SH improves runtime from 5511 ms to 565 ms

A loop from nbody.js for (let i: number = 0; i < size; i++) { const body: Body = bodies[i]; body.x += dt * body.vx; body.y += dt * body.vy; body.z += dt * body.vz; }

A loop from nbody.js for (let i: number = 0; i < size; i++) { const body: Body = bodies[i]; body.x += dt * body.vx; body.y += dt * body.vy; body.z += dt * body.vz; } ; body.x += dt * body.vx; ; body.y += dt * body.vy; and x8, x0, #0xffffffffffff ldur q0, [x8, #40] ldr q1, [x8, #64] ldr q2, [sp, #16] fmla.2d v0, v1, v2[0] stur q0, [x8, #40]

A loop from nbody.js for (let i: number = 0; i < size; i++) { const body: Body = bodies[i]; body.x += dt * body.vx; body.y += dt * body.vy; body.z += dt * body.vz; } ; body.x += dt * body.vx; ; body.y += dt * body.vy; and x8, x0, #0xffffffffffff ldur q0, [x8, #40] ldr q1, [x8, #64] ldr q2, [sp, #16] fmla.2d v0, v1, v2[0] stur q0, [x8, #40] ; body.z += dt * body.vz; ldr d0, [x8, #56] ldr d1, [x8, #80] fmadd d0, d2, d1, d0 str d0, [x8, #56]

A loop from nbody.js for (let i: number = 0; i < size; i++) { const body: Body = bodies[i]; body.x += dt * body.vx; body.y += dt * body.vy; body.z += dt * body.vz; } ; body.x += dt * body.vx; ; body.y += dt * body.vy; and x8, x0, #0xffffffffffff ldur q0, [x8, #40] ldr q1, [x8, #64] ldr q2, [sp, #16] fmla.2d v0, v1, v2[0] stur q0, [x8, #40] ; body.z += dt * body.vz; ldr d0, [x8, #56] ldr d1, [x8, #80] fmadd d0, d2, d1, d0 str d0, [x8, #56]

More Benchmarks

Zero-cost FFI Unsafe native extensions Soundly typed JavaScript

A Surprising Benefit: Native Platform Integration Sound typing enables ● Zero-cost FFI ● Direct calling into native APIs ● Platform integration can be implemented entirely in JavaScript ● Safe/unsafe code distinction

Understanding the problem 01 Writing native extensions is complex and error prone. It requires switching between languages. 02 Calling existing native APIs requires writing JSI wrappers in C++. 03 Crossing JSI is not practical when we need high frequency interop with JS. The cost of JSI itself starts to dominate performance.

Zero-cost FFI ● No conversions, indirections, allocations ● Just a C function call ● Call overhead between 15x to 80x lower than JSI. Not percent, times!

Zero-cost FFI ● Native functions can be directly invoked from “unsafe” sections of JS code ● Explicit distinction between “safe” and “unsafe” code ● Native functions and native types are 1st class citizens of the language, enabling no-overhead compilation

Example 1: Reading SQLite DB Fortunately, this was generated automatically from a C header, using our tool ffigen.

Reading SQLite DB in a few lines of JS, no C++ required // Open the database res = _sqlite3_open( stringToAsciiz(”file.db"), dbPtrBuffer ); if (res !== 0) throw Error(res); db = _sh_ptr_read_ptr(dbPtrBuffer, 0); // Prepare the SQL statement sqlPtr = stringToAsciiz( "SELECT id, name FROM my_table” ); res = _sqlite3_prepare_v2( db, sqlPtr, -1, dbPtrBuffer, c_null ); if (res !== 0) throw Error(res); stmt = _sh_ptr_read_ptr(dbPtrBuffer, 0); // Loop through all rows in the table let row = 1; while (_sqlite3_step(stmt) === SQLITE_ROW) { const id = _sqlite3_column_int(stmt, 0); const namePtr = _sqlite3_column_text(stmt, 1); const name = asciizToString(namePtr, 1024); print(`row${row++}:`, id, name); }

It actually works!

Example 2: getenv const _getenv = $SHBuiltin.extern_c({}, function getenv(name: c_ptr): c_ptr {throw 0;}); function getenv(name: string): string { "use unsafe"; let name_z = stringToAsciiz(name); try { let val_z = _getenv(name_z); return asciizToString_unsafe(val_z, 2048); } finally { free(name_z); } } print(getenv("PATH"));

Example: getenv const _getenv = $SHBuiltin.extern_c({}, function getenv(name: c_ptr): c_ptr {throw 0;}); function getenv(name: string): string { "use unsafe"; let name_z = stringToAsciiz(name); try { let val_z = _getenv(name_z); return asciizToString_unsafe(val_z, 2048); } finally { free(name_z); } } print(getenv("PATH")); Declaration of the native external function.

Example: getenv const _getenv = $SHBuiltin.extern_c({}, function getenv(name: c_ptr): c_ptr {throw 0;}); function getenv(name: string): string { "use unsafe"; let name_z = stringToAsciiz(name); try { let val_z = _getenv(name_z); return asciizToString_unsafe(val_z, 2048); } finally { free(name_z); } } print(getenv("PATH")); Convert the name from a JS string to a native C string

Example: getenv const _getenv = $SHBuiltin.extern_c({}, function getenv(name: c_ptr): c_ptr {throw 0;}); function getenv(name: string): string { "use unsafe"; let name_z = stringToAsciiz(name); try { let val_z = _getenv(name_z); return asciizToString_unsafe(val_z, 2048); } finally { free(name_z); } } print(getenv("PATH")); Call the native function.

Example: getenv const _getenv = $SHBuiltin.extern_c({}, function getenv(name: c_ptr): c_ptr {throw 0;}); function getenv(name: string): string { "use unsafe"; let name_z = stringToAsciiz(name); try { let val_z = _getenv(name_z); return asciizToString_unsafe(val_z, 2048); } finally { free(name_z); } } print(getenv("PATH")); Convert the result to a JS string.

Example: getenv const _getenv = $SHBuiltin.extern_c({}, function getenv(name: c_ptr): c_ptr {throw 0;}); function getenv(name: string): string { "use unsafe"; let name_z = stringToAsciiz(name); try { let val_z = _getenv(name_z); return asciizToString_unsafe(val_z, 2048); } finally { free(name_z); } } print(getenv("PATH")); Free the native memory buffer.

Anatomy of an Extern const _getenv = $SHBuiltin.extern_c({}, function getenv(name: c_ptr): c_ptr { throw 0; });

Anatomy of an Extern const _getenv = $SHBuiltin.extern_c({}, function getenv(name: c_ptr): c_ptr { throw 0; }); Options like calling convention

Anatomy of an Extern const _getenv = $SHBuiltin.extern_c({}, function getenv(name: c_ptr): c_ptr { throw 0; }); Name of the imported native function.

Anatomy of an Extern const _getenv = $SHBuiltin.extern_c({}, function getenv(name: c_ptr): c_ptr { throw 0; }); Type of the native parameter

Anatomy of an Extern const _getenv = $SHBuiltin.extern_c({}, function getenv(name: c_ptr): c_ptr { throw 0; }); Native return type

Anatomy of an Extern const _getenv = $SHBuiltin.extern_c({}, function getenv(name: c_ptr): c_ptr { throw 0; }); Throw away body to satisfy the type checker.

Thank you! ● For questions or suggestions, please use the Discussions tab on the Hermes GitHub repo. https://github.com/facebook/hermes/discussions/categories/static-hermes ● nbody.js: https://github.com/facebook/hermes/blob/static_h/benchmarks/nbody/fully-typed/nbody.js ● FFI: https://github.com/facebook/hermes/tree/static_h/examples/ffi