Calling Rust from Go, without cgo @ GothamGo 2017

Calling Rust from Go, without cgo @ GothamGo 2017

AKA "Building your own FFI"

https://blog.filippo.io/rustgo

9fdab9d005b82612cadbfe699b541f83?s=128

Filippo Valsorda

October 05, 2017
Tweet

Transcript

  1. 3.

    cgo Foreign function interface (FFI). Call C code from Go.

    • Code generator tool • FFI internals to make C happy • Safety features to make Go less unhappy • Utility libraries and types
  2. 5.

    FFI: starting kit A way to: • Compile the foreign

    code together with the Go code • Jump to the foreign code by function name • Share arguments, return values and memory • Bonus: packaging and redistribution
  3. 7.

    #![no_std] #[no_mangle] pub extern fn multiply_two(a: u64) -> u64 {

    return a * 2; } [package] name = "multiplication" version = "0.0.1" [lib] crate-type = ["staticlib"]
  4. 9.

    $GOROOT/pkg/tool/darwin_amd64/compile \ -o $WORK/.../rustgo.a [...] -pack ./main.go $GOROOT/pkg/tool/darwin_amd64/link \ -o

    $WORK/.../a.out -buildmode=exe [...] \ -linkmode=internal $WORK/.../rustgo.a mv $WORK/.../a.out rustgo go build -x (package main)
  5. 10.

    .go .go .go package .go package cmd/compile cmd/compile .o .o

    .a .o .o .a binary cmd/link symbol "x" symbol "y" symbol "foo" symbol "main" ...
  6. 14.

    $GOROOT/pkg/tool/darwin_amd64/compile \ -o $WORK/.../multiplication.a [...] -pack ./mul.go mv $WORK/.../multiplication.a \

    $GOPATH/pkg/darwin_amd64/.../multiplication.a go build -i -x (package multiplication)
  7. 15.

    .go .go .go package .go package cmd/compile cmd/compile .o .o

    .a .o .o .a binary cmd/link symbol "x" symbol "y" symbol "foo" symbol "main" ...
  8. 16.
  9. 17.
  10. 18.

    ar xv .../libmultiplication.a # the Rust library Stuffing .a files

    go tool pack r \ $GOPATH/pkg/darwin_amd64/.../multiplication.a *.o
  11. 19.

    .o .o .a .o .o the Rust static library .o

    .o .o .o ar cmd/pack .o .o .a .o .o $GOPATH/pkg/.../
  12. 20.

    ld -r -o .../libmultiplication.o --gc-sections \ -u multiply_two .../libmultiplication.a Stuffing

    .a files go tool pack r \ $GOPATH/pkg/darwin_amd64/.../multiplication.a \ .../libmultiplication.o
  13. 21.

    .o .o .a .o .o the Rust static library ld

    -r -u cmd/pack .o .o .a .o .o .o $GOPATH/pkg/.../
  14. 22.

    Compiling the foreign code together What we did: put the

    Rust code with the multiply_two function inside the Go package How we did it: ld -r -u and cmd/pack
  15. 25.
  16. 26.

    //go:cgo_import_static multiply_two //go:cgo_import_dynamic multiply_two //go:linkname multiply_two multiply_two var multiply_two uintptr

    var _multiply_two = &multiply_two .go TEXT MultiplyByTwo(SB), 0, $16384-8 MOVQ ·_multiply_two(SB), AX JMP AX RET .s
  17. 27.

    Calling the foreign symbol What we did: got the linker

    to put a pointer to the Rust function in _multiply_two How we did it: //go:cgo_import_xxx
  18. 29.
  19. 32.

    xxx`main.multiplyByTwo: movq %gs:0x8a0, %rcx leaq -0x8(%rsp), %rax cmpq 0x10(%rcx), %rax

    jbe 0x104f3e1 ; <+129> at main.go:9 [...] Stack size check preamble
  20. 34.
  21. 39.

    The C calling convention Or "sysv64" on x86_64. • Arguments:

    RDI, RSI, RDX, RCX, R8, and R9 • Return value: RAX •Stack alignment: 16 bytes Default for Rust extern functions.
  22. 40.

    TEXT ·MultiplyByTwo(SB), 0, $2048-16 MOVQ n+0(FP), DI // Load the

    argument before messing with SP MOVQ SP, BX // Save SP in a callee-saved registry ADDQ $2048, SP // Rollback SP to reuse this function's frame ANDQ $~15, SP // Align the stack to 16-bytes MOVQ ·_multiply_two(SB), AX CALL AX MOVQ BX, SP // Restore SP MOVQ AX, ret+8(FP) // Place the return value on the stack RET Trampoline
  23. 42.

    TEXT ·MultiplyByTwo(SB), 0, $2048-16 MOVQ n+0(FP), DI // Load the

    argument before messing with SP MOVQ SP, BX // Save SP in a callee-saved registry ADDQ $2048, SP // Rollback SP to reuse this function's frame ANDQ $~15, SP // Align the stack to 16-bytes MOVQ ·_multiply_two(SB), AX CALL AX MOVQ BX, SP // Restore SP MOVQ AX, ret+8(FP) // Place the return value on the stack RET
  24. 44.

    $ make install cargo build --release ld -r -u _multiply_two

    go tool asm go tool compile go tool pack cp multiplication/multiplication.a $GOPATH/pkg/... $ go build . $ ./rustgo 4
  25. 45.

    Go small stacks Each goroutine has its own stack. When

    it runs out, the preamble calls morestack. We don't, so we have to fit in a fixed stack size. (This is not ok.) TEXT ·MultiplyByTwo(SB), 0, $2048-16
  26. 46.

    Args, returns and calling conventions What we did: made a

    trampoline from the Go calling convention to the C one. How we did it: sacrilegious assembly.
  27. 48.

    //go:binary-only-package package multiplication import _ "unsafe" //go:cgo_import_static multiply_two //go:cgo_import_dynamic multiply_two

    //go:linkname multiply_two multiply_two var multiply_two uintptr var _multiply_two = &multiply_two func MultiplyByTwo(n uint64) uint64