Upgrade to PRO for Only $50/Year—Limited-Time Offer! 🔥

Calling Rust from Go, without cgo @ GothamGo 2017

Calling Rust from Go, without cgo @ GothamGo 2017

AKA "Building your own FFI"

https://blog.filippo.io/rustgo

Filippo Valsorda

October 05, 2017
Tweet

More Decks by Filippo Valsorda

Other Decks in Programming

Transcript

  1. cgo Foreign function interface (FFI). Call C code from Go.

    • Code generator tool • FFI internals to make C happy • Safety features to make Go less unhappy • Utility libraries and types
  2. FFI: starting kit A way to: • Compile the foreign

    code together with the Go code • Jump to the foreign code by function name • Share arguments, return values and memory • Bonus: packaging and redistribution
  3. #![no_std] #[no_mangle] pub extern fn multiply_two(a: u64) -> u64 {

    return a * 2; } [package] name = "multiplication" version = "0.0.1" [lib] crate-type = ["staticlib"]
  4. $GOROOT/pkg/tool/darwin_amd64/compile \ -o $WORK/.../rustgo.a [...] -pack ./main.go $GOROOT/pkg/tool/darwin_amd64/link \ -o

    $WORK/.../a.out -buildmode=exe [...] \ -linkmode=internal $WORK/.../rustgo.a mv $WORK/.../a.out rustgo go build -x (package main)
  5. .go .go .go package .go package cmd/compile cmd/compile .o .o

    .a .o .o .a binary cmd/link symbol "x" symbol "y" symbol "foo" symbol "main" ...
  6. $GOROOT/pkg/tool/darwin_amd64/compile \ -o $WORK/.../multiplication.a [...] -pack ./mul.go mv $WORK/.../multiplication.a \

    $GOPATH/pkg/darwin_amd64/.../multiplication.a go build -i -x (package multiplication)
  7. .go .go .go package .go package cmd/compile cmd/compile .o .o

    .a .o .o .a binary cmd/link symbol "x" symbol "y" symbol "foo" symbol "main" ...
  8. ar xv .../libmultiplication.a # the Rust library Stuffing .a files

    go tool pack r \ $GOPATH/pkg/darwin_amd64/.../multiplication.a *.o
  9. .o .o .a .o .o the Rust static library .o

    .o .o .o ar cmd/pack .o .o .a .o .o $GOPATH/pkg/.../
  10. ld -r -o .../libmultiplication.o --gc-sections \ -u multiply_two .../libmultiplication.a Stuffing

    .a files go tool pack r \ $GOPATH/pkg/darwin_amd64/.../multiplication.a \ .../libmultiplication.o
  11. .o .o .a .o .o the Rust static library ld

    -r -u cmd/pack .o .o .a .o .o .o $GOPATH/pkg/.../
  12. Compiling the foreign code together What we did: put the

    Rust code with the multiply_two function inside the Go package How we did it: ld -r -u and cmd/pack
  13. //go:cgo_import_static multiply_two //go:cgo_import_dynamic multiply_two //go:linkname multiply_two multiply_two var multiply_two uintptr

    var _multiply_two = &multiply_two .go TEXT MultiplyByTwo(SB), 0, $16384-8 MOVQ ·_multiply_two(SB), AX JMP AX RET .s
  14. Calling the foreign symbol What we did: got the linker

    to put a pointer to the Rust function in _multiply_two How we did it: //go:cgo_import_xxx
  15. xxx`main.multiplyByTwo: movq %gs:0x8a0, %rcx leaq -0x8(%rsp), %rax cmpq 0x10(%rcx), %rax

    jbe 0x104f3e1 ; <+129> at main.go:9 [...] Stack size check preamble
  16. The C calling convention Or "sysv64" on x86_64. • Arguments:

    RDI, RSI, RDX, RCX, R8, and R9 • Return value: RAX •Stack alignment: 16 bytes Default for Rust extern functions.
  17. TEXT ·MultiplyByTwo(SB), 0, $2048-16 MOVQ n+0(FP), DI // Load the

    argument before messing with SP MOVQ SP, BX // Save SP in a callee-saved registry ADDQ $2048, SP // Rollback SP to reuse this function's frame ANDQ $~15, SP // Align the stack to 16-bytes MOVQ ·_multiply_two(SB), AX CALL AX MOVQ BX, SP // Restore SP MOVQ AX, ret+8(FP) // Place the return value on the stack RET Trampoline
  18. TEXT ·MultiplyByTwo(SB), 0, $2048-16 MOVQ n+0(FP), DI // Load the

    argument before messing with SP MOVQ SP, BX // Save SP in a callee-saved registry ADDQ $2048, SP // Rollback SP to reuse this function's frame ANDQ $~15, SP // Align the stack to 16-bytes MOVQ ·_multiply_two(SB), AX CALL AX MOVQ BX, SP // Restore SP MOVQ AX, ret+8(FP) // Place the return value on the stack RET
  19. $ make install cargo build --release ld -r -u _multiply_two

    go tool asm go tool compile go tool pack cp multiplication/multiplication.a $GOPATH/pkg/... $ go build . $ ./rustgo 4
  20. Go small stacks Each goroutine has its own stack. When

    it runs out, the preamble calls morestack. We don't, so we have to fit in a fixed stack size. (This is not ok.) TEXT ·MultiplyByTwo(SB), 0, $2048-16
  21. Args, returns and calling conventions What we did: made a

    trampoline from the Go calling convention to the C one. How we did it: sacrilegious assembly.
  22. //go:binary-only-package package multiplication import _ "unsafe" //go:cgo_import_static multiply_two //go:cgo_import_dynamic multiply_two

    //go:linkname multiply_two multiply_two var multiply_two uintptr var _multiply_two = &multiply_two func MultiplyByTwo(n uint64) uint64