from binary - Let’s scroll the code - Let’s scroll the code - Let’s scroll the code - Let’s scroll the code - Let’s scroll the code - Let’s scroll the code - Let’s scroll the code Agenda 11
from binary - Let’s scroll the code - Let’s scroll the code - Let’s scroll the code - Let’s scroll the code - Let’s scroll the code - Let’s scroll the code - Let’s scroll the code - Inside stdlib Go 1.17.1, OS Darwin (macOS Big Sur), YMMV Agenda 12
there is a problem - But the problem is not so bad - Docker image is more important - if your base image is Ubuntu/Debian - also there is UPX - But what is inside?
there is a problem - But the problem is not so bad - Docker image is more important - if your base image is Ubuntu/Debian - also there is UPX - But what is inside? - Let’s talk about that
$0x3 TEXT main.main(SB) /Users/olegkovalov/code/funf/main.go func main() {} 0x1054be0 c3 RET 0x1054be1 cc INT $0x3 0x1054be2 cc INT $0x3 0x1054be3 cc INT $0x3 Lovely main() 55
$0x3 TEXT main.main(SB) /Users/olegkovalov/code/funf/main.go func main() {} 0x1054be0 c3 RET 0x1054be1 cc INT $0x3 0x1054be2 cc INT $0x3 0x1054be3 cc INT $0x3 Hm...main.main ? 56
// The main goroutine. func main() { g := getg() // ... fn := main_main // make an indirect call, // as the linker doesn't know the address // of the main package when laying down the runtime fn() // ...
CPU architecture (amd64, arm64, mips, …) - *.s - Assembler - 👀 - Let’s check asm_amd64.s - just 2k lines - others are + - 1k - arm64.s is also heavy But that's not all 67
when using // internal linking. This is the entry point for the program from the // kernel for an ordinary -buildmode=exe program. The stack holds the // number of arguments and the C-style argv. TEXT _rt0_amd64(SB),NOSPLIT,$-8 MOVQ 0(SP), DI // argc LEAQ 8(SP), SI // argv JMP runtime·rt0_go(SB) Ah, that the trick 70
stack-based Go ABI (and // in addition there are no calls to this entry point from Go code). TEXT runtime·rt0_go(SB),NOSPLIT|TOPFRAME,$0 // copy arguments forward on an even stack MOVQ DI, AX // argc MOVQ SI, BX // argv // ... // create istack out of the given (operating system) stack. MOVQ $runtime·g0(SB), DI // ... // find out information about the processor we're on MOVL $0, AX CPUID // ... Scroll further 71
Random - Thread sizes and other consts - Envs and cmd params - I mean environment variables и argc & argv - Machine sanity checks - Oh, interesting! runtime/os_<os>.go 77
argc = c argv = v sysargs(c, v) } // os_darwin.go func sysargs(argc int32, argv **byte) { // skip over argv, envv and the first string will be the path n := argc + 1 for argv_index(argv, n) != nil { n++ } executablePath = gostringnocopy(argv_index(argv, n+1)) // ... }
func osinit() { // pthread_create delayed until end of goenvs so that we // can look at the environment first. ncpu = getncpu() physPageSize = getPageSize() }
machine. - P - processor, a resource that is required to execute Go code. - getg() - returns the pointer to the current g - _g_ := getg() - p := getg().m.p.ptr() - tls - thread-local storage Everything else is if’s, atomics & unclear algorithms. Ez. Go runtime cheat sheet 90
10000 // maximum number of m's allowed (or die) // The world starts stopped. worldStopped() moduledataverify() // pclntab is correct stackinit() // mallocinit() fastrandinit() // must run before mcommoninit mcommoninit(_g_.m, -1) schedinit /1 91
10000 // maximum number of m's allowed (or die) // The world starts stopped. worldStopped() moduledataverify() // pclntab is correct stackinit() // mallocinit() fastrandinit() // must run before mcommoninit mcommoninit(_g_.m, -1) cpuinit() // must run before alginit alginit() // maps must not be used before this call modulesinit() // provides activeModules typelinksinit() // uses maps, activeModules itabsinit() // uses activeModules schedinit /1 92
// ... if procresize(procs) != nil { throw("unknown runnable goroutine during bootstrap") } unlock(&sched.lock) // World is effectively started now, as P's can run. worldStarted() // ... } schedinit /2 93
start program MOVQ $runtime·mainPC(SB), AX // entry PUSHQ AX PUSHQ $0 // arg size CALL runtime·newproc(SB) POPQ AX POPQ AX ... // mainPC is a function value for runtime.main, to be passed to newproc. // The reference to runtime.main is made via ABIInternal, since the // actual function (not the ABI0 wrapper) is needed by newproc. DATA runtime·mainPC+0(SB)/8,$runtime·main<ABIInternal>(SB) GLOBL runtime·mainPC(SB),RODATA,$8 A bit of assembler 94
// create a new goroutine to start program MOVQ $runtime·mainPC(SB), AX // entry PUSHQ AX PUSHQ $0 // arg size CALL runtime·newproc(SB) POPQ AX POPQ AX // start this M CALL runtime·mstart(SB) CALL runtime·abort(SB)// mstart should never return RET TEXT runtime·mstart(SB),NOSPLIT|TOPFRAME,$0 CALL runtime·mstart0(SB) RET // not reached
it on the queue of g's waiting to run. // The compiler turns a go statement into a call to this. func newproc(fn *funcval) { gp := getg() pc := getcallerpc() systemstack(func() { newg := newproc1(fn, gp, pc) _p_ := getg().m.p.ptr() runqput(_p_, newg, true) if mainStarted { wakep() } }) } 96
on the queue of g's waiting to run. // The compiler turns a go statement into a call to this. func newproc(fn *funcval) { gp := getg() pc := getcallerpc() systemstack(func() { newg := newproc1(fn, gp, pc) _p_ := getg().m.p.ptr() runqput(_p_, newg, true) if mainStarted { wakep() } }) } newproc 97
Put it on the queue of g's waiting to run. // The compiler turns a go statement into a call to this. func newproc(fn *funcval) { gp := getg() pc := getcallerpc() systemstack(func() { newg := newproc1(fn, gp, pc) _p_ := getg().m.p.ptr() runqput(_p_, newg, true) if mainStarted { wakep() } }) }
// If systemstack is called from the per-OS-thread (g0) stack, or // if systemstack is called from the signal handling (gsignal) stack, // systemstack calls fn directly and returns. // // Otherwise, systemstack is being called from the limited stack // of an ordinary goroutine. In this case, systemstack switches // to the per-OS-thread stack, calls fn, and switches back. // //go:noescape func systemstack(fn func())
goroutine and execute it. // Never returns. func schedule() { _g_ := getg() top: pp := _g_.m.p.ptr() pp.preempt = false var gp *g // get some gp execute(gp, inheritTime) }
you can easily find such lines - x86HasSSE41 = cpu.X86.HasSSE41 - to be honest a lot of them - Wasting our CPU on nothing - every nanosecond counts :( GOAMD64 proposal 110
them are OS/machine checks - type size - available CPU flags - But there is one interesting (then we will scroll through other stdlib packages) What about init()’s ?? 112
helper goroutine func init() { go forcegchelper() } // 5k lines later... // forcegcperiod is the maximum time in nanoseconds between garbage // collections. If we go this long without a garbage collection, one // is forced to run. // // This is a variable for testing purposes. It normally doesn't change. var forcegcperiod int64 = 2 * 60 * 1e9
package net // The build tag "netcgo" forces use of the cgo DNS resolver. // It is the opposite of "netgo". func init() { netCgo = true } // net/conf.go netGo bool // go DNS resolution forced netCgo bool // cgo DNS resolution forced
package net // The build tag "netcgo" forces use of the cgo DNS resolver. // It is the opposite of "netgo". func init() { netCgo = true } // net/conf.go netGo bool // go DNS resolution forced netCgo bool // cgo DNS resolution forced
flag lets you run // go test -run=BrokenTest -httptest.serve=127.0.0.1:8000 // to start the broken server so you can interact with it manually. // ... // isn't really part of our API. Don't depend on this. var serveFlag string func init() { if has(os.Args, "-httptest.serve=") { flag.StringVar(&serveFlag, "httptest.serve", "", "...") } } Inside net/http/httptest 124
p := range oldPools { // ... } for _, p := range allPools { // ... } // The pools with non-empty primary caches now have non-empty // victim caches and no pools have primary caches. oldPools, allPools = allPools, nil } Inside sync 127
database. // If this package is imported anywhere in the program, then if // the time package cannot find tzdata files on the system, // it will use this embedded information. // // Importing this package will increase the size of a program by about // 450 KB. // Inside time/tzdata 130
database. // If this package is imported anywhere in the program, then if // the time package cannot find tzdata files on the system, // it will use this embedded information. // // Importing this package will increase the size of a program by about // 450 KB. // Inside time/tzdata 131
of the timezone database. // If this package is imported anywhere in the program, then if // the time package cannot find tzdata files on the system, // it will use this embedded information. // // Importing this package will increase the size of a program by about // 450 KB. // // This package should normally be imported by a program's main package, // not by a library. Libraries normally shouldn't decide whether to // include the timezone database in a program. // // This package will be automatically imported if you build with // -tags timetzdata. package tzdata
of the timezone database. // If this package is imported anywhere in the program, then if // the time package cannot find tzdata files on the system, // it will use this embedded information. // // Importing this package will increase the size of a program by about // 450 KB. // // This package should normally be imported by a program's main package, // not by a library. Libraries normally shouldn't decide whether to // include the timezone database in a program. // // This package will be automatically imported if you build with // -tags timetzdata. package tzdata
first versions of Go compiler was in C ? :) - Many underscores in var and func names - again the C legacy - Windows, NetBSD, Wasm, JS & Plan9 are special - often requires special treatment 1 slide notes 137
first versions of Go compiler was in C ? :) - Many underscores in var and func names - again the C legacy - Windows, NetBSD, Wasm, JS & Plan9 are special - often requires special treatment - Cgo is not Go, as Rob Pike said - basically explains everything 1 slide notes 138
first versions of Go compiler was in C ? :) - Many underscores in var and func names - again the C legacy - Windows, NetBSD, Wasm, JS & Plan9 are special - often requires special treatment - Cgo is not Go, as Rob Pike said - basically explains everything - Runtime is all about corner cases - that’s my IMHO 1 slide notes 139
The Go Memory Model - https://golang.org/ref/mem - Go 1.2 Runtime Symbol Information (2013) - https://docs.google.com/document/d/1lyPIbmsYbXnpNj57a261hgOYVpNRcgydurVQIyZO z_o/pub - runtime/HACKING.md - https://github.com/golang/go/blob/master/src/runtime/HACKING.md References 146
SiteGround - Reliable hosting with speed, security, and support you can count on.
olegkovalov
macha64
pocke
satohjohn
hjmkth
hirokunimaeta
handlename
syumai
skrb
minodriven
mototakatsu
ioki
yukukotani
tenderlove
shlominoach
holman
cassininazir
jct
raygrieselhuber
eileencodes
danielanewman
kneath
samlambert
hannesfritz
csswizardry
