Compromising Readability in Search of Performance

We review a series of actual changes we've made the bleve and vellum projects to improve performance. Most of these relate to reducing memory allocations at runtime, so we look a little closer at those techniques. Finally, we consider the cost of readability to the codebase when we accept these changes.

Transcript

1. COMPROMISING READABILITY
2. Performance
3. USING PPROF FLAME GRAPH
4. USING PPROF FLAME GRAPH
5. USING PPROF FLAME GRAPH
6. RUNTIME SIGNATURES TO LOOK FOR makeslice growslice someslice := make([]*thing, 1024) someslice = append(someslice, anotherslice) newobject something := &thing{…}
8. REDUCE THE NEED TO GROW A SLICE Old version, calls to append() may have to grow slice
9. REDUCE THE NEED TO GROW A SLICE New version, original allocation is sufficient, no need to grow.
10. REDUCING ALLOCATION BY INTRODUCING A BACKING SLICE Old version, allocs new object every time through the loop
11. REDUCING ALLOCATION BY INTRODUCING A BACKING SLICE New version, allocates all objects in single call to runtime
12. REUSE (EXTERNAL) Old version always returns new instance of PostingsList
13. REUSE (EXTERNAL) New version let's caller decide if/when/how to reuse a PostingsList
14. REUSE (INTERNAL) Old version always returns new instance of DictEntry
15. REUSE (INTERNAL) New version ALWAYS reuses the same DictEntry
16. REUSING INTERNAL VS EXTERNAL Internal External Pro Simpler API Caller has full control of reuse (or never use it at all by passing in nil) Con Lifcecycle of reuse is dictated by the component Clutters the API NOTE: in both cases the caller can misuse and get into trouble
17. REUSE OFTEN REQUIRES RESET struct slice *thing = Thing{} someslice = someslice[:0] All struct members go back to zero-value. Slice len set to 0, cap remains
18. RESET IN PRACTICE Hang on to slice reference, reset struct to zero values, reset slice, associate struct reference to reset slice.
19. REUSING VIA POOLS (MANUAL) ALLOCATE IN BLOCKS OF 256 AT A TIME RESET ITEM ON WAY OUT OF POOL
20. REUSING VIA SYNC.POOL NOTICE this is package level variable
21. REUSING VIA SYNC.POOL … do work reusing vcd (visit document context) get from pool return to pool PRO: reuse spans goroutines PRO: runtime reclaims items in the pool during GC CON: hard to use well in practice
22. NECESSARY COMPLEXITY ▸ Everything you've seen here is real from Couchbase's bleve/vellum projects ▸ As ugly as they are, they've all shown measurable improvements to important metrics @mschoch marty.schoch@gmail.com