Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Go memory allocation

Deepu K Sasidharan
February 14, 2020
1.3k

Go memory allocation

Deepu K Sasidharan

February 14, 2020
Tweet

Transcript

  1. Process Virtual Memory
    Resident set
    mheap
    P1(Logical Processor/Hardware Thread)
    Goroutine 1
    stack
    arena
    arena ...
    mcentral mcentral mcentral
    ..
    mcache mspan
    8b
    mspan
    16b
    mspan
    32b
    mspan
    ..
    mspan
    ..
    mspan
    ..
    mspan
    ..
    mspan
    ..
    mspan
    32768b
    mspan 8b (non empty)
    mspan 8b (empty)
    mspan 16b
    mspan 16b
    mspan
    mspan
    P2
    G6
    stack
    mcache
    ..
    ..
    ..
    ..
    Pn
    Gn
    stack
    mcache
    mspan 8b
    mspan 8b
    mspan 8b
    mspan 8b
    mspan 8b
    mspan 8b

    View Slide

  2. mspan
    mspan 8b 8b 8b 8b …. 8b 8b 8b 1024 objects
    mspan 16b 16b 16b 16b …. 16b 16b 16b 512 objects
    mspan 32b 32b 32b 32b …. 32b 32b 32b 256 objects
    mspan 48b 48b 48b 48b …. 48b 48b 48b 170 objects
    mspan 32768 bytes 1 object
    mspan .. .. .. .. …. .. .. .. ...

    View Slide

  3. Go Scheduler
    P
    M
    G
    Processor
    Core
    G G
    P = Logical Processor per Hardware Thread
    M = Machine per OS Thread
    G = Goroutine (Coroutine)
    LRQ = Local Run Queue
    LRQ

    View Slide

  4. Tiny allocation

    View Slide

  5. Tiny allocation
    P1
    M
    G2
    Processor
    Core
    P = Logical Processor per Hardware Thread
    M = Machine per OS Thread
    G = Goroutine (Coroutine)
    mcache
    mheap
    arena
    mcentral
    mspan 8b (non
    empty)
    mspan 8b
    (empty)
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mcentral
    mspan 16b (non
    empty)
    mspan 16b
    (empty)
    mspan
    16b
    mspan
    16b
    mspan
    16b
    mspan
    16b
    mspan
    16b
    mspan
    16b
    mspan
    8b
    mspan
    16b
    mspan
    16b
    Tiny allocation slot
    12b

    View Slide

  6. Tiny allocation
    P1
    M
    G2
    Processor
    Core
    P = Logical Processor per Hardware Thread
    M = Machine per OS Thread
    G = Goroutine (Coroutine)
    mcache
    mheap
    arena
    mcentral
    mspan 8b (non
    empty)
    mspan 8b
    (empty)
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mcentral
    mspan 16b (non
    empty)
    mspan 16b
    (empty)
    mspan
    16b
    mspan
    16b
    mspan
    16b
    mspan
    16b
    mspan
    16b
    mspan
    16b
    mspan
    8b
    mspan
    16b
    mspan
    16b
    Tiny allocation slot
    12b
    4b

    View Slide

  7. Tiny allocation
    P1
    M
    G2
    Processor
    Core
    P = Logical Processor per Hardware Thread
    M = Machine per OS Thread
    G = Goroutine (Coroutine)
    mcache
    mheap
    arena
    mcentral
    mspan 8b (non
    empty)
    mspan 8b
    (empty)
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mcentral
    mspan 16b (non
    empty)
    mspan 16b
    (empty)
    mspan
    16b
    mspan
    16b
    mspan
    16b
    mspan
    16b
    mspan
    16b
    mspan
    16b
    mspan
    8b
    mspan
    16b
    mspan
    16b
    Tiny allocation slot
    12b 4b

    View Slide

  8. Tiny allocation
    P1
    M
    G2
    Processor
    Core
    P = Logical Processor per Hardware Thread
    M = Machine per OS Thread
    G = Goroutine (Coroutine)
    mcache
    mheap
    arena
    mcentral
    mspan 8b (non
    empty)
    mspan 8b
    (empty)
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mcentral
    mspan 16b (non
    empty)
    mspan 16b
    (empty)
    mspan
    16b
    mspan
    16b
    mspan
    16b
    mspan
    16b
    mspan
    16b
    mspan
    16b
    mspan
    8b
    mspan
    16b
    mspan
    16b
    Tiny allocation slot
    12b 4b
    10b

    View Slide

  9. Tiny allocation
    P1
    M
    G2
    Processor
    Core
    P = Logical Processor per Hardware Thread
    M = Machine per OS Thread
    G = Goroutine (Coroutine)
    mcache
    mheap
    arena
    mcentral
    mspan 8b (non
    empty)
    mspan 8b
    (empty)
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mcentral
    mspan 16b (non
    empty)
    mspan 16b
    (empty)
    mspan
    16b
    mspan
    16b
    mspan
    16b
    mspan
    16b
    mspan
    16b
    mspan
    16b
    mspan
    8b
    mspan
    16b
    mspan
    16b
    Tiny allocation slot
    12b 4b
    10b

    View Slide

  10. Tiny allocation
    P1
    M
    G2
    Processor
    Core
    P = Logical Processor per Hardware Thread
    M = Machine per OS Thread
    G = Goroutine (Coroutine)
    mcache
    mheap
    arena
    mcentral
    mspan 8b (non
    empty)
    mspan 8b
    (empty)
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mcentral
    mspan 16b (non
    empty)
    mspan 16b
    (empty)
    mspan
    16b
    mspan
    16b
    mspan
    16b
    mspan
    16b
    mspan
    16b
    mspan
    16b
    mspan
    8b
    mspan
    16b
    mspan
    16b
    Tiny allocation slot
    12b 4b 10b

    View Slide

  11. Tiny allocation
    P1
    M
    G2
    Processor
    Core
    P = Logical Processor per Hardware Thread
    M = Machine per OS Thread
    G = Goroutine (Coroutine)
    mcache
    mheap
    arena
    mcentral
    mspan 8b (non
    empty)
    mspan 8b
    (empty)
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mcentral
    mspan 16b (non
    empty)
    mspan 16b
    (empty)
    mspan
    16b
    mspan
    16b
    mspan
    16b
    mspan
    16b
    mspan
    16b
    mspan
    16b
    mspan
    8b
    mspan
    16b
    mspan
    16b
    Tiny allocation slot
    12b 4b 10b
    12b

    View Slide

  12. Tiny allocation
    P1
    M
    G2
    Processor
    Core
    P = Logical Processor per Hardware Thread
    M = Machine per OS Thread
    G = Goroutine (Coroutine)
    mcache
    mheap
    arena
    mcentral
    mspan 8b (non
    empty)
    mspan 8b
    (empty)
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mcentral
    mspan 16b (non
    empty)
    mspan 16b
    (empty)
    mspan
    16b
    mspan
    16b
    mspan
    16b
    mspan
    16b
    mspan
    16b
    mspan
    16b
    mspan
    8b
    mspan
    16b
    mspan
    16b
    Tiny allocation slot
    12b 4b 10b
    12b
    No
    free
    mspan
    16b

    View Slide

  13. Tiny allocation
    P1
    M
    G2
    Processor
    Core
    P = Logical Processor per Hardware Thread
    M = Machine per OS Thread
    G = Goroutine (Coroutine)
    mcache
    mheap
    arena
    mcentral
    mspan 8b (non
    empty)
    mspan 8b
    (empty)
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mcentral
    mspan 16b (non
    empty)
    mspan 16b
    (empty)
    mspan
    16b
    mspan
    16b
    mspan
    16b
    mspan
    16b
    mspan
    16b
    mspan
    16b
    mspan
    8b
    mspan
    16b
    mspan
    16b
    Tiny allocation slot
    12b 4b 10b
    12b

    View Slide

  14. Tiny allocation
    P1
    M
    G2
    Processor
    Core
    P = Logical Processor per Hardware Thread
    M = Machine per OS Thread
    G = Goroutine (Coroutine)
    mcache
    mheap
    arena
    mcentral
    mspan 8b (non
    empty)
    mspan 8b
    (empty)
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mcentral
    mspan 16b (non
    empty)
    mspan 16b
    (empty)
    mspan
    16b
    mspan
    16b
    mspan
    16b
    mspan
    16b
    mspan
    16b
    mspan
    16b
    mspan
    8b
    mspan
    16b
    mspan
    16b
    Tiny allocation slot
    12b 4b 10b
    12b

    View Slide

  15. Tiny allocation
    P1
    M
    G2
    Processor
    Core
    P = Logical Processor per Hardware Thread
    M = Machine per OS Thread
    G = Goroutine (Coroutine)
    mcache
    mheap
    arena
    mcentral
    mspan 8b (non
    empty)
    mspan 8b
    (empty)
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mcentral
    mspan 16b (non
    empty)
    mspan 16b
    (empty)
    mspan
    16b
    mspan
    16b
    mspan
    16b
    mspan
    16b
    mspan
    16b
    mspan
    16b
    mspan
    8b
    mspan
    16b
    mspan
    16b
    Tiny allocation slot
    12b 4b 10b 12b

    View Slide

  16. Small allocation

    View Slide

  17. Small allocation
    P1
    M
    G2
    Processor
    Core
    P = Logical Processor per Hardware Thread
    M = Machine per OS Thread
    G = Goroutine (Coroutine)
    mcache
    mheap
    arena
    mcentral
    mspan 8b (non
    empty)
    mspan 8b
    (empty)
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mcentral
    mspan 32b (non
    empty)
    mspan 32b
    (empty)
    mspan
    32b
    mspan
    32b
    mspan
    32b
    mspan
    32b
    mspan
    32b
    mspan
    32b
    mspan
    8b
    mspan
    16b
    mspan
    32b
    Tiny allocation slot
    12b

    View Slide

  18. Small allocation
    P1
    M
    G2
    Processor
    Core
    P = Logical Processor per Hardware Thread
    M = Machine per OS Thread
    G = Goroutine (Coroutine)
    mcache
    mheap
    arena
    mcentral
    mspan 8b (non
    empty)
    mspan 8b
    (empty)
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mcentral
    mspan
    8b
    mspan
    16b
    mspan
    32b
    Tiny allocation slot
    12b
    20b
    mspan 32b (non
    empty)
    mspan 32b
    (empty)
    mspan
    32b
    mspan
    32b
    mspan
    32b
    mspan
    32b
    mspan
    32b
    mspan
    32b

    View Slide

  19. Small allocation
    P1
    M
    G2
    Processor
    Core
    P = Logical Processor per Hardware Thread
    M = Machine per OS Thread
    G = Goroutine (Coroutine)
    mcache
    mheap
    arena
    mcentral
    mspan 8b (non
    empty)
    mspan 8b
    (empty)
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mcentral
    mspan
    8b
    mspan
    16b
    mspan
    32b
    Tiny allocation slot
    12b 20b
    mspan 32b (non
    empty)
    mspan 32b
    (empty)
    mspan
    32b
    mspan
    32b
    mspan
    32b
    mspan
    32b
    mspan
    32b
    mspan
    32b

    View Slide

  20. Small allocation
    P1
    M
    G2
    Processor
    Core
    P = Logical Processor per Hardware Thread
    M = Machine per OS Thread
    G = Goroutine (Coroutine)
    mcache
    mheap
    arena
    mcentral
    mspan 8b (non
    empty)
    mspan 8b
    (empty)
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mcentral
    mspan
    8b
    mspan
    16b
    mspan
    32b
    Tiny allocation slot
    12b 20b
    32b
    No
    free
    mspan
    32b
    mspan 32b (non
    empty)
    mspan 32b
    (empty)
    mspan
    32b
    mspan
    32b
    mspan
    32b
    mspan
    32b
    mspan
    32b
    mspan
    32b

    View Slide

  21. Small allocation
    P1
    M
    G2
    Processor
    Core
    P = Logical Processor per Hardware Thread
    M = Machine per OS Thread
    G = Goroutine (Coroutine)
    mcache
    mheap
    arena
    mcentral
    mspan 8b (non
    empty)
    mspan 8b
    (empty)
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mcentral
    mspan
    8b
    mspan
    16b
    mspan
    32b
    Tiny allocation slot
    12b 20b
    32b
    No
    free
    mspan
    32b
    mspan 32b (non
    empty)
    mspan 32b
    (empty)
    mspan
    32b
    mspan
    32b
    mspan
    32b
    mspan
    32b
    mspan
    32b
    mspan
    32b

    View Slide

  22. Small allocation
    P1
    M
    G2
    Processor
    Core
    P = Logical Processor per Hardware Thread
    M = Machine per OS Thread
    G = Goroutine (Coroutine)
    mcache
    mheap
    arena
    mcentral
    mspan 8b (non
    empty)
    mspan 8b
    (empty)
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mcentral
    mspan
    8b
    mspan
    16b
    mspan
    32b
    Tiny allocation slot
    12b 20b
    32b
    mspan 32b (non
    empty)
    mspan 32b
    (empty)
    mspan
    32b
    mspan
    32b
    mspan
    32b
    mspan
    32b
    mspan
    32b
    mspan
    32b

    View Slide

  23. Small allocation
    P1
    M
    G2
    Processor
    Core
    P = Logical Processor per Hardware Thread
    M = Machine per OS Thread
    G = Goroutine (Coroutine)
    mcache
    mheap
    arena
    mcentral
    mspan 8b (non
    empty)
    mspan 8b
    (empty)
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mcentral
    mspan
    8b
    mspan
    16b
    mspan
    32b
    Tiny allocation slot
    12b 20b
    mspan 32b (non
    empty)
    mspan 32b
    (empty)
    mspan
    32b
    mspan
    32b
    mspan
    32b
    mspan
    32b
    mspan
    32b
    mspan
    32b
    32b

    View Slide

  24. Large allocation

    View Slide

  25. Large allocation
    P1
    M
    G2
    Processor
    Core
    P = Logical Processor per Hardware Thread
    M = Machine per OS Thread
    G = Goroutine (Coroutine)
    mcache
    mheap
    arena
    mcentral
    mspan 8b (non
    empty)
    mspan 8b
    (empty)
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mcentral
    mspan
    8b
    mspan
    16b
    mspan
    32b
    Tiny allocation slot
    12b 20b
    mspan
    32b
    32b

    View Slide

  26. Large allocation
    P1
    M
    G2
    Processor
    Core
    P = Logical Processor per Hardware Thread
    M = Machine per OS Thread
    G = Goroutine (Coroutine)
    mcache
    mheap
    arena
    mcentral
    mspan 8b (non
    empty)
    mspan 8b
    (empty)
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mcentral
    mspan
    8b
    mspan
    16b
    mspan
    32b
    Tiny allocation slot
    12b 20b
    mspan
    32b
    32b
    40Kb

    View Slide

  27. Large allocation
    P1
    M
    G2
    Processor
    Core
    P = Logical Processor per Hardware Thread
    M = Machine per OS Thread
    G = Goroutine (Coroutine)
    mcache
    mheap
    arena
    mcentral
    mspan 8b (non
    empty)
    mspan 8b
    (empty)
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mcentral
    mspan
    8b
    mspan
    16b
    mspan
    32b
    Tiny allocation slot
    12b 20b
    mspan
    32b
    32b
    mspan
    40Kb

    View Slide