Go memory allocation

Transcript

1. Process Virtual Memory
   Resident set
   mheap
   P1(Logical Processor/Hardware Thread)
   Goroutine 1
   stack
   arena
   arena ...
   mcentral mcentral mcentral
   ..
   mcache mspan
   8b
   mspan
   16b
   mspan
   32b
   mspan
   ..
   mspan
   ..
   mspan
   ..
   mspan
   ..
   mspan
   ..
   mspan
   32768b
   mspan 8b (non empty)
   mspan 8b (empty)
   mspan 16b
   mspan 16b
   mspan
   mspan
   P2
   G6
   stack
   mcache
   ..
   ..
   ..
   ..
   Pn
   Gn
   stack
   mcache
   mspan 8b
   mspan 8b
   mspan 8b
   mspan 8b
   mspan 8b
   mspan 8b
   

   View Slide

2. mspan
   mspan 8b 8b 8b 8b …. 8b 8b 8b 1024 objects
   mspan 16b 16b 16b 16b …. 16b 16b 16b 512 objects
   mspan 32b 32b 32b 32b …. 32b 32b 32b 256 objects
   mspan 48b 48b 48b 48b …. 48b 48b 48b 170 objects
   mspan 32768 bytes 1 object
   mspan .. .. .. .. …. .. .. .. ...
   

   View Slide

3. Go Scheduler
   P
   M
   G
   Processor
   Core
   G G
   P = Logical Processor per Hardware Thread
   M = Machine per OS Thread
   G = Goroutine (Coroutine)
   LRQ = Local Run Queue
   LRQ
   

   View Slide

4. Tiny allocation
   

   View Slide

5. Tiny allocation
   P1
   M
   G2
   Processor
   Core
   P = Logical Processor per Hardware Thread
   M = Machine per OS Thread
   G = Goroutine (Coroutine)
   mcache
   mheap
   arena
   mcentral
   mspan 8b (non
   empty)
   mspan 8b
   (empty)
   mspan
   8b
   mspan
   8b
   mspan
   8b
   mspan
   8b
   mspan
   8b
   mspan
   8b
   mcentral
   mspan 16b (non
   empty)
   mspan 16b
   (empty)
   mspan
   16b
   mspan
   16b
   mspan
   16b
   mspan
   16b
   mspan
   16b
   mspan
   16b
   mspan
   8b
   mspan
   16b
   mspan
   16b
   Tiny allocation slot
   12b
   

   View Slide

6. Tiny allocation
   P1
   M
   G2
   Processor
   Core
   P = Logical Processor per Hardware Thread
   M = Machine per OS Thread
   G = Goroutine (Coroutine)
   mcache
   mheap
   arena
   mcentral
   mspan 8b (non
   empty)
   mspan 8b
   (empty)
   mspan
   8b
   mspan
   8b
   mspan
   8b
   mspan
   8b
   mspan
   8b
   mspan
   8b
   mcentral
   mspan 16b (non
   empty)
   mspan 16b
   (empty)
   mspan
   16b
   mspan
   16b
   mspan
   16b
   mspan
   16b
   mspan
   16b
   mspan
   16b
   mspan
   8b
   mspan
   16b
   mspan
   16b
   Tiny allocation slot
   12b
   4b
   

   View Slide

7. Tiny allocation
   P1
   M
   G2
   Processor
   Core
   P = Logical Processor per Hardware Thread
   M = Machine per OS Thread
   G = Goroutine (Coroutine)
   mcache
   mheap
   arena
   mcentral
   mspan 8b (non
   empty)
   mspan 8b
   (empty)
   mspan
   8b
   mspan
   8b
   mspan
   8b
   mspan
   8b
   mspan
   8b
   mspan
   8b
   mcentral
   mspan 16b (non
   empty)
   mspan 16b
   (empty)
   mspan
   16b
   mspan
   16b
   mspan
   16b
   mspan
   16b
   mspan
   16b
   mspan
   16b
   mspan
   8b
   mspan
   16b
   mspan
   16b
   Tiny allocation slot
   12b 4b
   

   View Slide

8. Tiny allocation
   P1
   M
   G2
   Processor
   Core
   P = Logical Processor per Hardware Thread
   M = Machine per OS Thread
   G = Goroutine (Coroutine)
   mcache
   mheap
   arena
   mcentral
   mspan 8b (non
   empty)
   mspan 8b
   (empty)
   mspan
   8b
   mspan
   8b
   mspan
   8b
   mspan
   8b
   mspan
   8b
   mspan
   8b
   mcentral
   mspan 16b (non
   empty)
   mspan 16b
   (empty)
   mspan
   16b
   mspan
   16b
   mspan
   16b
   mspan
   16b
   mspan
   16b
   mspan
   16b
   mspan
   8b
   mspan
   16b
   mspan
   16b
   Tiny allocation slot
   12b 4b
   10b
   

   View Slide

9. Tiny allocation
   P1
   M
   G2
   Processor
   Core
   P = Logical Processor per Hardware Thread
   M = Machine per OS Thread
   G = Goroutine (Coroutine)
   mcache
   mheap
   arena
   mcentral
   mspan 8b (non
   empty)
   mspan 8b
   (empty)
   mspan
   8b
   mspan
   8b
   mspan
   8b
   mspan
   8b
   mspan
   8b
   mspan
   8b
   mcentral
   mspan 16b (non
   empty)
   mspan 16b
   (empty)
   mspan
   16b
   mspan
   16b
   mspan
   16b
   mspan
   16b
   mspan
   16b
   mspan
   16b
   mspan
   8b
   mspan
   16b
   mspan
   16b
   Tiny allocation slot
   12b 4b
   10b
   

   View Slide

10. Tiny allocation
    P1
    M
    G2
    Processor
    Core
    P = Logical Processor per Hardware Thread
    M = Machine per OS Thread
    G = Goroutine (Coroutine)
    mcache
    mheap
    arena
    mcentral
    mspan 8b (non
    empty)
    mspan 8b
    (empty)
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mcentral
    mspan 16b (non
    empty)
    mspan 16b
    (empty)
    mspan
    16b
    mspan
    16b
    mspan
    16b
    mspan
    16b
    mspan
    16b
    mspan
    16b
    mspan
    8b
    mspan
    16b
    mspan
    16b
    Tiny allocation slot
    12b 4b 10b
    

    View Slide

11. Tiny allocation
    P1
    M
    G2
    Processor
    Core
    P = Logical Processor per Hardware Thread
    M = Machine per OS Thread
    G = Goroutine (Coroutine)
    mcache
    mheap
    arena
    mcentral
    mspan 8b (non
    empty)
    mspan 8b
    (empty)
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mcentral
    mspan 16b (non
    empty)
    mspan 16b
    (empty)
    mspan
    16b
    mspan
    16b
    mspan
    16b
    mspan
    16b
    mspan
    16b
    mspan
    16b
    mspan
    8b
    mspan
    16b
    mspan
    16b
    Tiny allocation slot
    12b 4b 10b
    12b
    

    View Slide

12. Tiny allocation
    P1
    M
    G2
    Processor
    Core
    P = Logical Processor per Hardware Thread
    M = Machine per OS Thread
    G = Goroutine (Coroutine)
    mcache
    mheap
    arena
    mcentral
    mspan 8b (non
    empty)
    mspan 8b
    (empty)
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mcentral
    mspan 16b (non
    empty)
    mspan 16b
    (empty)
    mspan
    16b
    mspan
    16b
    mspan
    16b
    mspan
    16b
    mspan
    16b
    mspan
    16b
    mspan
    8b
    mspan
    16b
    mspan
    16b
    Tiny allocation slot
    12b 4b 10b
    12b
    No
    free
    mspan
    16b
    

    View Slide

13. Tiny allocation
    P1
    M
    G2
    Processor
    Core
    P = Logical Processor per Hardware Thread
    M = Machine per OS Thread
    G = Goroutine (Coroutine)
    mcache
    mheap
    arena
    mcentral
    mspan 8b (non
    empty)
    mspan 8b
    (empty)
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mcentral
    mspan 16b (non
    empty)
    mspan 16b
    (empty)
    mspan
    16b
    mspan
    16b
    mspan
    16b
    mspan
    16b
    mspan
    16b
    mspan
    16b
    mspan
    8b
    mspan
    16b
    mspan
    16b
    Tiny allocation slot
    12b 4b 10b
    12b
    

    View Slide

14. Tiny allocation
    P1
    M
    G2
    Processor
    Core
    P = Logical Processor per Hardware Thread
    M = Machine per OS Thread
    G = Goroutine (Coroutine)
    mcache
    mheap
    arena
    mcentral
    mspan 8b (non
    empty)
    mspan 8b
    (empty)
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mcentral
    mspan 16b (non
    empty)
    mspan 16b
    (empty)
    mspan
    16b
    mspan
    16b
    mspan
    16b
    mspan
    16b
    mspan
    16b
    mspan
    16b
    mspan
    8b
    mspan
    16b
    mspan
    16b
    Tiny allocation slot
    12b 4b 10b
    12b
    

    View Slide

15. Tiny allocation
    P1
    M
    G2
    Processor
    Core
    P = Logical Processor per Hardware Thread
    M = Machine per OS Thread
    G = Goroutine (Coroutine)
    mcache
    mheap
    arena
    mcentral
    mspan 8b (non
    empty)
    mspan 8b
    (empty)
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mcentral
    mspan 16b (non
    empty)
    mspan 16b
    (empty)
    mspan
    16b
    mspan
    16b
    mspan
    16b
    mspan
    16b
    mspan
    16b
    mspan
    16b
    mspan
    8b
    mspan
    16b
    mspan
    16b
    Tiny allocation slot
    12b 4b 10b 12b
    

    View Slide

16. Small allocation
    

    View Slide

17. Small allocation
    P1
    M
    G2
    Processor
    Core
    P = Logical Processor per Hardware Thread
    M = Machine per OS Thread
    G = Goroutine (Coroutine)
    mcache
    mheap
    arena
    mcentral
    mspan 8b (non
    empty)
    mspan 8b
    (empty)
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mcentral
    mspan 32b (non
    empty)
    mspan 32b
    (empty)
    mspan
    32b
    mspan
    32b
    mspan
    32b
    mspan
    32b
    mspan
    32b
    mspan
    32b
    mspan
    8b
    mspan
    16b
    mspan
    32b
    Tiny allocation slot
    12b
    

    View Slide

18. Small allocation
    P1
    M
    G2
    Processor
    Core
    P = Logical Processor per Hardware Thread
    M = Machine per OS Thread
    G = Goroutine (Coroutine)
    mcache
    mheap
    arena
    mcentral
    mspan 8b (non
    empty)
    mspan 8b
    (empty)
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mcentral
    mspan
    8b
    mspan
    16b
    mspan
    32b
    Tiny allocation slot
    12b
    20b
    mspan 32b (non
    empty)
    mspan 32b
    (empty)
    mspan
    32b
    mspan
    32b
    mspan
    32b
    mspan
    32b
    mspan
    32b
    mspan
    32b
    

    View Slide

19. Small allocation
    P1
    M
    G2
    Processor
    Core
    P = Logical Processor per Hardware Thread
    M = Machine per OS Thread
    G = Goroutine (Coroutine)
    mcache
    mheap
    arena
    mcentral
    mspan 8b (non
    empty)
    mspan 8b
    (empty)
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mcentral
    mspan
    8b
    mspan
    16b
    mspan
    32b
    Tiny allocation slot
    12b 20b
    mspan 32b (non
    empty)
    mspan 32b
    (empty)
    mspan
    32b
    mspan
    32b
    mspan
    32b
    mspan
    32b
    mspan
    32b
    mspan
    32b
    

    View Slide

20. Small allocation
    P1
    M
    G2
    Processor
    Core
    P = Logical Processor per Hardware Thread
    M = Machine per OS Thread
    G = Goroutine (Coroutine)
    mcache
    mheap
    arena
    mcentral
    mspan 8b (non
    empty)
    mspan 8b
    (empty)
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mcentral
    mspan
    8b
    mspan
    16b
    mspan
    32b
    Tiny allocation slot
    12b 20b
    32b
    No
    free
    mspan
    32b
    mspan 32b (non
    empty)
    mspan 32b
    (empty)
    mspan
    32b
    mspan
    32b
    mspan
    32b
    mspan
    32b
    mspan
    32b
    mspan
    32b
    

    View Slide

21. Small allocation
    P1
    M
    G2
    Processor
    Core
    P = Logical Processor per Hardware Thread
    M = Machine per OS Thread
    G = Goroutine (Coroutine)
    mcache
    mheap
    arena
    mcentral
    mspan 8b (non
    empty)
    mspan 8b
    (empty)
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mcentral
    mspan
    8b
    mspan
    16b
    mspan
    32b
    Tiny allocation slot
    12b 20b
    32b
    No
    free
    mspan
    32b
    mspan 32b (non
    empty)
    mspan 32b
    (empty)
    mspan
    32b
    mspan
    32b
    mspan
    32b
    mspan
    32b
    mspan
    32b
    mspan
    32b
    

    View Slide

22. Small allocation
    P1
    M
    G2
    Processor
    Core
    P = Logical Processor per Hardware Thread
    M = Machine per OS Thread
    G = Goroutine (Coroutine)
    mcache
    mheap
    arena
    mcentral
    mspan 8b (non
    empty)
    mspan 8b
    (empty)
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mcentral
    mspan
    8b
    mspan
    16b
    mspan
    32b
    Tiny allocation slot
    12b 20b
    32b
    mspan 32b (non
    empty)
    mspan 32b
    (empty)
    mspan
    32b
    mspan
    32b
    mspan
    32b
    mspan
    32b
    mspan
    32b
    mspan
    32b
    

    View Slide

23. Small allocation
    P1
    M
    G2
    Processor
    Core
    P = Logical Processor per Hardware Thread
    M = Machine per OS Thread
    G = Goroutine (Coroutine)
    mcache
    mheap
    arena
    mcentral
    mspan 8b (non
    empty)
    mspan 8b
    (empty)
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mcentral
    mspan
    8b
    mspan
    16b
    mspan
    32b
    Tiny allocation slot
    12b 20b
    mspan 32b (non
    empty)
    mspan 32b
    (empty)
    mspan
    32b
    mspan
    32b
    mspan
    32b
    mspan
    32b
    mspan
    32b
    mspan
    32b
    32b
    

    View Slide

24. Large allocation
    

    View Slide

25. Large allocation
    P1
    M
    G2
    Processor
    Core
    P = Logical Processor per Hardware Thread
    M = Machine per OS Thread
    G = Goroutine (Coroutine)
    mcache
    mheap
    arena
    mcentral
    mspan 8b (non
    empty)
    mspan 8b
    (empty)
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mcentral
    mspan
    8b
    mspan
    16b
    mspan
    32b
    Tiny allocation slot
    12b 20b
    mspan
    32b
    32b
    

    View Slide

26. Large allocation
    P1
    M
    G2
    Processor
    Core
    P = Logical Processor per Hardware Thread
    M = Machine per OS Thread
    G = Goroutine (Coroutine)
    mcache
    mheap
    arena
    mcentral
    mspan 8b (non
    empty)
    mspan 8b
    (empty)
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mcentral
    mspan
    8b
    mspan
    16b
    mspan
    32b
    Tiny allocation slot
    12b 20b
    mspan
    32b
    32b
    40Kb
    

    View Slide

27. Large allocation
    P1
    M
    G2
    Processor
    Core
    P = Logical Processor per Hardware Thread
    M = Machine per OS Thread
    G = Goroutine (Coroutine)
    mcache
    mheap
    arena
    mcentral
    mspan 8b (non
    empty)
    mspan 8b
    (empty)
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mspan
    8b
    mcentral
    mspan
    8b
    mspan
    16b
    mspan
    32b
    Tiny allocation slot
    12b 20b
    mspan
    32b
    32b
    mspan
    40Kb
    

    View Slide