Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Out of memory or plenty to spare.

Joshua Thijssen
February 20, 2015
180

Out of memory or plenty to spare.

Joshua Thijssen

February 20, 2015
Tweet

Transcript

  1. Out of memory or plenty to spare?
    1
    Joshua Thijssen
    jaytaph
    The fine details of reading memory consumption

    View full-size slide

  2. Disclaimer
    2
    No PHP
    Pretty advanced stuff

    View full-size slide

  3. How much memory is
    our server using?
    3

    View full-size slide

  4. 6
    How much memory is
    our server using?

    View full-size slide

  5. $ free -m
    total used free shared buffers cached
    Mem: 3963 3500 462 0 722 1263
    -/+ buffers/cache: 1515 2448
    Swap: 400 20 379
    7

    View full-size slide

  6. Active / Total Objects (% used) : 2187767 / 2283870 (95.8%)
    Active / Total Slabs (% used) : 261417 / 261421 (100.0%)
    Active / Total Caches (% used) : 114 / 192 (59.4%)
    Active / Total Size (% used) : 1013948.21K / 1024061.72K (99.0%)
    Minimum / Average / Maximum Object : 0.02K / 0.45K / 4096.00K
    OBJS ACTIVE USE OBJ SIZE SLABS OBJ/SLAB CACHE SIZE NAME
    825808 825378 99% 0.98K 206452 4 825808K ext4_inode_cache
    522921 512677 98% 0.19K 24901 21 99604K dentry
    394290 364506 92% 0.10K 10110 39 40440K buffer_head
    166914 142785 85% 0.04K 1686 99 6744K ext4_extent_status
    143756 142975 99% 0.05K 1732 83 6928K jbd2_inode
    63700 62887 98% 0.56K 9100 7 36400K radix_tree_node
    42273 29552 69% 0.06K 671 63 2684K kmalloc-64
    23870 19345 81% 0.12K 770 31 3080K kmalloc-96
    16884 14623 86% 0.06K 268 63 1072K anon_vma_chain
    12144 12024 99% 0.12K 368 33 1472K kernfs_node_cache
    11424 10582 92% 0.19K 544 21 2176K vm_area_struct
    10044 7897 78% 0.03K 81 124 324K kmalloc-32
    9331 9135 97% 0.56K 1333 7 5332K inode_cache
    7434 6552 88% 0.06K 118 63 472K anon_vma
    6528 6528 100% 0.62K 1088 6 4352K proc_inode_cache
    3312 2330 70% 0.25K 207 16 828K filp
    2490 2458 98% 0.05K 30 83 120K ftrace_event_field
    2294 2207 96% 0.12K 74 31 296K kmalloc-128
    1596 1440 90% 0.19K 76 21 304K kmalloc-192
    1568 1476 94% 0.07K 28 56 112K Acpi-Operand
    1112 1068 96% 1.00K 278 4 1112K kmalloc-1024
    1104 1076 97% 0.09K 24 46 96K ftrace_event_file
    882 518 58% 0.19K 42 21 168K cred_jar
    880 644 73% 0.25K 55 16 220K skbuff_head_cache
    828 802 96% 0.65K 138 6 552K shmem_inode_cache
    648 552 85% 0.11K 18 36 72K jbd2_journal_head
    608 523 86% 0.25K 38 16 152K kmalloc-256
    600 500 83% 0.50K 75 8 300K kmalloc-512
    468 294 62% 0.11K 13 36 52K task_delay_info
    465 302 64% 0.12K 15 31 60K pid
    8
    slabtop

    View full-size slide

  7. 12
    Operating system (kernel)
    Process 1 Process 2 Process 3 Process 4
    Kernel POV

    View full-size slide

  8. 13
    Operating system (kernel)
    Process 1
    Process POV

    View full-size slide

  9. ➡ Every process can use up to 4 GB of
    memory.
    ➡ EVEN IF YOUR COMPUTER DOES
    NOT HAVE 4GB OF MEMORY!!!!!
    14

    View full-size slide

  10. 15
    0x00000000
    0x00010000
    0xC0000000
    0xFFFFFFFF
    1 GB
    3 GB
    program
    + data
    stack
    shared

    View full-size slide

  11. 17
    4 kb
    4 kb
    4 kb
    4 kb
    4 kb
    4 kb
    4 kb
    0x12340000
    0x12341000
    0x12342000
    0x12343000
    0x12344000
    0x12345000
    0x12346000
    0x12347000

    View full-size slide

  12. C
    0x00010000
    0x00011000
    B
    A
    0x00012000
    Virtual Physical
    B
    0x00054000
    A
    0x00CF2000
    C
    0x00EF7000

    View full-size slide

  13. 19
    C1
    0x00010000
    0x00011000
    B1
    A1
    0x00012000
    C2
    0x00010000
    0x00011000
    B2
    A2
    0x00012000
    Virtual Virtual
    B1
    A1
    A2
    B2
    C2
    C1
    Physical

    View full-size slide

  14. B1
    20
    C1
    B1
    A1
    A1
    C2
    B2
    A2
    A2
    B2
    C2
    C3
    B3
    A3
    A3
    B3
    C3
    C4
    B4
    A4
    C1
    Physical
    Virtual
    Virtual

    View full-size slide

  15. 21
    C1
    B1
    A1
    C2
    B2
    A2
    C3
    B3
    A3
    C4
    B4
    A4
    Virtual
    Virtual Swap
    B1
    A1
    B2
    C2
    A3
    B3
    C3
    A2
    C1
    Physical
    C4
    A4
    B4
    A2
    C1

    View full-size slide

  16. 22
    So how can we use
    more memory?

    View full-size slide

  17. Ask the OS
    brk() / mmap()
    23

    View full-size slide

  18. 24
    0x00000000
    0x00010000
    0xC0000000
    0xFFFFFFFF
    1 GB
    3 GB
    program
    + data
    stack
    Heap
    shared

    View full-size slide

  19. [root@localhost ~]# pmap -x 1271
    1271: php-fpm: master process (/etc/php-fpm.conf)
    Address Kbytes RSS Dirty Mode Mapping
    001ee000 248 8 0 r-x-- libgssapi_krb5.so.2.2
    0022c000 4 4 4 r---- libgssapi_krb5.so.2.2
    0022d000 4 4 4 rw--- libgssapi_krb5.so.2.2
    0022f000 28 4 0 r-x-- libcrypt-2.12.so
    00236000 4 4 4 r---- libcrypt-2.12.so
    00237000 4 4 4 rw--- libcrypt-2.12.so
    ....
    08048000 3400 204 0 r-x-- php-fpm
    0839a000 328 140 20 rw--- php-fpm
    083ec000 96 32 32 rw--- [ anon ]
    092b4000 1316 1176 1176 rw--- [ anon ]
    ...
    af483000 4 4 4 rw-s- zero (deleted)
    af484000 28 0 0 r--s- gconv-modules.cache
    af48b000 131072 0 0 rw-s- zero (deleted)
    b748b000 160 4 4 rw--- [ anon ]
    b74b3000 2048 8 0 r---- locale-archive
    b76b3000 1312 124 124 rw--- [ anon ]
    b77fb000 4 4 4 rw-s- zero (deleted)
    b77fc000 4 4 4 rw-s- zero (deleted)
    b77fd000 4 4 4 rw-s- zero (deleted)
    b77fe000 4 4 4 rw-s- zero (deleted)
    b77ff000 4 4 4 rw--- [ anon ]
    bf876000 84 48 48 rw--- [ stack ]
    -------- ------- ------- ------- -------
    total kB 155544 - - -
    26

    View full-size slide

  20. ...
    00aed000 36 4 0 r-x-- gmp.so
    00af6000 8 8 8 rw--- gmp.so
    00af8000 264 8 0 r-x-- libgmp.so.3.5.0
    00b3a000 4 4 4 rw--- libgmp.so.3.5.0
    00b3b000 36 4 0 r-x-- json.so
    00b44000 4 4 4 rw--- json.so
    00b45000 260 8 0 r-x-- phar.so
    00b86000 8 8 8 rw--- phar.so
    00b88000 72 4 0 r-x-- sockets.so
    00b9a000 8 8 8 rw--- sockets.so
    00b9c000 40 4 0 r-x-- xml.so
    00ba6000 4 4 4 rw--- xml.so
    00bad000 12 4 0 r-x-- libdl-2.12.so
    00bb0000 4 4 4 r---- libdl-2.12.so
    00bb1000 4 4 4 rw--- libdl-2.12.so
    00bb2000 16 4 0 r-x-- libplc4.so
    00bb6000 4 4 4 r---- libplc4.so
    00bb7000 4 4 4 rw--- libplc4.so
    00bb8000 88 8 0 r-x-- zip.so
    00bce000 8 8 8 rw--- zip.so
    00bd0000 48 20 0 r-x-- libnss_files-2.12.so
    00bdc000 4 4 4 r---- libnss_files-2.12.so
    00bdd000 4 4 4 rw--- libnss_files-2.12.so
    00c24000 8 4 0 r-x-- gettext.so
    00c26000 4 4 4 rw--- gettext.so
    00c5c000 8 4 0 r-x-- ctype.so
    00c5e000 4 4 4 rw--- ctype.so
    00d06000 40 8 0 r-x-- simplexml.so
    00d10000 4 4 4 rw--- simplexml.so
    ...
    27

    View full-size slide

  21. [root@localhost ~]# cat /proc/1271/status
    Name: php-fpm
    State: S (sleeping)
    Tgid: 1271
    Pid: 1271
    PPid: 1
    TracerPid: 0
    Uid: 0 0 0 0
    Gid: 0 0 0 0
    Utrace: 0
    FDSize: 32
    Groups: 0
    VmPeak: 155548 kB
    VmSize: 155548 kB
    VmLck: 0 kB
    VmHWM: 3484 kB
    VmRSS: 3484 kB
    VmData: 3160 kB
    VmStk: 88 kB
    VmExe: 3400 kB
    VmLib: 14668 kB
    VmPTE: 76 kB
    VmSwap: 0 kB
    Threads: 1
    SigQ: 0/3855
    SigPnd: 0000000000000000
    ShdPnd: 0000000000000000
    SigBlk: 0000000000000000
    SigIgn: 0000000000001000
    SigCgt: 0000000180014a06
    CapInh: 0000000000000000
    CapPrm: ffffffffffffffff
    CapEff: ffffffffffffffff
    CapBnd: ffffffffffffffff
    Cpus_allowed: 1
    Cpus_allowed_list: 0
    Mems_allowed: 1
    Mems_allowed_list: 0
    voluntary_ctxt_switches: 5155
    nonvoluntary_ctxt_switches: 3
    29

    View full-size slide

  22. 32
    Process 1 Process 2
    fork()

    View full-size slide

  23. 33
    C1
    B1
    A1
    C1`
    B1`
    A1`
    A1
    B1
    C1
    Physical
    Virtual
    Virtual
    fork() =>

    View full-size slide

  24. 34
    C1
    B1
    A1
    C1`
    B2
    A1`
    A1
    B1
    C1
    Physical
    Virtual
    Virtual
    fork() =>
    B2

    View full-size slide

  25. Everything is f*rked up
    35

    View full-size slide

  26. 37
    Most memory tools returns
    your virtual memory usage,
    which is pointless...

    View full-size slide

  27. http://farm1.static.flickr.com/73/163450213_18478d3aa6_d.jpg 38

    View full-size slide

  28. 39
    Find me on twitter: @jaytaph
    Find me for development and training: www.noxlogic.nl
    Find me on email: [email protected]
    Find me for blogs: www.adayinthelifeof.nl

    View full-size slide

  29. ➡ https://techtalk.intersec.com/2013/07/memory-part-2-
    understanding-process-memory/
    ➡ http://locklessinc.com/articles/memory_usage/
    ➡ http://rhaas.blogspot.nl/2012/01/linux-memory-
    reporting.html
    ➡ http://people.freebsd.org/~lstewart
    ➡ http://deathbytape.com/post/110371790629/intro-
    virtual-memoryarticles/cpumemory.pdf
    ➡ http://nikic.github.com/2011/12/12/How-big-are-PHP-
    arrays-really-Hint-BIG.html
    40
    303 See Other

    View full-size slide