Upgrade to Pro — share decks privately, control downloads, hide ads and more …

高集積コンテナホスティングにおけるボトルネックとその解法

 高集積コンテナホスティングにおけるボトルネックとその解法

話し手の所属するGMOペパボではロリポップ!マネージドクラウドというコンテナホスティングサービスを提供しており、1台のサーバに数千のコンテナを起動します。

コンテナはプロセスであるという認識は一般的になりつつありますが、コンテナプロセスを起動するにはネームスペースの作成や、マウントなどいくつもの処理が行われてプロセスを起動する必要があり、通常のプロセス起動と違う箇所にボトルネックが現れます。またコンテナ技術はLinuxのカーネルレイヤに近い領域で実装されているためそのボトルネックの解消には深いLinuxに対する理解が必要です。

本セッションにおいては話し手が1台のサーバで数千のコンテナを起動する際に直面したボトルネック、その調査方法、そして解決する方法を共有したいと思います。

昨今k8sの台頭により、あらゆるサービスでコンテナ技術が利用されつつありますが、コンテナ技術の深い理解は、今後のエンジニアのあらゆる開発において有用な知識となるはずです。ぜひ本セッションに足を運んでいただき、一歩進んだコンテナ利用をしていきましょう。

Kazuhiko Yamashita

September 08, 2018
Tweet

More Decks by Kazuhiko Yamashita

Other Decks in Technology

Transcript

  1. -JOVY MJCOTTTUOT JEFYBNQMF VJE FYBNQMF HJE FYBNQMF HSPVQT FYBNQMF VTFS

    OBNFFYBNQMF <\ OBNFFYBNQMF  JE   ^> γϯϓϧͰࣗ༝౓ͷߴ͍ઃܭ αʔό࣮૷΋͍ΘΏΔ3&45GVM෩ɺMJCOTTTUOT͔Β೚ҙͷίϚϯυΛ࣮ߦͨ͠ ݁ՌΛར༻Ͱ͖Δ࢓૊Έ΋͋ΔͷͰΫϥΠΞϯτଆͷ֦ு΋ࣗ༝ࣗࡏ <VTFSTFYBNQMF> JE LFZT<TTI> HSPVQ@JE TUOTDPOG
  2. EPDLFSSVOOBNFTUOTETUOTTUOTMBUFTU EPDLFSFYFDJUTUOTCJOCBTI JEFYBNQMF VJE FYBNQMF HJE FYBNQMF HSPVQT FYBNQMF ͙͢ʹࢼͤ·͢

    DVSMGT4-IUUQTSFQPTUOTKQTDSJQUTZVNSFQPTIcTI DVSMGT4-IUUQTSFQPTUOTKQTDSJQUTBQUSFQPTIcTI EPDLFS $FOU04 VCVOUVEFCJBO
  3. OBNFTQBDF IUUQTMJOVYKNPTEOKQIUNM-%1@NBOQBHFTNBOVOTIBSFIUNM SPPU!MPDBMIPTUdJQB MP-001#"$, 61 -08&3@61NUVREJTDOPRVFVF MJOLMPPQCBDLCSE JOFUTDPQFIPTUMP FOQT#30"%$"45 .6-5*$"45

    61 -08&3@61 MJOLFUIFSBCSE⒎⒎⒎⒎⒎⒎ JOFUCSETDPQFHMPCBMFOQT SPPU!MPDBMIPTUdCBTI SPPU!MPDBMIPTUdVOTIBSFOFU SPPU!MPDBMIPTUdJQB MP-001#"$,NUVREJTDOPPQTUBUF%08/HSPVQEFGBVMURMFO MJOLMPPQCBDLCSE
  4. DBQBCJMJUZ IUUQTMJOVYKNPTEOKQIUNM-%1@NBOQBHFTNBODBQBCJMJUJFTIUNM -JOVYͷݖݶ؅ཧͷ࢓૊Έɻϓϩηε΍ ϑΝΠϧ୯ҐͰ෇༩͢Δ͜ͱ͕Ͱ͖Δ QTFGcHSFQBQBDIF SPPU VTSTCJOBQBDIFLTUBSU XXXEBUB VTSTCJOBQBDIFLTUBSU XXXEBUB

    VTSTCJOBQBDIFLTUBSU SPPUQUTHSFQDPMPSBVUPBQBDIF SPPU!MPDBMIPTUdHFUQDBQT $BQBCJMJUJFTGPSA DBQ@DIPXO  DBQ@EBD@PWFSSJEF  DBQ@OFU@CJOE@TFSWJDF 
  5. wϓϩηεͷൃߦ͢ΔγεςϜίʔϧΛτϨʔε TUSBDF SPPU!MPDBMIPTUdTUSBDFUU5T4UJNFJQOFUOTBEEFYBNQMF FYFDWF TCJOJQ <JQ OFUOT BEE FYBNQMF> <

    WBST >   CSL /6-- YF BDDFTT FUDMETPOPIXDBQ '@0, &/0&/5 /PTVDIpMFPS EJSFDUPSZ  NNBQ /6--  1305@3&"%c1305@83*5& ."1@13*7"5&c ."1@"/0/:.064   YGCBD ʜ
  6. JQ<LFSOFMLBMMTZNT><L>FOUSZ@4:4$"--@@GBTUQBUI c FOUSZ@4:4$"--@@GBTUQBUI c cTZT@VOTIBSF cVOTIBSF@OTQSPYZ@OBNFTQBDFT cDSFBUF@OFX@OBNFTQBDFT cDPQZ@OFU@OT cc ccTFUVQ@OFU

    ccPQT@JOJU ccc cccJQUBCMF@pMUFS@OFU@JOJU cccJQU@SFHJTUFS@UBCMF cccc ccccYU@SFHJTUFS@UBCMF ccccYU@SFQMBDF@UBCMF cccc@@LNBMMPD@OPEF cccc@@TMBC@BMMPD ccccc ccccc@@@TMBC@BMMPD cccccEFBDUJWBUF@TMBC cccccc cccccc@@DNQYDIH@EPVCMF@TMBCJTSB ccccccVOTIBSF ccccccYB
  7. ϒϦοδΛ෼ࢄ -JOVY#SJEHF OFUOT DPOUBJOFS WFUIQBJS BQQMZOFUOT IBDPOJXB FUI@I@ DPOUBJOFS@ -JOVY#SJEHF

    IBDPOJXB999 OFUOT FUI@I@ DPOUBJOFS DPOUBJOFS@ BQQMZOFUOT OFUOT DPOUBJOFS WFUIQBJS BQQMZOFUOT FUI@I@ DPOUBJOFS@   
  8. 1*%64&313/*7*353&44)34$16.&.5*.& $0.."/% SPPUHH4DBEWJTPS SPPU3UPQ VCVOUV3BQBDIF SPPU%SPVUF SPPU3IBDPOJXB SPPU%SPVUF SPPU%SPVUF SPPU%JQ

    SPPU%SPVUF SPPU%JQ SPPU3IBDPOJXB SPPU3IBDPOJXB SPPU3IBDPOJXB SPPU3IBDPOJXB SPPU%SPVUF SPPU3IBDPOJXB UPQ
  9. TUSBDFJQOFUOTBEEFYBNQMF  PQFO WBSSVOOFUOTOFUOT@BECB 0@3%0/-:  TFOENTH  \NTH@OBNF 

    \TB@GBNJMZ"'@/&5-*/, QJE  HSPVQT^  SFDWNTH  \NTH@OBNF  \TB@GBNJMZ"'@/&5-*/, QJE  HSPVQT^  DMPTF   PQFO WBSSVOOFUOTOFUOT@DGDGEBC 0@3%0/-:  TFOENTH  \NTH@OBNF  \TB@GBNJMZ"'@/&5-*/, QJE  HSPVQT^  SFDWNTH  \NTH@OBNF  \TB@GBNJMZ"'@/&5-*/, QJE  HSPVQT^  DMPTF   طଘͷOFUOT͢΂ͯʹ ໰͍߹Θ͍ͤͯΔ
  10. HEC͔Β͋ͨΓΛ͚ͭΔ HECJQ HEC CTFOENTH HEC SVOOFUOTBEEUFTU@ 4UBSUJOHQSPHSBNTCJOJQOFUOTBEEUFTU@  #SFBLQPJOU TFOENTH

    BUTZTEFQTVOJYTZTDBMMUFNQMBUF4 TZTEFQTVOJYTZTDBMMUFNQMBUF4/PTVDIpMFPSEJSFDUPSZ HEC CU TFOENTH BUTZTEFQTVOJYTZTDBMMUFNQMBUF4 YDCJOSUOM@UBML  YCEJO   YGJOOFUOT@NBQ@JOJU  YDFJOEP@OFUOT  YJO  
  11. OFUMJOLΛར༻ͨ͠/4*%औಘ XIJMF FOUSZSFBEEJS EJS /6-- \ JG TUSDNQ FOUSZE@OBNF 

      DPOUJOVF JG TUSDNQ FOUSZE@OBNF    DPOUJOVF OTJEHFU@OFUOTJE@GSPN@OBNF FOUSZE@OBNF  JG OTJE  OFUOT@NBQ@BEE OTJE FOUSZE@OBNF  ^ JQJQOFUOTD
  12. OFUOTͷ௥Ճ͕ര଎ʹʂʂ̍ TUSBDFUUTPTUSBDF@JQOFUOTBEEUFTU@ IFBETUSBDF@ FYFDWF TCJOJQ <JQ OFUOT BEE UFTU@> <

    WBST >   UBJMTUSBDF@  FYJUFEXJUI  ύοέʔδόʔδϣϯൺֱͰඵ͔ΒඵʹͳΓ ഒ͘Β͍ૣ͘ͳͬͨ TUSBDFUUTPTUSBDF@JQOFUOTBEEUFTU@ IFBETUSBDF@ FYFDWF TCJOJQ <JQ OFUOT BEE UFTU@> < WBST >   UBJMTUSBDF@  FYJUFEXJUI CFGPSF BGUFS
  13. JQ<LFSOFMLBMMTZNT><L>FOUSZ@4:4$"--@@GBTUQBUI c FOUSZ@4:4$"--@@GBTUQBUI c cTZT@NPVOU cc ccEP@NPVOU ccc cccHSBGU@USFF cccBUUBDI@SFDVSTJWF@NOU

    cccc ccccQSPQBHBUF@NOU ccccc cccccQSPQBHBUF@POF cccccc ccccccDPQZ@USFF ccccccDMPOF@NOU ccccccc cccccccBMMPD@WGTNOU DPQZ@USFF͕ࢧ഑త
  14. XIJMF NPVOU  /&5/4@36/@%*3 OPOF .4@4)"3&%c.4@3&$ /6-- \  'BJMVOMFTTXFOFFEUPNBLFUIFNPVOUQPJOU

     JG FSSOP&*/7"-ccNBEF@OFUOT@SVO@EJS@NPVOU \ GQSJOUG TUEFSS NPVOUNBLFTIBSFETGBJMFETaO /&5/4@36/@%*3 TUSFSSPS FSSOP  SFUVSO ^  6QHSBEF/&5/4@36/@%*3UPBNPVOUQPJOU  JG NPVOU /&5/4@36/@%*3 /&5/4@36/@%*3 OPOF .4@#*/%c.4@3&$ /6-- \ GQSJOUG TUEFSS NPVOUCJOETTGBJMFETaO /&5/4@36/@%*3 /&5/4@36/@%*3  TUSFSSPS FSSOP  SFUVSO ^ NBEF@OFUOT@SVO@EJS@NPVOU ^ JQJQOFUOTD
  15. XIJMF NPVOU  /&5/4@36/@%*3 OPOF .4@4)"3&%c.4@3&$ /6-- \  'BJMVOMFTTXFOFFEUPNBLFUIFNPVOUQPJOU

     JG FSSOP&*/7"-ccNBEF@OFUOT@SVO@EJS@NPVOU \ GQSJOUG TUEFSS NPVOUNBLFTIBSFETGBJMFETaO /&5/4@36/@%*3 TUSFSSPS FSSOP  SFUVSO ^  6QHSBEF/&5/4@36/@%*3UPBNPVOUQPJOU  JG NPVOU /&5/4@36/@%*3 /&5/4@36/@%*3 OPOF .4@#*/%c.4@3&$ /6-- \ GQSJOUG TUEFSS NPVOUCJOETTGBJMFETaO /&5/4@36/@%*3 /&5/4@36/@%*3  TUSFSSPS FSSOP  SFUVSO ^ NBEF@OFUOT@SVO@EJS@NPVOU ^ JQJQOFUOTD WBSSVOOFUOTΛ࠶ؼతʹ 4)"3&%Ϛ΢ϯτ
  16. XIJMF NPVOU  /&5/4@36/@%*3 OPOF .4@4)"3&%c.4@3&$ /6-- \  'BJMVOMFTTXFOFFEUPNBLFUIFNPVOUQPJOU

     JG FSSOP&*/7"-ccNBEF@OFUOT@SVO@EJS@NPVOU \ GQSJOUG TUEFSS NPVOUNBLFTIBSFETGBJMFETaO /&5/4@36/@%*3 TUSFSSPS FSSOP  SFUVSO ^  6QHSBEF/&5/4@36/@%*3UPBNPVOUQPJOU  JG NPVOU /&5/4@36/@%*3 /&5/4@36/@%*3 OPOF .4@#*/%c.4@3&$ /6-- \ GQSJOUG TUEFSS NPVOUCJOETTGBJMFETaO /&5/4@36/@%*3 /&5/4@36/@%*3  TUSFSSPS FSSOP  SFUVSO ^ NBEF@OFUOT@SVO@EJS@NPVOU ^ JQJQOFUOTD WBSSVOOFUOTΛ ࣗ෼ࣗ਎ʹ#*/%Ϛ΢ϯτ͢Δ͜ ͱͰ4)"3&%Մೳʹ͢Δ
  17. JG VOTIBSF $-0/&@/&8/&5  \  GQSJOUG TUEFSS 'BJMFEUPDSFBUFBOFXOFUXPSLOBNFTQBDFaTaTaO 

      OBNF TUSFSSPS FSSOP   HPUPPVU@EFMFUF ^  #JOEUIFOFUOTMBTUTP*DBOXBUDIGPSJU  JG NPVOU QSPDTFMGOTOFU OFUOT@QBUI OPOF .4@#*/% /6--  \  GQSJOUG TUEFSS #JOEQSPDTFMGOTOFUTGBJMFETaO    OFUOT@QBUI TUSFSSPS FSSOP   HPUPPVU@EFMFUF ^ JQJQOFUOTD ͜ͷϓϩηεͷOFUOTΛ෼཭͢Δ
  18. JG VOTIBSF $-0/&@/&8/&5  \  GQSJOUG TUEFSS 'BJMFEUPDSFBUFBOFXOFUXPSLOBNFTQBDFaTaTaO 

      OBNF TUSFSSPS FSSOP   HPUPPVU@EFMFUF ^  #JOEUIFOFUOTMBTUTP*DBOXBUDIGPSJU  JG NPVOU QSPDTFMGOTOFU OFUOT@QBUI OPOF .4@#*/% /6--  \  GQSJOUG TUEFSS #JOEQSPDTFMGOTOFUTGBJMFETaO    OFUOT@QBUI TUSFSSPS FSSOP   HPUPPVU@EFMFUF ^ JQJQOFUOTD OFUOT@QBUIWBSSVOOFUOTOFUOT໊ ʹ෼཭ͨ͠OFUOTͷ'%Λ#*/%Ϛ΢ϯτ͢Δ ͜ͱͰϓϩηε͕ͳ͘ͳͬͯ΋OFUOT͕࢒Δ
  19. XIJMF NPVOU  /&5/4@36/@%*3 OPOF .4@4)"3&%c.4@3&$ /6-- \  'BJMVOMFTTXFOFFEUPNBLFUIFNPVOUQPJOU

     JG FSSOP&*/7"-ccNBEF@OFUOT@SVO@EJS@NPVOU \ GQSJOUG TUEFSS NPVOUNBLFTIBSFETGBJMFETaO /&5/4@36/@%*3 TUSFSSPS FSSOP  SFUVSO ^  6QHSBEF/&5/4@36/@%*3UPBNPVOUQPJOU  JG NPVOU /&5/4@36/@%*3 /&5/4@36/@%*3 OPOF .4@#*/%c.4@3&$ /6-- \ GQSJOUG TUEFSS NPVOUCJOETTGBJMFETaO /&5/4@36/@%*3 /&5/4@36/@%*3  TUSFSSPS FSSOP  SFUVSO ^ NBEF@OFUOT@SVO@EJS@NPVOU ^ JQJQOFUOTD .4@3&$͸࠶ؼతʹߦ͏ͷͰ 0 O ͷ͕͔͔࣌ؒΔ
  20. JQ<LFSOFMLBMMTZNT><L>FOUSZ@4:4$"--@@GBTUQBUI c FOUSZ@4:4$"--@@GBTUQBUI c cTZT@VOTIBSF cVOTIBSF@OTQSPYZ@OBNFTQBDFT cDSFBUF@OFX@OBNFTQBDFT cDPQZ@NOU@OT cc ccDPQZ@USFF

    ccc cccDMPOF@NOU cccc ccccBMMPD@WGTNOU ccccc ccccc@@BMMPD@QFSDQV cccccc ccccccQDQV@BMMPD ccccccc cccccccQDQV@BMMPD@BSFB cccccccc ccccccccVOTIBSF ccccccccOFUOT@TXJUDI
  21.  TOQSJOUG OFU@QBUI TJ[FPG OFU@QBUI TT /&5/4@36/@%*3 OBNF  

    OFUOTPQFO OFU@QBUI 0@3%0/-:c0@$-0&9&$   JG OFUOT \   GQSJOUG TUEFSS $BOOPUPQFOOFUXPSLOBNFTQBDFaTaTaO     OBNF TUSFSSPS FSSOP    SFUVSO  ^  JG TFUOT OFUOT $-0/&@/&8/&5  \   GQSJOUG TUEFSS TFUUJOHUIFOFUXPSLOBNFTQBDFaTaGBJMFETaO     OBNF TUSFSSPS FSSOP    DMPTF OFUOT    SFUVSO  ^ MJCOBNFTQBDFD
  22.  TOQSJOUG OFU@QBUI TJ[FPG OFU@QBUI TT /&5/4@36/@%*3 OBNF  

    OFUOTPQFO OFU@QBUI 0@3%0/-:c0@$-0&9&$   JG OFUOT \   GQSJOUG TUEFSS $BOOPUPQFOOFUXPSLOBNFTQBDFaTaTaO     OBNF TUSFSSPS FSSOP    SFUVSO  ^  JG TFUOT OFUOT $-0/&@/&8/&5  \   GQSJOUG TUEFSS TFUUJOHUIFOFUXPSLOBNFTQBDFaTaGBJMFETaO     OBNF TUSFSSPS FSSOP    DMPTF OFUOT    SFUVSO  ^ MJCOBNFTQBDFD ࡞੒ࡁΈͷOFUOTʹ VOTIBSF
  23.  JG VOTIBSF $-0/&@/&8/4  \   GQSJOUG TUEFSS

    VOTIBSFGBJMFETaO TUSFSSPS FSSOP    SFUVSO  ^   %POUMFUBOZNPVOUTQSPQBHBUFCBDLUPUIFQBSFOU   JG NPVOU   OPOF .4@4-"7&c.4@3&$ /6-- \   GQSJOUG TUEFSS aNPVOUNBLFSTMBWFaGBJMFETaO     TUSFSSPS FSSOP    SFUVSO  ^ MJCOBNFTQBDFD NPVOUOTΛ෼཭͢Δ
  24.  JG VOTIBSF $-0/&@/&8/4  \   GQSJOUG TUEFSS

    VOTIBSFGBJMFETaO TUSFSSPS FSSOP    SFUVSO  ^   %POUMFUBOZNPVOUTQSPQBHBUFCBDLUPUIFQBSFOU   JG NPVOU   OPOF .4@4-"7&c.4@3&$ /6-- \   GQSJOUG TUEFSS aNPVOUNBLFSTMBWFaGBJMFETaO     TUSFSSPS FSSOP    SFUVSO  ^ MJCOBNFTQBDFD .4@4-"7&ʹΑͬͯ͜ͷ/4ʹ ͓͚Δ͋ΒΏΔϚ΢ϯτͷมߋΛ ਌ʹ఻ൖ͠ͳ͍
  25.  JG NPVOU OBNF TZT TZTGT NPVOUqBHT /6--  \

      GQSJOUG TUEFSS NPVOUPGTZTGBJMFETaO TUSFSSPS FSSOP    SFUVSO  ^ MJCOBNFTQBDFD TZTGTΛϚ΢ϯτ͢Δ͜ͱͰ ࣮࣭ίϯςφԽ
  26. VOTIBSF OFUOT VOTIBSF NPVOUOT TMBWF SPPUTZTUFN TZT NPVOU JQOFUOTFYFD͸ ίϯςφͰ͋Δ

    OFUOTͷ*'ʹ*1௥ՃͷΈ΍Γ͍ͨ ͷͰίϯςφͷػೳ͸͍Βͳ͍
  27. WBMHSJOEͰϝϞϦͷΞϩέʔγϣϯΛݟΔ # valgrind curl http://localhost/resource/example > /dev/null … ==18619== HEAP

    SUMMARY: ==18619== in use at exit: 0 bytes in 0 blocks ==18619== total heap usage: 1,863 allocs, 1,863 frees, 211,125 bytes allocated # valgrind /home/pyama/mruby test.rb … ==12811== HEAP SUMMARY: ==12811== in use at exit: 0 bytes in 0 blocks ==12811== total heap usage: 51,721 allocs, 51,721 frees, 12,028,345,116 bytes allocated
  28. TUSBDFͰγεςϜίʔϧͷ౷ܭΛऔΔ # strace -fc /home/pyama/mruby test.rb % time seconds usecs/call

    calls errors syscall ------ ----------- ----------- --------- --------- ---------------- 73.00 0.006317 2 3361 munmap 23.27 0.002014 11 183 brk 2.62 0.000227 0 1734 read 1.11 0.000096 0 3380 mmap 0.00 0.000000 0 3 write 0.00 0.000000 0 16 open 0.00 0.000000 0 23 close NVONBQ͕ࢧ഑త IUUQTMJOVYKNPTEOKQIUNM-%1@NBOQBHFTNBONNBQIUNM
  29. QFSGΛར༻ͯ͠ίʔϧάϥϑΛऔಘ͢Δ # perf record --call-graph dwarf -- /home/pyama/mruby test.rb #

    perf report -g -G —stdio 94.35% 0.03% mruby mruby [.] mrb_str_plus | ---mrb_str_plus | --20.06%-- mrb_vm_exec mrb_vm_run … NSC@TUS@QMVT͕ࢧ഑త IUUQNSVCZPSHEPDTBQJIFBEFSTNSVCZ@'TUSJOHIIUNMNSC@TUS@QMVTGVODUJPO
  30. NSVCZTJNQMFIUUQ while (t = socket.read(1024)) if block_given? yield t next

    end response_text += t end CPEZΛCZUFͣͭಡΈɺ จࣈྻ݁߹͍ͯ͠Δ
  31. NSVCZTJNQMFIUUQ while (t = socket.read(4096)) if block_given? yield t next

    end response_text += t end ୯७ʹ͸όοϑΝαΠζΛ֦ு͠ɺ ಡΈࠐΈճ਺ΛݮΒͤ͹ϧʔϓ͸ݮΔ
  32. NSC@TUS@QMVT MRB_API mrb_value mrb_str_plus(mrb_state *mrb, mrb_value a, mrb_value b) {

    struct RString *s = mrb_str_ptr(a); struct RString *s2 = mrb_str_ptr(b); struct RString *t; t = str_new(mrb, 0, RSTR_LEN(s) + RSTR_LEN(s2)); memcpy(RSTR_PTR(t), RSTR_PTR(s), RSTR_LEN(s)); memcpy(RSTR_PTR(t) + RSTR_LEN(s), RSTR_PTR(s2), RSTR_LEN(s2)); return mrb_obj_value(t); } ݁߹ޙจࣈྻͷϝϞϦΛ Ξϩέʔτ
  33. NSC@TUS@QMVT B C  BC BC D  BCD BCD

    E  BCDE ϧʔϓ໨ όΠτ ϧʔϓ໨ όΠτ ϧʔϓ໨ όΠτ ݁ՌจࣈྻͷྖҬ͸৽ن࡞੒
  34. NSC@TUS@DBU MRB_API mrb_value mrb_str_cat(mrb_state *mrb, mrb_value str, const char *ptr,

    size_t len) { struct RString *s = mrb_str_ptr(str); size_t capa; size_t total; ptrdiff_t off = -1; ... resize_capa(mrb, s, capa); memcpy(RSTR_PTR(s) + RSTR_LEN(s), ptr, len); mrb_assert_int_fit(size_t, total, mrb_int, MRB_INT_MAX); RSTR_SET_LEN(s, total); RSTR_PTR(s)[total] = '\0'; /* sentinel */ return str; } ͍ΘΏΔSFBMMPDͰ طଘͷྖҬΛ֦ு͍ͯ͠Δ
  35. NSC@TUS@DBU B C  BC BC D  BCD BCD

    E  BCDE ϧʔϓ໨ όΠτ ϧʔϓ໨ όΠτ ϧʔϓ໨ όΠτ ݁ՌจࣈྻͷྖҬ͸ݩจࣈྻΛ֦ு
  36. ͲΕ͘Β͍͕ࠩ͋Δͷ͔ʁ a = "aaa" b = "bbb" 100000.times do |n|

    a += b end a = "aaa" b = "bbb" 100000.times do |n| a << b end TVEPWBMHSJOENSVCZOFXSC  )&"146.."3: JOVTFBUFYJUCZUFTJOCMPDLT UPUBMIFBQVTBHF   CZUFTBMMPDBUFE TVEPWBMHSJOENSVCZDPODBUSC  )&"146.."3: JOVTFBUFYJUCZUFTJOCMPDLT UPUBMIFBQVTBHF  CZUFTBMMPDBUFE NSC@TUS@QMVT NSC@TUS@DPODBU ສϧʔϓͩͱ ϝϞϦΞϩέʔγϣϯഒ
  37. QTFGcHSFQBQBDIFcHSFQ9 OTDE BQBDIF%'03&(306/%9 QFSGSFDPSEDBMMHSBQIEXBSGQ QFSGSFQPSUH(TUEJP BQBDIFJPODVCF@MPBEFS@MJO@TP<>YCGD c YGEGGD c c;&/%@*/$-6%&@03@&7"-@41&$@$0/45@)"/%-&3

    cFYFDVUF@FY cYGEGGD cc cc;&/%@*/$-6%&@03@&7"-@41&[email protected]"3@)"/%-&3 ccFYFDVUF@FY ccYGEGGD cc;&/%@*/$-6%&@03@&7"-@41&[email protected]"3@)"/%-&3 ccFYFDVUF@FY ccYGEGGD cc;&/%@*/$-6%&@03@&7"-@41&[email protected]"3@)"/%-&3 ccFYFDVUF@FY "QBDIF)551%͸9Λ͚ͭΔͱ γϯάϧϞʔυͰىಈ͢Δ
  38. 2"