Upgrade to Pro — share decks privately, control downloads, hide ads and more …

自宅スケーラブル・ファイルシステムのご紹介

 自宅スケーラブル・ファイルシステムのご紹介

2012年3月17-18日
Open Source Conference 2012 Tokyo/Spring

Avatar for Kentaro Mitsuyasu

Kentaro Mitsuyasu

June 09, 2026

More Decks by Kentaro Mitsuyasu

Other Decks in Technology

Transcript

  1. ·ͣ͸ϐΞπʔϐΞ͔Β ݕূͯ͠Θ͔ͬͨ͜ͱɿ ݁ہΧʔυʹ͸*1ΛৼΔ͠ɺৼͬͯ͠·͑͹௨ৗͷ /*$ͱมΘΒͳ͍ ରԠ04͸$FOU04ܥͳΒ໰୊ͳ͠ɺԾ૝ԽϋΠύʔ όΠβʔͷରԠ΋ਐΜͰ͍Δ *OGJOJCBOEωοτϫʔΫ಺ʹ͸4VCOFU.BOBHFS͕࠷௿ͭ ىಈ͍ͯ͠ͳ͚Ε͹ͳΒͳ͍ɻ 4VCOFU.BOBHFS͸*OGJOJCBOEεΠονʹ಺ଂͨ͠੡඼΋͋Δɻ 7

    <SPPU!NMd>JGDPOGJH B *GDPOGJH VTFTUIFJPDUM BDDFTTNFUIPEUPHFUUIFGVMMBEESFTTJOGPSNBUJPO XIJDIMJNJUTIBSEXBSFBEESFTTFTUP CZUFT #FDBVTF*OGJOJCBOE BEESFTTIBTCZUFT POMZUIFGJSTUCZUFTBSFEJTQMBZFEDPSSFDUMZ *GDPOGJH JTPCTPMFUF'PSSFQMBDFNFOUDIFDLJQ JC-JOLFODBQ*OGJOJ#BOE )8BEES 999999 JOFU BEES#DBTU.BTL JOFUBEES999999999999G4DPQF-JOL 61#30"%$"4536//*/(.6-5*$"45.56.FUSJD 39QBDLFUTFSSPSTESPQQFEPWFSSVOTGSBNF 59QBDLFUTFSSPSTESPQQFEPWFSSVOTDBSSJFS DPMMJTJPOTUYRVFVFMFO 39CZUFT (J# 59CZUFT .J#
  2. ϐΞπʔϐΞݕূͰ͸·ͬͨϙΠϯτ ఺ ̎ɺϑΝʔϜ΢ΣΞͷߋ৽ NTU TUBUVTίϚϯυͰσόΠεΛௐ΂ΔˠͦΜͳ΋ͷ͸ͳ͍ʂ ɾɾɾNTU TUBSUͰαʔϏεΛ։࢝͢Δͱ໰୊ͳ͘σόΠεൃݟɻ 9 [root@ml115 ~]#

    mst status MST modules: ------------ MST PCI module is not loaded MST PCI configuration module is not loaded MST devices: ------------ No MST devices found [root@ml115 ~]# mst start Starting MST (Mellanox Software Tools) driver set Loading MST PCI module - Success Loading MST PCI configuration module - Success Create devices [root@ml115 ~]# mst status MST modules: ------------ MST PCI module loaded MST PCI configuration module loaded MST devices: ------------ /dev/mst/mt25218_pci_cr0 - PCI direct access. bus:dev.fn=12:00.0 bar=0xfdf00000 size=0x100000 Chip revision is: A0 /dev/mst/mt25218_pciconf0 - PCI configuration cycles access. bus:dev.fn=12:00.0 addr.reg=88 data.reg=92 Chip revision is: A0
  3. ϐΞπʔϐΞݕূͰ͸·ͬͨϙΠϯτ ఺ ̏ɺωοτϫʔΫઃఆʢ*1ΞυϨεʣ ॳΊ͸TZTUFNDPOGJHOFUXPSLUVJͰઃఆ͕ͨ͠ɺͭͳ͕Βͣɻ ݁ہԼهͷઃఆͰಈ࡞Λ֬ೝͨ͠ɻ 11 /etc/sysconfig/network-scripts/ifcfg-ib0 # Please read

    /usr/share/doc/initscripts-*/sysconfig.txt # for the documentation of these parameters. DEVICE=ib0 BOOTPROTO=static NETMASK=255.255.255.0 TYPE=Infiniband #HWADDR=80:00:04:04:fe:80 IPADDR=192.168.50.60 BROADCAST=192.168.50.255 NETWORK=192.168.50.0 ONBOOT=yes
  4. ͭͷಈ࡞Ϟʔυ 14 Server1 Server2 Server3 Server1 Server2 Server3 Distributed Striped

    ϑΝΠϧసૹ଎౓ɿ֤ϊʔυ୯ମͷೳྗʹ ࠨӈʹଞͷαʔόʔͷ*0ͱ଍ฒΈΛͦΖ͑ Δඞཁ͕ͳ͍ ϑΝΠϧసૹ଎౓ɿ֤ϊʔυ୯ମͷೳྗͱ ετϥΠϓʹࢀՃ͢Δ୆਺ʹࠨӈʹ*0ͷ஗ ͍αʔόʔʹ଍ΛҾͬுΒΕΔ
  5. ֤ϊʔυͷߏ੒ 17 brick1 brick2 brick3 brick4 server1 IB IP: 192.168.50.10

    IB IP: 192.168.50.20 IB IP: 192.168.50.1 IB IP: 192.168.50.60 IB IP: 192.168.50.70
  6. 70-6.&ͷ࡞੒ 18 #SJDLdɿ 044DJFOUJGJD -JOVYY@ HMVTUFSHMVTUFSGT CFUB JOGJOJCBOEPGFE )$".FMMBOPY .)&"95$(C

    4FSWFS 04$FOU04Y@ )$".FMMBOPY .)&"95$(C EJTUSJCVUFEEVQMJDBUFͷ৔߹ͷίϚϯυ HMVTUFS WPMVNFDSFBUFHMVTUFSWPM SFQMJDBUSBOTQPSUUDQ SENB EBUBCSJDLEBUBCSJDL IPNFCSJDLIPNFCSJDL 4USJQFEEVQMJDBUFͷ৔߹ͷίϚϯυ HMVTUFS WPMVNFDSFBUFHMVTUFSWPM TUSJQFSFQMJDBUSBOTQPSUUDQ SENB EBUBCSJDLEBUBCSJDL IPNFCSJDLIPNFCSJDL
  7. ෳ਺ಉ࣌Ϛ΢ϯτ ͭͷϘϦϡʔϜʹରͯ͠ɺͭͷϚγϯ͔ΒϚ΢ϯτͰ͖Δ͔ɻ ʢͲͪΒ΋HMVTUFSωΠςΟϒΫϥΠΞϯτʣ ˠ ໰୊ͳ͘Ϛ΢ϯτͰ͖͕ͨɺ౰વͳ͕ΒϩοΫػೳ͸ͳ͠ɻ 21 brick1 brick2 brick3 brick4

    server1 server2 gluster volume start glustervol mount -t glusterfs -o mountproto=rdma 192.168.50.10:/glustervol /root/ibtest mount -t glusterfs -o mountproto=rdma 192.168.50.10:/glustervol /root/ibtest
  8. ωοτϫʔΫɺػثߏ੒ 29 brick1 brick2 brick3 brick4 server1 server2 CPU:Core i5-670

    Mem:8GB DDR3 PC1333 HDD: HITACHI 2TB 7200rpm SATA x4 M/B Intel DQ57TM CPU:Pentium G620 Mem:4GB DDR3 PC1333 HDD: Intel SSD 80GB M/B Intel DQ67EP CPU:Celeron G1101 Mem:4GB DDR3 PC1333 HDD: HITACHI 1TB SATA HDD M/B HP ML110G6 CPU:Intel Core-i5 2400S Mem:8GB DDR3 PC1333 HDD: HITACHI 1TB SATA HDD x2 M/B WADE-8012 CPU:Celeron G1101 Mem:4GB DDR3 PC1333 HDD: SEAGATE 1TB SATA HDD M/B NEC Express5800 CPU:AMD Athlon 3core Mem:6GB DDR3 PC1333 HDD: SEAGATE 2TB SATA HDD M/B HP ML115 Router: Cisco ASA5505 To internet