Using Ceph-Deploy

JTF 2014 での発表資料

Transcript

1. Using Ceph-Deploy Tomokazu HIRAI @jedipunkz
2. Index * ࣗݾ঺հ * Ceph-Deploy ͱ͸ ? * ߏ੒ * Ceph-Deploy Λ༻͍Δ४උ * खॱ : conf ϑΝΠϧͷੜ੒ * खॱ : Ceph ͷΠϯετʔϧ * खॱ : MON ͷՔಇ * खॱ : 伴ͷڞ༗ * खॱ : OSD ͷՔಇ * खॱ : MDS ͷՔಇ * खॱ : ֤छϚ΢ϯτํ๏
3. ࣗݾ঺հ ! * ໊લ: Tomokazu HIRAI * ࢓ࣄ: ϗεςΟϯάձࣾۈ຿ * ೔ຊ OpenStack Ϣʔβձ * #Ceph, #Chef, #Fog, #Swift, #Sensu, #Ruby, #docker, #Mesos, … * http://jedipunkz.github.io/ * https://twitter.com/jedipunkz
4. ࠓճͷखॱͲ͓Γʹૢ࡞ ͢Ε͹ɺͱΓ͋͑ͣಈ͖·͢.
5. Ceph-Deploy ͱ͸ʁ * Ceph ͷެࣜσϓϩΠπʔϧ * Python ੡ * ެ։৔ॴ : https://github.com/ceph/ceph-deploy * ϚχϡΞϧͰΠϯετʔϧ͢Δͱ ‘Ceph-Deploy ࢖͑’ ͱ ౖΒΕΔ !
6. ߏ੒ EFWTEC EFWTED EFWTTE DFQI
   OPEF NPO PTE PTE EFWTEC EFWTED EFWTTE DFQI
   OPEF NPO PTE PTE EFWTEC PTE EFWTED EFWTTE DFQI
   OPEF NPO PTE DFQI
   NET $MVTUFS
   /FUXPSL 1VCMJD
   /FUXPSL DFQI NET
7. Ceph-Deploy Λ༻͍Δ४උ #1 Ceph-Deploy ͸ Public Network ্ͷͲ͔͜ͷϊʔυͰ࣮ߦ ͜ͷखॱͰ͸ ceph05 Ͱ࣮ߦ͢Δ͜ͱΛલఏʹ ! % sudo apt-get update ; sudo apt-get -y install python-virtualenv % git clone https://github.com/ceph/ceph-deploy.git ~/ceph-deploy % cd ~/ceph-deploy % ./bootstrap ! % sudo ${EDITOR} /etc/hosts 10.200.10.1 ceph01 10.200.10.2 ceph02 10.200.10.3 ceph03 10.200.10.4 ceph04 10.200.10.5 ceph05
8. Ceph-Deploy Λ༻͍Δ४උ #2 % ssh-keygen % ssh-copy-id ceph@ceph01 % ssh-copy-id ceph@ceph02 % ssh-copy-id ceph@ceph03 % ssh-copy-id ceph@ceph04 ! targets# ${EDITOR} /etc/sudoers.d/ceph cephALL=(ALL) NOPASSWD:ALL
9. Conf ϑΝΠϧͷੜ੒ #1 % ceph-deploy --cluster cluster01 new \ ceph01 ceph02 ceph03 ceph04 ceph05 ! * ΧϨϯτσΟϨΫτϦʹ cluster01.conf ͕ੜ੒͞ΕΔ * ߲࣍ͷίϯϑΟΪϡϨʔγϣϯΛ௥ه͢Δ
10. Conf ϑΝΠϧͷੜ੒ #2 public network = <network_addr> cluster network = <network_addr> ! [mon.a] host = ceph01 mon addr = <ceph01_ip_addr>:6789 ! [mon.b] host = ceph02 mon addr = <ceph02_ip_addr>:6789 ! [mon.c] host = ceph03 mon addr = <ceph03_ip_addr>:6789 ! <ӈ΁ଓ͘> [osd.0] public addr = <ceph01_public_ip_addr> cluster addr = <ceph01_cluster_ip_addr> ! [osd.1] public addr = <ceph01_public_ip_addr> cluster addr = <ceph01_cluster_ip_addr> ! [osd.2] public addr = <ceph01_public_ip_addr> cluster addr = <ceph01_cluster_ip_addr> ! [mds.a] host = ceph04 ! [mds.b] host = ceph05
11. Ceph ͷΠϯετʔϧ % ceph-deploy --cluster cluster01 install \ ceph01 ceph02 ceph03 ceph04 ceph05 ! Πϯετʔϧ͢Δόʔδϣϯͷࢦఆ͸Լهͷ௨Γߦ͏ ! --release <code-name> --testing --dev <branch-or-tag>
12. MON ͷՔಇ % ceph-deploy --cluster cluster01 mon create \ ceph01 ceph02 ceph03
13. 伴ͷڞ༗ MON ϗετ͔Β伴Λऔಘ͠ OSD, MDS ϒʔτετϥοϓʹඋ͑Δ ! % ceph-deploy --cluster cluster01 gatherkeys \ ceph01 ceph02 ceph03
14. OSD ͷՔಇ Disk Ұཡͷ֬ೝ % ceph-deploy --cluster cluster01 disk list ceph01 % ceph-deploy --cluster cluster01 disk list ceph02 % ceph-deploy --cluster cluster01 disk list ceph03 ! Disk ΛॳظԽ % ceph-deploy --cluster cluster01 disk zap ceph01:/dev/sdb ceph01:/dev/sdc % ceph-deploy --cluster cluster01 disk zap ceph02:/dev/sdb ceph02:/dev/sdc % ceph-deploy --cluster cluster01 disk zap ceph03:/dev/sdb ceph03:/dev/sdc ! ϝλσʔλΛ֨ೲ͢Δ Disk ͷύʔςΟγϣχϯά (sdb, sdc ༻ 2 ͭ) targets% sudo gdisk /dev/ssd #<= ֤ϊʔυͰ࣮ࢪ /dev/ssd1, ssd2 Λੜ੒ ! OSD ͷՔಇ % ceph-deploy --cluster cluster01 osd create ceph01:sdb:/dev/ssd1 \ ceph02:sdb:/dev/ssd1 ceph03:sdb:/dev/ssd1 % ceph-deploy --cluster cluster01 osd create ceph01:sdc:/dev/ssd2 \ ceph02:sdc:/dev/ssd2 ceph03:sdc:/dev/ssd2
15. MDS ͷՔಇ % ceph-deploy --cluster cluster01 mds create \ ceph04 ceph05
16. ֤छϚ΢ϯτํ๏
17. ֤छϚ΢ϯτํ๏ Block Device Mount % sudo rbd -c /etc/ceph/cluster01.conf create foo --keyring \ /etc/ceph/cluster01.client.admin.keyring --size 4096 ! % sudo modprobe rbd ! % sudo rbd map foo --pool rbd --name client.admin --keyring \ /etc/ceph/cluster01.client.admin.keyring -c \ /etc/ceph/cluster01.conf ! % sudo mkfs.ext4 -m0 /dev/rbd/rbd/foo % sudo mkdir /mnt/myrbd % sudo mount /dev/rbd/rbd/foo /mnt/myrbd
18. ֤छϚ΢ϯτํ๏ Kernel Driver Mount % sudo mkdir /mnt/mykernelfs % sudo mount -t ceph 10.200.10.1:6789:/ \ /mnt/mykernelfs -o \ name=admin,secret=`sudo ceph-authtool \ -p /etc/ceph/cluster01.client.admin.keyring `
19. ֤छϚ΢ϯτํ๏ Fuse Driver Mount % sudo apt-get install ceph-fuse % sudo ceph-fuse -m 10.200.9.116:6789 \ /mnt/myfusefs \ -c /etc/ceph/cluster01.conf —keyring /etc/ceph/cluster01.client.admin.keyring
20. ·ͱΊ
21. ·ͱΊ * ϚχϡΞϧૢ࡞ېࢭ -> Ceph-Deploy Λ༻͍Δͷ͕ެࣜखॱ * ΄΅શͯͷૢ࡞Λ Ceph-Deploy Λ༻͍ͯ֎෦͔Βߦ͏ * OSD ͸ Disk 1ຊʹରͯ͠ 1 ϓϩηεͷؔ܎ * OSD, MON, MDS ͸ͦΕͧΕϨϓϦέʔγϣϯ͕औΕΔ Ceph ͸ SPOF ͕ແ͍ Distributed Storage System Ͱ͋Δ