Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Using Ceph-Deploy

Using Ceph-Deploy

JTF 2014 での発表資料

Tomokazu HIRAI

June 05, 2014
Tweet

More Decks by Tomokazu HIRAI

Other Decks in Technology

Transcript

  1. Index * ࣗݾ঺հ * Ceph-Deploy ͱ͸ ? * ߏ੒ *

    Ceph-Deploy Λ༻͍Δ४උ * खॱ : conf ϑΝΠϧͷੜ੒ * खॱ : Ceph ͷΠϯετʔϧ * खॱ : MON ͷՔಇ * खॱ : 伴ͷڞ༗ * खॱ : OSD ͷՔಇ * खॱ : MDS ͷՔಇ * खॱ : ֤छϚ΢ϯτํ๏
  2. ࣗݾ঺հ ! * ໊લ: Tomokazu HIRAI * ࢓ࣄ: ϗεςΟϯάձࣾۈ຿ *

    ೔ຊ OpenStack Ϣʔβձ * #Ceph, #Chef, #Fog, #Swift, #Sensu, #Ruby, #docker, #Mesos, … * http://jedipunkz.github.io/ * https://twitter.com/jedipunkz
  3. Ceph-Deploy ͱ͸ʁ * Ceph ͷެࣜσϓϩΠπʔϧ * Python ੡ * ެ։৔ॴ

    : https://github.com/ceph/ceph-deploy * ϚχϡΞϧͰΠϯετʔϧ͢Δͱ ‘Ceph-Deploy ࢖͑’ ͱ ౖΒΕΔ !
  4. ߏ੒ EFWTEC EFWTED EFWTTE DFQIOPEF NPO PTE PTE EFWTEC EFWTED

    EFWTTE DFQIOPEF NPO PTE PTE EFWTEC PTE EFWTED EFWTTE DFQIOPEF NPO PTE DFQI NET $MVTUFS/FUXPSL 1VCMJD/FUXPSL DFQI NET
  5. Ceph-Deploy Λ༻͍Δ४උ #1 Ceph-Deploy ͸ Public Network ্ͷͲ͔͜ͷϊʔυͰ࣮ߦ ͜ͷखॱͰ͸ ceph05

    Ͱ࣮ߦ͢Δ͜ͱΛલఏʹ ! % sudo apt-get update ; sudo apt-get -y install python-virtualenv % git clone https://github.com/ceph/ceph-deploy.git ~/ceph-deploy % cd ~/ceph-deploy % ./bootstrap ! % sudo ${EDITOR} /etc/hosts 10.200.10.1 ceph01 10.200.10.2 ceph02 10.200.10.3 ceph03 10.200.10.4 ceph04 10.200.10.5 ceph05
  6. Ceph-Deploy Λ༻͍Δ४උ #2 % ssh-keygen % ssh-copy-id ceph@ceph01 % ssh-copy-id

    ceph@ceph02 % ssh-copy-id ceph@ceph03 % ssh-copy-id ceph@ceph04 ! targets# ${EDITOR} /etc/sudoers.d/ceph cephALL=(ALL) NOPASSWD:ALL
  7. Conf ϑΝΠϧͷੜ੒ #1 % ceph-deploy --cluster cluster01 new \ ceph01

    ceph02 ceph03 ceph04 ceph05 ! * ΧϨϯτσΟϨΫτϦʹ cluster01.conf ͕ੜ੒͞ΕΔ * ߲࣍ͷίϯϑΟΪϡϨʔγϣϯΛ௥ه͢Δ
  8. Conf ϑΝΠϧͷੜ੒ #2 public network = <network_addr> cluster network =

    <network_addr> ! [mon.a] host = ceph01 mon addr = <ceph01_ip_addr>:6789 ! [mon.b] host = ceph02 mon addr = <ceph02_ip_addr>:6789 ! [mon.c] host = ceph03 mon addr = <ceph03_ip_addr>:6789 ! <ӈ΁ଓ͘> [osd.0] public addr = <ceph01_public_ip_addr> cluster addr = <ceph01_cluster_ip_addr> ! [osd.1] public addr = <ceph01_public_ip_addr> cluster addr = <ceph01_cluster_ip_addr> ! [osd.2] public addr = <ceph01_public_ip_addr> cluster addr = <ceph01_cluster_ip_addr> ! [mds.a] host = ceph04 ! [mds.b] host = ceph05
  9. Ceph ͷΠϯετʔϧ % ceph-deploy --cluster cluster01 install \ ceph01 ceph02

    ceph03 ceph04 ceph05 ! Πϯετʔϧ͢Δόʔδϣϯͷࢦఆ͸Լهͷ௨Γߦ͏ ! --release <code-name> --testing --dev <branch-or-tag>
  10. OSD ͷՔಇ Disk Ұཡͷ֬ೝ % ceph-deploy --cluster cluster01 disk list

    ceph01 % ceph-deploy --cluster cluster01 disk list ceph02 % ceph-deploy --cluster cluster01 disk list ceph03 ! Disk ΛॳظԽ % ceph-deploy --cluster cluster01 disk zap ceph01:/dev/sdb ceph01:/dev/sdc % ceph-deploy --cluster cluster01 disk zap ceph02:/dev/sdb ceph02:/dev/sdc % ceph-deploy --cluster cluster01 disk zap ceph03:/dev/sdb ceph03:/dev/sdc ! ϝλσʔλΛ֨ೲ͢Δ Disk ͷύʔςΟγϣχϯά (sdb, sdc ༻ 2 ͭ) targets% sudo gdisk /dev/ssd #<= ֤ϊʔυͰ࣮ࢪ /dev/ssd1, ssd2 Λੜ੒ ! OSD ͷՔಇ % ceph-deploy --cluster cluster01 osd create ceph01:sdb:/dev/ssd1 \ ceph02:sdb:/dev/ssd1 ceph03:sdb:/dev/ssd1 % ceph-deploy --cluster cluster01 osd create ceph01:sdc:/dev/ssd2 \ ceph02:sdc:/dev/ssd2 ceph03:sdc:/dev/ssd2
  11. ֤छϚ΢ϯτํ๏ Block Device Mount % sudo rbd -c /etc/ceph/cluster01.conf create

    foo --keyring \ /etc/ceph/cluster01.client.admin.keyring --size 4096 ! % sudo modprobe rbd ! % sudo rbd map foo --pool rbd --name client.admin --keyring \ /etc/ceph/cluster01.client.admin.keyring -c \ /etc/ceph/cluster01.conf ! % sudo mkfs.ext4 -m0 /dev/rbd/rbd/foo % sudo mkdir /mnt/myrbd % sudo mount /dev/rbd/rbd/foo /mnt/myrbd
  12. ֤छϚ΢ϯτํ๏ Kernel Driver Mount % sudo mkdir /mnt/mykernelfs % sudo

    mount -t ceph 10.200.10.1:6789:/ \ /mnt/mykernelfs -o \ name=admin,secret=`sudo ceph-authtool \ -p /etc/ceph/cluster01.client.admin.keyring `
  13. ֤छϚ΢ϯτํ๏ Fuse Driver Mount % sudo apt-get install ceph-fuse %

    sudo ceph-fuse -m 10.200.9.116:6789 \ /mnt/myfusefs \ -c /etc/ceph/cluster01.conf —keyring /etc/ceph/cluster01.client.admin.keyring
  14. ·ͱΊ * ϚχϡΞϧૢ࡞ېࢭ -> Ceph-Deploy Λ༻͍Δͷ͕ެࣜखॱ * ΄΅શͯͷૢ࡞Λ Ceph-Deploy Λ༻͍ͯ֎෦͔Βߦ͏

    * OSD ͸ Disk 1ຊʹରͯ͠ 1 ϓϩηεͷؔ܎ * OSD, MON, MDS ͸ͦΕͧΕϨϓϦέʔγϣϯ͕औΕΔ Ceph ͸ SPOF ͕ແ͍ Distributed Storage System Ͱ͋Δ