OpenStack環境で利用されるハイパフォーマンスストレージの紹介

ddn.com © 2015 DataDirect Networks, Inc. * Other names and
brands may be claimed as the property of others. Any statements or representations around future events are subject to change. OpenStack 環境で利用されるハイパフォーマンスストレージの紹介 2016/01/29 DataDirect Networks Japan, Inc. 塩入ヶ谷寛 Hiroshi Shioirigatani

brands may be claimed as the property of others. Any statements or representations around future events are subject to change. 自己紹介 ► DataDirect Networks Japan • 塩入ヶ谷寛 (しおいりがたにひろし） o プリセルースエンジニア ► 主な担当 • 製造・自動車 CAE • 通信事業 Big Data • クラウド事業 • 仮想化・OpenStack 2

brands may be claimed as the property of others. Any statements or representations around future events are subject to change. Agenda ► DDN とは ► DDN 製品ポートフォリオ ► DDN GRIDScaler について • GPFS とは • GPFS の基本構成 ► OpenStack 環境での GPFS の利用 • メリット • Storage Pool • Replica ► Performance • GPFS Local Read Only Cache vs DDN SFX 3

brands may be claimed as the property of others. Any statements or representations around future events are subject to change. 4 DDN とは

brands may be claimed as the property of others. Any statements or representations around future events are subject to change. DDN について 5 顧客： 1,100以上、50カ国大規模環境におけるビッグデータ・ライフサイクルマネージメントの課題を解決します数々の受賞歴従業員： 650名以上、20カ国ビジネスモデル：パートナービジネス・一部ダイレクト歴史： 1998年設立本社：米国カリフォルニアサンタクララ製造拠点：米国カリフォルニアチャッツワース

brands may be claimed as the property of others. Any statements or representations around future events are subject to change. DDN Japan 6 代表者代表取締役社長：ロベルト・トリンドル取締役：Alex Bouzari （DDN米国本社CEO) 取締役：Paul Bloch （DDN米国本社ボードメンバー) 社員 30 名以上事業内容 DDN ストレージ・ソリューションの販売, DDN ストレージ・ソリューションの設計, 構築、保守国内代理店 SCSK, CTCSP, 日本 SGI, 朋栄、加賀ソルネット、, 伊藤忠ケーブルシステム、HPC ソリューションズ OEM販売 SONY, 日本 SGI、クレイ・ジャパン・インク、日本 IBM、日本 HP

brands may be claimed as the property of others. Any statements or representations around future events are subject to change. DDN とは ► ハイパフォーマンスストレージ • SFA7700X • SFA12KX • SFA14K ► 高密度ディスクエンクロージャ • 4 U で 3.5 インチディスクを 84 本搭載可能 • 6Gb SAS または、 12Gb SAS ► ファイル・オブジェクトアクセスソリューションの提供 • GPFS (IBM Spectrum Scale) • Lustre • IME （Infinite Memory Engine） • WOS （Web Object Scaler） ► HPC 業界での実績 7

brands may be claimed as the property of others. Any statements or representations around future events are subject to change. GPFS with DDN SFA12KX パフォーマンス 8 38 Pool 53 Pool 76 Pool Write (GB/s) 26.57 32.31 34.11 Read (GB/s) 28.80 35.61 39.17 26.57 32.31 34.11 28.80 35.61 39.17 0.00 5.00 10.00 15.00 20.00 25.00 30.00 35.00 40.00 45.00 GB/sec

brands may be claimed as the property of others. Any statements or representations around future events are subject to change. DDN SFA12KXi All Flash Array 9 0 500,000 1,000,000 1,500,000 2,000,000 2,500,000 3,000,000 3,500,000 4,000,000 4,500,000 1 8 16 32 40 48 56 64 IOPS Pool Count Random Read IOPS 4K I/O Size 8K I/O Size 16K I/O Size 32K I/O Size 64K I/O Size 256K I/O Size 0 5,000 10,000 15,000 20,000 25,000 30,000 35,000 40,000 45,000 50,000 1 8 16 32 40 48 56 64 72 Rate (MB/s) Pool Count Random Read Throughput 64K I/O Size 256K I/O Size 512K I/O Size 1M I/O Size 2M I/O Size 4M I/O Size 0 500,000 1,000,000 1,500,000 2,000,000 2,500,000 3,000,000 1 8 16 32 40 48 56 64 72 IOPS Pool Count Random 50%-50% Read-Write 4K I/O Size 8K I/O Size 16K I/O Size 32K I/O Size 64K I/O Size 256K I/O Size TEST CONFIGURATION: SFX12KXi w 5x Shelves 144x SSD (RAID-1)

brands may be claimed as the property of others. Any statements or representations around future events are subject to change. 高密度エンクロージャ DDN SS84xx 10 SS8460 Internals Completely Fault Tolerant IO Module A 42 x Dual-Port HDDs/SSDs 42 x Dual-Port HDDs/SSDs IO Module B Multipath Controller I/O 16 x SAS 6Gb 4 x SAS 6Gb 4 x SAS 6Gb 4 x SAS 6Gb 4 x SAS 6Gb 4 x SAS 6Gb 4 x SAS 6Gb 4 x SAS 6Gb 4 x SAS 6Gb 42 x SAS 6Gb 42 x SAS 6Gb 42 x SAS 6Gb 42 x SAS 6Gb

brands may be claimed as the property of others. Any statements or representations around future events are subject to change. DDN ストレージ HPC 業界での実績 11 80% of the Top 10 67% of the Top 100 32% of the Top 500

brands may be claimed as the property of others. Any statements or representations around future events are subject to change. 12 DDN 製品ポートフォリオ

brands may be claimed as the property of others. Any statements or representations around future events are subject to change. ペタスケール Lustre® ストレージ Up to 100,000 Clients Open Source Leading metadata & small file performance EXAScaler™ DDN 製品ポートフォリオ 13 コアストレージプラットフォーム 48GB/s, 1.7M IOPS 1,680 Drives in 2 Racks Embedded Computing 12.5GB/s, 450K IOPS 60 Drives in 4U 396 Drives in 20U SFA™12KX SFA™7700X SFA14K 60GB/s, 6M IOPS 72 Drives in 4U 2.5in 1680 Drives in 2 Racks エンタープライズスケールアウト GPFS ファイルストレージ Up to ~16,000 Clients Integrated Object Storage, Tape NFS & CIFS & Swift & HDFS Snapshots, Replication GRIDScaler™ / GS7K* SFA Software SSD Cache SFX™ Intelligent Cache Management ReACT™ Full Data Protection Direct Protect™ ストレージエンクロージャ 4 U 84 Drives 6 Gbps SAS SS8460 4 U 84 Drives 12 Gbps SAS SS8412 柔軟なドライブ構成 1 つのディスクエンクロージャに混在可能 SAS SSD , NVMe NL-SAS Virtualization Environment VMware Cinder Swift Glance Manila NFS FC

brands may be claimed as the property of others. Any statements or representations around future events are subject to change. 通常のストレージ 14 Cache Link Write I/O Cache Battery-Backed L2 Cache Battery-Backed L2 Cache

brands may be claimed as the property of others. Any statements or representations around future events are subject to change. DDN SFA ReACT 15 IOサイズに応じ、シーケンシャルデータはキャッシュを介さず直接ディスクに書き込み、小さなランダムデータはキャッシュに書き込まれます。その結果、ライトキャッシュを有効利用する事でシステム全体の書き込みの最適化を実現します。 Aligned I/O  シーケンシャルデータ(フルストライプ)はキャッシュを利用せず、ディスクに直接ストライプ書込され、キャッシュリンクを利用せずオーバヘッドを軽減する Unaligned I/O  ランダムデータはキャッシュに書き込み、ランダムWrite I/O性能の低下を防ぐ High-Speed Cache Link Full Stripe Small Write I/O Cache Battery-Backed SFA L2 Cache Battery-Backed SFA L2 Cache

brands may be claimed as the property of others. Any statements or representations around future events are subject to change. 16 DDN GRIDScaler について GPFS (IBM Spectrum Scale)

brands may be claimed as the property of others. Any statements or representations around future events are subject to change. GPFS とは 17 ► 複数のアクセス方式を有する、スケールアウト型の SDS • 高速なシミュレーションや解析、Cloud環境で必要なパラレルI/Oを提供 • 汎用的にアクセスするためのNAS を同時に提供 o GPFS , NFS , CIFS , Swift ► DDN SFA ストレージとの組み合わせで、省スペース・ハイパフォーマンス I/Oを提供 • 1 クライアントから、最大 4GB/sec の I/O を実現※1 ► 豊富な機能 • Snapshot/Rollback • File Level Clone • Quota • Data and Metadata Replica • Active File Management • Disaster Recovery • HSM (ILM) • Local Read Only Cache • HAWC • NFS / CIFS / Swift / HDFS RPC Access • CLI / GUI Management ※1 Infiniband FDR を使用 NSD#1 NSD#2 NSD#3 NSD#4 DDN SFA GPFS Client 複数のサーバから同時に Read

brands may be claimed as the property of others. Any statements or representations around future events are subject to change. GPFS 基本構成 18 NSD Server 1 NSD Server 2 ・・・・ NSD Server N Protocol Node NFS Protocol Node CIFS Protocol Node Swift GPFS Client GPFS Client GPFS Client ・・・・ Protocol Client Protocol Client Protocol Client ・・・・・・・・ LROC GPFS File System NSD NSD NSD LROC NFS CIFS Swift NSD: Network Shared Disk Ethernet Infiniband FC or IB LROC: Local Read-Only Cache Ethernet IPoIB HDFS Client GUI Node WOS Core Tape Library TSM Bridge Migration Cinder Glance Object Storage

brands may be claimed as the property of others. Any statements or representations around future events are subject to change. 19 OpenStack 環境における GPFS

brands may be claimed as the property of others. Any statements or representations around future events are subject to change. OpenStack 環境における GPFS メリット ► Compute Node からのパラレル I/O ► GPFS Client （Compute Node）のパフォーマンス監視が可能な GUI ► Storage Pool (ILM) を利用したデータマネージメント ► 利用用途に合わせた、データレプリケーション ► GPFS Client （Compute Node）が GPFS で利用するメモリを制限 ► 複数の Cinder Volume の作成速度 ► Swift でのアクセス、 Manila によるファイルサービスの提供も可能 20

brands may be claimed as the property of others. Any statements or representations around future events are subject to change. GPFS GUI 22 ► カスタマイズ可能なダッシュボード

brands may be claimed as the property of others. Any statements or representations around future events are subject to change. Performance Metrics 23

brands may be claimed as the property of others. Any statements or representations around future events are subject to change. GPFS Storage Pool 25 SSD NL-SAS SSD SSD NL-SAS NL-SAS NL-SAS NL-SAS NL-SAS GPFS File System System Storage Pool NL-SAS Storage Pool /gpfs glance cinder ・・・さまざまなポリシーに基づきデータをマイグレーションスタブ [gpfs_ssd] gpfs_storage_pool = system [gpfs_nl-sas] gpfs_storage_pool = NL-SAS Volume Type Fast Normal Metadata Volume-xxxxxxx Cinder.conf

brands may be claimed as the property of others. Any statements or representations around future events are subject to change. cinder.conf • volume_driver = cinder.volume.drivers.ibm.gpfs.GPFSDriver • gpfs_images_dir = /gpfs/glance • gpfs_images_share_mode = copy_on_write • gpfs_max_clone_depth = 0 • gpfs_mount_point_base /gpfs • gpfs_sparse_volumes = True • gpfs_storage_pool = system 26

brands may be claimed as the property of others. Any statements or representations around future events are subject to change. GPFS Replica 28 GPFS File System Failure Group 100 Failure Group 200 Replica # cinder create --metadata fstype=ext4 fslabel=newfs dio=yes replicas=1 --display-name vol01 50 # cinder create --metadata fstype=ext4 fslabel=newfs dio=yes replicas=2 --display-name vol02 50 vol01 vol02 Replica NSD Server SFA7700X

brands may be claimed as the property of others. Any statements or representations around future events are subject to change. GPFS Cinder Create • fstype • fslabel • data_pool_name • replicas • dio • write_affinity_depth • block_group_factor • write_affinity_failure_group 29 # cinder create --metadata fstype=ext4 fslabel=newfs dio=yes replicas=2 --display-name volume_1 50

brands may be claimed as the property of others. Any statements or representations around future events are subject to change. 30 構成例

brands may be claimed as the property of others. Any statements or representations around future events are subject to change. 構成例 2 x NSD Server with SFA7700X 31 Pool 0 Vdisk 0 (LUN0) Raid6 (8d+2p) Pool 1 Vdisk 1 (LUN1) Raid6 (8d+2p) Pool 8 Vdisk 8 (LUN8) Raid6 (8d+2p) Pool 9 Vdisk 9 (LUN9) Raid6 (8d+2p) nsd01 21 TiB nsd02 21 TiB nsd08 21 TiB nsd09 21 TiB SFA7700X 8 TB NL-SAS x120 750 TiB /gpfs glance cinder volume-5cbd0b01-6a20-4b00-adb6-c3d06be83b45 volume-8ba2535d-f4b4-477e-a013-275db98f71e3 snap-8ba2535d-f4b4-477e-a013-275db98f71e3 ・・・・・・ 10 TiB 30TiB share1 100 GB Compute Nodes GPFS Clients Quota 値仮想マシン用ボリュームファイル仮想マシン用ボリュームファイル仮想マシン用スナップショットファイル Cinder Glance Controller Node GPFS Client ・・・ 10 LUN ・・・ GPFS File System GPFS Filesystem は任意の LUN を組みあわせて作成することが可能です。 Virtual Machines GPFS Server (NSD Server) NSD Server NSD Server NSD Server NSD Server NSD Server NSD Server NSD Server NSD Server

brands may be claimed as the property of others. Any statements or representations around future events are subject to change. 構成例 GPFS アプライアンス GS7K 32 Pool 0 Vdisk 0 (LUN0) Raid6 (8d+2p) Pool 1 Vdisk 1 (LUN1) Raid6 (8d+2p) Pool 8 Vdisk 8 (LUN8) Raid6 (8d+2p) Pool 9 Vdisk 9 (LUN9) Raid6 (8d+2p) nsd01 21 TiB nsd02 21 TiB nsd08 21 TiB nsd09 21 TiB GPFS Apliance GS7K 8 TB NL-SAS x120 750 TiB /gpfs glance cinder volume-5cbd0b01-6a20-4b00-adb6-c3d06be83b45 volume-8ba2535d-f4b4-477e-a013-275db98f71e3 snap-8ba2535d-f4b4-477e-a013-275db98f71e3 ・・・・・・ 10 TiB 30TiB share1 100 GB Compute Nodes GPFS Clients Quota 値仮想マシン用ボリュームファイル仮想マシン用ボリュームファイル仮想マシン用スナップショットファイル Cinder Glance Controller Node GPFS Client ・・・ 10 LUN ・・・ GPFS File System GPFS Filesystem は任意の LUN を組みあわせて作成することが可能です。 Virtual Machines 10GbE or 40GbE 10GbE or 40GbE 10GbE or 40GbE 10GbE or 40GbE NSD Server NSD Server NSD Server NSD Server NSD Server NSD Server NSD Server NSD Server NSD Server NSD Server SFA7700X

brands may be claimed as the property of others. Any statements or representations around future events are subject to change. 33 Performance GPFS LROC vs DDN SFX

brands may be claimed as the property of others. Any statements or representations around future events are subject to change. Benchmark Result From 32 VM and 64 VM Client with LROC 34 Linux GPFS Clients 2 x Dell R630 １０GbE Swich 1 x Brocade TurboIron NSD Servers 2 x Dell R630 NSD 1 x SF7700X-FC 1 x SS8460 100 x 900 GB 10krpm SAS 8 Gb FC 10 Gb Ethernet Benchmark Environment SSD SSD SSD SSD VM VM VM VM VM VM ・・・ VM VM VM VM VM VM ・・・ 32 VM 32 VM 0 2,000 4,000 6,000 8,000 10,000 12,000 14,000 16,000 10 20 30 40 50 60 70 80 90 100 110 120 130 140 150 160 170 180 190 200 210 220 230 240 250 260 270 280 290 300 310 320 330 340 350 360 370 380 390 400 410 420 430 440 450 460 470 480 490 500 510 520 530 540 550 560 570 580 590 600 Random 4KB Read IOPS from VM on GPFS Client 32 VM(1 GPFS Client) Read 64 VM(2 GPFS Client) Read Read Read 32 VM(1 GPFS Client) 64 VM(2 GPFS Client) Avarage 9,106 14,417 0 2,000 4,000 6,000 8,000 10,000 12,000 14,000 16,000 IOPS Random 4KB Read IOPS from VM on GPFS Client

brands may be claimed as the property of others. Any statements or representations around future events are subject to change. Benchmark Result From 32 VM and 64 VM Client with SFX 35 Linux GPFS Clients 2 x Dell R630 １０GbE Swich 1 x Brocade TurboIron NSD Servers 2 x Dell R630 NSD 1 x SF7700X-FC 1 x SS8460 100 x 900 GB 10krpm SAS 8 Gb FC 10 Gb Ethernet Benchmark Environment SSD SSD VM VM VM VM VM VM ・・・ VM VM VM VM VM VM ・・・ 32 VM 32 VM SSD SSD 4x Pliant LB206S 0 10,000 20,000 30,000 40,000 50,000 60,000 10 20 30 40 50 60 70 80 90 100 110 120 130 140 150 160 170 180 190 200 210 220 230 240 250 260 270 280 290 300 310 320 330 340 350 360 370 380 390 400 410 420 430 440 450 460 470 480 490 500 510 520 530 540 550 560 570 580 590 600 Random 4KB Read IOPS from VM on GPFS Client with SFX 32 VM(1 GPFS Client) Read IOPS 64 VM(2 GPFS Client) Read IOPS 32 VM(1 GPFS Client) 64 VM(2 GPFS Client) Avarage 18,738 39,187 0 5,000 10,000 15,000 20,000 25,000 30,000 35,000 40,000 45,000 IOPS Random 4KB Read IOPS from VM on GPFS Client with SFX

brands may be claimed as the property of others. Any statements or representations around future events are subject to change. Benchmark Result compare LROC and SFX From 32 VM and 64 VM Client 36 0 10,000 20,000 30,000 40,000 50,000 60,000 10 20 30 40 50 60 70 80 90 100 110 120 130 140 150 160 170 180 190 200 210 220 230 240 250 260 270 280 290 300 310 320 330 340 350 360 370 380 390 400 410 420 430 440 450 460 470 480 490 500 510 520 530 540 550 560 570 580 590 600 Random 4KB Read IOPS from VM on GPFS Client LROC 32 VM(1 GPFS Client) LROC 64 VM(2 GPFS Client) SFX 32 VM(1 GPFS Client) SFX 64 VM(2 GPFS Client)

brands may be claimed as the property of others. Any statements or representations around future events are subject to change. 37 まとめ

brands may be claimed as the property of others. Any statements or representations around future events are subject to change. OpenStack 環境における GPFS のメリット ► Compute Node からのパラレル I/O ► GPFS Client （Compute Node）のパフォーマンス監視が可能な GUI ► Storage Pool (ILM) を利用したデータマネージメント ► 利用用途に合わせた、データレプリケーション ► GPFS Client （Compute Node）が GPFS で利用するメモリを制限 ► 複数の Cinder Volume の作成速度 ► Swift でのアクセス、 Manila によるファイルサービスの提供も可能 ► DDN SFA と使用することで最大限のパフォーマンスが発揮できる 38

brands may be claimed as the property of others. Any statements or representations around future events are subject to change. 102-0081 東京都千代田区四番町6-2 東急番町ビル 8F TEL:03-3261-9101 FAX：03-3261-9140 company/datadirect-networks @ddn_limitless [email protected] Thank You! Keep in touch with us

brands may be claimed as the property of others. Any statements or representations around future events are subject to change. GPFS Cinder Driver Functions Function Y/N Comments Create Volume OK Delete Volume OK Attach Volume OK Detach Volume OK Extend Volume OK Create Snapshot OK Delete Snapshot OK List Snapshots OK Create Volume from Snapshot OK Create Volume from Image OK Create Volume from Volume (Clone) OK Create Image from Volume OK Volume Migration (host assisted) OK

OpenStack環境で利用されるハイパフォーマンスストレージの紹介

OpenStack環境で利用されるハイパフォーマンスストレージの紹介

More Decks by DataDirect Networks Japan

Other Decks in Technology

Featured

Transcript