OpenStackクラスタ間マイグレーション事例 Havana to Mitaka / OpenStack Migration Case (Shift from Havana to Mitaka)

OpenStack Days Tokyo 2017 で発表した、Havana/Mitaka の並行運用+その間のマイグレーションについて

Transcript

1. )BWBOBUP.JUBLB
   ৗদ৳࠸(.01FQBCP *OD
   0QFO4UBDL%BZT5PLZP
   0QFO4UBDLΫϥελؒ

   ϚΠάϨʔγϣϯ
   

   View Slide

2. ϓϦϯγύϧΤϯδχΞ
   ৗদ৳࠸!UONU
   ٕज़෦ٕज़ج൫νʔϜ
   IUUQTCMPHUONUJOGP
   

   View Slide

3. (.0ϖύϘ
   اۀཧ೦΋ͬͱ͓΋͠Ζ͘Ͱ͖Δ
   ϛογϣϯΠϯλʔωοτͰՄೳੑΛͭͳ͛ΔɺͻΖ͛Δ
   

   View Slide

4. View Slide

5. -0-*101ϚωʔδυΫϥ΢υϦϦʔε
   Ћςετதʂ
   

   View Slide

6. ΞδΣϯμ
   w ϓϥΠϕʔτΫϥ΢υಋೖܦҢͱࠓ·Ͱ
   w όʔδϣϯฒߦӡ༻ʹ͍ͭͯ
   w ࣄྫ τϥϒϧγϡʔτ
   w ฒߦɾࣗࣾӡ༻Λܦͯࠓޙ
   

   View Slide

7. ϓϥΠϕʔτΫϥ΢υಋೖܦҢ
   

   View Slide

8. View Slide

9. /ZBIJT
   w(.0ϖύϘͷϓϥΠϕʔτΫϥ΢υͷίʔυωʔϜ
   w0QFO4UBDLͰߏங͞ΕͨԾ૝Πϯϑϥج൫
   w֤αʔϏεɾ঎ࡐͷαʔόʔ؀ڥͱͯ͠ར༻த
   w೥ߏ૝݄ΑΓ0QFO4UBDLݕ౼։࢝
   w೥݄άϧʔϓձࣾͷࢧԉΛड͚ɺ)BWBOBελοΫӡ༻։࢝
   w೥݄ࣗࣾߏஙʹͯɺ.JUBLBελοΫͷӡ༻։࢝
   

   View Slide

10. ೥݄
    0QFO4UBDLӡ༻೥ΛৼΓฦͬͨ
    ·ͱΊΛൃද
    IUUQTTQFBLFSEFDLDPNUONUQFQBCPT
    QSJWBUFDMPVEOZBIBGUFSUIBU
    

    View Slide

11. ೥݄ൃදͷৼΓฦΓ
    

    View Slide

12. ৼΓฦΓಋೖܦҢ
    

    View Slide

13. ৼΓฦΓޮՌ
    

    View Slide

14. ৼΓฦΓ՝୊
    

    View Slide

15. ৼΓฦΓ೥ํ਑
    

    View Slide

16. ৼΓฦΓࣗࣾͰͷ0QFO4UBDLߏங
    

    View Slide

17. ݱࡏͷ/ZBIͷঢ়گ
    

    View Slide

18. /ZBICFGPSFBGUFS
    w0QFO4UBDLόʔδϣϯ)BWBOBˠ.JUBLB
    w$JOEFSར༻ແˠ༗ %FMM&.$4DBMF*0
    
    w/FVUSPOҰ෦ར༻ˠ%734/"5)"
    

    View Slide

19. %73
    IUUQTXXXTDIPPMDUDHDPKQDPMVNOTOBLBJOBLBJIUNM
    

    View Slide

20. 4/"5)"
    DPNQVUF
    WN
    OFVUSPO OFVUSPO
    LFFQBMJWFE
    .JUBLBҎ߱͸OFVUSPOΛ৑௕Խ͢Δ͜ͱͰɺ4/"5ϧʔλʔΛ෼ࢄ഑ஔ͠
    LFFQBMJWFEͰ৑௕Խ͢Δ͜ͱ͕ग़དྷΔΑ͏ʹͳͬͨ
    

    View Slide

21. όʔδϣϯؒϚΠάϨʔγϣϯ
    

    View Slide

22. ϚΠάϨʔγϣϯʁ
    wطଘͷ)BWBOB؀ڥΛΞοϓάϨʔυग़དྷͳ͍
    w04͕ҟͳΔ $FOU04ˠ6CVOUV
    
    w)BWBOBUP.JUBLB͸Ұ౓Ͱ͸ΞοϓάϨʔυෆՄ
    w৽نϥοΫɾαʔόʹͯ.JUBLB؀ڥΛผ్ߏங
    wಉ%$಺ผϥοΫͳͷͰωοτϫʔΫ͸ૄ௨ग़དྷΔ
    

    View Slide

23. ؀ڥͷωοτϫʔΫ઀ଓ
    

    View Slide

24. ωοτϫʔΫ઀ଓྫ
    WN WN
    
    OFUXPSL
    7-"/
    %)$1QPPM
    
    OFUXPSL
    7-"/
    %)$1QPPM
    QPSUSFTFSWF
    /ZBI .JUBLB
    
    /ZBIDMBTTJD )BWBOB
    
    -εΠον
    ʢ෺ཧ
    
    Ͱͭͳ͕͍ͬͯΔ
    .ɾ)
    ؒͰॏෳ͠ͳ͍Α͏
    ʹ*1ઃܭ
    

    View Slide

25. ωοτϫʔΫ઀ଓྫ
    wϧʔλʔΛར༻͠ͳ͍ߏ੒
    w. JUBLB
    ɾ) BWBOB
    ؒͰಉ͡7-"/*% αϒωοτΛར༻
    wޓ͍ͷ؀ڥͰ؅ཧ*1͕ॏෳ͠ͳ͍Α͏ʹ؅ཧ
    w)BWBOBSFTFSWFEQPSU %)$1ϨϯδΛޙ͔ΒมߋͰ͖ͳ͍ʜ
    
    wγϯϓϧͰ෼͔Γ΍͍͢
    wQSPKFDUͷن໛͕େ͖͍ͱ*1͕଍Βͳ͘ͳΔՄೳੑ͕͋Δ
    

    View Slide

26. ωοτϫʔΫ઀ଓྫ
    WN
    WN
    ˒UFTUMBO7-"/
    ˒
    ˒DMBTTJDUFTUMBO7-"/
    ˒
    ˒UFTUMBOSPVUFS
    ˒UFTUMBO7-"/
    /ZBIDMBTTJD )BWBOB
    /ZBI .JUBLB
    
    

    View Slide

27. ωοτϫʔΫ઀ଓྫ
    wϧʔλʔΛར༻͢Δߏ੒
    w.ɾ)؀ڥͰͷ؅ཧ*1ॏෳΛؾʹ͠ͳ͘ͱ΋ྑ͍
    wτϙϩδ͸एׯෳࡶʹ͸ͳΔ͕ཧղ͕ग़དྷͳ͍΄ͲͰ͸ͳ͍
    wར༻QPSU਺ͷ૿Ճ͕༧૝͞ΕΔ৔߹ɺผαϒωοτʹͳΔͷͰϨϯδ
    Λ޿͘औΔ͜ͱ͕ग़དྷΔ
    wݱࡏ͸ͪ͜ΒΛར༻͢Δํ਑ʹ͍ͯ͠Δ
    

    View Slide

28. ໰୊ύέοτॏෳ໰୊
    WNˠWNSPVUFS͔Βग़Δύέοτͷ43$."$͸%73͕ಠࣗʹ
    ෇༩͢Δ."$
    WNˠWNύέοτͷ%45͕SPVUFSͷ."$ΞυϨε
    w %45͸%73͕ಠࣗʹ෇༩͢Δ."$ʹͯ͠ཉ͍͠
    %73Ͱߏ੒͞ΕͨSPVUFS͸֤DPNQVUFOPEF্ʹଘࡏ͠ɺ.ɾ)ؒ
    ͸-Ͱͭͳ͕͍ͬͯΔͨΊɺ͢΂ͯͷSPVUFSʹύέοτ͕ಧ͘
    WNʹಧ͘ύέοτ͕ॏෳ͠௨৴͕ෆ҆ఆʹͳΔ
    

    View Slide

29. ໰୊ύέοτॏෳ໰୊
    w.JUBLBͰͷچϓϩδΣΫτωοτϫʔΫʢલड़7-"/ʣ͸֎෦
    FYUFSOBM
    ωοτϫʔΫͱͯ͠ఆٛ͢Δඞཁ͕͋Δ
    w֎෦ωοτϫʔΫͱͯ͠ઃఆͨ͠৔߹ύέοτͷ%61͸ൃੜ͠ͳ͍
    w಺෦ωοτϫʔΫ֎෦ωοτϫʔΫ΁ͷ௨৴͕4/"5ϧʔλʔΛ
    ܦ༝͢Δ
    wͭ·Γ%73Λ࢖༻͠ͳ͍ͨΊ
    

    View Slide

30. 5FSSBGPSN
    

    View Slide

31. 5FSSBGPSN
    w.JUBLBΑΓຊ֨ར༻։࢝
    w0QFO4UBDL1SPWJEFS
    wJOTUBODFҎ֎ʹ΋༷ʑͳϦιʔεʹରͯ͠ɺએݴతʹهड़ɾ؅ཧ͕ߦ
    ͑ΔΑ͏ʹͳͬͨ
    wQPSU OFUXPSL WPMVNF౳ʑ
    w؀ڥͷωοτϫʔΫ઀ଓ༻ͷSPVUFS΋࡞੒ग़དྷΔΑ͏ʹͳͬͨ
    

    View Slide

32. ໰୊ෆਖ਼ͳϧʔλʔ͕࡞੒͞ΕΔ
    w 4/"5)"ߏ੒ͰSPVUFS࡞੒࣌ʹFYUFSOBM@HBUFXBZ@JOGPΛࢦఆ͢Δ
    ͱωοτϫʔΫ௨৴͕ग़དྷͳ͍SPVUFS͕࡞੒͞Εͯ͠·͏
    w OFVUSPOϊʔυ্ͰΤϥʔ͕ϧʔϓ͠ϩά͕ංେԽ͢Δ
    w ରॲํ๏ͱͯ͠͸OFVUSPOMBHFOUͷ࠶ىಈͷΈ
    w )PSJ[PO͔ΒͷSPVUFS࡞੒࣌ʹ͸໰୊ʹͳΒͳ͔ͬͨ
    w 5FSSBGPSNΛར༻͢ΔΑ͏ʹͳ͔ͬͯΒ໰୊͕ൃ֮
    w 5'͕FYUSB@HBUFXBZΛઃఆ͢ΔλΠϛϯά͕ૣ͗͢Δ͜ͱ͕ݪҼ
    

    View Slide

33. ໰୊ෆਖ਼ͳϧʔλʔ͕࡞੒͞ΕΔ
    wରॲํ๏ͱͯ͠͸5'ͰͷSPVUFS࡞੒࣌ʹ͸FYUFSOBM@HBUFXBZ͸ࢦఆ
    ͠ͳ͍
    w݄಄ɺCBDLQPSUpYͰࠜຊରԠͱͳΔमਖ਼͕ߦΘΕͨ
    wIUUQTCVHTMBVODIQBEOFUOFVUSPOCVH
    wOFVUSPOϊʔυ͸৑௕ߏ੒͕औΒΕ͍ͯΔҝɺ୆ͣͭϑΣΠϧΦʔ
    όʔͤͯ͞ύοέʔδΛΞοϓσʔτ
    

    View Slide

34. ໰୊4/"57331QPSU%08/
    w4/"5)"ͷखಈϑΣΠϧΦʔόʔ࣌ʹൃ֮
    wOFVUSPOϊʔυؒͰLFFQBMJWFEʹΑΔ৑௕Խ͕औΒΕ͍ͯΔ
    w7331Λૹड৴͢ΔQPSU͕ىಈ͍ͯ͠Δͷ͕લఏ͕ͩɺԿނ͔୆Ͱ
    ͦͷQPSU͕μ΢ϯ͍ͯͨ͠
    wQPSU͕μ΢ϯ͍ͯ͠ΔͨΊɺखಈϑΣΠϧΦʔόʔ͕ߦ͑ͳ͍
    w݁ՌΠϯελϯε͔Β֎෦ωοτϫʔΫ΁ͷ௨৴͕ߦ͑ͳ͘ͳΔ
    

    View Slide

35. ໰୊4/"57331QPSU%08/
    wࠓͷͱ͜ΖݪҼ͸ෆ໌
    wରԠΛݕ౼த
    wQPSU͕%08/͍ͯ͠Δ͜ͱ͕ݕ஌ग़དྷΔΑ͏ʹ؂ࢹΛߦ͏
    wݕ஌͞ΕͨΒࣗಈతʹىಈ͢Δ
    

    View Slide

36. 0DUBWJB -#BB4
    
    

    View Slide

37. 0DUBWJB
    w-#BB4Wͷ࣮૷ 0QFO4UBDLίϯϙʔωϯτͷҰͭ
    w.JUBLB͔Βඪ४Ͱαϙʔτ͞Ε͍ͯΔ
    w֓ཁɺڍಈʹ͍ͭͯ͸ฐࣾϝϯόʔʹΑΔϒϩάهࣄࢀর
    wIUUQCVUZIBUFOBCMPHDPNFOUSZ
    w)BWBOB؀ڥͰ͸֤QSPKFDU΍ཁ݅͝ͱʹϩʔυόϥϯα༻ͷΠϯελϯεΛ࡞
    ੒͍ͯͨ͠
    w͜ΕΛ0QFO4UBDLίϯϙʔωϯτͱͯ͠ղܾ͍ͨ͠
    

    View Slide

38. ൺֱ IBQSPYZCBDLFOE
    
    

    View Slide

39. ൺֱ IBQSPYZCBDLFOE
    
    w'MPBUJOH*1 pQ
    ͸-#ͷ7*1ʹඥ෇͍͍ͯΔ
    wΠϯλʔωοτ͔Βͷύέοτ͸-#Λ௨͠Πϯελϯε΁ϩʔυόϥ
    ϯε͞ΕΔ
    wOFUXPSLOPEF͕γϯάϧϙΠϯτʹͳΓ΍͍͢
    

    View Slide

40. ൺֱ 0DUBWJB
    
    

    View Slide

41. ൺֱ 0DUBWJB
    
    w-#͕IZQFSWJTPS DPNVUFOPEF
    ্ʹ͋Δ
    w-#Λ௨ৗͷΠϯελϯεͱͯ͠ىಈ͢Δ͜ͱͰOFUXPSLOPEF΁ͷτ
    ϥϑΟοΫूதΛආ͚ΕΔ
    wOFUXPSLOPEFͷ410'΋ղফग़དྷΔ
    

    View Slide

42. ໰୊0DUBWJBW
    w.JUBLBͷ0DUBWJBόʔδϣϯ͕
    w/FXUPOW 0DBUBW
    wMCBBTMPBECBMBODFSDSFBUF࣌ʹ"NQIPSBΠϯελϯε͕࡞੒͞ΕΔ
    ͕1307*4*0/*/(@45"564͕&3303
    w-#΍QPPMͷ࡟আ͕ࣦഊ͠࡟আग़དྷͳ͍
    w ݫີʹ͸τϥϒϧͰ͸ͳ͍
    1309:1SPUPDPM͕ར༻ग़དྷͳ͍
    w)5514ΛΠϯελϯεͰॲཧ͍ͨ͠৔߹རศੑ͕௿Լ͢Δ
    

    View Slide

43. ໰୊0DUBWJBW
    wݱ࣌఺Ͱ࠷৽ͷCΛར༻
    w/FVUSPO-#BB4ͱͷ࿈ܞ͸ఘΊͨ "1*όʔδϣϯ͕߹க͠ͳ͍
    
    w0DUBWJBΛ؆қʹѻ͑ΔΫϥΠΞϯτΛ࡞੒͠Α͏ͱݕ౼த
    wݱঢ়ͩͱDVSMͰ0DUBWJB"1*Λૢ࡞͢Δ͜ͱͰ-#Λ࡞͍ͬͯΔ
    wͦ΋ͦ΋1309:1SPUPDPMΛ࢖༻͢Δʹ͸0DUBWJB"1*Λ௚઀ୟ͘
    ͔͠ͳ͍
    

    View Slide

44. ϚΠάϨʔγϣϯঢ়گ
    

    View Slide

45. ϚΠάϨʔγϣϯਐḿ
    w؀ڥؒͷ௨৴͕ߦ͑ΔΑ͏ʹͳͬͨ
    wঢ়ଶ΍σʔλΛอ͍࣋ͯ͠ͳ͍ϩʔϧ͔Βॱ࣍Ҡߦ
    w8FCαʔό΍"1*αʔό
    w"1*αʔό͕.JUBLBʹ͋Γɺσʔλϕʔεαʔό͕)BWBOBʹ͋Δɺ
    ͱ͍͏αʔϏε΋૿͖͑ͯͨ
    

    View Slide

46. ϚΠάϨʔγϣϯํ๏
    w.ɾ)ؒͰ௚઀ΠϯελϯεΛҠಈͤ͞Δํ๏͸ࠓͷͱ͜Ζͳ͍
    wͦͷͨΊɺ࠷ॳ͸ঢ়ଶΛ͍࣋ͬͯͳ͍ϩʔϧ͔ΒऔΓ૊ΜͰ͍Δ
    w)BWBOBͷΠϯελϯεΛ(MBODFΠϝʔδԽͯ͠.JUBLBͰͨͯΔɺͱ
    ͍͏πʔϧͷ࡞੒Λݕ౼
    w͜ͷ৔߹͸Πϯελϯεͷఀࢭ͕൐͏
    wσʔλϕʔεαʔόͷҠߦͳͲࠓޙͷ՝୊
    

    View Slide

47. ن໛ײ
    w/ZBIDMBTTJD )BWBOB
    
    wDPNQVUFOPEF
    wJOTUBODF
    w/ZBI .JUBLB
    
    wDPNQVUFOPEF
    wJOTUBODF
    

    View Slide

48. ͦͷଞվળঢ়گ
    

    View Slide

49. -JWF.JHSBUJPO CMPDLCBTFE
    
    wແఀࢭͷΠϯελϯεϚΠάϨʔγϣϯ
    w$JOEFS7PMVNFͷΑ͏ͳڞ༗ετϨʔδΛར༻͠ͳ͍ύλʔϯ
    wΠϯελϯεͷσΟεΫΠϝʔδ΍ϝϞϦ৘ใ͕RFNVΛհͯ͠సૹ
    ͞Ε੾ΓସΘΔ
    wݕূͰ͸੾ΓସΘΓ࣌͸ύέοτ΄ͲͰɺ௨৴ʹେ͖ͳӨڹ͸ݟΒ
    Εͳ͍
    wTTIEͷηογϣϯ΍σʔλϕʔεͷϨϓϦέʔγϣϯ͕੾Εͳ͍
    

    View Slide

50. ໰୊ίϯιʔϧϙʔτিಥ
    wϚΠάϨʔγϣϯ࣌ʹΠϯελϯεʹඥ෇͍͍ͯΔ4FSJBMDPOTPMFϙʔ
    τ͕DPNQVUFOPEFؒͰিಥ͢Δ৔߹͕͋Δ
    wϙʔτ͕িಥ͢ΔͱϚΠάϨʔγϣϯʹࣦഊ͢Δ
    wIUUQTCVHTMBVODIQBEOFUOPWBCVHʹͯόάใࠂ
    ͕͞Ε͍ͯΔ
    

    View Slide

51. ໰୊ίϯιʔϧϙʔτিಥ
    w/PWBͷઃఆͰΠϯελϯεʹ෷͍ग़͞ΕΔDPOTPMFQPSUͷൣғࢦఆ
    ͕Մೳ
    w֤DPNQVUFOPEFͰൣғ͕ॏෳ͠ͳ͍Α͏ʹগͣͭ͠εϥΠυͤͯ͞
    ઃఆ͢Δ͜ͱͰճආग़དྷͨ
    w/FXUPO͔Β͸pY͞Ε͍ͯΔ
    

    View Slide

52. $JOEFS
    w$JOEFS͕ར༻Մೳʹͳͬͨ
    wݱࡏͷόοΫΤϯυ͸%FMM&.$4DBMF*0
    wฐࣾ(JU)VC&OUFSQSJTFͷΠϯελϯε͸ىಈΠϝʔδɾσʔλྖҬ
    ྆ํͱ΋͜ͷ$JOEFS7PMVNFʹ഑ஔ͠Քಇ͍ͤͯ͞Δ
    wσʔλϕʔεͷσʔλͳͲ*014͕ඞཁͳγʔϯͰར༻͍ͯ͘͠༧ఆ
    

    View Slide

53. )$* )ZQFS$POWFSHFE*OGSBTUSVDUVSF
    
    wDPNQVUFOPEF͕4DBMF*0ͷTUPSBHFOPEF΋݉ͶΔ
    w෼ࢄσʔλΛஔ͘σόΠεʹ͸/7.FΛར༻͍ͯ͠Δ
    wIEQBSNͷ؆қ଎౓νΣοΫฏۉ.#TFD.CQT
    w4"5"ͷ(CQTΛ௒͑ΔεϖοΫ
    w4DBMF*0ͷ෼ࢄͰ(CBTFͷετϨʔδ༻ωοτϫʔΫͷํ͕ઌʹଳ
    Ҭ͕଍Βͳ͘ͳΓͦ͏͕ͩɺ·ͩͦ͜·ͰϔϏʔʹ͸࢖ΘΕ͍ͯͳ͍
    

    View Slide

54. ·ͱΊ
    

    View Slide

55. ࣗલߏஙɾ؀ڥฒྻӡ༻Λܦͯ
    

    View Slide

56. 0QFO4UBDLͷίʔυ͕ಡΊͳ͍ͱπϥ͍
    wόάΛ౿Μͩ࣌ɺ৽͍͠ػೳΛ࢖͓͏ͱͨ࣌͠ɺڞʹ
    wޙऀͷ৔߹υΩϡϝϯτ͞Ε͍ͯͳ͍ɺ#MVFQSJOU͚ͩ΋ଟ͍
    wࠓճಛʹ/FVUSPOؚΊωοτϫʔΫͷτϥϒϧγϡʔτʹ࣌ؒΛׂ͍
    ͨ
    wϩάͷௐࠪʹՃ͑ͯ0QFO4UBDLͷίʔυΛಡΉ͕࣌ؒ૿͑ͨ
    wࣗલͰߏஙɾӡ༻͢Δͱ͍͏৔߹ʹ͸֮ޛ͕ඞཁ
    

    View Slide

57. ܧଓతʹόʔδϣϯΞοϓ͠ͳ͍ͱπϥ͍
    w঺հͨ͠໰୊΍ػೳͰଟ͔ͬͨʮ৽͍͠όʔδϣϯͩͬͨΒ௚͍ͬͯ
    Δɾ࢖͑ΔʯʹରԠ͍͖͍ͯͨ͠
    wϢʔβ؅ཧ΍ηΩϡϦςΟରԠͳͲ؀ڥΛฒߦӡ༻͢Δखؒ͸͔͔
    Δɻ؀ڥΛूதͯ͠ӡ༻͍ͨ͠ɻ
    w؀ڥؒϚΠάϨʔγϣϯʹ͸ύϫʔ͕ඞཁɻ೥಺͍ͬͺ͍͸ଓ͍ͯ
    ͍͖ͦ͏ͳݟ௨͠ͳͷͰɺຖ౓͸ग़དྷͳ͍ɻ
    wࠓ͋Δ؀ڥΛஈ֊తʹΞοϓάϨʔυ͍ͨ͠
    

    View Slide

58. ָ͍͜͠ͱ΋͋Δ
    wlΫϥ΢υͷϓϩόΠμzͱͯ͠ͷ΍Γ͕͍
    wΫϥ΢υͷ࡞Γํֶ͕΂Δ
    wΫϥ΢υͷதͷਓͷؾ͕࣋ͪ෼͔Δ
    w໰୊ղܾͷϨΠϠ͕૿΍ͤΔ
    w4%4 4%/ͳͲιϑτ΢ΣΞͰͷ࣮૷Λ஌Δ͜ͱ͕ग़དྷΔ
    w૯߹ྗ͕ࢼ͞ΕΔ
    

    View Slide

59. ࠓޙͷ༧ఆ
    w0QFO4UBDLΞοϓάϨʔυ͠΍͍͢࢓૊Έͮ͘Γ
    wίϯϙʔωϯτͷίϯςφԽ ,PMMB ,VCFSOFUFT
    
    w4DBMF*0Ҏ֎ͷ4%4Λಋೖ͠શ୆)$*ʹ͢Δ
    wݱࡏͷͱ͜Ζ%3#%Λݕ౼த
    w)$*ʹ൐ͬͨ7.)"
    wΠϯελϯεͷΑΓΑ͍εέδϡʔϦϯάํ๏
    

    View Slide

60. ϓϥΠϕʔτΫϥ΢υӡ༻΋
    z΋ͬͱ͓΋͠Ζ͘Ͱ͖Δz
    ࠷৽ͷ࠾༻৘ใΛνΣοΫˠ !QC@SFDSVJU
    

    View Slide