OpenStack Preemptible Instance

Transcript

1. Copyright©2019 NTT corp. All Rights Reserved. OpenStack Preemptible Instance OpenStack

   Days Tokyo 2019   
    https://speakerdeck.com/tetsuro0907/openstack-preemptible-instance

2. 1 Copyright©2019 NTT corp. All Rights Reserved. 
   

   : Tetsuro Nakamura (@tetsuro) NTT +'*24 "4-$#$(0 • 74/3%4$%.* ü DPDK 5SPP7 )4&,$6 ü OpenStack 5PlacementBlazar6 • 7 ü 1!-4

3. 2 Copyright©2019 NTT corp. All Rights Reserved.  1. Preemptible

   instance 2. OpenStack  3. Preemptible instance 4. Part Time Reservation
    5.  

4. 3 Copyright©2019 NTT corp. All Rights Reserved.  1. Preemptible

   instance 2. OpenStack  3. Preemptible instance 4. Part Time Reservation
    5.  

5. 4 Copyright©2019 NTT corp. All Rights Reserved. 1. HFB&#/-3'!6<
   371,6HFB<

   +:(3" 2. 8383PTIHFB< 9%:AOUXJ=WCEFJQ6.&!BR?M6AXDFLUX H5,6& );00#:" A) Preemptible Instance: $ 6PEW<KNV>+:-76SGXF&94$!SGXF<+: -75
   57:(362':6PEW6(3" B) Kubernetes PreemptionYv1.9-Z: BRFH6SGXF&*-5!
   5 &$ Pod <$ *1 & $ Pod <+:" Preemption   HFBB HFBB HFBA NS@WNL

6. 5 Copyright©2019 NTT corp. All Rights Reserved. Preemptible Instance •

   An]od^mVqs`psi\),UtUu8 Ø .:0tbjm9%B(*:0/u Ø lfagsd9%tWeb`psnrau Ø kfe * etc. 1. Google Compute PlatformtPreemptible VMu i. ;V3 V1V<Q +6 ii. FOQW 2K[Z6HEZ t!QW24 >Q 2u 2. Amazon Web ServicestSpot Instanceu i. +MZ^rcdrcV +&HLPGYB7 H $LN&\ +&H
   ZR ,U^rcdrcH@ ii. ^rcdrcH +=-U@K[ZV\?JC34#>V1DXBAuto Scaling\ +QIZCn`_chR5DTSV'6"

7. 6 Copyright©2019 NTT corp. All Rights Reserved. OpenStack Preemptible Instance
   

8. 7 Copyright©2019 NTT corp. All Rights Reserved. 
   

9. 8 Copyright©2019 NTT corp. All Rights Reserved.  1. Preemptible

   instance 2. OpenStack  Ø Placement Project Ø Blazar Project 3. Preemptible instance 4. Part Time Reservation
    5.  

10. 9 Copyright©2019 NTT corp. All Rights Reserved. Placement 1. &98:OFRD5

    <(JPC?@I 2. 5SOpenStack4!93'T
     *;-AQEJI</ A) Resource ProviderU OFRD5 B) Resource Class: OFRD5# C) Inventory: OFRD5 D) Consumer: OFRD5 3.  $Nova)Placement53@N>=QI A) NovaU MBQ5HJP><(JPC?@I B) LDI5 4Placement<':% OFRD VHRGKRD5 S72726Nova54&.-)$SQL)" ,:51REST-IF<.0Nova2 4+-T

11. 10 Copyright©2019 NTT corp. All Rights Reserved. Nova-Placement Interaction 1.

    OBH.
    +4“nova-compute”( /Placement2TDXB6 Ø I9KJ9@0  Ø libvirt (KVM, QEMU etc.), Xen Server, Hyper-V etc. 2. VM0"1#VM0 BNG=6Nova'3Placement/)-OBH6 4 3. Nova0 L79U/ *5,*3/'%L8UETW>1Nova0B?AQXSMVCB Y"nova-scheduler"Z.&$ nova-api nova- conductor nova- scheduler placement RPC REST nova- compute TDXB OBH B<XPF:G= BNG=
    9WBEWB OBH0! RX@T=;BH

12. 11 Copyright©2019 NTT corp. All Rights Reserved. Placement Extraction 1.

    Newton-Rocky L Placement in Nova A) <KC'Nova/HF>BH,#AK@EK?2Nova B) Nova-/IF0REST,7+$6'#DI>9;B
    0' " 2. Stein- L Placement DI>9;B A) HF>BH#AK@EK?%41DI>9;B/ 8Nova&5. B) DI>9;B/ .46 3 /' C) Nova/Placement<KC0SteinHHK?,0 Ø TrainHHK?&5!(7*Placement8)6:D=GJ2 

13. 12 Copyright©2019 NTT corp. All Rights Reserved.  1. Preemptible

    instance 2. OpenStack  Ø Placement Project Ø Blazar Project 3. Preemptible instance 4. Part Time Reservation
     5.  

14. 13 Copyright©2019 NTT corp. All Rights Reserved. Blazar 1. OFZMHL\d_$D

    +=82;:-<^NdM>&eh^NdM fD#0 2. Nova@Neutron;*4>^NdM> DIXdR. A) FcMOcM i. WMR>^NdM>
    +> Ø  ,=YKc19ACBe7?>^NdM1%3CBf2;D& ii. WMR>^NdM> 9D eWMR  !f Ø  47]dJ>' >YKc1 5=QUaF3C</2;D&6B B) TPRbdG g i. Floating IPES`M 3. -Ironic;*49-VE[O_> DIXdR6B.e(f A) ]dMHdMg FcMOcMD9B >):"=VE[O_D0.

15. 14 Copyright©2019 NTT corp. All Rights Reserved. User :10000 Feet

    Overview Reservation ID Reservation ID Reservation ID Reservation ID   3+7
     
    ID !# 
    ID% 8
     9  2,/  3+7
    %5*0-4   3+7  2,/ 
    2,/ 
    "'$2,/& 3+7(.16) 
     Reservation ID

16. 15 Copyright©2019 NTT corp. All Rights Reserved. Blazar – Ironic

    Usecase • IronicO – F9M%+(F9M1 OpenStack*CL:45= – F9M% OS('1
    1D2G<K&-!.D2G<KCLB:I?M 61/8MEN@M=& – D2G<K1VM& ) /!.*Nova*>J3A"#!)$ ”Ironic stand alone” &,0/HN;7N;  • Nova Ironic driver • Ironic stand alone OS Container HW OS libvirt VM HW OS HW

17. 16 Copyright©2019 NTT corp. All Rights Reserved. User @10000 Feet

    Overview Reservation ID  5,81:
     
    ID $  90=/  
     (6/3  Ironic stand alone'!"*5,81:
      5,81:24;-  90=/  
    #+%7.<)5,81:24;- 
    ID&6/3 >
     ? Reservation ID 
     Reservation ID

18. 17 Copyright©2019 NTT corp. All Rights Reserved.  1. Preemptible

    instance 2. OpenStack   3. Preemptible instance Ø Nova
     4. Part Time Reservation   5. 

19. 18 Copyright©2019 NTT corp. All Rights Reserved. PLACEMENT cell1 Preemtible

    OpenStack Preemptible Instance • ,8941*# ?Rocky- : @ 1.  7-=26<+NoValidHost)&Nova
     7-=)Pending State" 2. Reaper()Preemptible Instance)%'! :/>.) 3. :/>. " & ReaperPending State#+=.0=.):5;3 ?$+=.0=.)Error State"@ ? REAPER 1. VM create HW OS HW OS VM VM cell0 Failed VM HW OS VM VM 3. Error State Pending State 2. Delete 2. NoValidHost 1. Detect 3 Rebuild Preemtible

20. 19 Copyright©2019 NTT corp. All Rights Reserved. • 4<>:71, 

    • Nova+ A • Introduce Pending VM stateA • https://review.opendev.org/#/c/648687/ • Enable Rebuild for Instances in cell0A • https://review.opendev.org/#/c/648686 • Nova,+" %" A • Reaper FunctionA • https://gitlab.cern.ch/ttsiouts/ReaperServicePrototype • A • Reaper '$'$!'OpenStack+ %";?5238
    0  0B =@79@6- /0,B • * *"Preemptible Instance &#(0$).0 OpenStack Preemptible Instance ? REAPER

21. 20 Copyright©2019 NTT corp. All Rights Reserved.  1. Preemptible

    instance 2. OpenStack   3. Preemptible instance 4. Part Time Reservation  Ø OpenStack Blazar
    5. 

22. 21 Copyright©2019 NTT corp. All Rights Reserved. Proposal 1. :Blazar9.1>ASGISG!VFull

    Time Reservation Ø OTD/!'*"
    6#8QHTG&? : ,7/24. 9:00 - 7/27. 9:00- 5 ,4C@ 16GB LES 1 -?! ⇒ Blazar/!: * ?% 2. 3>48Blazar:ASGISG!V Part Time Reservation Ø OTD/JTIR:(?20 V,7/24. 9:00 - 7/31. 9:00- :(5JTIR5,72(-* ,4C@ 16GB LES 1 -?! ⇒ !: ;*Blazar/!?%3+ )Q@RIAM@KQBTEPS/$: *QHTG9<=* !(?24=*QGBFNTR2>+ ,7/25 . 9:00 - 7/26 9:00-U,7/28 . 9:00 - 7/30 9:00-87+ Blazar Part Time Reservation Feature

23. 22 Copyright©2019 NTT corp. All Rights Reserved. Preemtible Instance Part

    Time Instance Part Time Reservation – Work Flow Preemtible Instance Preemtible Instance
    ,   … ,      ,   
       Part Time Instance     Part Time Instance • Preemptible Instance • Part Time Reservation 

24. 23 Copyright©2019 NTT corp. All Rights Reserved. 1. Preemptible Instance62BLT5V5RG2IH<0Part

    Time ReservationDJLAC &2oq[nqfW HeqdmI#-IM) A) Preemptible Instancew Ø rjq_s VM6K<21"($GIDLACOQ21 Ø r[kZfs=94L=602BLT5JV5SL>X1 B) Part Time Reservationw Ø rjq_sVM6K<21"($GIDLACOQ260*:LDHv- 5<@21 Ø r[kZfsDJ0+Ht-E0Hu-0 5<L=1 2. Part Time ReservationI w / 5=eqdmI#-6 5R);UT1 /
    LA@g`pI ,Wjq_DPT-6!:T1 / BlazarIb]aiqlp\Ym^lchID=9'D7?31 . HNC&2E23 %DJG80;KFHJD7G21 Part Time Reservation - Summary

25. 24 Copyright©2019 NTT corp. All Rights Reserved.  1. Preemptible

    instance
     2. OpenStack  
     3. Preemptible instance  4.

26. 25 Copyright©2019 NTT corp. All Rights Reserved. /*0 • Blazar*Placement,'(

    • Nova) !2(1Preemptible Instance3 • Blazar)+Part Time Reservation 3 "
    2."/#?$-RFE@Request for EnhancementA 2.4576;>:=- -upstream developer@*A 3&( %!@-<983/+)?A

27. 26 Copyright©2019 NTT corp. All Rights Reserved.  ?7-=452.6<3 A)

    Part Time Reservation ':=0/=0? Ø GCP$AWS*## +!@ Ø Spot Instance !@ Ø Part Time Reservation %@ Ø Preemptible Instance@ Ø ")Reaper*Blazar/Tacker&
    &+!@ B) 8,91;  ':=0/=0? Ø Ironic *## +!@ Ø virt driverNova !@stand alone@ Ø Baremetal Reservation %@ C)  ( openstack-jp-ops@slack.com  @tetsuro !>

28. 27 Copyright©2019 NTT corp. All Rights Reserved. 1. OpenStack Placement:

    A) 756@DAC4%  -,  E i. SteinAAD7!%:>B2?7;% : Ø https://review.opendev.org/#/c/646029/ ii. RockyAAD7!%<4=1937$+. : Ø https://review.opendev.org/#/c/659207/ B) 0##756@DAC4E"/& 'E 2. DPDK SPP: A) DPDK>B83;% ) ' ( E Ø http://git.dpdk.org/apps/spp/ Ø https://www.youtube.com/watch?v=RIOHSPrvyXU
    F0#")* '

29. 28 Copyright©2019 NTT corp. All Rights Reserved.

30. 29 Copyright©2019 NTT corp. All Rights Reserved. : Extraction Step.

    1. "Placement5H=(:1GFH=$0G7<HD 2. Placement >.0D(
     A) ``nova.conf``([placement]836AG(``placement.conf``*( B) ``placement-policy.yaml``(placement;/E3<C*( 3. Placement(;H9?H7(
     4. ;H9?H7@04EH6AG(nova-api -> placement) A) mysql-migrate-db.sh, postgre-migrate-db.sh 5. ;H9?H7 (ICCH7 J A) 2>B0G@04EH6AGK CCH7#"! '&,(
    &% B) 2GB0G@04EH6AGK TrainCCH7+Extracted Placement- )2>B0G#   https://docs.openstack.org/placement/latest/upgrade/to-stein.html