ONOS

Transcript

1. ONOS   Open  Network  Opera.ng  System     An  Experimental  Open-­‐Source  Distributed  SDN  OS   Pankaj  Berde,  Umesh  Krishnaswamy,  Jonathan  Hart,  Masayoshi  Kobayashi,  Pavlin  Radoslavov,   Pingping  Lin,  Sean  Corcoran,  Tim  Lindberg,  Rachel  Sverdlov,  Suibin  Zhang,  William  Snow,  Guru   Parulkar  
2. Agenda   §  Overview  of  ONRC  (Open  Networking  Research  Center)     §  ONOS     •  Architecture   •  Scale-­‐out  and  high  availability     •  Network  graph  as  north-­‐bound  abstracOon   •  DEMO   •  Consistency  models     •  Development  and  test  environment     •  Performance     •  Next  steps    
3. Leadership   NaOonal  Academy  of  Engineering   ACM  SIGCOMM  Award  Winners       Fellow  of  IEEE  and  ACM   Entrepreneurs   Impact  on  pracOce  of  networking/cloud   Nick  Mckeown   KP,  Mayfield,  Sequoia   Professor,  Stanford   Larry  Peterson   Bob  Kahn  Professor  Princeton   Chief  Architect,  ON.LAB   ScoY  Shenker   Professor,  UC  Berkeley   Chief  ScienOst,  ICSI    
4. Stanford/Berkeley  SDN  AcOviOes     With  Partners   2007   2011   2008   2009   2010   Ethane   Demo   Deployment   Pla`orm    Development   OpenFlow  Spec   v0.8.9   v1.0   v1.1   Reference  Switch   NetFPGA   Soaware   Network  OS   NOX   SNAC   Beacon   VirtualizaOon   FlowVisor   FlowVisor  (Java)   Tools   Test  Suite   oarace   Mininet   Measurement  tools   GENI  soaware  suite   Expedient/Opt-­‐in  Manager/FOAM   Stanford  University   ~45  switch/APs  ~25user   In  McKeown  Group   CIS/EE  Building   ProducOon  Network   US  R&E  Community   GENI:  8  UniversiOes  +  Internet2  +  NLR   Many  other  campuses   Other  countries   Over  68  countries   (Europe,  Japan,  China,  Korea,   Brazil,  etc.)   VM  MigraOon   (Best  Demo)   Trans-­‐Pacific   VM  MigraOon   Baby  GENI   NaOon  Wide  GENI   “The  OpenFlow  Show”   –  IT  World   SDN  Concept   (Best  Demo)   SIGCOMM08   GEC3   SIGCOMM09   GEC6   GEC9   Interop   2011   +Broadcom  
5. Scaling  of  SDN  InnovaOon     Standardize  OpenFlow  and  promote  SDN   ~100  Members  from  all  parts  of  the  industry   Bring  best  SDN  content;  facilitate  high  quality  dialogue   3  successive  sold  out  events;  parOcipaOon  of  ecosys   Build  strong  intellectual  foundaOon   Bring  open  source  SDN  tools/pla`orms  to  community   SDN     Academy   Bring  best  SDN  training  to  companies   to  accelerate  SDN  development  and  adopOon  
6. ONRC Organizational Structure Berkeley Scott Shenker Sylvia Ratnasamy Open Network Lab Exec Director: Guru Parulkar VP Eng: Bill Snow Chief Architect: Larry Peterson 16-19 Engineers/Tech Leads (includes PlanetLab team) Tools/Platforms for SDN community OpenCloud demonstration of XaaS and SDN   PhD/Postdocs     Research   Stanford Nick McKeown Guru Parulkar Sachin Katti
7.       Mission   Bring  innovaOon  and  openness  to  internet   and  cloud  infrastructure  with  open  source   tools  and  pla`orms   7  
8. Tools  &  Pla`orms   3rd  party   components   Network  OS   Apps   Apps   Network  OS   Apps   Apps   Open  Interfaces   Open  Interfaces   Network  Hypervisor   Forwarding   FlowVisor,  OpenVirteX   MININET,  Cluster  Edi.on   ONOS   SDN-­‐IP  Peering   TestON  with  debugging  support   NetSight  
9. Open  Network  OS  (ONOS)   §  Architecture     §  Scale-­‐out  and  high  availability     §  Network  graph  as  north-­‐bound  abstracOon   §  DEMO   §  Consistency  models     §  Development  and  test  environment     §  Performance     §  Next  steps    
10. ONOS:  ExecuOve  Summary   ONOS   Status   Distributed  Network  OS   Network  Graph  Northbound  Abstrac.on   Horizontally  Scalable   Highly  Available   Built  using  open  source  components   Version  0.1    -­‐  Flow  API,  Shortest  Path  computa.on,  Sample  applica.on   -­‐  Build  &  QA    (  Jenkins,  Sanity  Tests,  Perf/Scale  Tests,  CHO)   -­‐  Deployment  in  progress  at  REANNZ  (SDN-­‐IP  peering)     Next   Exploring  performance  &  reac.ve  computa.on  frameworks   Expand  graph  abstrac.on  for  more  types  of  network  state   Control  func.ons:  intra-­‐domain  &  inter-­‐domain  rou.ng   Example  use  cases:  traffic  engineering,    dynamic  virtual  networks   on  demand,  …  
11. ONOS  –  Architecture  Overview  
12. RouOng   TE   Network  OS   Packet   Forwarding       Packet   Forwarding       Packet   Forwarding       Mobility   Programmable   Base  StaOon     Openflow   Scale-­‐out   Design   Fault  Tolerance   Global  network  view   Open  Network  OS  Focus   (Started  in  Summer  2012)   Global  Network  View  
13. Prior  Work     Distributed  control  plaaorm  for  large-­‐scale  networks   Focus  on  reliability,  scalability,  and  generality   Scale-­‐out  NOS  focused  on  network  virtualiza.on  in  data  centers   State  distribu.on  primi.ves,  global  network  view,  ONIX  API   ONIX   Other  Work   Helios  (NEC),  Midonet  (Midokura),  Hyperflow,  Maestro,  Kandoo       NOX,  POX,  Beacon,  Floodlight,  Trema  controllers   Community  needs  an  open  source  distributed  SDN  OS  
14. Host   Host   Host   Titan  Graph  DB   Cassandra  In-­‐Memory  DHT   Instance  1   Instance  2   Instance  3   Network  Graph   Eventually  consistent   Distributed  Registry   Strongly  Consistent   Zookeeper   OpenFlow     Controller+   OpenFlow   Controller+   OpenFlow   Controller+   ONOS  High  Level  Architecture   +Floodlight   Drivers  
15. Scale-­‐out  &  HA  
16. ONOS  Scale-­‐Out   Distributed   Network  OS   Instance  2     Instance  3                 Instance  1     Network    Graph   Global  network  view   An  instance  is  responsible  for  maintaining  a  part  of  network  graph   Control  capacity  can  grow  with  network  size  or  applicaOon  need     Data  plane  
17. Master    Switch  A  =  ONOS  1   Candidates  =  ONOS  2,   ONOS  3   Master    Switch  A  =  ONOS  1   Candidates  =    ONOS  2,   ONOS  3   Master    Switch  A  =  ONOS  1   Candidates  =  ONOS  2,   ONOS  3   ONOS  Control  Plane  Failover   Distributed   Network  OS   Instance  2     Instance  3     Instance  1     Distributed   Registry   Host   Host   Host   A   B   C   D   E   F   Master    Switch  A  =  NONE       Candidates  =  ONOS  2,   ONOS  3   Master    Switch  A  =  NONE   Candidates  =  ONOS  2,   ONOS  3   Master    Switch  A  =  NONE   Candidates  =    ONOS  2,   ONOS  3   Master    Switch  A  =  ONOS  2     Candidates  =  ONOS  3   Master    Switch  A  =  ONOS  2     Candidates  =  ONOS  3   Master    Switch  A  =  ONOS  2     Candidates  =  ONOS  3  
18. Network  Graph  
19. Cassandra     In-­‐memory  DHT   Id:  1   A   Id:  101,  Label   Id:  103,  Label   Id:  2   C   Id:  3   B   Id:  102,  Label   Id:  104,  Label   Id:  106,  Label   Id:  105,  Label   Network  Graph     Titan  Graph  DB   ONOS  Network  Graph  AbstracOon  
20. Network  Graph   port switch port device port on port port port link switch on device host host Ø  Network  state  is  naturally  represented  as  a  graph   Ø  Graph  has  basic  network  objects  like  switch,  port,  device  and  links   Ø  Applica.on  writes  to  this  graph  &  programs  the  data  plane  
21. Example:  Path  ComputaOon  App  on  Network   Graph   port switch port device Flow  path Flow  entry port on port port port link switch inport on Flow  entry device outport switch switch host host flow flow •  Applica.on  computes  path  by  traversing  the  links  from  source  to  des.na.on   •  Applica.on  writes  each  flow  entry  for  the  path   Thus  path  computa.on  app  does  not  need  to  worry  about  topology  maintenance    
22. Example:  A  simpler  abstracOon  on  network   graph?     Logical  Crossbar port switch port device Edge  Port port on port port port link switch physical on Edge  Port device physical host host •  App  or  service  on  top  of  ONOS     •  Maintains  mapping  from  simpler  to  complex     Thus  makes  applica.ons  even  simpler  and  enables  new  abstrac.ons   Virtual  network  objects Real  network  objects
23. Network  Graph  RepresentaOon   Flow  path Flow  entry Flow  entry flow flow Vertex  with     10  properOes   Vertex  with     11  properOes   Vertex     represented  as   Cassandra  row   Property   (e.g.   dpid)   Property   (e.g.   state)   Property   …   Edge   Edge   Edge  represented   as  Cassandra   column   Column   Value   Label  id  +   direcOon   Primary   key   Edge  id   Vertex  id   Signature   properOes   Other   properOes   Switch Vertex  with     3  properOes   Row  indices  for  fast  vertex  centric  queries  
24. Switch  Manager   Switch  Manager   Switch  Manager   Network  Graph:  Switches   OF   OF   OF   OF   OF   OF   Network  Graph  and  Switches  
25. SM   Network  Graph:  Links   SM   SM   Link  Discovery   Link  Discovery   Link  Discovery   LLDP   LLDP   Network  Graph  and  Link  Discovery  
26. Network  Graph:  Devices   SM   SM   SM   LD   LD   LD   Device  Manager   Device  Manager   Device  Manager   PKTIN   PKTIN   PKTIN   Host   Host   Host   Devices  and  Network  Graph  
27. SM   SM   SM   LD   LD   LD   Host   Host   Host   DM   DM   DM   Path  Computa.on   Path  Computa.on   Path  Computa.on   Network  Graph:  Flow  Paths   Flow  1   Flow  4   Flow  7   Flow  2   Flow  5   Flow  3   Flow  6   Flow  8   Flow  entries   Flow  entries   Flow  entries   Flow  entries   Flow  entries   Flow  entries   Flow  entries   Flow  entries   Flow  entries   Flow  entries   Flow  entries   Flow  entries   Flow  entries   Flow  entries   Flow  entries   Flow  entries   Flow  entries   Flow  entries   Flow  entries   Flow  entries   Flow  entries   Flow  entries   Flow  entries   Flow  entries   Path  ComputaOon  with  Network  Graph  
28. SM   SM   SM   LD   LD   LD   Host   Host   Host   DM   DM   DM   Flow  Manager   Network  Graph:  Flows   PC   PC   PC   Flow  Manager   Flow  Manager   Flowmod   Flowmod   Flowmod   Flow  1   Flow  4   Flow  7   Flow  2   Flow  5   Flow  3   Flow  6   Flow  8   Flow  entries   Flow  entries   Flow  entries   Flow  entries   Flow  entries   Flow  entries   Flow  entries   Flow  entries   Flow  entries   Flow  entries   Flow  entries   Flow  entries   Flow  entries   Flow  entries   Flow  entries   Flow  entries   Flow  entries   Flow  entries   Flow  entries   Flow  entries   Flow  entries   Flow  entries   Flow  entries   Flow  entries   Network  Graph  and  Flow  Manager  
29. Host   Host   Host   Titan  Graph  DB   Cassandra  In-­‐Memory  DHT   Instance  1   Instance  2   Instance  3   Network  Graph   Eventually  consistent   Distributed  Registry   Strongly  Consistent   Zookeeper   OpenFlow     Controller+   OpenFlow   Controller+   OpenFlow   Controller+   ONOS  High  Level  Architecture   +Floodlight   Drivers  
30. DEMO  
31. Consistency  Deep  Dive  
32. Consistency  DefiniOon   •  Strong  Consistency:  Upon  an  update  to  the  network  state  by  an  instance,   all  subsequent  reads  by  any  instance  returns  the  last  updated  value.       •  Strong  consistency  adds  complexity  and  latency  to  distributed  data   management.     •  Eventual  consistency  is  slight  relaxaOon  –  allowing  readers  to  be  behind   for  a  short  period  of  Ome.    
33. Strong  Consistency  using  Registry   Distributed   Network  OS       Instance  2     Instance  3             Network  Graph   Instance  1     A  =     Switch  A   Master  =  NONE     A  =  ONOS  1     Timeline   All  instances      Switch  A    Master  =  NONE   Instance  1  Switch  A  Master  =  ONOS  1   Instance  2  Switch  A  Master  =  ONOS  1   Instance  3  Switch  A  Master  =  ONOS  1   Master  elected  for  switch  A   Registry   Switch  A     Master  =  NONE   Switch  A     Master  =  ONOS  1     Switch  A     Master  =  ONOS  1     Switch  A   Master  =  NONE       Switch  A    Master  =  ONOS  1     Delay  of  Locking  &   Consensus   All  instances      Switch  A    Master  =  NONE  
34. Why  Strong  Consistency  is  needed  for   Master  ElecOon   §  Weaker  consistency  might  mean  Master  elecOon  on   instance  1  will  not  be  available  on  other  instances.   §  That  can  lead  to  having  mulOple  masters  for  a  switch.   §  MulOple  Masters  will  break  our  semanOc  of  control   isolaOon.   §  Strong  locking  semanOc  is  needed  for  Master  ElecOon  
35. Eventual  Consistency  in  Network  Graph   Distributed   Network  OS       Instance  2     Instance  3             Network  Graph   Instance  1     SWITCH  A     STATE=  INACTIVE   Switch  A   State  =  INACTIVE   Switch  A   STATE  =  INACTIVE   All  instances                    Switch  A  STATE  =  ACTIVE   Instance  1  Switch  A  =  ACTIVE   Instance  2  Switch  A  =  INACTIVE   Instance  3  Switch  A  =  INACTIVE   DHT   Switch  Connected  to  ONOS   Switch  A    State  =  ACTIVE   Switch  A     State  =  ACTIVE   Switch  A   STATE    =  ACTIVE   Timeline   All  instances              Switch  A    STATE  =  INACTIVE   Delay  of  Eventual   Consensus  
36. Cost  of  Eventual  Consistency   §  Short  delay  will  mean  the  switch  A  state  is  not  ACTIVE  on   some  ONOS  instances  in  previous  example.   §  ApplicaOons  on  one  instance  will  compute  flow  through  the   switch  A  while  other  instances  will  not  use  the  switch  A  for   path  computaOon.   §  Eventual  consistency  becomes  more  visible  during  control   plane  network  congesOon.  
37. Why  is  Eventual  Consistency  good   enough  for  Network  State?     §  Physical  network  state  changes  asynchronously     §  Strong  consistency  across  data  and  control  plane  is  too  hard   §  Control  apps  know  how  to  deal  with  eventual  consistency   §  In  the  current  distributed  control  plane,  each  router  makes  its   own  decision  based  on  old  info  from  other  parts  of  the  network   and  it  works  fine       §  Strong  Consistency  is  more  likely  to  lead  to  inaccuracy  of   network  state  as  network  congesOons  are  real.  
38. Consistency  learning   §  One  Consistency  does  not  fit  all   §  Consequences  of  delays  need  to  be  well  understood   §  More  research  needs  to  be  done  on  various  states   using  different  consistency  models  
39. Development  &  test  environment  
40. ONOS  Development  &  Test  cycle   §  Source  code  on  github     §  Agile:  3-­‐4  week  sprints   §  Mostly  Java  and  many  uOlity  scripts   §  CI:  Maven,  Jenkins,  JUnit,  Coverage,  TestON   §  Vagrant-­‐based  development  VM   §  Daily  4  hour  of  ConOnuous  Hours  of  OperaOons  (CHO)  tests  as   part  of  build   §  Several  CHO  cycles  simulaOng  rapid  churns  in  network  &   failures  on  ONOS  instances  
41. ONOS  Development  Environment   Single  installa.on  script  creates  a  cluster  of  Virtual  Box  VMs  
42. Test  Lab  Topology  
43. ON.LAB ONOS Test implementation §  ON.LAB  team  has  implemented  following   aspects  of  automated  tests   •  ONOS  Unit  Tests  (70%  coverage)   •  ONOS  System  Tests  for  FuncOonality,  Scale,   Performance  and  Resiliency  test  (85%  coverage)   •  White  Box  Network  Graph  Performance   Measurements   §  All  tests  are  executed  nightly  in  Jenkins   ConOnuous  IntegraOon  environment.      
44. Performance  
45. Key  performance  metrics  in  Network  OS   §  Network  scale  (#  switches,  #  ports)  -­‐>  Delay  and  Throughput   •  Link  failure,  switch  failure,  switch  port  failure   •  Packet_in  (request  for  sexng  reacOve  flows)   •  Reading  and  searching  network  graph   •  Network  Graph  Traversals   •  Setup  of  proacOve  flows   §  ApplicaOon  scale  (#  operaOons,  #  applicaOons)     •  Number  of  network  events  propagated  to  applicaOons  (delay  &   throughput)   •  Number  of  operaOons  on  Network  Graph  (delay  &  throughput)   •  Parallelism/threading  for  applicaOons  (parallelism  on  Network  Graph)   •  Parallel  path  computaOon  performance  
46. Performance:  Hard  Problems   §  Off  the  shelf  open  source  does  not  perform   §  Ultra  low-­‐latency  requirements  are  unique   §  Need  to  apply  distributed/parallel  programming  techniques   to  scale  control  applicaOons   §  ReacOve  control  applicaOons  need  event-­‐driven  framework   which  scale   46  
47. ONOS:  Summary   ONOS   Status   Distributed  Network  OS   Network  Graph  Northbound  Abstrac.on   Horizontally  Scalable   Highly  Available   Built  using  open  source  components   Version  0.1    -­‐  Flow  API,  Shortest  Path  computa.on,  Sample  applica.on   -­‐  Build  &  QA    (  Jenkins,  Sanity  Tests,  Perf/Scale  Tests,  CHO)   -­‐  Deployment  in  progress  at  REANNZ  (SDN-­‐IP  peering)     Next   Exploring  performance  &  reac.ve  computa.on  frameworks   Expand  graph  abstrac.on  for  more  types  of  network  state   Control  func.ons:  intra-­‐domain  &  inter-­‐domain  rou.ng   Example  use  cases:  traffic  engineering,    dynamic  virtual  networks   on  demand,  …  
48. www.onlab.us