Upgrade to Pro — share decks privately, control downloads, hide ads and more …

ONOS Distributed Core and performance- AT&T talk (May 7th)

ONOS Distributed Core and performance- AT&T talk (May 7th)

Madan Jampani, ON.Lab

ONOS Project

May 07, 2015
Tweet

More Decks by ONOS Project

Other Decks in Technology

Transcript

  1. ONOS Distributed Core
    ONOS Performance
    Madan Jampani
    [email protected]
    May 7th, 2015
    AT&T Talk

    View Slide

  2. Control Plane: Design Principles
    ● Distributed
    ○ Divide and conquer the problem space
    ● Symmetric
    ○ Instances are identical with respect to form and function
    ● Fault-tolerant
    ○ Handle failures seamlessly through built-in redundancy
    ● Decentralized
    ○ System is self-organizing and lacks any centralized control
    ● Incrementally Scalable
    ○ Capacity can be introduced gradually
    ● Operational Simplicity
    ○ Easy to deploy, no special hardware, no synchronized clocks, etc.

    View Slide

  3. Key Challenges
    ● Preserve simple SDN abstractions without
    compromising performance at scale
    ● Match problems to solutions that can meet their
    consistency/availability/durability needs
    ● Provide strong consistency at scale
    ● Expose distributed state management and coordination
    primitives as key application building blocks

    View Slide

  4. App
    App
    App
    App
    App
    App
    App
    App
    App
    System Model
    ● Each controller manages a
    portion of network
    ● Controllers communicate
    with each other via RPC
    ● All core services are
    accessible on any instance
    ● Applications are
    distribution transparent

    View Slide

  5. Topology (Global Network View)
    ● Partitioned by Controller
    ○ no single controller has direct visibility over entire network
    ● Reasonable size
    ○ can fit into main memory on a given controller
    ● Read-intensive
    ○ low latency access to GNV is critical
    ● Consistent with Environment
    ○ incorporate network updates as soon as possible

    View Slide

  6. Topology State Management
    1
    2
    3
    3
    4
    4
    (term,seq)

    View Slide

  7. Flow State
    ● Per switch state
    ● Exhibits strong read locality
    ● Authoritative source of what data plane should contain
    ● Backup data for quick recovery
    ● High level application edicts
    ● Intents are immutable and durable while Intent states
    are eventually consistent
    ● Topology events can trigger intent rerouting enmasse
    Intent State

    View Slide

  8. Flow State
    ● Primary/backup replication. Switch master is primary
    ● Backup location is the node that will most likely succeed
    the current master.
    ● Fully replicated using an Eventually Consistent Map
    ● Partitioned Logical Queues enable synchronization free
    execution
    Intent State

    View Slide

  9. State Management Primitives
    ● EventuallyConsistentMap
    ● ConsistentMap
    ● LeadershipService
    ● AtomicCounter *
    ● DistributedSet *
    * Will be available in Cardinal release

    View Slide

  10. Performance Metrics
    ● Device & link sensing latency
    ○ measure how fast can controller react to environment changes, such
    as switch or port down to rebuild the network graph and notify apps
    ● Flow rule operations throughput
    ○ measure how many flow rule operations can be issued against the
    controller and characterize relationship of throughput with cluster size
    ● Intent operations throughput
    ○ measure how many intent operations can be issued against controller
    cluster and characterize relationship of throughput with cluster size
    ● Intent operations latency
    ○ measure how fast can the controller react to environment changes and
    reprovision intents on the data-plane and characterize scalability

    View Slide

  11. Topology Latency
    ● Verify:
    ○ observe effect of distributed state management on latency
    ○ react faster to negative events than positive ones
    ● Results consist of multiple parts:
    ○ Switch up latency
    ○ Switch down latency
    ○ Link up/down latency
    ● Experimental setup:
    ○ Two OVS switches connected to each other.
    ○ Events are generated from the switch
    ○ Elapsed time is measured from the switch until ONOS triggers a
    corresponding topology event

    View Slide

  12. Device & Link Sensing Latency

    View Slide

  13. Switch Up Latency
    ● Most of the time is spent waiting for
    the switch to respond to a features
    request. (~53ms)
    ● ONOS spends under 25ms with most
    of it’s time electing a master for the
    device.
    ○ Which is a strongly consistent
    operation

    View Slide

  14. Switch Down Latency
    ● Significantly faster because there is
    no negotiation with the switch
    ● A terminating TCP connection
    unequivocally indicates that the
    switch is gone

    View Slide

  15. Link Up/Down Latency
    ● The increase from single to multi instance is
    being investigated
    ● Since we use LLDP to discover links, it
    takes longer to discover a link coming up
    than going down
    ● Port down event trigger immediate teardown
    of the link.

    View Slide

  16. Flow Throughput
    ● ONOS may have to provision flows for many devices
    ● Objective is to understand how flow installation scales:
    ○ with increased east/west communication with the cluster
    ○ number of devices connected to each instance
    ● Experimental setup:
    ○ Constant number of flows
    ○ Constant number of devices attached to cluster
    ○ Mastership evenly distributed
    ○ Variable number for flow installers
    ○ Variable number separate device masters traversed.

    View Slide

  17. Flow Rule Operations Throughput

    View Slide

  18. Flow Throughput results
    ● Single instance can install over 500K
    flows per second
    ● ONOS can handle 3M local and 2M
    non local flow installations
    ● With 1-3 ONOS instances, the flow
    setup rate remains constant no
    matter how many neighbours are
    involved
    ● With more than 3 instances injecting
    load the flow performance drops off
    due to extra coordination requires.

    View Slide

  19. Intent Framework Performance
    ● Intents are high-level, network-level policy definitions
    ○ e.g. provide connectivity between two hosts, or
    route all traffic that matches this prefix to this edge port
    ● All objects are distributed for high availability
    ○ Synchronously written to at least one other node
    ○ Work is divided among all instances in the cluster
    ● Intents must be compiled into device-specific rules
    ○ Paths are computed and selected, reservations made, etc.
    ● Device-specific rules are installed
    ○ Leveraging other asynchronous subsystems (e.g. Flow Rule Service)
    ● Intents react to network events ("reroute")
    ○ e.g. device or link failure, host movement, etc.

    View Slide

  20. Intent Framework Latency
    ● API calls are asynchronous and return after storing the
    intent
    ● After an intent has been submitted, the framework starts
    compiling and installing
    ● An event is generated after the framework confirms that
    the policy has been written to the devices
    ● Experiment shows how quickly an application's policy can
    be reflected in the network ("install" and "withdrawn"), as
    well as how long it takes to react to a network event
    ("reroute")

    View Slide

  21. Intent Latency Experiment

    View Slide

  22. Intent Latency Results
    ● Less than 100ms to install or withdraw a batch of intents
    ● Less than 50ms to process and react to network events
    ○ Slightly faster because intent objects are already replicated

    View Slide

  23. Intent Framework Throughput
    ● Dynamic networks undergo changes of policies (e.g.
    forwarding decision) on an ongoing basis
    ● Framework needs to be able to cope with a stream of
    requests and catch-up if it ever falls behind
    ● Capacity needs to scale with growth of the cluster

    View Slide

  24. Intent Throughput Experiment

    View Slide

  25. Intent Throughput Results
    ● Processing clearly scales as cluster size increases

    View Slide

  26. View Slide