$30 off During Our Annual Pro Sale. View Details »

OpenStack @ IIIT-H

OpenStack @ IIIT-H

OpenStack usage at IIIT-H presented at Hyderabad OpenStack Meetup.
https://www.meetup.com/Indian-OpenStack-User-Group/events/106526922/

dharmeshkakadia

March 31, 2013
Tweet

More Decks by dharmeshkakadia

Other Decks in Technology

Transcript

  1. OpenStack@IIIT-H
    Dharmesh Kakadia (@dharmeshkakadia)
    Shashank Sahni (@shredder12)

    View Slide

  2. What we do
    ● Run an Indian Languages Search Engine
    ● Research
    ○ Information Extraction
    ○ Information Retrieval
    ○ Information Access
    ○ Virtualization and Cloud
    ● Users of
    ○ OpenStack
    ○ Hadoop
    ○ and lot of other FOSS

    View Slide

  3. Before OpenStack...

    View Slide

  4. Before OpenStack
    source: http://www.codeproject.com/KB/threads/hxgrid/image4.jpg

    View Slide

  5. Problems
    ● Provisioning
    ○ Adhoc
    ○ Time consuming
    ○ Unmanaged
    ● User Management
    ○ No resource accounting
    ○ Access Control
    ○ Usage Restriction
    ● Storage
    ○ Data reliability
    ○ Duplication

    View Slide

  6. More Problems...
    ● Cluster
    ○ Terrible Resource Utilization
    ○ New deployment => Too much time
    ○ Data Redundancy
    ○ Non-optimal deployments
    ● Academic
    ○ No cloud platform for experimentation
    ○ Large Scale sandboxed resource provisioning for
    students.

    View Slide

  7. After OpenStack

    View Slide

  8. OpenStack(KVM)

    7 Compute nodes
    (8GB, quad-core)

    1 nova-volume(2 TB,
    Raid-1)
    Swift
    ● 3 storage nodes (2TB
    each)
    OpenStack(LXC)
    ● 16 Compute nodes
    (6GB, dual core)

    View Slide

  9. Provisioning
    ● Pre-configured images to quickly get started.
    ● VM of any capacity available at any time( 2
    a.m. Sunday morning)
    ● Snapshots

    View Slide

  10. User Management
    ● Resource restrictions using Quota
    ● Project based collaboration and private
    resources
    ● Usage monitoring

    View Slide

  11. Storage
    This wasn't easy. We experimented with
    ● nova-volume
    ● Swift(diablo)
    ● GlusterFS
    ● Swift(Folsom)(current)

    View Slide

  12. Storage
    ● Hadoop compatible distributed storage
    ● Glance image store
    ● Desktop backup utility using CloudFuse
    ● Data reliability
    ● No more Data Fragmentation

    View Slide

  13. OpenStack in Academia
    ● Research
    ○ Inter cloud migration
    ○ Inter cloud scheduling
    ○ Performance Evaluation
    ● Resource provisioning for course
    assignments and projects.
    ○ 3 courses
    ○ 350+ students
    ○ 20+ projects

    View Slide

  14. HadoopStack
    ● Big Data processing on Demand
    ● Entire ecosystem for Big Data - Hadoop
    Family, Spark, Mahout, R
    ● Multi-Cloud - OpenStack and AWS.

    View Slide

  15. HadoopStack

    View Slide

  16. Conclusion
    ● Using OpenStack
    ● Working with and around OpenStack

    OpenStack is Awesome !!

    View Slide

  17. Questions/Feedback ?

    View Slide