OpenStack @ IIIT-H

OpenStack usage at IIIT-H presented at Hyderabad OpenStack Meetup.
https://www.meetup.com/Indian-OpenStack-User-Group/events/106526922/

Transcript

1. OpenStack@IIIT-H
   Dharmesh Kakadia (@dharmeshkakadia)
   Shashank Sahni (@shredder12)
   

   View Slide

2. What we do
   ● Run an Indian Languages Search Engine
   ● Research
   ○ Information Extraction
   ○ Information Retrieval
   ○ Information Access
   ○ Virtualization and Cloud
   ● Users of
   ○ OpenStack
   ○ Hadoop
   ○ and lot of other FOSS
   

   View Slide

3. Before OpenStack...
   

   View Slide

4. Before OpenStack
   source: http://www.codeproject.com/KB/threads/hxgrid/image4.jpg
   

   View Slide

5. Problems
   ● Provisioning
   ○ Adhoc
   ○ Time consuming
   ○ Unmanaged
   ● User Management
   ○ No resource accounting
   ○ Access Control
   ○ Usage Restriction
   ● Storage
   ○ Data reliability
   ○ Duplication
   

   View Slide

6. More Problems...
   ● Cluster
   ○ Terrible Resource Utilization
   ○ New deployment => Too much time
   ○ Data Redundancy
   ○ Non-optimal deployments
   ● Academic
   ○ No cloud platform for experimentation
   ○ Large Scale sandboxed resource provisioning for
   students.
   

   View Slide

7. After OpenStack
   

   View Slide

8. OpenStack(KVM)
   ●
   7 Compute nodes
   (8GB, quad-core)
   ●
   1 nova-volume(2 TB,
   Raid-1)
   Swift
   ● 3 storage nodes (2TB
   each)
   OpenStack(LXC)
   ● 16 Compute nodes
   (6GB, dual core)
   

   View Slide

9. Provisioning
   ● Pre-configured images to quickly get started.
   ● VM of any capacity available at any time( 2
   a.m. Sunday morning)
   ● Snapshots
   

   View Slide

10. User Management
    ● Resource restrictions using Quota
    ● Project based collaboration and private
    resources
    ● Usage monitoring
    

    View Slide

11. Storage
    This wasn't easy. We experimented with
    ● nova-volume
    ● Swift(diablo)
    ● GlusterFS
    ● Swift(Folsom)(current)
    

    View Slide

12. Storage
    ● Hadoop compatible distributed storage
    ● Glance image store
    ● Desktop backup utility using CloudFuse
    ● Data reliability
    ● No more Data Fragmentation
    

    View Slide

13. OpenStack in Academia
    ● Research
    ○ Inter cloud migration
    ○ Inter cloud scheduling
    ○ Performance Evaluation
    ● Resource provisioning for course
    assignments and projects.
    ○ 3 courses
    ○ 350+ students
    ○ 20+ projects
    

    View Slide

14. HadoopStack
    ● Big Data processing on Demand
    ● Entire ecosystem for Big Data - Hadoop
    Family, Spark, Mahout, R
    ● Multi-Cloud - OpenStack and AWS.
    

    View Slide

15. HadoopStack
    

    View Slide

16. Conclusion
    ● Using OpenStack
    ● Working with and around OpenStack
    ●
    OpenStack is Awesome !!
    

    View Slide

17. Questions/Feedback ?
    

    View Slide