CyVerse overview - CSHL Hackathon

Transcript

1. Overview of CyVerse CyVerse 2016 Transforming Science Through Data-Driven Discovery Vision: Transforming science through data-driven discovery Mission: Design, develop, deploy, and expand a naAonal cyberinfrastructure for life science research, and train scienAsts Funding: NaAonal Science FoundaAon. $100 Million in investment Usage: More than 30K users, PB of data, and hundreds of publicaAons, courses, and discoveries
2. What is cyberinfrastructure? PlaPorms Data Storage & HPC People
3. CyVerse supports all domains Plant / Microbial Animal Biomedical Ecological/Climate CyVerse is built for data
4. CyVerse product stack Ready to use Pla:orms Founda<onal Capabili<es Established CI Components Extensible Services Ease of Use Flexibility
5. CyVerse products Data Store Discovery Environment Atmosphere BisQue Data commons
6. Discovery Environment Hundreds of bioinformaAcs apps in an easy-to-use interface ü  A plaPorm that can run almost any bioinformaAcs applicaAon ü  Seamlessly integrated with data and high performance compuAng ü  User extensible – add, edit and share your applicaAons
7. Discovery Environment
8. •  Upload / Download files and folders •  Share files via URL (Public Links) •  Share files/folders with other users Data Manage data Discovery Environment
9. Apps •  Run hundreds of bioinformaAcs Apps •  Build automated workflows •  Modify Apps or integrate new ones Customize ApplicaAons and launch jobs Discovery Environment
10. Analyses •  Monitor job status and find results •  Cancel jobs or re-launch jobs •  Detailed job history View history, find results, reproduce analyses, opAmize parameters Discovery Environment
11. Discovery Environment Get Science Done Reproducibility ProducAvity •  Use hundreds of bioinformaAcs Apps without the command line •  Add your own applicaAons – an extensible, scalable plaPorm •  Analysis history and provenance •  Create and publish Apps and workflows so anyone can use them •  High-performance compuAng – not dependent on your hardware •  Manage a secure data repository and share data easily Access your computaAonal science through a single portal
12. The Reality + + •  R •  PERL •  Python •  ARCGIS •  Java •  Fortran C C# C++ •  etc. •  Amazon •  Azure •  Rackspace •  Campus HPC •  XSEDE •  Etc.
13. •  Docker is a type of virtualizaAon for sogware distribuAon. •  “Docker is an open plaPorm for developing, shipping, and running applicaAons as a light weight containers” – From Docker website. •  Docker allow you to create and manage containers •  Container consists of: •  The base image (Ex: Ubuntu 14.04) •  Your applicaAon source code •  Dependencies/Libraries/Binaries
14. How does it work together? Source:www.docker.com
15. Why use Docker? ü  Portable deployments ü  Fast applicaAon delivery ü  Scale and deploy with ease ü  Easy sogware versioning ü  Reproducibility
16. Design app UI User Prepare Dockerfile User Build Docker image CyVerse Staff The process of Dockerizing tools in DE à à FROM r-base:latest MAINTAINER Upendra Devisemy <upendra@cyverse.org> LABEL DescripAon "This Dockerfile is for IUTA-1.0” RUN apt-get update && apt-get install –y r-base-dev && libxml2 && libxml2-dev RUN wget hmp://www.niehs.nih.gov/resources/files/IUTA_1.0.tar.gz RUN tar zxvf IUTA_1.0.tar.gz ADD run_IUTA.R / RUN chmod +x /run_IUTA.R && cp /run_IUTA.R /usr/bin ENTRYPOINT ["run_IUTA.R”] à
17. How to get started? •  Install Docker on your local machine •  Figure out dependencies/binaries/source code •  Submit the request for a “new tool” •  Design UI for your app and use it •  Create a Dockerfile, build and test the built image +
18. Help with DockerizaAon