Tweet
Tweet

Slide 1

Slide 1 text

Nispero Marina Manrique 2013-08-28

Slide 2

Slide 2 text

content • What is this? • Architecture and players Nispero + Statika • Nispero + Statika • Why Nispero is cool for NGS data analysis? • Hands-on Nispero

Slide 3

Slide 3 text

What is this? loquats

Slide 4

Slide 4 text

What is this? A component to scale independent tasks Basic building block to implement distributed systems

Slide 5

Slide 5 text

Architecture and players

Slide 6

Slide 6 text

No content

Slide 7

Slide 7 text

Nispero + Statika You need to configure the workers!!

Slide 8

Slide 8 text

Why Nispero is cool for NGS data analysis?

Slide 9

Slide 9 text

Why Nispero is cool for NGS data analysis? parallel automatic tasks horizontal scaling horizontal scaling - Easy to use - Easy to reuse - Robust

Slide 10

Slide 10 text

Why Nispero is cool for NGS data analysis? parallel automatic tasks horizontal scaling horizontal scaling - Easy to use - Easy to reuse - Robust

Slide 11

Slide 11 text

Why Nispero is cool for NGS data analysis? usual parallel tasks in NGS data analysis analysis

Slide 12

Slide 12 text

Why Nispero is cool for NGS data analysis? usual parallel tasks in NGS data analysis analysis - QA - Reads Preprocessing: trimming, filtering - Blast

Slide 13

Slide 13 text

Why Nispero is cool for NGS data analysis? Samples are parallel Reads are parallel

Slide 14

Slide 14 text

Why Nispero is cool for NGS data analysis? 100Gb X min

Slide 15

Slide 15 text

Why Nispero is cool for NGS data analysis? 100Gb X min X/4 min

Slide 16

Slide 16 text

Why Nispero is cool for NGS data analysis? 100Gb X min Samples Reads ~X/16 min

Slide 17

Slide 17 text

Why Nispero is cool for NGS data analysis? horizontal scaling limited number of different instances types

Slide 18

Slide 18 text

Why Nispero is cool for NGS data analysis? horizontal scaling limited number of different instances types

Slide 19

Slide 19 text

Why Nispero is cool for NGS data analysis? horizontal scaling limited number of different instances types : cr1.8xlarge

Slide 20

Slide 20 text

No content

Slide 21

Slide 21 text

No content

Slide 22

Slide 22 text

No content

Slide 23

Slide 23 text

Hands-on Nispero Now it’s your turn You’re going to try to use Nispero to run two parallel blast run two parallel blast And Kim please support the others but let them try by themselves :)

Slide 24

Slide 24 text

Hands-on Nispero Now it’s your turn You’re going to try to use Nispero to run two parallel blast run two parallel blast And Kim please support the others but let them try by themselves :)

Slide 25

Slide 25 text

Hands-on Nispero How to run a Nispero 1. Set up the environment 2. Prepare the tasks file 3. Prepare the scripts 3. Prepare the scripts 4. Download 5. Set up the configuration file 6. Publish the folder 7. Run Nispero 8. Terminate nispero-usage.md

Slide 26

Slide 26 text

Hands-on Nispero 1. Set up the environment aws-linux-env-setup.md Standard Linux Amazon m2.xlarge Spot request God mode + right key pair

Slide 27

Slide 27 text

Hands-on Nispero 2. Prepare the tasks file Put the tasks file in a bucket in S3 Put the tasks file in a bucket in S3

Slide 28

Slide 28 text

"id": "task1", "inputObjects":{ "database":{ "bucket":"team1-resources", "key":"input/NC_000913.fna" }, "query":{ "bucket":"team1-resources", "key":"input/E-coli-rna.frn" "key":"input/E-coli-rna.frn" } }, "outputObjects":{ "results":{ "bucket":"team1-resources", "key":"results/ecoli-blastresults.txt" } }

Slide 29

Slide 29 text

Hands-on Nispero 2. Prepare the tasks file Put the tasks file in a bucket in S3 Put the tasks file in a bucket in S3

Slide 30

Slide 30 text

Hands-on Nispero 3. Prepare the scripts - Workers configuration (Statika?) - Workers configuration (Statika?) - Run the task (Blast)

Slide 31

Slide 31 text

Hands-on Nispero 4. Download 4.1 Connect to the instance you launched at 1. 4.2 Download Nispero to the instance https://github.com/ohnosequences/cloud-ngs- course/blob/master/nispero-task/nispero- usage.md#download Name + mail

Slide 32

Slide 32 text

Hands-on Nispero 5. Set up the configuration.scala file 5.1 Use VIM to edit the config file 5.2 https://github.com/ohnosequences/cloud- ngs-course/blob/master/nispero- task/nispero-usage.md#setup-your- configuration

Slide 33

Slide 33 text

Hands-on Nispero 6. Publish https://github.com/ohnosequences/cloud -ngs-course/blob/master/nispero- task/nispero-usage.md#publish task/nispero-usage.md#publish 7. Run https://github.com/ohnosequences/cloud -ngs-course/blob/master/nispero- task/nispero-usage.md#run

Slide 34

Slide 34 text

Hands-on Nispero 8. Terminate / Undeploy

Slide 35

Slide 35 text

No content