FGED Keynote

Keynote at FGED 2013

Transcript

1. Deepak Singh Principal Product Manager, Amazon EC2 deesingh@amazon.com @mndoci Science without constraints
2. Good morning!
7. Bioinformatics
8. collection
9. curation
10. analysis
11. So what?
12. Image: Yael Fitzpatrick (AAAS)
13. lots of data
14. lots of people
15. lots of places
16. constant change
17. How can we be more effective with data?
18. Versioning
19. Provenance
20. Filter
21. Aggregate Image: Chris Heiler
22. Image: Bethan Extend
23. Image: Sebastian Anthony Interfaces
24. Share
25. image: Leo Reynolds Communicate
26. Hard Problem
27. Hard Problem Really ^
28. How do we scale?
29. How can we be more effective with data?
30. Wrong question
31. How can we be more effective with our people?
32. Image: Pieter Musterd Remove constraints
33. Move data to the users?
34. Move data to the users? X
35. Move tools to the data
36. Give tools access to data
37. Give data access to tools
38. APIs
39. Small things, loosely coupled
41. Amazon EC2
42. ec2-run-instances
43. starcluster start
45. Programmable
48. Elastic
49. Go from one instance...
50. To Thousands
51. Building Blocks
52. Instance Types
53. General Purpose (M1, M3) Memory Optimized (M2,CR1) Compute Optimized (C1, CC2) Storage Optimized (HI1, HS1) GPU Optimized (CG1)
54. 240 Teraflops
55. 120,000 IOPS
56. Amazon S3
57. > 2 Trillion Objects
59. Secured Uplink Planning
60. JPL Data Center Decider File Transfer Workers Data Processing Workers Polyphony Amazon SWF Decider Data Processing Tasks File Transfer Tasks Decision Tasks Create EC2 Instances Upload and Download File Chunks Data Processing Workers EC2 EC2 EC2 EC2 S3
61. SWF EC2 S3 SimpleDB CloudWatch IAMs ELB 5 Giga-pixels in 5 minutes!
63. Zero to Internet-Scale in One Week!
64. Mars Science Laboratory - Live Video Streaming Architecture Availability Zone: us-east-1a Adobe Flash Media Server Availability Zone: us-west-1b Telestream Wirecast CloudFront streaming for museum partners Adobe Flash Media Server Elastic Load Balancer Tier 2 Nginx Cache Tier 1 Nginx Cache Cloud Formation Stack Elastic Load Balancer Tier 2 Nginx Cache Tier 1 Nginx Cache Cloud Formation Stack
66. Reproduce. Reuse. Remix
69. [plugin ipcluster] setup_class = ipcluster.IPCluster enable_notebook = true notebook_passwd = YOUR-PASS [cluster qiime] node_image_id = ami-2faa7346 keyname = YOUR-KEY cluster_size = 4 node_instance_type = m2.4xlarge plugins = ipcluster $ starcluster start -c qiime myqiime Source: Justin Riley
71. 1000 Genomes
72. 1000 Genomes Cloud BioLinux
74. Your HiSeq data Illumina BaseSpace
75. Your data GenomeSpace
80. Reproduce. Reuse. Remix
81. Image: Pieter Musterd Making people more effective
82. Thank You