ESA

Transcript

1. Community   Profiling  via   QIIME   Dorota  Porazinska  and  Zech  Xu   University  of  Colorado   Boulder,  CO    
2. File  Download   •  View  slides  at:   –  hAp://goo.gl/4duXII   •  Raw  files:     –  hAps://app.box.com/s/kwzjd1go2g8cmic59xcd   –  Extract  it:   !tar zxf crawford_mice.tar.gz! •  View  IPython  Notebook   –  hAp://nbviewer.ipython.org/gist/RNAer/d8e7cbd7b68a273d2269   –  Also  inside  the  downloaded  files  (require  ipython  to  open  it)   •  Processed  file:   –  hAps://app.box.com/s/3a6gvuyn8crjamx7uqte   –  Run:   !mv output.tar.gz crawford_mice! !tar zxf output.tar.gz!
3. Sequencing  cost  ge]ng  cheaper   hAp://goo.gl/rWW1Ay  
4. Tsunami  of  sequence  data   ??? 3
5. 1st  vs.  NGS  technologies     hAp://www.patrickwardphd.com/wp-­‐content/uploads/2012/05/sprinkler-­‐kids-­‐l.jpg   hAp://1000awesomethings.com/2011/06/21/218-­‐drinking-­‐from-­‐the-­‐hose/  
6. A  classic  microbial  ecology  study  
7. A  classic  microbial  ecology  study  
8. A  classic  microbial  ecology  study  
9. A  classic  microbial  ecology  study  
10. A  classic  microbial  ecology  study  
11. A  classic  microbial  ecology  study   Bacterial  Community  Variacon  in  Human  Body  Habitats   Across  Space  and  Time,  Costello  et  al.,  Science  2009   Modified  from  Hamady  et  al.  Genome  Research.  2009  
12. Datasets  with  billions  of  sequences:     •  Human  Microbiome  Project:  Largest   characterizacon  of  the  microbiome  of  healthy   individuals   – NIH  sponsored,  $185  million  project   – Samples  from  300  adults  and  18  body  sites   – Raw  data:  ~232  GB  
13.  Earth  Microbiome  Project  
15. Coursera  Course   hAps://www.coursera.org/course/microbiome  
16. …  accumulacng  data   Healthy  individual  traveling     from  the  US  to  Bangladesh    
17. Healthy  individual  traveling     from  the  US  to  Bangladesh     Relacves  of  Crohn's  disease   pacents  
18. Healthy  individual  traveling     from  the  US  to  Bangladesh     Relacves  of  Crohn's  disease   pacents   Pacents  
19. …  so  what  can  we  tell  from  all  this   work?   Healthy  individual  traveling     from  the  US  to  Bangladesh     Relacves  of  Crohn's  disease   pacents   Pacents   USA   Global  gut  
20. Healthy  individual  traveling     from  the  US  to  Bangladesh     Relacves  of  Crohn's  disease   pacents   Pacents   USA   Venezuela   Malawi   Global  gut  
21. Healthy  individual  traveling     from  the  US  to  Bangladesh     Relacves  of  Crohn's  disease   pacents   Pacents   USA   Venezuela   Malawi   Global  gut   HMP  
22. …  so  what  can  we  tell  from  all  this   work?   Healthy  individual  traveling     from  the  US  to  Bangladesh     Relacves  of  Crohn's  disease   pacents   Pacents   USA   Venezuela   Malawi   Global  gut   HMP  
23. …  so  what  can  we  tell  from  all  this   work?   Healthy  individual  traveling     from  the  US  to  Bangladesh     Relacves  of  Crohn's  disease   pacents   Pacents   USA   Venezuela   Malawi   Global  gut   HMP  
24. hAp://qiime.org   hAp://forum.qiime.org   hAp://blog.qiime.org  
25. Graphical  User  Interface  
26. Command  line  
27. Perform  idenccal  operacons  
28. Paths  (absolute)   /Users/yoshiki/evident-data/hmp-v13_arare/alpha_div $HOME/evident-data/hmp-v13_arare/alpha_div ~/evident-data/hmp-v13/alpha_div A  slash  at  the  beginning  of  a  path  denotes  it  as  an  absolute  path,  i.  e.  from  the  base  of  your   hard  drive.  
29. Paths  (relacve)   evident-data/hmp-v13_arare/alpha_div On  the  other  side  relacve  paths  are  not  preceeded  by  a  slash  
30. QIIME  
31. QIIME  Structure   •  Integrates  other  somware   •  Set  of  scripts  to  perform  certain  funccons   •  Allows  an  easy  workflow   •  Keys,  wallet,  phone:   print_qiime_config.py  
32. QIIME  somware  dependencies   [data-­‐lanemask]   [data-­‐core]   [python]   [setuptools]   [MySQL-­‐python]   [SQLAlchemy]   [pycogent]   [pynast]   [numpy]   [matplotlib]   [mpi4py]   [lxml]   [sphinx]   [raxml]   [fasFree]   [cdbtools]   [chimeraslayer]   [cdhit]   [rdpclassifier]   [blast]   [muscle]   [infernal]   [cytoscape]   [clearcut]   [mothur]   [uclust]   [r]   [ampliconnoise]   [vienna]   [pprospector]  
33. Script  types   Single  Task    One  step    Most  of  them     Workflows    MulGple  scripts  in  one    Uses  a  log  file    Indicated  in  the  script  descripcon  
34. QIIME  commands   Get  help  with  index  site   hAp://qiime.org/genindex.html   Get  help  with  the  -­‐h  opcon    pick_otus.py -h Command  names  are  self-­‐explanatory   Filtering   filter_fasta.py filter_otus_by_sample.py filter_distance_matrix.py Sorcng   sort_otu_table.py
35. Ge]ng  help   hAp://qiime.org/genindex.html    
36. These  opGons  are  required,  else  the  script  will   not  funcGon  correctly   These  arguments   are  opGonal,  you   can  either  use   them  or  not,  some   default  values  are   explained  here.  
37. QIIME   •  The  code  is  tested  (properly)   •  The  documentacon  is  updated  constantly   based  on  users  suggescons   •  The  help  in  the  QIIME-­‐forum  has  a   collaboracve  spirit  (developers  &  users   sharing  their  research  experiences)  
38. print_qiime_config.py  
39. QIIME  
40. Sequencing output (454, Illumina, Sanger) fastq, fasta, qual, or sff/trace files Metadata mapping file Pre-processing e.g., remove primer(s), demultiplex, quality filter Denoise 454 Data PyroNoise, Denoiser Reference based BLAST, UCLUST, USEARCH Pick OTUs and representative sequences De novo e.g., UCLUST, CD-HIT, MOTHUR, USEARCH Assign taxonomy BLAST, RDP Classifier Align sequences e.g., PyNAST, INFERNAL, MUSCLE, MAFFT Build 'OTU table' i.e., sample by observation matrix Build phylogenetic tree e.g., FastTree, RAxML, ClearCut Database Submission (In development) OTU (or other sample by observation) table Phylogenetic Tree Evolutionary relationship between OTUs α-diversity and rarefaction e.g., Phylogenetic Diversity, Chao1, Observed Species β-diversity and rarefaction e.g., Weighted and unweighted UniFrac, Bray- Curtis, Jaccard Interactive visualizations e.g., PCoA plots, distance histograms, taxonomy charts, rarefaction plots, network visualization, jackknifed hierarchical clustering. Legend Required step or input Optional step or input Currently supported for marker-gene data only (i.e., 'upstream' step) Currently supported for general sample by observation data (i.e., 'downstream' step) www.QIIME.org Upstream  analyses     Downstream  analyses    
41. Sequencing output (454, Illumina, Sanger) fastq, fasta, qual, or sff/trace files Metadata mapping file Pre-processing e.g., remove primer(s), demultiplex, quality filter Denoise 454 Data PyroNoise, Denoiser Reference based BLAST, UCLUST, USEARCH Pick OTUs and representative sequences De novo e.g., UCLUST, CD-HIT, MOTHUR, USEARCH Assign taxonomy BLAST, RDP Classifier Align sequences e.g., PyNAST, INFERNAL, MUSCLE, MAFFT Build 'OTU table' i.e., sample by observation matrix Build phylogenetic tree e.g., FastTree, RAxML, ClearCut Database Submission (In development) OTU (or other sample by observation) table Phylogenetic Tree Evolutionary relationship between OTUs α-diversity and rarefaction e.g., Phylogenetic Diversity, Chao1, Observed Species β-diversity and rarefaction e.g., Weighted and unweighted UniFrac, Bray- Curtis, Jaccard Interactive visualizations e.g., PCoA plots, distance histograms, taxonomy charts, rarefaction plots, network visualization, jackknifed hierarchical clustering. Legend Required step or input Optional step or input Currently supported for marker-gene data only (i.e., 'upstream' step) Currently supported for general sample by observation data (i.e., 'downstream' step) www.QIIME.org QC  and  split  libraries  
42. Sequencing output (454, Illumina, Sanger) fastq, fasta, qual, or sff/trace files Metadata mapping file Pre-processing e.g., remove primer(s), demultiplex, quality filter Denoise 454 Data PyroNoise, Denoiser Reference based BLAST, UCLUST, USEARCH Pick OTUs and representative sequences De novo e.g., UCLUST, CD-HIT, MOTHUR, USEARCH Assign taxonomy BLAST, RDP Classifier Align sequences e.g., PyNAST, INFERNAL, MUSCLE, MAFFT Build 'OTU table' i.e., sample by observation matrix Build phylogenetic tree e.g., FastTree, RAxML, ClearCut Database Submission (In development) OTU (or other sample by observation) table Phylogenetic Tree Evolutionary relationship between OTUs α-diversity and rarefaction e.g., Phylogenetic Diversity, Chao1, Observed Species β-diversity and rarefaction e.g., Weighted and unweighted UniFrac, Bray- Curtis, Jaccard Interactive visualizations e.g., PCoA plots, distance histograms, taxonomy charts, rarefaction plots, network visualization, jackknifed hierarchical clustering. Legend Required step or input Optional step or input Currently supported for marker-gene data only (i.e., 'upstream' step) Currently supported for general sample by observation data (i.e., 'downstream' step) www.QIIME.org Building  an  OTU  table  
43. Alpha  and  Beta  diversity   Sequencing output (454, Illumina, Sanger) fastq, fasta, qual, or sff/trace files Metadata mapping file Pre-processing e.g., remove primer(s), demultiplex, quality filter Denoise 454 Data PyroNoise, Denoiser Reference based BLAST, UCLUST, USEARCH Pick OTUs and representative sequences De novo e.g., UCLUST, CD-HIT, MOTHUR, USEARCH Assign taxonomy BLAST, RDP Classifier Align sequences e.g., PyNAST, INFERNAL, MUSCLE, MAFFT Build 'OTU table' i.e., sample by observation matrix Build phylogenetic tree e.g., FastTree, RAxML, ClearCut Database Submission (In development) OTU (or other sample by observation) table Phylogenetic Tree Evolutionary relationship between OTUs α-diversity and rarefaction e.g., Phylogenetic Diversity, Chao1, Observed Species β-diversity and rarefaction e.g., Weighted and unweighted UniFrac, Bray- Curtis, Jaccard Interactive visualizations e.g., PCoA plots, distance histograms, taxonomy charts, rarefaction plots, network visualization, jackknifed hierarchical clustering. Legend Required step or input Optional step or input Currently supported for marker-gene data only (i.e., 'upstream' step) Currently supported for general sample by observation data (i.e., 'downstream' step) www.QIIME.org
44. Sequencing output (454, Illumina, Sanger) fastq, fasta, qual, or sff/trace files Metadata mapping file Pre-processing e.g., remove primer(s), demultiplex, quality filter Denoise 454 Data PyroNoise, Denoiser Reference based BLAST, UCLUST, USEARCH Pick OTUs and representative sequences De novo e.g., UCLUST, CD-HIT, MOTHUR, USEARCH Assign taxonomy BLAST, RDP Classifier Align sequences e.g., PyNAST, INFERNAL, MUSCLE, MAFFT Build 'OTU table' i.e., sample by observation matrix Build phylogenetic tree e.g., FastTree, RAxML, ClearCut Database Submission (In development) OTU (or other sample by observation) table Phylogenetic Tree Evolutionary relationship between OTUs α-diversity and rarefaction e.g., Phylogenetic Diversity, Chao1, Observed Species β-diversity and rarefaction e.g., Weighted and unweighted UniFrac, Bray- Curtis, Jaccard Interactive visualizations e.g., PCoA plots, distance histograms, taxonomy charts, rarefaction plots, network visualization, jackknifed hierarchical clustering. Legend Required step or input Optional step or input Currently supported for marker-gene data only (i.e., 'upstream' step) Currently supported for general sample by observation data (i.e., 'downstream' step) www.QIIME.org Visualizacons  
45. Sequencing output (454, Illumina, Sanger) fastq, fasta, qual, or sff/trace files Metadata mapping file Pre-processing e.g., remove primer(s), demultiplex, quality filter Denoise 454 Data PyroNoise, Denoiser Reference based BLAST, UCLUST, USEARCH Pick OTUs and representative sequences De novo e.g., UCLUST, CD-HIT, MOTHUR, USEARCH Assign taxonomy BLAST, RDP Classifier Align sequences e.g., PyNAST, INFERNAL, MUSCLE, MAFFT Build 'OTU table' i.e., sample by observation matrix Build phylogenetic tree e.g., FastTree, RAxML, ClearCut Database Submission (In development) OTU (or other sample by observation) table Phylogenetic Tree Evolutionary relationship between OTUs α-diversity and rarefaction e.g., Phylogenetic Diversity, Chao1, Observed Species β-diversity and rarefaction e.g., Weighted and unweighted UniFrac, Bray- Curtis, Jaccard Interactive visualizations e.g., PCoA plots, distance histograms, taxonomy charts, rarefaction plots, network visualization, jackknifed hierarchical clustering. Legend Required step or input Optional step or input Currently supported for marker-gene data only (i.e., 'upstream' step) Currently supported for general sample by observation data (i.e., 'downstream' step) www.QIIME.org QC  and  split  libraries  
46. Data   Sequences  are  in  FASTA  format  
47. Data   •  Quality  scores  are  in  the  .qual  file,  similar  to   FASTA  
48. Metadata  (mapping  file)  
49. validate_mapping_file.py  
50. Split  libraries   • Demulcplex     • Quality  trim   • Quality  filter   split_libraries.py   hAp://qiime.org/scripts/split_libraries.html   Output  files:   seqs.fna  –  demulcplexed  sequences   histograms.txt  –  histogram  of  read  lengths   split_library_log.txt  –  detailed  informacon  about  the   demulcplexing  and  quality  of  reads  
51. Error-­‐correccng  codes  allow  mulcplex  sequencing   Micah  Hamady,  et  al.,  Nature  Methods,  2008.   Error-­‐correccng  barcodes  for  pyrosequencing  hundreds  of  samples  in  mulcplex.   >GCACCTGAGGACAGGCATGAGGAA…   >GCACCTGAGGACAGGGGAGGAGGA…   >TCACATGAACCTAGGCAGGACGAA…   >CTACCGGAGGACAGGCATGAGGAT…   >TCACATGAACCTAGGCAGGAGGAA…   >GCACCTGAGGACACGCAGGACGAC…   >CTACCGGAGGACAGGCAGGAGGAA…   >CTACCGGAGGACACACAGGAGGAA…   >GAACCTTCACATAGGCAGGAGGAT…   >TCACATGAACCTAGGGGCAAGGAA…   >GCACCTGAGGACAGGCAGGAGGAA…   >PC.634_1 FLP3FBN01ELBSX CTGGGCCGTGTCTCAGTCCCAATGTGGCCGTTTACCCTCTCAGGCCGGCTAC GCATCATCGCCTTGGTGGGCCGTTACCTCACCAACTAGCTAATGCGCCGCAG GTCCATCCATGTTCACGCCTTGATGGGCGCTTTAATATACTGAGCATGCGCT CTGTATACCTATCCGGTTTTAGCTACCGTTTCCAGCAGTTATCCCGGACACA TGGGCTAGG! >PC.354_3 FLP3FBN01EEWKD ! TTGGACCGTGTCTCAGTTCCAATGTGGGGGCCTTCCTCTCAGAACCCCTATC CATCGAAGGCTTGGTGGGCCGTTACCCCGCCAACAACCTAATGGAACGCATC CCCATCGATGACCGAAGTTCTTTAATAGTTCTACCATGCGGAAGAACTATGC CATCGGGTATTAATCTTTCTTTCGAAAGGCTATCCCCGAGTCATCGGCAGGT TGGATACGTGTTACTCACCCGTGCGCCGGT!
52. split_libraries.py   •  seqs.fna  –  demulcplexed  sequences    
53. Sequencing output (454, Illumina, Sanger) fastq, fasta, qual, or sff/trace files Metadata mapping file Pre-processing e.g., remove primer(s), demultiplex, quality filter Denoise 454 Data PyroNoise, Denoiser Reference based BLAST, UCLUST, USEARCH Pick OTUs and representative sequences De novo e.g., UCLUST, CD-HIT, MOTHUR, USEARCH Assign taxonomy BLAST, RDP Classifier Align sequences e.g., PyNAST, INFERNAL, MUSCLE, MAFFT Build 'OTU table' i.e., sample by observation matrix Build phylogenetic tree e.g., FastTree, RAxML, ClearCut Database Submission (In development) OTU (or other sample by observation) table Phylogenetic Tree Evolutionary relationship between OTUs α-diversity and rarefaction e.g., Phylogenetic Diversity, Chao1, Observed Species β-diversity and rarefaction e.g., Weighted and unweighted UniFrac, Bray- Curtis, Jaccard Interactive visualizations e.g., PCoA plots, distance histograms, taxonomy charts, rarefaction plots, network visualization, jackknifed hierarchical clustering. Legend Required step or input Optional step or input Currently supported for marker-gene data only (i.e., 'upstream' step) Currently supported for general sample by observation data (i.e., 'downstream' step) www.QIIME.org Building  an  OTU  table  
54. OTU  Picking  -­‐  “de-­‐novo”   •  Pros   –  Vast  majority  of  reads  are  clustered     –  No  reference  database  bias   •  Cons   –  Speed;  not  easily  parallelizable     –  Erroneous  reads  get  clustered   CTGGGCCGTGTCTCAGTCCCAA TTGGAAGATGTCTCAGTTCCAG TTGGGCCGTATGTCAGTCCCTA CTGGGCCGTGTCTCAGTCCCAA TTGGAAGATGTCTCAGTTCCAG TTGGGCCGTATGTCAGTCCCTA   Clustered Sequences OTUS OTU1 OTU2 OTU3 Clustering Algorithm CTGGGCCGTGTCTCAGTCCCAAACA TTGGAAGATGTCTCAGTTCCAGACA CTGGGCCGTGTCTCAGTCCCAAACA TTGGAAGATGTCTCAGTTCCAGACA CTGGGCCGTGTCTCAGTCCCAAACA TTGGAAGATGTCTCAGTTCCAGACA Experimental Sequences
55. OTU  Picking  -­‐  “closed-­‐reference”   •  Pros   –  Reference  database  is  a  quality  filter   –  Speed;  easily  parallelizable   •  Cons   –  No  new  OTUs  can  be  observed   –  Reference  database  bias   CTGGGCCGTGTCTCAGTCCCAA CTGGGCCGTGTCTCAGTCCCAA TTGGAAGATGTCTCAGTTCCAG TTGGGCCGTATGTCAGTCCCTA CTGGGCCGTGTCTCAGTCCCAA TTGGAAGATGTCTCAGTTCCAG TTGGGCCGTATGTCAGTCCCTA   Experimental Sequences Reference  Sequences CTGGGCCGTGTCTCAGTCCCAA CTGGGCCGTGTCTCAGTCCCAA TTGGAAGATGTCTCAGTTCCAG CTGGGCCGTGTCTCAGTCCCAA TTGGAAGATGTCTCAGTTCCAG CTGGGCCGTGTCTCAGTCCCAA TTGGAAGATGTCTCAGTTCCAG Sequences that hit a reference CTGGGCCGTGTCTCAGTCCCAA Sequences that failed to hit CTGGGCCGTGTCTCAGTCCCAA TTGGAAGATGTCTCAGTTCCAG TTGGGCCGTATGTCAGTCCCTA CTGGGCCGTGTCTCAGTCCCAA TTGGAAGATGTCTCAGTTCCAG TTGGGCCGTATGTCAGTCCCTA   OTUS OTU1 OTU1 OTU1
56. Reference  database  
57. Percentage  of  reads   that  do  not  hit  the   reference  colleccon,   by  environment  type.  
58. Other  databases   •  hAp://www.arb-­‐silva.de   hAp://qiime.org/home_stacc/dataFiles.html   •  hAp://ssu-­‐rrna.org  
59. OTU  Picking  -­‐  “open-­‐reference”   •  Pros   –  Best  of  both  worlds   •  Cons   –  Downsides  of  de-­‐novo   CTGGGCCGTGTCTCAGTCCCAA CTGGGCCGTGTCTCAGTCCCAA TTGGAAGATGTCTCAGTTCCAG TTGGGCCGTATGTCAGTCCCTA CTGGGCCGTGTCTCAGTCCCAA TTGGAAGATGTCTCAGTTCCAG TTGGGCCGTATGTCAGTCCCTA   Experimental Sequences Reference   Sequences CTGGGCCGTGTCTCAGTCCCAA CTGGGCCGTGTCTCAGTCCCAA TTGGAAGATGTCTCAGTTCCAG CTGGGCCGTGTCTCAGTCCCAA TTGGAAGATGTCTCAGTTCCAG CTGGGCCGTGTCTCAGTCCCAA TTGGAAGATGTCTCAGTTCCAG Sequences that hit a reference CTGGGCCGTGTCTCAGTCCCAA Sequences that failed to hit CTGGGCCGTGTCTCAGTCCCAA TTGGAAGATGTCTCAGTTCCAG TTGGGCCGTATGTCAGTCCCTA CTGGGCCGTGTCTCAGTCCCAA TTGGAAGATGTCTCAGTTCCAG TTGGGCCGTATGTCAGTCCCTA   OTUS OTU1 OTU2 OTU3 OTU4 OTU5 OTU6 Clustering Algorithm
60. pick_open_reference_otus.py   •  hAp://qiime.org/scripts/pick_open  _reference_otus.html   •  Workflow  script,  performs  all  steps  through  building  an  OTU   table  (see  the  log  file)   –  pick_otus.py:  determine  the  OTU  clusters   –  pick_rep_set.py:  pick  the  representacve  sequence  for  each  OTU  cluster   –  align_seqs.py:  align  the  sequences  to  a  template  or  other  reference  alignment   –  assign_taxonomy.py:  allot  a  taxonomy  to  the  representacve  sequences   –  filter_alignment.py:  remove  non-­‐phylogeneccally  informacve  posicons   –  make_phylogeny.py:  construct  a  phylogeny  from  an  alignment   –  make_otu_table.py:  constructs  the  actual  OTU  table  object  
61. QIIME parameters •  hAp://qiime.org/documentacon/ qiime_parameters_files.html   •  Modify  the  default  behavior  of  a  workflow  script.   •   Blank  lines  and  those  starcng  with  ‘#’  are  ignored   •   Format   –  script:parameter  value  
62. OTU  Table  in  BIOM  format   •  Opcmized  and  efficient  data  abstraccon   •  Can  be  used  with  many  types  of  data,  but  to   make  it  Excel  'readable’  use:  biom  convert  
63. biom  convert     •  hAp://biom-­‐format.org   •  Converts  the  BIOM  format  OTU  table  to  an  Excel   readable  format   •  biom  convert  –i  otu_table_mc2_w_tax.biom  –o   otu_table.txt  -­‐b  
64. OTU  table   sample  idencfiers  
65. Taxonomic  Assignment   •  Kingdom   •  Phylum   •  Class   •  Order   •  Family   •  Genus   •  Species   Sequence  16S  gene  and   compare  to  16S  database  with   taxonomic  assignments  
66. Taxonomic  Assignment  using     e.g.  Uclust     CTGGGCCGTGTCTCAGTCCCAA TTGGAAGATGTCTCAGTTCCAG TTGGGCCGTATGTCAGTCCCTA CTGGGCCGTGTCTCAGTCCCAA TTGGAAGATGTCTCAGTTCCAG TTGGGCCGTATGTCAGTCCCTA   Experimental Sequences Reference  Sequences CTGGGCCGTGTCTCAGTCCCAA TTGGAAGATGTCTCAGTTCCAG TTGGGCCGTATGTCAGTCCCTA CTGGGCCGTGTCTCAGTCCCAA TTGGAAGATGTCTCAGTTCCAG TTGGGCCGTATGTCAGTCCCTA  
67. Biom  summary   •  Basic  stacsccs  on  the  OTU  table   –  Num  samples,  OTUs,  sequences  in  OTUs   –  Sequences  per  sample   –  Useful  to  determine  values  to  use  in  downstream  analyses  
68. Alpha  and  Beta  diversity   Sequencing output (454, Illumina, Sanger) fastq, fasta, qual, or sff/trace files Metadata mapping file Pre-processing e.g., remove primer(s), demultiplex, quality filter Denoise 454 Data PyroNoise, Denoiser Reference based BLAST, UCLUST, USEARCH Pick OTUs and representative sequences De novo e.g., UCLUST, CD-HIT, MOTHUR, USEARCH Assign taxonomy BLAST, RDP Classifier Align sequences e.g., PyNAST, INFERNAL, MUSCLE, MAFFT Build 'OTU table' i.e., sample by observation matrix Build phylogenetic tree e.g., FastTree, RAxML, ClearCut Database Submission (In development) OTU (or other sample by observation) table Phylogenetic Tree Evolutionary relationship between OTUs α-diversity and rarefaction e.g., Phylogenetic Diversity, Chao1, Observed Species β-diversity and rarefaction e.g., Weighted and unweighted UniFrac, Bray- Curtis, Jaccard Interactive visualizations e.g., PCoA plots, distance histograms, taxonomy charts, rarefaction plots, network visualization, jackknifed hierarchical clustering. Legend Required step or input Optional step or input Currently supported for marker-gene data only (i.e., 'upstream' step) Currently supported for general sample by observation data (i.e., 'downstream' step) www.QIIME.org
69.  How  do  we  describe  and  compare   diversity?   •  α  Diversity:   –   “How  many  species  (taxa)  are  in  a  sample?”       •  e.g.  6  colors  in  A  and  6  in  B   •  Are  polluted  environments  less  diverse  than  priscne?   •  β  Diversity:   –  “How  many  species  are  shared  between  samples?”   •  e.g.  2  shared  colors  between  A  and  B   •  Do  the  microbiota  differ  among  different  disease   states?     A   B  
70. Qualitacve  vs.  Quanctacve  measures   •  Qualitacve:  Considers  presence/absence   –  α:  How  many  species  are  in  a  sample?   •  e.g.:  6  species  (colors)  in  both  A  and  B.   –  β: How  many  species  are  shared  between  samples?   •  e.g.:  A  and  B  are  idenccal  because  the  same  colors  are  present  in   both.     •  Quanctacve:  Considers  abundance   –  α:  Accounts  for  distribucon:   •  e.g.  in  B,  6  species  are  evenly  distributed  and  thus  the  co                                                         community  is  more  diverse  than  in  A  where  1                                                                                   species  dominates  over  other  5.   –  β: Samples  will  be  considered  more  similar  if  the                    same   distribucon  of  species  is  similar.   •  e.g.  B  and  A  no  longer  look  idenccal  because  of  differences  in   abundance.   A   B  
71.  What  is  a  phylogenecc  diversity   measure?   •  α  Diversity:   –  Taxon:  “How  many  species  are  in  a  sample?”       –  Phylogenecc:  “How  much  phylogenecc  divergence  is  in  a   sample?”     •  e.g.  B  more  diverse  than  A  -­‐  more  divergent  colors   •  β  Diversity:   –  Taxon:  “How  many  species  are  shared  between  samples?”   –  Phylogenecc:  “How  much  phylogenecc  distance  is  shared   between  samples?”   •  only  related  colors  from  B  are  in  A   A   B  
72. UniFrac  distance  matrix  
73. core_diversity_analyses.py •  Workflow  script   –  filter_samples_from_otu_table.py:  Filter  samples  with  low  sequence  count   from  table     –  single_rarefaccon.py:  sample  the  table  at  specified  sequencing  depth   –  beta_diversity.py:  use  the  sampled  table  for  beta  diversity  calculacon   –  principal_coordinates.py:  perform  PCoA  analysis   –  make_emperor.py:  make  plots  for  principal  coordinates   –  mulcple_rarefaccons.py:  make  mulcple  subsamplings/rarefaccons  on  an  otu   table  at  various  sequencing  depths   –  alpha_diversity.py  and  collate_alpha.py:  calculate  alpha  diversices  at  those   depths  and  collate  them   –  make_rarefaccon_plots.py:  plot  the  rarefaccon  curves   –  summarize_taxa.py  and  plot_taxa_summary.py:  summarize  taxa  and  plot   them  
74. Alpha  diversity   Basic  alpha  diversity  measure:   count  number  of  OTUs.     other  measures  can  be:   •  phylogenecc  (PD)   •  escmators  (chao1)   •  other  stacsccs  (evenness)   •  …    
75. Beta  diversity       orange1   orange2   blue1   OTU1   4   4   0   OTU2   4   4   0   OTU3   0   1   7   OTU4   0   0   7  
76. Summarize  Taxa   •  Calculates  proporcon  of  taxa  per  sample,  at   different  taxonomic  levels   •  summarize_taxa_through_plots.py  
78. Taxa  Summarized  by  Category    
79. Procrustes  Analysis   hAp://qiime.org/tutorials/procrustes_analysis.html   transform_coordinate_matrices.py   compare_3d_plots.py   Muegge,  B.  D.  et  al.  Science  332,  970–974  (2011).  
81. Stacsccally  Different?   •  group_significance.py   •  Parametric   –  G-­‐test   –  ANOVA   –  T-­‐test   •  Non  parametric   –  Kruskal-­‐Wallis   –  Mann-­‐Whitney-­‐U   –  Bootstrap  Mann-­‐Whitney-­‐U   –  Bootstrap  T-­‐test   •  compare_categories.py   •  make_distance_boxplots.py   •  …    
82. Acknowledgements   Rob  Knight   Antonio  Gonzalez   Meg  Pirrung   Adam  Robbins-­‐Pianka   Luke  Ursell   Tony  Walters   Doug  Wendel   Daniel  McDonald   Yoshiki  Vázquez  Baeza   Will  Van  Treuren   Laura  Wegener  Parfery   Kris  Mayer     Merete  Eggesbo   Jessica  Metcalf   Ulla  Westermann   Zhenjiang  Zech  Xu   Jose  Navas   Chris  Lauber   MaA  Gebert   Greg  C  Humphrey   Hongwei  Zhou   Rick  Stevens  (Argonne),  Jack  Gilbert  (Argonne),   Folker  Meyer  (Argonne),  Janet  Jansson  (LBNL),  Jed   Fuhrman  (USC),  Jonathan  Eisen  (UC  Davis),  many,   many  sample  donors.   Other  collaborators:   Noah  Fierer  (CU,  EEB),  Jeff  Gordon  (Wash  U),  Ruth   Ley  (Cornell),  Peter  Turnbaugh(Harvard),  Maria   Gloria  Dominguez  (UPR),  Catherine  Lozupone  (CU)   ...