Workflow mangers

DLSM Seminar @uni.lu
snakemake and targets

Transcript

1. Work ow managers Empower users with reproducible pipelines Aurélien Ginolhac | DLSM Seminars | 2021-03-03
2. example from Lasse and me course. initial work ow: QC runs separately and not in parallel trimming / mapping ok (paleomix) peak calling separately Bunch of serial manual computing steps at least, singularity was already used Bioinformatic analyses source: ChIP-seq doctoral-course (2018) 2/33
3. sh/bash/csh/zsh scripts We start from #!/bin/csh # Created by: Chandra Sekhar Pedamallu @ DFCI, The Broad Inst. # Date: June 2016 # Full PathSeq pipeline # time $pdir/FullPathSeq_June2016.sh $bamloc/SN218_Run0771_Lane # set start_time=`date +%s` @ noargs=$# #############################PLEASE SET THESE ENVIRONMENTAL VAR # Program Settings set Institute="BROAD" [...] if($noargs < 4) then echo "Please check your argunments" echo "Usage : ./FullPathSeq_xxxx.sh <Input file in BAM or F echo "Example : ./FullPathSeq_xxxx.sh unmappedreads.10K.fq1 exit endif ###################INPUT FILES################################# # Present Directory set pdir = `pwd` rm $pdir"/clean.files" rm -r $pdir"/Commands/" [...] FullPathSeq_June2016.sh 3/33
4. sh/bash/csh/zsh scripts Issues mix of code and parameters common actions are mingled (here: settings) software/input de ned as absolute paths comments are instructions software dependencies not included (csh!) on HPC, no admin rights any issue implies to start over version in lename (see the usage with xxxx) le management (here: rm recursive! 😳) requires many effort to port over We start from #!/bin/csh # Created by: Chandra Sekhar Pedamallu @ DFCI, The Broad Inst. # Date: June 2016 # Full PathSeq pipeline # time $pdir/FullPathSeq_June2016.sh $bamloc/SN218_Run0771_Lane # set start_time=`date +%s` @ noargs=$# #############################PLEASE SET THESE ENVIRONMENTAL VAR # Program Settings set Institute="BROAD" [...] if($noargs < 4) then echo "Please check your argunments" echo "Usage : ./FullPathSeq_xxxx.sh <Input file in BAM or F echo "Example : ./FullPathSeq_xxxx.sh unmappedreads.10K.fq1 exit endif ###################INPUT FILES################################# # Present Directory set pdir = `pwd` rm $pdir"/clean.files" rm -r $pdir"/Commands/" [...] FullPathSeq_June2016.sh 3/33
5. Wor ow managers (snakemake) What we want 4/33
6. Wor ow managers (snakemake) Solved issues run only what needs to be, stop micromanaging your analyses use relative paths clear instructions in a README code in one folder, users edit 3 text les software installed in singularity image makes seamless deployment on HPC 🥴 singularity images are versioned, code and reports too ongoing work in temporary folders multi-platform (Windows, MacOS, GNU/Linux) What we want 4/33
7. this is the work ow being used: not just a diagram Explicit dependencies Parallelization: independent branches, like fastqc and adapter_removal inside a step, speci ed number of cores to used RNA-seq work ow, steps From sequence reads to differential gene expression step dependencies from my snakemake template 5/33
8. RNA-seq work ow, samples From sequence reads to differential gene expression Directed Acyclic Graph from the snakemake template using Marios Gavriil yeast partial dataset 6/33
9. Example of rule to index the genome with STAR input de nes both upstream dependencies output de nes the produced le(s) container tells to use a speci c singularity image threads speci ed. Over total of available, decide what to run if 28 cores booked, 2 indexing can be started simultaneously params are fetch from config (adv. users) next rule align_multi will use this output for its input Rule example Remember, you don't need to edit this rule star_index: input: fasta = "refs/{}.fasta".format(config["ref"]["build"]), gtf = "refs/{}.gtf".format(config["ref"]["build"]) output: directory("refs/{}".format(config["ref"]["build"])) message: "STAR indexing {}".format(config["ref"]["build"]) container: "docker://ginolhac/snake-rna-seq:0.4" threads: 12 params: build = config["ref"]["build"], extra = "" log: "logs/star_index_{}.log".format(config["ref"]["build"]) wrapper: "0.65.0/bio/star/index" 7/33
10. 1. HPC account 2. clone repository from template 3. copy FASTQ les in fastq RNA-seq analysis from the user perspective 8/33
11. 1. HPC account 2. clone repository from template 3. copy FASTQ les in fastq Folder tree RNA-seq analysis from the user perspective . ├── CHANGELOG.md ├── config/ │ ├── config.yaml | FOR │ ├── samples.tsv | USERS │ └── units.tsv | EDIT ├── fastq/ │ ├── BY_A-2.fastq.gz | USERS │ ├── BY_A.fastq.gz | COPY │ ├── BY_B.fastq.gz | DATA │ ├── BY_Gis1_A.fastq.gz | │ └── BY_Gis1_C.fastq.gz | ├── LICENSE ├── README.md └── workflow/ ├── [...] <- Snakemake machinery there 8/33
12. le config/config.yaml From the user perspective 4. edit the 3 les in config trimming: skip: false # true to skip trimming threads: 2 adapter1: AGATCGGAAGAGCACACGTCTGAACTCCAGTCAC adapter2: AGATCGGAAGAGCGTCGTGTAGGGAAAGAGTGTAGATCTCGGTGGTCGCCG min_length: 35 ref: species: "saccharomyces_cerevisiae" build: "R64-1-1" release: "99" pca: labels: - condition # column of samples.tsv diffexp: contrasts: # contrasts for the deseq2 results method KO_vs_WT: - WT - KO 9/33
13. le config/config.yaml le config/samples.tsv le config/units.tsv Main issue is to use proper tabs and not spaces for .tsv From the user perspective 4. edit the 3 les in config trimming: skip: false # true to skip trimming threads: 2 adapter1: AGATCGGAAGAGCACACGTCTGAACTCCAGTCAC adapter2: AGATCGGAAGAGCGTCGTGTAGGGAAAGAGTGTAGATCTCGGTGGTCGCCG min_length: 35 ref: species: "saccharomyces_cerevisiae" build: "R64-1-1" release: "99" pca: labels: - condition # column of samples.tsv diffexp: contrasts: # contrasts for the deseq2 results method KO_vs_WT: - WT - KO sample condition WT_A WT WT_B WT KO_A KO KO_C KO sample unit fq1 fq2 strandedness WT_A 1 fastq/BY_A.fastq.gz reverse WT_A 2 fastq/BY_A-2.fastq.gz reverse WT_B 1 fastq/BY_B.fastq.gz reverse KO_A 1 fastq/BY_Gis1_A.fastq.gz reverse KO_C 1 fastq/BY_Gis1_C.fastq.gz reverse 9/33
14. Why you don't need/want to install any software? it's boring 🥱 On HPC one doesn't have admin rights modules prepared by the HPC teams are great, but more speci c software are missing docker is great but requires admin rights singularity is kind of docker for HPC 10/33
15. Why you don't need/want to install any software? it's boring 🥱 On HPC one doesn't have admin rights modules prepared by the HPC teams are great, but more speci c software are missing docker is great but requires admin rights singularity is kind of docker for HPC Of course, someone has to install software, it doesn't have to be you — Aurélien adapted from Jenny Bryan hosted at DockerHub and versioned. 10/33
16. From the user perspective 5. install snakemake in a conda environment 11/33
17. option -n for a dry run From the user perspective 5. install snakemake in a conda environment 6. run snakemake on a computing node (here with 12 cores) module load tools/Singularity conda activate snakemake snakemake --use-singularity \ --singularity-args "-B /scratch/users/aginolhac:/scratch/user -j 12 -n 11/33
18. option -n for a dry run From the user perspective 5. install snakemake in a conda environment 6. run snakemake on a computing node (here with 12 cores) module load tools/Singularity conda activate snakemake snakemake --use-singularity \ --singularity-args "-B /scratch/users/aginolhac:/scratch/user -j 12 -n Job counts: count jobs 5 adapter_removal_se 4 align_multi 1 all 1 deseq2 1 deseq2_init 5 fastq_screen 5 fastqc 1 feature_count 1 get_annotation 1 get_genome 1 multiqc 1 pca 1 session 1 star_index 29 This was a dry-run (flag -n). The order of jobs does not reflec 11/33
19. Demo on one rule: indexing the yeast genome 12/33
20. Directed Acyclic Graph, dashed = done Completed tasks are marked as such snakemake --dag | dot -Tpdf > dag.pdf 13/33
21. Directed Acyclic Graph, dashed = done Eventually, we obtain: a QC report a result report the matrix of counts ( object) Most analyses need more complex design than KO vs WT start from counts in can be run on a laptop now we still NEED a work ow manager for downstream analyses Completed tasks are marked as such snakemake --dag | dot -Tpdf > dag.pdf 13/33
22. Features PhD project of Wendkouni Nadège Minoungou LncRNA focus 3 cell lines Hyp-IL6 treatment 6 time points paired-end ➡ 72samples ➡ 144 FASTQ les Experimental design Scaling up seamlessly source: Nadège Minoungou 14/33
23. Features PhD project of Wendkouni Nadège Minoungou LncRNA focus 3 cell lines Hyp-IL6 treatment 6 time points paired-end ➡ 72samples ➡ 144 FASTQ les Experimental design Scaling up seamlessly Without snakemake running such large experiments would be painful source: Nadège Minoungou 14/33
24. Computing After editing the 3 con g les and 37 hours of computing (28 cores) Figure is truncated... references were not included as Nadège wanted speci c ENCODE ones Walltime for initial submission was 24h timeout 😑. Had to resubmit only the same job launcher 😎 condition is simpli ed on purpose. DGE will be redone locally TODO: helpers for editing the config les. 72 * 2 lines by hand is error-prone (and tiring) 15/33
25. Computing After editing the 3 con g les and 37 hours of computing (28 cores) Figure is truncated... references were not included as Nadège wanted speci c ENCODE ones Walltime for initial submission was 24h timeout 😑. Had to resubmit only the same job launcher 😎 condition is simpli ed on purpose. DGE will be redone locally TODO: helpers for editing the config les. 72 * 2 lines by hand is error-prone (and tiring) today solution, using BASH ls fastq/*z | paste - - | \ awk 'BEGIN{OFS="\t";print "sample\tunit\tfq1\tfq2\tstrandedness"}{id=substr($1,index($1,"/")+1,index($1,"_")-7);print id,"1",$1,$2, " sed 's/\-/_/' > config/units.tsv ls fastq/*z | paste - - | \ awk 'BEGIN{OFS="\t";print "sample\tcondition"}{id=substr($1,index($1,"/")+1,index($1,"_")-7);print id,id}' | \ sed 's/[0-9]$//; s/\-/_/g' > config/samples.tsv 15/33
26. Con g les and ease of extension samples.tsv units.tsv sample condition Hep3B_I1 Hep3B_I Hep3B_I2 Hep3B_I Hep3B_I3 Hep3B_I [...] (66 lines) HuH7_IV4 HuH7_IV HuH7_IV5 HuH7_IV HuH7_IV6 HuH7_IV sample unit fq1 fq2 strandedness Hep3B_I1 1 fastq/Hep3B-I1_Iris_S1_R1_001.fastq.gz fastq/Hep3B-I1_Iris_S1_R2_001.fastq.gz reverse Hep3B_I2 1 fastq/Hep3B-I2_Iris_S2_R1_001.fastq.gz fastq/Hep3B-I2_Iris_S2_R2_001.fastq.gz reverse Hep3B_I3 1 fastq/Hep3B-I3_Iris_S3_R1_001.fastq.gz fastq/Hep3B-I3_Iris_S3_R2_001.fastq.gz reverse [...] (66 lines) HuH7_IV4 1 fastq/HuH7-IV4_Iris_S10_R1_001.fastq.gz fastq/HuH7-IV4_Iris_S10_R2_001.fastq.gz reverse HuH7_IV5 1 fastq/HuH7-IV5_Iris_S11_R1_001.fastq.gz fastq/HuH7-IV5_Iris_S11_R2_001.fastq.gz reverse HuH7_IV6 1 fastq/HuH7-IV6_Iris_S12_R1_001.fastq.gz fastq/HuH7-IV6_Iris_S12_R2_001.fastq.gz reverse 16/33
27. Con g les and ease of extension samples.tsv units.tsv Nadège asked me to summarized counts at the transcript level change one option in featureCounts rule re-run the whole pipeline change in rule is detected re-run one counting (on transcripts) downstream steps (DESeq2) Results: 2 levels of counting are available Now go to sample condition Hep3B_I1 Hep3B_I Hep3B_I2 Hep3B_I Hep3B_I3 Hep3B_I [...] (66 lines) HuH7_IV4 HuH7_IV HuH7_IV5 HuH7_IV HuH7_IV6 HuH7_IV sample unit fq1 fq2 strandedness Hep3B_I1 1 fastq/Hep3B-I1_Iris_S1_R1_001.fastq.gz fastq/Hep3B-I1_Iris_S1_R2_001.fastq.gz reverse Hep3B_I2 1 fastq/Hep3B-I2_Iris_S2_R1_001.fastq.gz fastq/Hep3B-I2_Iris_S2_R2_001.fastq.gz reverse Hep3B_I3 1 fastq/Hep3B-I3_Iris_S3_R1_001.fastq.gz fastq/Hep3B-I3_Iris_S3_R2_001.fastq.gz reverse [...] (66 lines) HuH7_IV4 1 fastq/HuH7-IV4_Iris_S10_R1_001.fastq.gz fastq/HuH7-IV4_Iris_S10_R2_001.fastq.gz reverse HuH7_IV5 1 fastq/HuH7-IV5_Iris_S11_R1_001.fastq.gz fastq/HuH7-IV5_Iris_S11_R2_001.fastq.gz reverse HuH7_IV6 1 fastq/HuH7-IV6_Iris_S12_R1_001.fastq.gz fastq/HuH7-IV6_Iris_S12_R2_001.fastq.gz reverse . ├── CHANGELOG.md ├── config │ ├── config.yaml │ ├── samples.tsv │ └── units.tsv ├── counts │ ├── fc.rds │ ├── fc_stats.summary │ ├── fc_tr.rds │ └── fc_tr_stats.summary [...] 16/33
28. targets Same ideas as in snakemake but for R (actually more elegant) 17/33
29. targets and companion package tarchetypes A work ow manager for R Saving you time and stress Understand how it is implemented in targets de ne your targets connect targets to create the dependencies check dependencies with visnetwork embrace dynamic branching run only what needs to be executed bundle dependencies in a Rmarkdown document with tar_render() increase reproducibility with the package manager renv Example with RNA-seq data from Wendkouni Nadège MINOUNGOU 18/33
30. with renv. Snapshot your package environment (and restore! 😌) _targets.R is the only mandatory le use a R sub-folder for functions, gets closer to a package Rmarkdown le allows to gather results in a report in a RStudio project version tracked with Folder structure ├── .git/ ├── run.R ├── _targets.R ├── _targets/ ├── Repro.Rproj ├── R │ ├── functions.R │ └── utils.R ├── run.R* ├── renv/ ├── renv.lock └── report.Rmd 19/33
31. First aim Split counts and metadata from the big DESeq2 object As difference between cells >> biological effect PCA from snakemake pipeline 20/33
32. _targets.R, de ne 4 targets Last target depends on the 3 upstreams LncH6 project, starting from snakemake results library(targets) source("R/functions.R") source("R/plotting.R") list( tar_target(cells, c("HepG2", "HuH7", "Hep3B")), tar_qs(dds, read_rds(here::here("data", "all.rds")), packages = "DESeq2"), tar_fst_tbl(annotation, gtf_to_tbl(here::here("data", "gencode.v36.an packages = c("tibble", "rtracklayer")), tar_qs(sub_dds, subset_dds(dds, filter(annotation, type == "gene") .cell = cells), pattern = map(cells), # dynamic branching packages = c("DESeq2", "tidyverse")) [...] ) 21/33
33. _targets.R, de ne 4 targets Last target depends on the 3 upstreams Dynamic branching makes dependencies easier to read. Of course, someone has to write for loops, it doesn't have to be you — Jenny Bryan LncH6 project, starting from snakemake results library(targets) source("R/functions.R") source("R/plotting.R") list( tar_target(cells, c("HepG2", "HuH7", "Hep3B")), tar_qs(dds, read_rds(here::here("data", "all.rds")), packages = "DESeq2"), tar_fst_tbl(annotation, gtf_to_tbl(here::here("data", "gencode.v36.an packages = c("tibble", "rtracklayer")), tar_qs(sub_dds, subset_dds(dds, filter(annotation, type == "gene") .cell = cells), pattern = map(cells), # dynamic branching packages = c("DESeq2", "tidyverse")) [...] ) Figure from tar_visnetwork() 21/33
34. Running targets • run target annotation • run target cells • run target dds • run branch sub_dds_3078b1e0 condition time_h HepG2_I1 control 0 HepG2_I2 HIL6 2 using pre-existing size factors estimating dispersions gene-wise dispersion estimates: 2 workers mean-dispersion relationship final dispersion estimates, fitting model and testing: 2 worker • run branch sub_dds_d05c5da7 condition time_h HuH7_I1 control 0 HuH7_I2 HIL6 2 using pre-existing size factors estimating dispersions gene-wise dispersion estimates: 2 workers mean-dispersion relationship final dispersion estimates, fitting model and testing: 2 worker • run branch sub_dds_c60d7096 condition time_h Hep3B_I1 control 0 Hep3B_I2 HIL6 2 using pre-existing size factors estimating dispersions gene-wise dispersion estimates: 2 workers mean-dispersion relationship final dispersion estimates, fitting model and testing: 2 worker • end pipeline 22/33
35. Options to display time and object sizes Running targets • run target annotation • run target cells • run target dds • run branch sub_dds_3078b1e0 condition time_h HepG2_I1 control 0 HepG2_I2 HIL6 2 using pre-existing size factors estimating dispersions gene-wise dispersion estimates: 2 workers mean-dispersion relationship final dispersion estimates, fitting model and testing: 2 worker • run branch sub_dds_d05c5da7 condition time_h HuH7_I1 control 0 HuH7_I2 HIL6 2 using pre-existing size factors estimating dispersions gene-wise dispersion estimates: 2 workers mean-dispersion relationship final dispersion estimates, fitting model and testing: 2 worker • run branch sub_dds_c60d7096 condition time_h Hep3B_I1 control 0 Hep3B_I2 HIL6 2 using pre-existing size factors estimating dispersions gene-wise dispersion estimates: 2 workers mean-dispersion relationship final dispersion estimates, fitting model and testing: 2 worker • end pipeline 22/33
36. All good, nothing to be done ✔. Actually targets tracks all objects and so functions A more complete dependency graph shows functions Re-running ✓ skip target annotation ✓ skip target cells ✓ skip target dds ✓ skip branch sub_dds_3078b1e0 ✓ skip branch sub_dds_d05c5da7 ✓ skip branch sub_dds_c60d7096 ✓ skip pipeline 23/33
37. All good, nothing to be done ✔. Actually targets tracks all objects and so functions A more complete dependency graph shows functions Let's add the PCA per cell type now Re-running ✓ skip target annotation ✓ skip target cells ✓ skip target dds ✓ skip branch sub_dds_3078b1e0 ✓ skip branch sub_dds_d05c5da7 ✓ skip branch sub_dds_c60d7096 ✓ skip pipeline 23/33
38. smaller targets avoid unnecessary re-running steps Translate into: for every cell data, compute regularized counts (vst: variance stabilization) for every regularized counts, compute PCA (df: data.frame, i. e a table) for every cell names / PCA tables, plot PCA in a table for easier labeling PCA, add 4 targets [...] tar_target(rcounts, vst(sub_dds, blind = TRUE), pattern = map(sub_dds), packages = c("DESeq2")), tar_target(pca_df, pca_meta(rcounts), pattern = map(rcounts), packages = c("DESeq2", "tidyr", "dplyr")), tar_target(pca_cell, tibble(cell = cells, pca = list(plot_pca_meta(pca_df))), pattern = map(cells, pca_df), packages = c("ggplot2", "tibble")) [...] 24/33
39. smaller targets avoid unnecessary re-running steps Translate into: for every cell data, compute regularized counts (vst: variance stabilization) for every regularized counts, compute PCA (df: data.frame, i. e a table) for every cell names / PCA tables, plot PCA in a table for easier labeling PCA, add 4 targets [...] tar_target(rcounts, vst(sub_dds, blind = TRUE), pattern = map(sub_dds), packages = c("DESeq2")), tar_target(pca_df, pca_meta(rcounts), pattern = map(rcounts), packages = c("DESeq2", "tidyr", "dplyr")), tar_target(pca_cell, tibble(cell = cells, pca = list(plot_pca_meta(pca_df))), pattern = map(cells, pca_df), packages = c("ggplot2", "tibble")) [...] 24/33
40. PCA results Running Awesome feature: load results IN a Rmarkdown document separate code from content How to display a plot 25/33
41. The full picture Adding step by step desired analyses Whole analysis takes 24 minutes and 4.54 seconds Of course, someone has to remember the dependencies, it doesn't have to be you — could be William Landau via Jenny Bryan 26/33
42. How it sounds 27/33
43. How it sounds 27/33
44. The bad news is that when ever you learn a new skill you’re going to suck. It’s going to be frustrating. The good news is that is typical and happens to everyone and it is only temporary. You can’t go from knowing nothing to becoming an expert without going through a period of great frustration and great suckiness. — Hadley Wickham How it sounds source: Hadley Wickham dplyr tutorial at useR 2014 27/33
45. Is it worth the effort? 28/33
46. Yes For you independence autonomy skills control your data For me free time con dence over results reproducibility fun 🥳 Is it worth the effort? 28/33
47. Yes For you independence autonomy skills control your data For me free time con dence over results reproducibility fun 🥳 Reproducibility snakemake (actually the Singularity image is providing those too) targets via git Is it worth the effort? STAR 2.7.4a samtools 1.10 FastQC v0.11.9 FastQ Screen v0.14.0 AdapterRemoval ver. 2.3.1 R version 4.0.0 (2020-04-24) Ubuntu 20.04 LTS package version date source ggplot2 3.3.2 2020-06-19 CRAN (R 4.0.0) DESeq2 1.28.1 2020-05-12 Bioconductor apeglm 1.10.0 2020-04-27 Bioconductor Rsubread 2.2.2 2020-05-15 Bioconductor > renv::history() commit author_date committer_date subject 1e8dd2278 2021-02-23 15:29:57 2021-02-23 15:29:57 reformat creating config files 24c1222db 2021-02-15 17:07:01 2021-02-15 17:07:01 highlight gene type in the DEG patterns 326c8a726 2021-02-04 16:16:38 2021-02-04 16:16:38 cluster LRT genes by they dynamic patterns 4c6791796 2021-01-26 13:08:15 2021-01-26 13:08:15 gene types in upset plots for lengths 5865ee70b 2021-01-21 16:36:48 2021-01-21 16:37:08 add upset plots 6c06b496e 2021-01-20 17:03:29 2021-01-20 17:03:29 add upset protoype [...] 28/33
48. renv parses your code and nds library calls install a pckage including its dependencies snapshot registers changes, hashes and origin restore to a certain point in time (snakemake also using with the image tag) renv features > renv::snapshot() The following package(s) will be updated in the lockfile: # CRAN =============================== - RcppParallel [5.0.2 -> 5.0.3] - cli [2.3.0 -> 2.3.1] - pkgload [1.1.0 -> 1.2.0] - tint [0.1.3 -> *] # GitHub ============================= - targets [ropensci/targets@main: 598d7a23 -> bdc1b29c] Do you want to proceed? [y/N]: 29/33
49. renv parses your code and nds library calls install a pckage including its dependencies snapshot registers changes, hashes and origin restore to a certain point in time (snakemake also using with the image tag) renv.lock le after a snapshot renv features > renv::snapshot() The following package(s) will be updated in the lockfile: # CRAN =============================== - RcppParallel [5.0.2 -> 5.0.3] - cli [2.3.0 -> 2.3.1] - pkgload [1.1.0 -> 1.2.0] - tint [0.1.3 -> *] # GitHub ============================= - targets [ropensci/targets@main: 598d7a23 -> bdc1b29c] Do you want to proceed? [y/N]: "R": { "Version": "4.0.3", "Repositories": [ { "Name": "CRAN", "URL": "https://cloud.r-project.org" } ] }, "Bioconductor": { "Version": "3.12" }, "Packages": { "AnnotationDbi": { "Package": "AnnotationDbi", "Version": "1.52.0", "Source": "Bioconductor", "Hash": "ca5106b296b3aa6af713ce197be547c1" }, "BH": { "Package": "BH", "Version": "1.75.0-0", "Source": "Repository", "Repository": "CRAN", "Hash": "e4c04affc2cac20c8fec18385cd14691" }, "targets": { "Package": "targets", "Version": "0.1.0.9000", "Source": "GitHub", "RemoteType": "github", "RemoteUsername": "ropensci", 29/33
50. snakemake is great but report is not exible and works like a dry/black box. targets, written by William Landau (pictured), is exible, robust and still allows for a customized report. All computing is done only when needed, and code is away from writing content. Once knitted the report can be sent to the inquirer. The report as Rmarkdown documents 30/33
51. snakemake is great but report is not exible and works like a dry/black box. targets, written by William Landau (pictured), is exible, robust and still allows for a customized report. All computing is done only when needed, and code is away from writing content. Once knitted the report can be sent to the inquirer. The report as Rmarkdown documents 30/33
52. snakemake is great but report is not exible and works like a dry/black box. targets, written by William Landau (pictured), is exible, robust and still allows for a customized report. All computing is done only when needed, and code is away from writing content. Once knitted the report can be sent to the inquirer. The report as Rmarkdown documents 30/33
53. targets events watched live 🎞 here, after changing a threshold in the LRT step branches can be monitored too 2 videos joined as I xed an error at 1'42" option to display functions (unset here) tar_watch() shiny app from targets Bonus: watch the pipeline running live 🍿 31/33
54. Future work Enhancements 💅✨ Use it! For all projects. Big regrets when returning to previous set-up Add shiny app by Anthoula Gaigneaux to the snakemake template Needs to have docker containers on the shiny server rst practise targets to run on HPC and slurm Link targets upstream les from snakemake at cluster (hash tracked) Design more snakemake templates (ChIP-seq) Advertise and bring in more users 32/33
55. Before we stop Acknowledgments 🙏 👏 Anthoula Gaigneaux Eric Koncina early adopter of targets Marios Gavriil for yeast data Wendkouni Nadège Minoungou for the Lnc6 data Jochen Ohnmacht, Vincent Gureghian & Yujuan Gui for testing the template Sarah Peter for snakemake introduction William Landau main developer of targets Johannes Köster main developer of snakemake Thank you for your attention! 4-days R workshop in May 2021 Elixir-LU event 6th - 11th May 33/33
56. Before we stop Acknowledgments 🙏 👏 Anthoula Gaigneaux Eric Koncina early adopter of targets Marios Gavriil for yeast data Wendkouni Nadège Minoungou for the Lnc6 data Jochen Ohnmacht, Vincent Gureghian & Yujuan Gui for testing the template Sarah Peter for snakemake introduction William Landau main developer of targets Johannes Köster main developer of snakemake Thank you for your attention! 4-days R workshop in May 2021 Elixir-LU event 6th - 11th May 33/33