Beyond Prototypes:A Journey to The Production Land_EARL2018

Slides of a talk presented at EARL London 2018 (https://earlconf.com)

Transcript

1. Beyond Prototypes A Journey to The Production Land Omayma Said OmaymaS
2. The Land of PROTOTYPES The Land of PRODUCTION
3. An excited Data Scientist!
4. Begin with the DATA PRODUCT in mind Bridging the prototypes and production lands
5. Web API Begin with the DATA PRODUCT in mind Web Application Reproducible Report Package Other
6. Think of the DATA PRODUCT as an answer to a question Bridging the prototypes and production lands
7. Q: What is the matching score of application X in job Y? A: Value returned from an endpoint (WEB API) Think of the DATA PRODUCT As an answer to a question
8. Q: How does the daily activity of the subscribed companies look like ? A: Metrics shown and updated in a Dashboard Think of the DATA PRODUCT As an answer to a question
9. Q: How do different segments respond to our recommendations ? A: Report/Presentation Think of the DATA PRODUCT As an answer to a question
10. Question +Data IN Answers/ Solutions OUT
11. Question +Data IN Answers/ Solutions OUT NOT always straightforward in real life!
12. Real Life Challenges Reproducibility Portability Accessibility
13. Make the DATA PRODUCT accessible and user friendly Bridging the prototypes and production lands
14. Rstudio Connect Rstudio/Shiny Server (open source) Docker Experimenting in the land of production
15. Docker (Commands to build an image) (Executable package) (Running instance of an image) Docker file Docker Image Docker Container
16. - Build this Dockerfile! - Install R - Install system packages - Install shiny server - Install R packages - Expose port - Run Versus
17. - Run this docker image! - Install R - Install system packages - Install shiny server - Install R packages - Expose port - Run Versus
18. FROM rocker/r-base MAINTAINER Jeff Allen <docker@trestletech.com> RUN apt-get update -qq && apt-get install -y \ git-core \ libssl-dev \ libcurl4-gnutls-dev RUN install2.r plumber EXPOSE 8000 ENTRYPOINT ["R", "-e", "pr <- plumber::plumb(commandArgs()[4]); pr$run(host='0.0.0.0', port=8000)"] CMD ["/usr/local/lib/R/site-library/plumber/examples/04-mean-sum/plumber.R"] How does a Dockerfile look like? https://hub.docker.com/r/trestletech/plumber
19. FROM rocker/r-base MAINTAINER Jeff Allen <docker@trestletech.com> RUN apt-get update -qq && apt-get install -y \ git-core \ libssl-dev \ libcurl4-gnutls-dev RUN install2.r plumber EXPOSE 8000 ENTRYPOINT ["R", "-e", "pr <- plumber::plumb(commandArgs()[4]); pr$run(host='0.0.0.0', port=8000)"] CMD ["/usr/local/lib/R/site-library/plumber/examples/04-mean-sum/plumber.R"] Install R How does a Dockerfile look like? https://hub.docker.com/r/trestletech/plumber
20. FROM rocker/r-base MAINTAINER Jeff Allen <docker@trestletech.com> RUN apt-get update -qq && apt-get install -y \ git-core \ libssl-dev \ libcurl4-gnutls-dev RUN install2.r plumber EXPOSE 8000 ENTRYPOINT ["R", "-e", "pr <- plumber::plumb(commandArgs()[4]); pr$run(host='0.0.0.0', port=8000)"] CMD ["/usr/local/lib/R/site-library/plumber/examples/04-mean-sum/plumber.R"] Install System Packages How does a Dockerfile look like? https://hub.docker.com/r/trestletech/plumber
21. FROM rocker/r-base MAINTAINER Jeff Allen <docker@trestletech.com> RUN apt-get update -qq && apt-get install -y \ git-core \ libssl-dev \ libcurl4-gnutls-dev RUN install2.r plumber EXPOSE 8000 ENTRYPOINT ["R", "-e", "pr <- plumber::plumb(commandArgs()[4]); pr$run(host='0.0.0.0', port=8000)"] CMD ["/usr/local/lib/R/site-library/plumber/examples/04-mean-sum/plumber.R"] Install plumber How does a Dockerfile look like? https://hub.docker.com/r/trestletech/plumber
22. FROM rocker/r-base MAINTAINER Jeff Allen <docker@trestletech.com> RUN apt-get update -qq && apt-get install -y \ git-core \ libssl-dev \ libcurl4-gnutls-dev RUN install2.r plumber EXPOSE 8000 ENTRYPOINT ["R", "-e", "pr <- plumber::plumb(commandArgs()[4]); pr$run(host='0.0.0.0', port=8000)"] CMD ["/usr/local/lib/R/site-library/plumber/examples/04-mean-sum/plumber.R"] Expose port How does a Dockerfile look like? https://hub.docker.com/r/trestletech/plumber
23. FROM rocker/r-base MAINTAINER Jeff Allen <docker@trestletech.com> RUN apt-get update -qq && apt-get install -y \ git-core \ libssl-dev \ libcurl4-gnutls-dev RUN install2.r plumber EXPOSE 8000 ENTRYPOINT ["R", "-e", "pr <- plumber::plumb(commandArgs()[4]); pr$run(host='0.0.0.0', port=8000)"] CMD ["/usr/local/lib/R/site-library/plumber/examples/04-mean-sum/plumber.R"] Run How does a Dockerfile look like? https://hub.docker.com/r/trestletech/plumber
24. Could someone help me write Dockerfiles?
25. 2 1 Could someone help me write Dockerfiles? Use/Modify available Dockerfiles Use helper packages
26. 1 Use/Modify available Dockerfiles Could someone help me write Dockerfiles?
27. The Rocker Project github.com/rocker-org
28. The Rocker Project hub.docker.com/u/rocker
29. The Rocker Project docker pull <user/repo> docker run <image> hub.docker.com/u/rocker
30. 2 Use helper packages Could someone help me write Dockerfiles?
31. Containerit containerit::dockerfile() Capture dependencies in session file workspace github.com/o2r-project/containerit
32. liftr github.com/road2stat/liftr liftr::lift(“foo.rmd”)
33. liftr --- title: "User Engagement Analysis" author: "Sara K." output: rmarkdown::html_document liftr: maintainer: "Sara K." email: "sara.k@gmail.com" cran: - dplyr --- github.com/road2stat/liftr
34. rize rize::shiny_dockerize() BUILD docker image CREATE Dockerfile github.com/cole-brokamp/rize app.R
35. The Land of PROTOTYPES The Land of PRODUCTION INTERCONNECTED
36. Begin with the DATA PRODUCT in mind Think of the DATA PRODUCT As an answer to a question Make the DATA PRODUCT Accessible and user friendly https://speakerdeck.com/omaymas