Tinkering with Serverless Computing and R in the Cloud

Transcript

1. Tinkering with Serverless Computing and R in the Cloud Analytic Administration
2. Lower the Cost of Turning Ideas into Realities If you have something of value and a PayPal account - you can: Start an online business in minutes
3. Lower the Cost of Turning Ideas into Realities If you have data analysis of value and the R data product toolchain - you can: Share that analysis with everyone you know in minutes
4. The Worst Ever: TRAPPED - have a cool idea and not the tools or means to execute in the manner you wanted BUT HOW??
5. Local Environment Promotion Strategies Local Data Science Environment Post a Screenshot to Twitter Write a Blog Post Create a GitHub Repository Publish to RPubs / Shinyapps.io Publish to an Analytic Sandbox (Tinker-Space) Publish to Professional Analytic Infrastructure
6. Local Environment Promotion Strategies Local Data Science Environment Post a Screenshot to Twitter Write a Blog Post Create a GitHub Repository Publish to RPubs / Shinyapps.io Publish to an Analytic Sandbox (Tinker-Space) Publish to Professional, Secure Analytic Infrastructure Possible Blockers: • Fire • Dragons • Linux Administration
7. Role - Analytic Administrator A data scientist who: - Onboards new tools, deploys solutions, supports existing standards - Works closely with IT to maintain, upgrade and scale analytic environments - Influences others in the organization to be more effective - (Passionate about making R a legitimate analytic standard within the organization) Check out Nathan Stephens on the RViews Blog - Analytics Administration for R
8. Becoming an Analytic Admin for R Local Data Science Environment Publish to an Analytic Sandbox (Tinker-Space) Publish to Professional, Secure Analytic Infrastructure - Learn the ropes of Analytic Administration! (Linux/DevOps) - Practice with open source tools and technology - Build POCs and develop a sense for best practices Apply what you’ve learned to deliver analytic infrastructure solutions to your organization
9. Motivated by Practical Need or Burning Curiosity Tinkering! A totally valid way to learn “this stuff”
10. Plumber APIs A package for creating REST APIs out of existing R code Jeff Allen gave an RStudio Webinar on Plumber (July 2018) - Now Available On Demand
11. Serverless Compute Abstract away all your server management and low level infrastructure decisions Let the cloud do it.
12. Google App Engine Fully managed serverless application platform Bring your code, have GCP manage all the infrastructure, Pay for what you use Famously (Infamously?) used by Snapchat 2008 2018 Now with Custom Runtimes!!
13. Google App Engine - R not Included (as default) Can it be used to deploy docker + R + plumber? “Popular Crowd”
14. Plumber REST API NGINX Serving Static Rendered Rmarkdown
15. You don’t always need to Docker all the things But in this particular instance we do.
16. Create a Google Cloud Project
17. Two Custom App Engine Projects Two Directories: - Dockerfile - App.yaml - Assets runtime: custom env: flex
18. Docker Prep - Plumber FROM trestletech/plumber RUN R -e 'install.packages(c("ggplot2"))' COPY [".", "./"] EXPOSE 8080 ENTRYPOINT ["R", "-e", "pr <- plumber::plumb(commandArgs()[4]); pr$run(host='0.0.0.0', port=8080)"] CMD ["plumber.R"]
19. Docker Prep - NGINX FROM nginx COPY nginx.conf /etc/nginx/nginx.conf RUN mkdir -p /var/log/app_engine # Add the static content ADD www/ /usr/share/nginx/www/ RUN chmod -R a+r /usr/share/nginx/www The NGINX website has introductory webinars available on demand + loads of documentation
20. Run App Engine! 1. Prep your Dockerfile 2. Create an app.yaml file 3. Gather any asset files $ gcloud app create $ gcloud app deploy
21. $300 Free GCP Credits console.developers.google.com/billing/
22. *Not an instantaneous deployment process Leave yourself at least 15 minutes for gcloud app deploy to run. If you need something with fast push button deploy... (I Work for RStudio)
23. Basic Bonus Fun: LAY SIEGE
24. Advanced Fun: Script Scheduling Integration Plumber REST API Google Cloud Composer - Managed Apache Airflow Service
25. Plumber REST API Google Cloud Composer - Managed Apache Airflow Service DAGS! - Do you know when CRON jobs fail? - Handle long running tasks and dependencies? - Spot when your tasks become 3x slower? - Can you visualize what’s currently running? What’s queued? Apache Airflow
26. Orchestrate and Burn Down Proof of Concepts take time and energy - but most of the time it’s best not to get too attached. When it comes time for the real deal, use what you learned to start fresh. The cleansing power of BURNING IT DOWN Settings and Utilities > Project Settings > Shutdown
27. Check Out - R Views Blog: Learning Analytic Administration through a Sandbox #radmins
28. Kelly O’Briant RStudio Solutions Engineer kelly@rstudio.com @kellrstats on Twitter