Share
Embed
Tinkering with Serverless Computing and R in the Cloud
kellobri
November 09, 2018
Transcript
28 slides
-
Lower the Cost of Turning Ideas into Realities If you
have something of value and a PayPal account - you can: Start an online business in minutes -
Lower the Cost of Turning Ideas into Realities If you
have data analysis of value and the R data product toolchain - you can: Share that analysis with everyone you know in minutes -
The Worst Ever: TRAPPED - have a cool idea and
not the tools or means to execute in the manner you wanted BUT HOW?? -
Local Environment Promotion Strategies Local Data Science Environment Post a
Screenshot to Twitter Write a Blog Post Create a GitHub Repository Publish to RPubs / Shinyapps.io Publish to an Analytic Sandbox (Tinker-Space) Publish to Professional Analytic Infrastructure -
Local Environment Promotion Strategies Local Data Science Environment Post a
Screenshot to Twitter Write a Blog Post Create a GitHub Repository Publish to RPubs / Shinyapps.io Publish to an Analytic Sandbox (Tinker-Space) Publish to Professional, Secure Analytic Infrastructure Possible Blockers: • Fire • Dragons • Linux Administration -
Role - Analytic Administrator A data scientist who: - Onboards
new tools, deploys solutions, supports existing standards - Works closely with IT to maintain, upgrade and scale analytic environments - Influences others in the organization to be more effective - (Passionate about making R a legitimate analytic standard within the organization) Check out Nathan Stephens on the RViews Blog - Analytics Administration for R -
Becoming an Analytic Admin for R Local Data Science Environment
Publish to an Analytic Sandbox (Tinker-Space) Publish to Professional, Secure Analytic Infrastructure - Learn the ropes of Analytic Administration! (Linux/DevOps) - Practice with open source tools and technology - Build POCs and develop a sense for best practices Apply what you’ve learned to deliver analytic infrastructure solutions to your organization -
Motivated by Practical Need or Burning Curiosity Tinkering! A totally
valid way to learn “this stuff” -
Plumber APIs A package for creating REST APIs out of
existing R code Jeff Allen gave an RStudio Webinar on Plumber (July 2018) - Now Available On Demand -
Serverless Compute Abstract away all your server management and low
level infrastructure decisions Let the cloud do it. -
Google App Engine Fully managed serverless application platform Bring your
code, have GCP manage all the infrastructure, Pay for what you use Famously (Infamously?) used by Snapchat 2008 2018 Now with Custom Runtimes!! -
Google App Engine - R not Included (as default) Can
it be used to deploy docker + R + plumber? “Popular Crowd” -
You don’t always need to Docker all the things But
in this particular instance we do. -
Two Custom App Engine Projects Two Directories: - Dockerfile -
App.yaml - Assets runtime: custom env: flex -
Docker Prep - Plumber FROM trestletech/plumber RUN R -e 'install.packages(c("ggplot2"))'
COPY [".", "./"] EXPOSE 8080 ENTRYPOINT ["R", "-e", "pr <- plumber::plumb(commandArgs()[4]); pr$run(host='0.0.0.0', port=8080)"] CMD ["plumber.R"] -
Docker Prep - NGINX FROM nginx COPY nginx.conf /etc/nginx/nginx.conf RUN
mkdir -p /var/log/app_engine # Add the static content ADD www/ /usr/share/nginx/www/ RUN chmod -R a+r /usr/share/nginx/www The NGINX website has introductory webinars available on demand + loads of documentation -
Run App Engine! 1. Prep your Dockerfile 2. Create an
app.yaml file 3. Gather any asset files $ gcloud app create $ gcloud app deploy -
*Not an instantaneous deployment process Leave yourself at least 15
minutes for gcloud app deploy to run. If you need something with fast push button deploy... (I Work for RStudio) -
Advanced Fun: Script Scheduling Integration Plumber REST API Google Cloud
Composer - Managed Apache Airflow Service -
Plumber REST API Google Cloud Composer - Managed Apache Airflow
Service DAGS! - Do you know when CRON jobs fail? - Handle long running tasks and dependencies? - Spot when your tasks become 3x slower? - Can you visualize what’s currently running? What’s queued? Apache Airflow -
Orchestrate and Burn Down Proof of Concepts take time and
energy - but most of the time it’s best not to get too attached. When it comes time for the real deal, use what you learned to start fresh. The cleansing power of BURNING IT DOWN Settings and Utilities > Project Settings > Shutdown -
Check Out - R Views Blog: Learning Analytic Administration through
a Sandbox #radmins