R & Microservices

Db8efd836c9a09b71e3d8e1c60d6ea84?s=47 Colin Fay
September 29, 2019

R & Microservices

Talk about R and microservices at WhyR 2019

Db8efd836c9a09b71e3d8e1c60d6ea84?s=128

Colin Fay

September 29, 2019
Tweet

Transcript

  1. R & Microservices Small is beautiful Colin Fay - ThinkR

    Colin FAY (@_ColinFay) - WhyR 2019 - https://rtask.thinkr.fr 1 / 24
  2. $ whoami Colin FAY Data Scientist & R-Hacker at ThinkR,

    a french company focused on Data Science & R. Hyperactive open source developer. http://thinkr.fr http://rtask.thinkr.fr http://twitter.com/_colinfay http://github.com/colinfay Colin FAY (@_ColinFay) - WhyR 2019 - https://rtask.thinkr.fr 2 / 24
  3. ThinkR Colin FAY (@_ColinFay) - WhyR 2019 - https://rtask.thinkr.fr 3

    / 24
  4. Data Science engineering, focused on R.  Training  Software

    Engineering  R in production  Consulting ThinkR Colin FAY (@_ColinFay) - WhyR 2019 - https://rtask.thinkr.fr 4 / 24
  5. Today's schedule What this talk is not about: Technical details

    Microservices deployment What this talk is about: What's a microservice Why you should care How to get started building an R microservices Colin FAY (@_ColinFay) - WhyR 2019 - https://rtask.thinkr.fr 5 / 24
  6. Microservices Colin FAY (@_ColinFay) - WhyR 2019 - https://rtask.thinkr.fr 6

    / 24
  7. What's a Microservice? Microservices are a software development technique (...)

    that structures an application as a collection of loosely coupled services. In a microservices architecture, services are fine-grained and the protocols are lightweight. https://en.wikipedia.org/wiki/Microservices Something which is: Lightweight Good for one task Easy to scale Colin FAY (@_ColinFay) - WhyR 2019 - https://rtask.thinkr.fr 7 / 24
  8. The good Lightweight Easier to maintain, to change, and to

    scale Not linked to an infrastructure The bad More communication management More architecture needs Microservices The ugly The "distributed monolith" Colin FAY (@_ColinFay) - WhyR 2019 - https://rtask.thinkr.fr 8 / 24
  9. Colin FAY (@_ColinFay) - WhyR 2019 - https://rtask.thinkr.fr 9 /

    24
  10. R and Microservices, why? Make your R algorithm, graph, data

    output... language agnostic: integrate it to other services Don't depend on your machine's configuration Build tools with different versions of a package which still work together Simplify your code base You're going to sound cool Colin FAY (@_ColinFay) - WhyR 2019 - https://rtask.thinkr.fr 10 / 24
  11. R and Microservices, how? The central thing about microservice is

    communication. Don't think about the language, think about the data Take text as input, and output text (JSON) Or use a "universal" format like arrow Every piece should fail gracefully Colin FAY (@_ColinFay) - WhyR 2019 - https://rtask.thinkr.fr 11 / 24
  12. R and Microservices, how? The UNIX philosophy: Unix tradition strongly

    encourages writing programs that read and write simple, textual, stream-oriented, device-independent formats. Under classic Unix, as many programs as possible are written as simple filters, which take a simple text stream on input and process it into another simple text stream on output. The Art of UNIX Programming Colin FAY (@_ColinFay) - WhyR 2019 - https://rtask.thinkr.fr 12 / 24
  13. R and Microservices, how? # In R Rscript -e "jsonlite::write_json(head(iris),

    'iris.json')" # In NodeJS node -e "const readFileSync = require('fs').readFileSync; var iris = readFileSync('iris.json'); iris = JSON.parse(iris); iris.forEach( (x) => { console.log(x['Sepal.Length']) })" #> 5.1 #> 4.9 #> 4.7 #> 4.6 #> 5 #> 5.4 Notes: IRL, this service should handle errors gracefully Colin FAY (@_ColinFay) - WhyR 2019 - https://rtask.thinkr.fr 13 / 24
  14. R and Microservices, how? # In R Rscript -e "arrow::write_arrow(head(iris),

    'iris.arrow')"; # In NodeJS node -e "var readFileSync = require('fs').readFileSync var Table = require('apache-arrow').Table const arrow = readFileSync('iris.arrow'); const table = Table.from([arrow]); console.log( table.toArray()[0].toString() );" #> 5.1, 3.5, 1.4, 0.2, "setosa" Notes: IRL, this service should handle errors gracefully Colin FAY (@_ColinFay) - WhyR 2019 - https://rtask.thinkr.fr 14 / 24
  15. R and Microservices, tools RScript & littler: Command line tool

    to execute R code const exec = require('child_process').exec; exec("RScript -e 'head(iris, 4)'", function(error, stdout, stderr){ console.log(stdout); }) node ./script.js #> Sepal.Length Sepal.Width Petal.Length Petal.Width Species #> 1 5.1 3.5 1.4 0.2 setosa #> 2 4.9 3.0 1.4 0.2 setosa #> 3 4.7 3.2 1.3 0.2 setosa #> 4 4.6 3.1 1.5 0.2 setosa Notes: IRL, this service should handle errors gracefully Colin FAY (@_ColinFay) - WhyR 2019 - https://rtask.thinkr.fr 15 / 24
  16. R and Microservices, tools R package is a micro service

    by itself package::function() Colin FAY (@_ColinFay) - WhyR 2019 - https://rtask.thinkr.fr 16 / 24
  17. R and Microservices, tools RScript & littler - Command line

    tools to execute R code: const exec = require('child_process').exec; exec("RScript -e 'myservice::computation()'", function(error, stdout, stderr){ var res = JSON.parse(stdout); res.forEach((x) => { console.log(x["mpg"]) }) }) node ./script.js #> 21 #> 21 #> 22.8 #> 21.4 #> 18.7 #> 18.1 Colin FAY (@_ColinFay) - WhyR 2019 - https://rtask.thinkr.fr 17 / 24
  18. Colin FAY (@_ColinFay) - WhyR 2019 - https://rtask.thinkr.fr 18 /

    24
  19. {plumber} opencpu {RestRServe} Native Node API IPA API Colin FAY

    (@_ColinFay) - WhyR 2019 - https://rtask.thinkr.fr 19 / 24
  20. Docker Colin FAY (@_ColinFay) - WhyR 2019 - https://rtask.thinkr.fr 20

    / 24
  21. Colin FAY (@_ColinFay) - WhyR 2019 - https://rtask.thinkr.fr 21 /

    24
  22. r-online (https://srv.colinfay.me/r-online) Colin FAY (@_ColinFay) - WhyR 2019 - https://rtask.thinkr.fr

    22 / 24
  23. r-online internals Colin FAY (@_ColinFay) - WhyR 2019 - https://rtask.thinkr.fr

    23 / 24
  24. colin@thinkr.fr http://twitter.com/_colinfay http://twitter.com/thinkr_fr https://github.com/ColinFay https://thinkr.fr/ https://rtask.thinkr.fr/ https://colinfay.me/ Thx! Questions? Colin

    Fay Colin FAY (@_ColinFay) - WhyR 2019 - https://rtask.thinkr.fr 24 / 24