R & Microservices

Talk about R and microservices at WhyR 2019

Transcript

1. R & Microservices Small is beautiful Colin Fay - ThinkR Colin FAY (@_ColinFay) - WhyR 2019 - https://rtask.thinkr.fr 1 / 24
2. $ whoami Colin FAY Data Scientist & R-Hacker at ThinkR, a french company focused on Data Science & R. Hyperactive open source developer. http://thinkr.fr http://rtask.thinkr.fr http://twitter.com/_colinfay http://github.com/colinfay Colin FAY (@_ColinFay) - WhyR 2019 - https://rtask.thinkr.fr 2 / 24
3. ThinkR Colin FAY (@_ColinFay) - WhyR 2019 - https://rtask.thinkr.fr 3 / 24
4. Data Science engineering, focused on R.  Training  Software Engineering  R in production  Consulting ThinkR Colin FAY (@_ColinFay) - WhyR 2019 - https://rtask.thinkr.fr 4 / 24
5. Today's schedule What this talk is not about: Technical details Microservices deployment What this talk is about: What's a microservice Why you should care How to get started building an R microservices Colin FAY (@_ColinFay) - WhyR 2019 - https://rtask.thinkr.fr 5 / 24
6. Microservices Colin FAY (@_ColinFay) - WhyR 2019 - https://rtask.thinkr.fr 6 / 24
7. What's a Microservice? Microservices are a software development technique (...) that structures an application as a collection of loosely coupled services. In a microservices architecture, services are fine-grained and the protocols are lightweight. https://en.wikipedia.org/wiki/Microservices Something which is: Lightweight Good for one task Easy to scale Colin FAY (@_ColinFay) - WhyR 2019 - https://rtask.thinkr.fr 7 / 24
8. The good Lightweight Easier to maintain, to change, and to scale Not linked to an infrastructure The bad More communication management More architecture needs Microservices The ugly The "distributed monolith" Colin FAY (@_ColinFay) - WhyR 2019 - https://rtask.thinkr.fr 8 / 24
9. Colin FAY (@_ColinFay) - WhyR 2019 - https://rtask.thinkr.fr 9 / 24
10. R and Microservices, why? Make your R algorithm, graph, data output... language agnostic: integrate it to other services Don't depend on your machine's configuration Build tools with different versions of a package which still work together Simplify your code base You're going to sound cool Colin FAY (@_ColinFay) - WhyR 2019 - https://rtask.thinkr.fr 10 / 24
11. R and Microservices, how? The central thing about microservice is communication. Don't think about the language, think about the data Take text as input, and output text (JSON) Or use a "universal" format like arrow Every piece should fail gracefully Colin FAY (@_ColinFay) - WhyR 2019 - https://rtask.thinkr.fr 11 / 24
12. R and Microservices, how? The UNIX philosophy: Unix tradition strongly encourages writing programs that read and write simple, textual, stream-oriented, device-independent formats. Under classic Unix, as many programs as possible are written as simple filters, which take a simple text stream on input and process it into another simple text stream on output. The Art of UNIX Programming Colin FAY (@_ColinFay) - WhyR 2019 - https://rtask.thinkr.fr 12 / 24
13. R and Microservices, how? # In R Rscript -e "jsonlite::write_json(head(iris), 'iris.json')" # In NodeJS node -e "const readFileSync = require('fs').readFileSync; var iris = readFileSync('iris.json'); iris = JSON.parse(iris); iris.forEach( (x) => { console.log(x['Sepal.Length']) })" #> 5.1 #> 4.9 #> 4.7 #> 4.6 #> 5 #> 5.4 Notes: IRL, this service should handle errors gracefully Colin FAY (@_ColinFay) - WhyR 2019 - https://rtask.thinkr.fr 13 / 24
14. R and Microservices, how? # In R Rscript -e "arrow::write_arrow(head(iris), 'iris.arrow')"; # In NodeJS node -e "var readFileSync = require('fs').readFileSync var Table = require('apache-arrow').Table const arrow = readFileSync('iris.arrow'); const table = Table.from([arrow]); console.log( table.toArray()[0].toString() );" #> 5.1, 3.5, 1.4, 0.2, "setosa" Notes: IRL, this service should handle errors gracefully Colin FAY (@_ColinFay) - WhyR 2019 - https://rtask.thinkr.fr 14 / 24
15. R and Microservices, tools RScript & littler: Command line tool to execute R code const exec = require('child_process').exec; exec("RScript -e 'head(iris, 4)'", function(error, stdout, stderr){ console.log(stdout); }) node ./script.js #> Sepal.Length Sepal.Width Petal.Length Petal.Width Species #> 1 5.1 3.5 1.4 0.2 setosa #> 2 4.9 3.0 1.4 0.2 setosa #> 3 4.7 3.2 1.3 0.2 setosa #> 4 4.6 3.1 1.5 0.2 setosa Notes: IRL, this service should handle errors gracefully Colin FAY (@_ColinFay) - WhyR 2019 - https://rtask.thinkr.fr 15 / 24
16. R and Microservices, tools R package is a micro service by itself package::function() Colin FAY (@_ColinFay) - WhyR 2019 - https://rtask.thinkr.fr 16 / 24
17. R and Microservices, tools RScript & littler - Command line tools to execute R code: const exec = require('child_process').exec; exec("RScript -e 'myservice::computation()'", function(error, stdout, stderr){ var res = JSON.parse(stdout); res.forEach((x) => { console.log(x["mpg"]) }) }) node ./script.js #> 21 #> 21 #> 22.8 #> 21.4 #> 18.7 #> 18.1 Colin FAY (@_ColinFay) - WhyR 2019 - https://rtask.thinkr.fr 17 / 24
18. Colin FAY (@_ColinFay) - WhyR 2019 - https://rtask.thinkr.fr 18 / 24
19. {plumber} opencpu {RestRServe} Native Node API IPA API Colin FAY (@_ColinFay) - WhyR 2019 - https://rtask.thinkr.fr 19 / 24
20. Docker Colin FAY (@_ColinFay) - WhyR 2019 - https://rtask.thinkr.fr 20 / 24
21. Colin FAY (@_ColinFay) - WhyR 2019 - https://rtask.thinkr.fr 21 / 24
22. r-online (https://srv.colinfay.me/r-online) Colin FAY (@_ColinFay) - WhyR 2019 - https://rtask.thinkr.fr 22 / 24
23. r-online internals Colin FAY (@_ColinFay) - WhyR 2019 - https://rtask.thinkr.fr 23 / 24
24. colin@thinkr.fr http://twitter.com/_colinfay http://twitter.com/thinkr_fr https://github.com/ColinFay https://thinkr.fr/ https://rtask.thinkr.fr/ https://colinfay.me/ Thx! Questions? Colin Fay Colin FAY (@_ColinFay) - WhyR 2019 - https://rtask.thinkr.fr 24 / 24