R & Microservices

Talk about R and microservices at WhyR 2019

Transcript

1. R & Microservices
   Small is beautiful
   Colin Fay - ThinkR
   Colin FAY (@_ColinFay) - WhyR 2019 - https://rtask.thinkr.fr 1 / 24
   

   View Slide

2. $ whoami
   Colin FAY
   Data Scientist & R-Hacker at ThinkR, a french company focused on Data Science & R.
   Hyperactive open source developer.
   http://thinkr.fr
   http://rtask.thinkr.fr
   http://twitter.com/_colinfay
   http://github.com/colinfay
   Colin FAY (@_ColinFay) - WhyR 2019 - https://rtask.thinkr.fr 2 / 24
   

   View Slide

3. ThinkR
   Colin FAY (@_ColinFay) - WhyR 2019 - https://rtask.thinkr.fr 3 / 24
   

   View Slide

4. Data Science engineering, focused on R.
    Training
    Software Engineering
    R in production
    Consulting
   ThinkR
   Colin FAY (@_ColinFay) - WhyR 2019 - https://rtask.thinkr.fr 4 / 24
   

   View Slide

5. Today's schedule
   What this talk is not about:
   Technical details
   Microservices deployment
   What this talk is about:
   What's a microservice
   Why you should care
   How to get started building an R microservices
   Colin FAY (@_ColinFay) - WhyR 2019 - https://rtask.thinkr.fr 5 / 24
   

   View Slide

6. Microservices
   Colin FAY (@_ColinFay) - WhyR 2019 - https://rtask.thinkr.fr 6 / 24
   

   View Slide

7. What's a Microservice?
   Microservices are a software development technique (...) that structures an
   application as a collection of loosely coupled services. In a microservices
   architecture, services are fine-grained and the protocols are lightweight.
   https://en.wikipedia.org/wiki/Microservices
   Something which is:
   Lightweight
   Good for one task
   Easy to scale
   Colin FAY (@_ColinFay) - WhyR 2019 - https://rtask.thinkr.fr 7 / 24
   

   View Slide

8. The good
   Lightweight
   Easier to maintain, to change, and to
   scale
   Not linked to an infrastructure
   The bad
   More communication management
   More architecture needs
   Microservices
   The ugly
   The "distributed monolith"
   Colin FAY (@_ColinFay) - WhyR 2019 - https://rtask.thinkr.fr 8 / 24
   

   View Slide

9. Colin FAY (@_ColinFay) - WhyR 2019 - https://rtask.thinkr.fr 9 / 24
   

   View Slide

10. R and Microservices, why?
    Make your R algorithm, graph, data output... language agnostic: integrate it to
    other services
    Don't depend on your machine's configuration
    Build tools with different versions of a package which still work together
    Simplify your code base
    You're going to sound cool
    Colin FAY (@_ColinFay) - WhyR 2019 - https://rtask.thinkr.fr 10 / 24
    

    View Slide

11. R and Microservices, how?
    The central thing about microservice is communication.
    Don't think about the language, think about the data
    Take text as input, and output text (JSON)
    Or use a "universal" format like arrow
    Every piece should fail gracefully
    Colin FAY (@_ColinFay) - WhyR 2019 - https://rtask.thinkr.fr 11 / 24
    

    View Slide

12. R and Microservices, how?
    The UNIX philosophy:
    Unix tradition strongly encourages writing programs that read and write
    simple, textual, stream-oriented, device-independent formats. Under classic
    Unix, as many programs as possible are written as simple filters, which take
    a simple text stream on input and process it into another simple text stream
    on output.
    The Art of UNIX Programming
    Colin FAY (@_ColinFay) - WhyR 2019 - https://rtask.thinkr.fr 12 / 24
    

    View Slide

13. R and Microservices, how?
    # In R
    Rscript -e "jsonlite::write_json(head(iris), 'iris.json')"
    # In NodeJS
    node -e "const readFileSync = require('fs').readFileSync;
    var iris = readFileSync('iris.json');
    iris = JSON.parse(iris);
    iris.forEach( (x) => {
    console.log(x['Sepal.Length'])
    })"
    #> 5.1
    #> 4.9
    #> 4.7
    #> 4.6
    #> 5
    #> 5.4
    Notes: IRL, this service should handle errors gracefully
    Colin FAY (@_ColinFay) - WhyR 2019 - https://rtask.thinkr.fr 13 / 24
    

    View Slide

14. R and Microservices, how?
    # In R
    Rscript -e "arrow::write_arrow(head(iris), 'iris.arrow')";
    # In NodeJS
    node -e "var readFileSync = require('fs').readFileSync
    var Table = require('apache-arrow').Table
    const arrow = readFileSync('iris.arrow');
    const table = Table.from([arrow]);
    console.log( table.toArray()[0].toString() );"
    #> 5.1, 3.5, 1.4, 0.2, "setosa"
    Notes: IRL, this service should handle errors gracefully
    Colin FAY (@_ColinFay) - WhyR 2019 - https://rtask.thinkr.fr 14 / 24
    

    View Slide

15. R and Microservices, tools
    RScript & littler: Command line tool to execute R code
    const exec = require('child_process').exec;
    exec("RScript -e 'head(iris, 4)'", function(error, stdout, stderr){
    console.log(stdout);
    })
    node ./script.js
    #> Sepal.Length Sepal.Width Petal.Length Petal.Width Species
    #> 1 5.1 3.5 1.4 0.2 setosa
    #> 2 4.9 3.0 1.4 0.2 setosa
    #> 3 4.7 3.2 1.3 0.2 setosa
    #> 4 4.6 3.1 1.5 0.2 setosa
    Notes: IRL, this service should handle errors gracefully
    Colin FAY (@_ColinFay) - WhyR 2019 - https://rtask.thinkr.fr 15 / 24
    

    View Slide

16. R and Microservices, tools
    R package is a micro service by itself
    package::function()
    Colin FAY (@_ColinFay) - WhyR 2019 - https://rtask.thinkr.fr 16 / 24
    

    View Slide

17. R and Microservices, tools
    RScript & littler - Command line tools to execute R code:
    const exec = require('child_process').exec;
    exec("RScript -e 'myservice::computation()'",
    function(error, stdout, stderr){
    var res = JSON.parse(stdout);
    res.forEach((x) => { console.log(x["mpg"])
    })
    })
    node ./script.js
    #> 21
    #> 21
    #> 22.8
    #> 21.4
    #> 18.7
    #> 18.1
    Colin FAY (@_ColinFay) - WhyR 2019 - https://rtask.thinkr.fr 17 / 24
    

    View Slide

18. Colin FAY (@_ColinFay) - WhyR 2019 - https://rtask.thinkr.fr 18 / 24
    

    View Slide

19. {plumber}
    opencpu
    {RestRServe}
    Native Node API
    IPA API
    Colin FAY (@_ColinFay) - WhyR 2019 - https://rtask.thinkr.fr 19 / 24
    

    View Slide

20. Docker
    Colin FAY (@_ColinFay) - WhyR 2019 - https://rtask.thinkr.fr 20 / 24
    

    View Slide

21. Colin FAY (@_ColinFay) - WhyR 2019 - https://rtask.thinkr.fr 21 / 24
    

    View Slide

22. r-online (https://srv.colinfay.me/r-online)
    Colin FAY (@_ColinFay) - WhyR 2019 - https://rtask.thinkr.fr 22 / 24
    

    View Slide

23. r-online internals
    Colin FAY (@_ColinFay) - WhyR 2019 - https://rtask.thinkr.fr 23 / 24
    

    View Slide

24. colin@thinkr.fr
    http://twitter.com/_colinfay
    http://twitter.com/thinkr_fr
    https://github.com/ColinFay
    https://thinkr.fr/
    https://rtask.thinkr.fr/
    https://colinfay.me/
    Thx! Questions?
    Colin Fay
    Colin FAY (@_ColinFay) - WhyR 2019 - https://rtask.thinkr.fr 24 / 24
    

    View Slide