Containers: Practices, Problems, and Possiblilities

WOSSS19 presentation

Transcript

1. 

   Jupyter Team
   Carol Willing
   WOSSS19
   April 25, 2019
   @WillingCarol
   Containers
   Practices,
   Problems, and
   Possibilities
   

   View Slide

2. Jupyter
   Steering Council
   Core Developer
   JupyterHub, BinderHub,
   mybinder.org
   Python
   Steering Council
   Core Developer
   PSF Fellow
   Carol Willing
   

   View Slide

3. Jupyter
   ‣ Growth
   ‣ ACM Award
   ‣ Industry adoption
   ‣ Creative uses
   ‣ Open Source Book
   

   View Slide

4. Project Jupyter
   Containers
   2014-Present
   @WillingCarol
   

   View Slide

5. Evolution
   • jupyter/docker-stacks: Docker containers for
   common user environments
   • JupyterHub: Docker containers for ease of
   deployment and to provide isolated
   environments for users
   • Zero to JupyterHub with Kubernetes: Use
   Kubernetes to scale JupyterHub from a
   single class to a large scale MOOCs
   • BinderHub and mybinder.org: Kubernetes
   and Docker containers used to offer a
   service
   @WillingCarol
   • Environment
   • Deployment
   • Scaling
   • Service
   

   View Slide

6. zero-to-jupyterhub.readthedocs.io
   

   View Slide

7. https://blog.jupyter.org/binder-2-0-a-tech-guide-2017-fd40515a3a84
   https://elifesciences.org/labs/8653a61d/introducing-binder-2-0-share-your-
   interactive-research-environment
   https://www.nature.com/articles/d41586-018-01322-9
   mybinder.org
   

   View Slide

8. Practices
   @WillingCarol
   

   View Slide

9. Understand the use
   case before selecting a
   container solution.
   @WillingCarol
   Practices
   

   View Slide

10. Converge on solutions
    for deployment and
    share openly.
    @WillingCarol
    Practices
    

    View Slide

11. https://zonca.github.io/
    @WillingCarol
    Practices
    

    View Slide

12. Use standard tools.
    Don't build software from
    scratch when standard
    tools are available.
    @WillingCarol
    Practices
    

    View Slide

13. When running a service,
    apply Site Reliability best
    practices.
    Document processes and
    perform blameless post-
    mortems.
    @WillingCarol
    Practices
    

    View Slide

14. Problems
    @WillingCarol
    

    View Slide

15. Learning Curve
    and
    Software Skills
    Investment
    @WillingCarol
    Problems
    

    View Slide

16. Data and
    Access
    Controls
    @WillingCarol
    Problems
    

    View Slide

17. Scaling
    @WillingCarol
    Problems
    

    View Slide

18. Vendor Lock-in
    @WillingCarol
    Problems
    

    View Slide

19. Possibilities
    @WillingCarol
    

    View Slide

20. Tailored
    Workflows
    @WillingCarol
    Possibilities
    

    View Slide

21. http://genepattern-notebook.org/
    @WillingCarol
    

    View Slide

22. nteract
    Enterprise data workflows
    

    View Slide

23. Accessible
    Science
    @WillingCarol
    Possibilities
    

    View Slide

24. Introduction to
    Mechanical
    Vibration
    (Jason Moore &
    Kenneth Lyons)
    http://www.moorepants.info/blog/introducing-resonance.html
    

    View Slide

25. https://jupyter4edu.github.io/jupyter-edu-book/
    @WillingCarol
    

    View Slide

26. Visualization
    and
    WebAssembly
    @WillingCarol
    Possibilities
    

    View Slide

27. https://github.com/data-
    exp-lab/rust-yt-tools/
    npm
    package @data-
    exp-lab/yt-tools
    Irber Junior LC.
    Oxidizing Python:
    writing extensions in
    Rust [version 1; not
    peer
    reviewed]. F1000Rese
    arch 2018, 7(ISCB
    Comm J):955 (poster)
    (https://doi.org/
    10.7490/
    f1000research.
    1115726.1)
    https://github.com/
    munkm/widgyts
    

    View Slide

28. ipyvolume
    https://
    towardsdatascience.com/
    multivolume-rendering-in-
    jupyter-with-ipyvolume-
    cross-language-3d-
    visualization-64389047634a
    

    View Slide

29. View Slide

30. View Slide

31. Matplotlib documentation
    
    Carol Willing papers and photos
    
    Project Jupyter
    
    nteract project
    
    Python website
    
    Fernando Perez website
    
    ipyvolume project
    
    yt project and Madickin Munk
    
    Jason Moore
    

    View Slide

32. 

    Jupyter Team
    Carol Willing
    WOSSS19
    April 25, 2019
    @WillingCarol
    Containers
    Practices,
    Problems, and
    Possibilities
    

    View Slide