Upgrade to Pro — share decks privately, control downloads, hide ads and more …

2019-12-26 Programming Environment - Interoperable Computing by Federated Kubernetes

2019-12-26 Programming Environment - Interoperable Computing by Federated Kubernetes

TAKAHASHI Shuuji

December 26, 2019
Tweet

More Decks by TAKAHASHI Shuuji

Other Decks in Technology

Transcript

  1. Interoperable Computing
    by Federated Kubernetes
    2019-12-26 Programming Environment
    Graduate School of Systems & Information Engineering M1
    201920673 TAKAHASHI Shuuji

    View Slide

  2. Problem: Drawback and missing feature of Grid, HPC, Cloud
    ● Many interoperability problems
    ○ Various system has a various environment
    ■ Cannot resolve complex library dependencies
    ■ Cannot run program on other operating system
    ○ On HPC: each site has a different type of the batch
    job system
    ● This can be solved by
    ○ Multi-sites federated Kubernetes clusters using
    container technology

    View Slide

  3. Clouds Can Down
    ● Clouds can down even if major providers such as AWS, GCP,
    and Azure though they has a great experience on
    deployment of the cloud
    ● AWS Tokyo regional catastrophe on Friday afternoon,
    August 23, 2019[1]
    ○ duration: ~10 hours
    ● GCP Google Cloud Storage Incident on 2019-03-13[2]
    ○ duration: ~4 hours
    [1] https://publickey1.jp/blog/19/aws23.html
    [2] https://www.publickey1.jp/blog/19/sregmail313google.html

    View Slide

  4. What is Container?
    ● Container
    ○ Provide an isolated system environment
    ■ Popular container runtime: Docker, rkt
    ○ Users can create containerized program including
    libraries it depends
    ○ Users can run it on any
    environment
    Cloud On-premise HPC
    Open Container Runtime
    (Docker, rkt)
    Ubuntu CentOS
    Go Python Fortran
    C

    View Slide

  5. What is Kubernetes?
    ● Kubernetes
    ○ Container orchestration system
    ○ Open source & initially
    developed by Google
    ○ Can run any containerized
    application automatically
    ● Provided features
    ○ Service discovery
    ○ Storage orchestration
    ○ Secret & config management
    ○ Batch execution Source: Least Privilege Container Orchestration - Docker Blog
    https://www.docker.com/blog/least-privilege-container-orchestration/

    View Slide

  6. New Grid Computing by Federated Kubernetes
    ● There are researches about
    federation between cloud and
    grid computing[3]
    ● However, it can be enhanced
    with Kubernetes
    HPC
    On-prem
    ise
    HPC
    Cloud
    Cloud
    [3] Buyya, Rajkumar, and Rajiv Ranjan.
    "Federated resource management in grid and
    cloud computing systems." Future Generation
    Computer Systems 26.8 (2010): 1189-1191.

    View Slide

  7. New Grid Computing by Federated Kubernetes
    ● Kubernetes cluster federation
    on various environments:
    ○ Cloud: for flexible
    on-demand computing and
    storage
    ○ On-premise: for
    cost-efficient/special
    hardwares
    ○ HPC: for the large-scale
    compute-intensive programs
    HPC
    On-prem
    ise
    HPC
    Cloud
    Cloud

    View Slide

  8. Fin.

    View Slide