Lock in $30 Savings on PRO—Offer Ends Soon! ⏳

2019-12-26 Programming Environment - Interopera...

2019-12-26 Programming Environment - Interoperable Computing by Federated Kubernetes

TAKAHASHI Shuuji

December 26, 2019
Tweet

More Decks by TAKAHASHI Shuuji

Other Decks in Technology

Transcript

  1. Interoperable Computing by Federated Kubernetes 2019-12-26 Programming Environment Graduate School

    of Systems & Information Engineering M1 201920673 TAKAHASHI Shuuji
  2. Problem: Drawback and missing feature of Grid, HPC, Cloud •

    Many interoperability problems ◦ Various system has a various environment ▪ Cannot resolve complex library dependencies ▪ Cannot run program on other operating system ◦ On HPC: each site has a different type of the batch job system • This can be solved by ◦ Multi-sites federated Kubernetes clusters using container technology
  3. Clouds Can Down • Clouds can down even if major

    providers such as AWS, GCP, and Azure though they has a great experience on deployment of the cloud • AWS Tokyo regional catastrophe on Friday afternoon, August 23, 2019[1] ◦ duration: ~10 hours • GCP Google Cloud Storage Incident on 2019-03-13[2] ◦ duration: ~4 hours [1] https://publickey1.jp/blog/19/aws23.html [2] https://www.publickey1.jp/blog/19/sregmail313google.html
  4. What is Container? • Container ◦ Provide an isolated system

    environment ▪ Popular container runtime: Docker, rkt ◦ Users can create containerized program including libraries it depends ◦ Users can run it on any environment Cloud On-premise HPC Open Container Runtime (Docker, rkt) Ubuntu CentOS Go Python Fortran C
  5. What is Kubernetes? • Kubernetes ◦ Container orchestration system ◦

    Open source & initially developed by Google ◦ Can run any containerized application automatically • Provided features ◦ Service discovery ◦ Storage orchestration ◦ Secret & config management ◦ Batch execution Source: Least Privilege Container Orchestration - Docker Blog https://www.docker.com/blog/least-privilege-container-orchestration/
  6. New Grid Computing by Federated Kubernetes • There are researches

    about federation between cloud and grid computing[3] • However, it can be enhanced with Kubernetes HPC On-prem ise HPC Cloud Cloud [3] Buyya, Rajkumar, and Rajiv Ranjan. "Federated resource management in grid and cloud computing systems." Future Generation Computer Systems 26.8 (2010): 1189-1191.
  7. New Grid Computing by Federated Kubernetes • Kubernetes cluster federation

    on various environments: ◦ Cloud: for flexible on-demand computing and storage ◦ On-premise: for cost-efficient/special hardwares ◦ HPC: for the large-scale compute-intensive programs HPC On-prem ise HPC Cloud Cloud