2019-12-26 Programming Environment - Interoperable Computing by Federated Kubernetes

This is a short fun presentation for my graduate school lecture: https://www.cs.tsukuba.ac.jp/syllabus/current/01CH301_E.html

Source: https://docs.google.com/presentation/d/1oWD_ti4YsQeTnB82TwSIcqTqa5RjkzlP-HeIxWNWSls/edit?usp=sharing

Transcript

1. Interoperable Computing by Federated Kubernetes 2019-12-26 Programming Environment Graduate School of Systems & Information Engineering M1 201920673 TAKAHASHI Shuuji
2. Problem: Drawback and missing feature of Grid, HPC, Cloud • Many interoperability problems ◦ Various system has a various environment ▪ Cannot resolve complex library dependencies ▪ Cannot run program on other operating system ◦ On HPC: each site has a different type of the batch job system • This can be solved by ◦ Multi-sites federated Kubernetes clusters using container technology
3. Clouds Can Down • Clouds can down even if major providers such as AWS, GCP, and Azure though they has a great experience on deployment of the cloud • AWS Tokyo regional catastrophe on Friday afternoon, August 23, 2019[1] ◦ duration: ~10 hours • GCP Google Cloud Storage Incident on 2019-03-13[2] ◦ duration: ~4 hours [1] https://publickey1.jp/blog/19/aws23.html [2] https://www.publickey1.jp/blog/19/sregmail313google.html
4. What is Container? • Container ◦ Provide an isolated system environment ▪ Popular container runtime: Docker, rkt ◦ Users can create containerized program including libraries it depends ◦ Users can run it on any environment Cloud On-premise HPC Open Container Runtime (Docker, rkt) Ubuntu CentOS Go Python Fortran C
5. What is Kubernetes? • Kubernetes ◦ Container orchestration system ◦ Open source & initially developed by Google ◦ Can run any containerized application automatically • Provided features ◦ Service discovery ◦ Storage orchestration ◦ Secret & config management ◦ Batch execution Source: Least Privilege Container Orchestration - Docker Blog https://www.docker.com/blog/least-privilege-container-orchestration/
6. New Grid Computing by Federated Kubernetes • There are researches about federation between cloud and grid computing[3] • However, it can be enhanced with Kubernetes HPC On-prem ise HPC Cloud Cloud [3] Buyya, Rajkumar, and Rajiv Ranjan. "Federated resource management in grid and cloud computing systems." Future Generation Computer Systems 26.8 (2010): 1189-1191.
7. New Grid Computing by Federated Kubernetes • Kubernetes cluster federation on various environments: ◦ Cloud: for flexible on-demand computing and storage ◦ On-premise: for cost-efficient/special hardwares ◦ HPC: for the large-scale compute-intensive programs HPC On-prem ise HPC Cloud Cloud
8. Fin.