2019-12-26 Programming Environment - Interoperable Computing by Federated Kubernetes

This is a short fun presentation for my graduate school lecture: https://www.cs.tsukuba.ac.jp/syllabus/current/01CH301_E.html

Source: https://docs.google.com/presentation/d/1oWD_ti4YsQeTnB82TwSIcqTqa5RjkzlP-HeIxWNWSls/edit?usp=sharing

Transcript

1. Interoperable Computing
   by Federated Kubernetes
   2019-12-26 Programming Environment
   Graduate School of Systems & Information Engineering M1
   201920673 TAKAHASHI Shuuji
   

   View Slide

2. Problem: Drawback and missing feature of Grid, HPC, Cloud
   ● Many interoperability problems
   ○ Various system has a various environment
   ■ Cannot resolve complex library dependencies
   ■ Cannot run program on other operating system
   ○ On HPC: each site has a different type of the batch
   job system
   ● This can be solved by
   ○ Multi-sites federated Kubernetes clusters using
   container technology
   

   View Slide

3. Clouds Can Down
   ● Clouds can down even if major providers such as AWS, GCP,
   and Azure though they has a great experience on
   deployment of the cloud
   ● AWS Tokyo regional catastrophe on Friday afternoon,
   August 23, 2019[1]
   ○ duration: ~10 hours
   ● GCP Google Cloud Storage Incident on 2019-03-13[2]
   ○ duration: ~4 hours
   [1] https://publickey1.jp/blog/19/aws23.html
   [2] https://www.publickey1.jp/blog/19/sregmail313google.html
   

   View Slide

4. What is Container?
   ● Container
   ○ Provide an isolated system environment
   ■ Popular container runtime: Docker, rkt
   ○ Users can create containerized program including
   libraries it depends
   ○ Users can run it on any
   environment
   Cloud On-premise HPC
   Open Container Runtime
   (Docker, rkt)
   Ubuntu CentOS
   Go Python Fortran
   C
   

   View Slide

5. What is Kubernetes?
   ● Kubernetes
   ○ Container orchestration system
   ○ Open source & initially
   developed by Google
   ○ Can run any containerized
   application automatically
   ● Provided features
   ○ Service discovery
   ○ Storage orchestration
   ○ Secret & config management
   ○ Batch execution Source: Least Privilege Container Orchestration - Docker Blog
   https://www.docker.com/blog/least-privilege-container-orchestration/
   

   View Slide

6. New Grid Computing by Federated Kubernetes
   ● There are researches about
   federation between cloud and
   grid computing[3]
   ● However, it can be enhanced
   with Kubernetes
   HPC
   On-prem
   ise
   HPC
   Cloud
   Cloud
   [3] Buyya, Rajkumar, and Rajiv Ranjan.
   "Federated resource management in grid and
   cloud computing systems." Future Generation
   Computer Systems 26.8 (2010): 1189-1191.
   

   View Slide

7. New Grid Computing by Federated Kubernetes
   ● Kubernetes cluster federation
   on various environments:
   ○ Cloud: for flexible
   on-demand computing and
   storage
   ○ On-premise: for
   cost-efficient/special
   hardwares
   ○ HPC: for the large-scale
   compute-intensive programs
   HPC
   On-prem
   ise
   HPC
   Cloud
   Cloud
   

   View Slide

8. Fin.
   

   View Slide