Let's start Cloud Native with Rancher !!

Let's start Cloud Native with Rancher !!

#rancherjp #cndjp

7bcf6f29a0b9a348fe422a31a4f44752?s=128

cyberblack28

January 26, 2019
Tweet

Transcript

  1. Let’s start Cloud Native with Rancher !! © 2019 cyberblack28

    #rancherjp #cndjp
  2. Profile Twitter : cyberblack28 Blog : https://cyberblack28.hatenablog.com/ Job Educational Solution

    Architect Developer Advocate/Technical Evangelist Infrastructure Engineer Frontend Engineer !! Music,Live,FES !! Take a picture !! https://www.flickr.com/photos/cyberblack ISBN-10: 4798155373 ISBN-13: 978-4798155371 I wrote the Chapter 6 Rancher 2.0 part. Release on March 15,2018 #rancherjp #kujiraya #deepcn
  3. Company We’re hireling !! Docker Kubernetes Rancher Cloud Native

  4. KubeCon + CloudNativeCon China 2018 Recap http://u0u0.net/PGHs Rancher Yes !!

  5. KubeCon + CloudNativeCon NA 2018 Recap http://ur0.link/PkTG Vitess MySQL Operator

    http://ur0.link/PkUm
  6. Agenda 1. Market trend of Kubernetes 2. Introduction of Kubernetes

    3. Introduction of Rancher 4. Kubernetes & Rancher Handson 5. Rancher 2.2 Preview 6. Information
  7. Market trend of Kubernetes

  8. Market trend of Kubernetes Kubernetes<17I $(&H?  CNCF. ,)"-.AEG/25 by

    Publickey 2018.3.8 Incubation Stage2 /,"02GJ( /.2' D6;= H <3?!-&% L97 >  CA   *) 2#!-&'+1$# 8B: 5F ECNCFTechnical Operating CommitteeTOC@IK4 A G/ Kubernetes@70 ;8D C4 B:=>!+,'%F9   63*#"-.
  9. Market trend of Kubernetes AWSKubernetes   Amazon EKS by

    Publickey 2018.6.7 • Amazon EKS(11" AWS re:Invent 2017 $ • "! ( &!)  "%! ()  '#*)201812GA
  10. Market trend of Kubernetes Azure Kubernetes Service (AKS) GA –

    New regions, more features, increased productivity by Microsoft Azure Blog 2018.6.13 • Microsoft Build 2018 Azure Container Service Azure Kubernetes Service ./ • "61"$'"$'"2( 2  ,&30#)*! 10  4+5% -
  11. Market trend of Kubernetes    Kubernetes Google Cloud

    Google Kubernetes Engine on-PremGoogle Cloud Next ’18 by Publickey 2018.7.25 •  #"GKE(Google Kubernetes Engine)$ Kubernetes#" &  • Google Cloud Console Kubernetes! • Google Cloud Platform MarketplaceKubernetes  % 
  12. Market trend of Kubernetes Google    Serverless containers

    Google Cloud Next ’18 by Publickey 2018.7.25 • Google Kubernetes Engine( )2$ &%+-  'GKE serverless add-on*, • '!$ #"'Kubernetes0/ +- KnativeGKE serverless add-on  .1*,
  13. Market trend of Kubernetes GitHub Goes All in on Kubernetes

    by THENEWSTACK 2017.8.16 SRESite Reliability Engineering   
  14. Introduction of Kubernetes

  15. What’s kubernetes ? kubernetesGoogle14/"$ #.,/ "*Borg / (  52

    OSS/"$ #.,/!-  B3'&" '&" '&" A CDk8skubernetes 8<@ 7=CNCF(Cloud Native Computing Foundation)?6  :8;0 >9 )+%"52   
  16. What’s kubernetes ? • " " • ! " 

    •  " • 0*-1 • "#&43 •    • !"" • ! 2% • &%     )5 • $)5 • /6,"#)5 Kubernetes! '.+ $(
  17. Kubernetes Architecture Master API UI CLI Node 3 Node 2

    Node 1 KubernetesF=:K MasterNode 1+(1&  • Master /#$;I> ? #"-01*1&AB/#$854G7D  • Node )0!.1 1%'LH3<  1%'/1$,LH Master@C> 1%'E9J62D  Node n Image Registry
  18. Kubernetes Architecture API UI CLI API Server Scheduler Controller-Manager etcd

    Master   API Server kubernetesAPI B<69AREST,CRUD,FIFL5  Scheduler PodNode(,3 $ #.14   Controller-Manager 0$&?W !4-+4)Node[TNQMP!4'*2,1 "/4X1%$7O U@:   etcd 0$&=ECJZ (&$) V!4-+4)API Server>K etcd;S ECMP Y8H 9GRD  
  19. Kubernetes Architecture Node   Kubelet NodeLG Pod&#%247"JM YAML.5B 

    Container RuntimeRD  Pod9?P:NQ8  Container Runtime kubeletRD1%>C$7),;88 Kubernetes$7),37'0> OIE Dockercontainerdrktcri-o$7),37'0@<  Kube-proxy FNode=D Service /%!+A Cluster IPHK  -(+6!Proxy *.5 +iptables@<  Master Kube-proxy Container Runtime Kubelet Node 1,2,3,n Pod Image Registry Pod Pod Pod Pod Pod Pod Pod Pod
  20. Kubernetes Architecture Pod • 1 >3 ,!*21=6 • Pod/ ,."7

    IP)5 • Pod/ ,.Node- ?< • Pod/ ,90 & (5 Container Volume Pod Container Container Pod#'%$ +981:4- ,(! IP ※;  Container 3:  
  21. Kubernetes Architecture Pod Node2 Container Pod Container Pod Node 

       Node1 Node1 Container Pod Container  localhost IP Container Pod Node2 IP
  22. Kubernetes Architecture Deployment & ReplicaSet Master Node 1,2,3,n Deployment .<%(

    replicas:3 Controllers • ReplicationController D RS Kube proxy Container Runtime Kubelet Pod1 Pod2 Pod3 Controller-Manager D RS Deployment ReplicaSet  DeploymentReplicaSet84A/CB?1   ReplicaSet79 Pod)&'52> ,.<%()&'5(replicas)0:   Pod- *'+ !&"#*( $!,  ReplicaSet.< 5Pod=;3 @6   Image Registry
  23. Kubernetes Architecture Service Client Node Service Pod Pod Pod 

     ClusterIP /"$7 Pod5C9>  -/.'IP(2" ExternalIP @6 NodeIP:Port ? '/+%4&)EG =89> NodePort :Node IP:Port ? '/+%4&)EG =89> LoadBalancer LB;D #"FBServiceL43(*/4 <B endpoint Pod Pod Node Node LoadBalancer Service/"$7=0"'Pod#" I1&4DNS<BAH , !' 
  24. Kubernetes Architecture Ingress Ingress *( HTTP(S)  '(Service "$ ""+&

    ,% ")Kubernetes! Service#4(TCP/UDP over IP)Ingress#7(HTTP(S))   Service Pod Internet Pod Pod Ingress endpoint
  25. Kubernetes Architecture Others PersistentVolumeClaim,PersistentVolume,StrageClass,Namespace,Label,Selector,ConfigMap,Secret% kubernetes  "!"  )&($ #'

  26. Introduction of Rancher

  27. What’s Rancher ? • ,"0&.)80#@? =B:6 0#>5&,! %* • '!$"

    0# >5&-+0 &/!.+0@ ? 17=B49 • 32 Rancher Labshttp://rancher.com/ • &0%!(!< ;AC 
  28. Difference between Rancher 1.6 and 2.1   • *14docker-compose-7

    +, '!* • Network ipsec,vxlan# • Storage NFS,EBS# • Architecture Docker" • Java Go5MySQL60 • $( )%,# • Architecture kubernetes" • 100%Go5etcd60 •  )%, 100% kubernetes • *14Helm-7 +,'!* • Network flannel,canal,Calico9kubernetes!& ,:.-73/ • Storage kubernetes2084  :.-73/
  29. Rancher v2.1.0 • Rancher1$ Cattle Orchestrator Rancher2$ Kubernetes  •

    Windows Kubernetes # Rancher 2.1 !Windows Node Kubernetes " http://ur0.link/PjA3 Main outline • Latest - v2.1.5 - rancher/rancher:latest • Stable - v2.1.5 - rancher/rancher:stable Version Rancher Release - v2.1.0 http://urx3.nu/MJA0 Details
  30. Feature of Rancher2.1 Kubernetes Clusters Create Import   

      Bare Metal Create,Import,Manage Kubernetes Clusters !! Manage
  31. Feature of Rancher2.1 One Click Deployment ! !  #-$,

    (%!  !    Creating Private Catalogs ! ! )" '&  +*  
  32. Feature of Rancher2.1 Major developer catalogs

  33. Rancher2.1 Technical Architecture Cluster Controller Auth Proxy Rancher Server RKE

    Nodes Kubectl Kubernetes API Rancher UI CLI API Rancher Component Other Kubernetes Master Legend Rancher API Server etcd Cluster Agent 1 Cluster Agent 2 RKE Kubernetes Master GKE Nodes GKE Kubernetes Master Kubelet Kubelet   a x v x tiy Rd l o b e N H r c b e Re d c Dk AKC e d r b n b I Hus oC Gg eP e d v v S h
  34. Rancher2.1 Deep Dive Documents https://www.slideshare.net/linecorp/lets-unbox-rancher-20-v200

  35. Rancher Labs New Project “Rio” Kubernetes, Istio, Service Mesh like

    Docker Darren Shepherd Co-founder and Chief Architect at Rancher Labs, Inc. @ibuildthecloud https://github.com/rancher/rio https://www.slideshare.net/cyberblackvoom/whats-rio-112779732 Rio!!
  36. Get started with Rancher 2.1 OS Hardware Software Ubuntu 16.04

    (64-bit) Memory: 4GB • 1.12.6 • 1.13.1 • 17.03.2 Red Hat Enterprise Linux 7.5 (64-bit) RancherOS 1.3.0 (64-bit) Step 2: Start the server $ sudo docker run -d --restart=unless-stopped -p 80:80 -p 443:443 rancher/rancher Web Access!! https://<server_ip> Step 1: Prepare a Linux Host
  37. Kubernetes & Rancher Handson

  38. Kubernetes & Rancher Handson 1. Rancher Server 2. Kubernetes Cluster

    3. weavescope   4. RancherUIkubectl  5. Guestbook    rancher-server rancher-host Guestbook Guestbook Rancher Server weavescope Agent Server Qiita URL http://u0u1.net/PnyX
  39. Kubernetes & Rancher Handson 1.Rancher Server 1.Instance2   4/

    rancher-server )'+ asia-northeast1,2 + asia-northeast1-b &+% vCPUx1 $"! Ubuntu 16.04 LTS !80GB #* HTTP"(# 87 HTTPS"(# 87 6.  -  GCP(Google Cloud Platform)30 #* ** 187/5    4/ rancher-host )'+ asia-northeast1,2 + asia-northeast1-b &+% vCPUx1 $"! Ubuntu 16.04 LTS !80GB #* HTTP"(# 87 HTTPS"(# 87 6.  -  1 2
  40. Kubernetes & Rancher Handson 2.Docker 2:rancher-server,rancher-host $ curl https://releases.rancher.com/install-docker/17.03.2.sh |

    sh 3.Rancher2.0 Server (1:rancher-server ) $ sudo docker run -d --restart=unless-stopped -p 80:80 -p 443:443 rancher/rancher:v2.2.0-alpha3 4.  RancherUI rancher-server https://EXTERNAL-IP/ 5. 
  41. 6.Helm Catalogs  Enable Global  Kubernetes & Rancher Handson

  42. Kubernetes & Rancher Handson 1.Add Cluster 2.Kubernetes Cluster

  43. Kubernetes & Rancher Handson 2.CUSTOM Cluster Name  Next E

    C POF , , 2 E C POF SG E C POF G A K N D C D , , , , D C
  44. Kubernetes & Rancher Handson 3.Show advanced options

  45. Kubernetes & Rancher Handson 4. Node Role etcd Control Plane

    Node Address Public Address Internal Address 2 (rancher-host) IP Copy to Clipboard 2 (rancher-host) - Done 1 2 3 5 4
  46. Kubernetes & Rancher Handson   

  47. Kubernetes & Rancher Handson   

  48. Kubernetes & Rancher Handson 1.DefaultCatalog Apps 3.weavescope   

    2.Launch 
  49. Kubernetes & Rancher Handson 3.weave-scopeView Details 

  50. Kubernetes & Rancher Handson 4.Launch

  51. Kubernetes & Rancher Handson 5.  Workloads

  52. Kubernetes & Rancher Handson 6.Load Balancing

  53. Kubernetes & Rancher Handson 7.Add Ingress

  54. Kubernetes & Rancher Handson 9.Name Namespaceweave-scope Targetweave-scope- frontend-weave-scope Port4040 Save

     1 2 3 4 5
  55. Kubernetes & Rancher Handson 10. URL

  56. Kubernetes & Rancher Handson 11.weavescope

  57. Kubernetes & Rancher Handson 1. rancher-kubernetes-cluster 4.RancherUIkubectl   RancherUI

    kubectl    2.Launch kubectl 
  58. Kubernetes & Rancher Handson 3. 

  59. Kubernetes & Rancher Handson 3.kubectl get nodes > kubectl get

    nodes NAME STATUS ROLES AGE VERSION rancher-host Ready controlplane,etcd,worker 39m v1.10.5 > kubectl version Client Version: version.Info{Major:"1", Minor:"10", GitVersion:"v1.10.5", GitCommit:"32ac1c9073b132b8ba18aa830f46b77dcceb0723", GitTreeState:"clean", BuildDate:"2018-06-21T11:46:00Z", GoVersion:"go1.9.3", Compiler:"gc", Platform:"linux/amd64"} Server Version: version.Info{Major:"1", Minor:"12", GitVersion:"v1.12.3", GitCommit:"435f92c719f279a3a67808c80521ea17d5715c66", GitTreeState:"clean", BuildDate:"2018-11-26T12:46:57Z", GoVersion:"go1.10.4", Compiler:"gc", Platform:"linux/amd64"} 4.kubectl version
  60. Kubernetes & Rancher Handson 5.Guestbook 2 .1 Guestbook  yaml

       https://raw.githubusercontent.com/kubernetes/examples/master/guestbook/all-in-one/guestbook-all-in-one.yaml 2 .1 > kubectl apply -f https://raw.githubusercontent.com/kubernetes/examples/master/guestbook/all-in-one/guestbook-all-in-one.yaml service "redis-master" created deployment "redis-master" created service "redis-slave" created deployment "redis-slave" created service "frontend" created deployment "frontend" created > kubectl get services NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE frontend ClusterIP 10.43.52.144 <none> 80/TCP 1m kubernetes ClusterIP 10.43.0.1 <none> 443/TCP 47m redis-master ClusterIP 10.43.31.61 <none> 6379/TCP 1m redis-slave ClusterIP 10.43.55.29 <none> 6379/TCP 1m
  61. Kubernetes & Rancher Handson 3.RancherUIIngress  defaultAdd Ingress  

    2 1
  62. Kubernetes & Rancher Handson 4.NameTargetfrontendPort80Save   1 2 3

    4
  63. Kubernetes & Rancher Handson 5. URL

  64. Kubernetes & Rancher Handson 6.Guestbook 

  65. Kubernetes & Rancher Handson 7.weavescopeGuestbook  

  66. Kubernetes & Rancher Handson 1guestbook-all-in-one.yaml apiVersion: v1 kind: Service metadata:

    name: redis-master labels: app: redis tier: backend role: master spec: ports: - port: 6379 targetPort: 6379 selector: app: redis tier: backend role: master --- apiVersion: apps/v1 # for k8s versions before 1.9.0 use apps/v1beta2 and before 1.8.0 use extensions/v1beta1 kind: Deployment metadata: name: redis-master spec: selector: matchLabels: app: redis role: master tier: backend replicas: 1 template: metadata: labels: app: redis role: master tier: backend spec: containers: - name: master image: k8s.gcr.io/redis:e2e # or just image: redis resources: requests: cpu: 100m memory: 100Mi ports: - containerPort: 6379 --- redis-master
  67. Kubernetes & Rancher Handson 1guestbook-all-in-one.yaml redis-slave apiVersion: v1 kind: Service

    metadata: name: redis-slave labels: app: redis tier: backend role: slave spec: ports: - port: 6379 selector: app: redis tier: backend role: slave --- apiVersion: apps/v1 # for k8s versions before 1.9.0 use apps/v1beta2 and before 1.8.0 use extensions/v1beta1 kind: Deployment metadata: name: redis-slave spec: selector: matchLabels: app: redis role: slave tier: backend replicas: 2 template: metadata: labels: app: redis role: slave tier: backend spec: containers: - name: slave image: gcr.io/google_samples/gb-redisslave:v1 resources: requests: cpu: 100m memory: 100Mi env: - name: GET_HOSTS_FROM value: dns # If your cluster config does not include a dns service, then to # instead access an environment variable to find the master # service's host, comment out the 'value: dns' line above, and # uncomment the line below: # value: env ports: - containerPort: 6379 ---
  68. Kubernetes & Rancher Handson 1guestbook-all-in-one.yaml frontend apiVersion: v1 kind: Service

    metadata: name: frontend labels: app: guestbook tier: frontend spec: # if your cluster supports it, uncomment the following to automatically create # an external load-balanced IP for the frontend service. # type: LoadBalancer ports: - port: 80 selector: app: guestbook tier: frontend --- apiVersion: apps/v1 # for k8s versions before 1.9.0 use apps/v1beta2 and before 1.8.0 use extensions/v1beta1 kind: Deployment metadata: name: frontend spec: selector: matchLabels: app: guestbook tier: frontend replicas: 3 template: metadata: labels: app: guestbook tier: frontend spec: containers: - name: php-redis image: gcr.io/google-samples/gb-frontend:v4 resources: requests: cpu: 100m memory: 100Mi env: - name: GET_HOSTS_FROM value: dns # If your cluster config does not include a dns service, then to # instead access environment variables to find service host # info, comment out the 'value: dns' line above, and uncomment the # line below: # value: env ports: - containerPort: 80
  69. Kubernetes & Rancher Handson 2Guestbook  

  70. Rancher 2.2 Preview

  71. Rancher 2.2 Preview Rancher’s New Mluti-Tenant Prometheus Support

  72. Rancher 2.2 Preview PrometheusGoogleBorgmon!5'-! 8PullKHTTP1,3#'HI &'*0KubernetesLBA9 CN Service Discovery@EHI:J6? ;7

       GrafanaPrometheus>MHI+( 9<GI=)4 Web/2"%$2.= +(3 4 (!0FD  Rancher’s New Mluti-Tenant Prometheus Support
  73. Rancher 2.2 Preview    PrometheusGrafana   Deploy

    from Catalog function
  74. Rancher 2.2 Preview http://ur0.link/PjFf Rancher Official Blog

  75. Rancher 2.2 Preview Rancher 2.2 simplifies and cuts this process

    down to just 2 steps
  76. Rancher 2.2 Preview Rancher Official Blog http://ur0.link/PjHe

  77. 1.  Tools-Monitoring Prometheus Rancher 2.2 Preview

  78. 2.Prometheus  Save   1 2 Rancher 2.2 Preview

  79. 3.   Prometheus + Grafana  Rancher 2.2 Preview

  80. 4.        Rancher 2.2

    Preview
  81. 5.Grafana   Grafana  Rancher 2.2 Preview

  82. Nodes Cluster 6.Launch kubectl Rancher 2.2 Preview

  83. 7.kubectl get pods --all-namespaces  # Run kubectl commands inside

    here # e.g. kubectl get all > kubectl get pods --all-namespaces NAMESPACE NAME READY STATUS RESTARTS AGE cattle-prometheus exporter-kube-state-cluster-monitoring-779b96cf75-gg272 1/1 Running 0 20m cattle-prometheus exporter-node-cluster-monitoring-ljhch 1/1 Running 0 20m cattle-prometheus grafana-cluster-monitoring-5b4bc74756-hx7mx 3/3 Running 0 20m cattle-prometheus prometheus-cluster-monitoring-0 4/4 Running 1 20m cattle-prometheus prometheus-cluster-monitoring-nginx-5ff785dd4d-d6l8l 1/1 Running 3 20m cattle-prometheus prometheus-operator-system-monitor-77df6db6df-gdmmz 1/1 Running 0 20m cattle-system cattle-cluster-agent-77498f79c-5w78s 1/1 Running 0 30m cattle-system cattle-node-agent-s8gnz 1/1 Running 0 30m ingress-nginx default-http-backend-5bdd9fdd69-2ntmw 1/1 Running 0 30m ingress-nginx nginx-ingress-controller-xdwnk 1/1 Running 0 30m kube-system canal-wzw4r 3/3 Running 0 30m kube-system kube-dns-autoscaler-689f6f9756-zqn6f 1/1 Running 0 30m kube-system kube-dns-ddddcfcc8-xbpwn 3/3 Running 0 30m kube-system metrics-server-5444cf6dfc-g7xvp 1/1 Running 0 30m kube-system rke-ingress-controller-deploy-job-bshtx 0/1 Completed 0 30m kube-system rke-kubedns-addon-deploy-job-k9vxg 0/1 Completed 0 30m kube-system rke-metrics-addon-deploy-job-hmr7k 0/1 Completed 0 30m kube-system rke-network-plugin-deploy-job-tjqhn 0/1 Completed 0 31m Rancher 2.2 Preview
  84. http://ur0.link/PjJn • December 2018 Online Meetup Introducing Rancher's New Multi

    Tenant Prometheus Support Webinar • Rancher v2.2.0-Alpha3 Multi-Tenant Prometheus Support Docs http://ur0.link/PjK6 Rancher 2.2 Preview
  85. Information

  86. Information Docker  $;&:   ISBN-10: 4844339621 ISBN-13: 978-4844339625

    Docker,"  439#0)<?  81765(. 2019/2/18+-;'* Windows $;&:   ISBN-10: 4295000957 ISBN-13: 978-4295000952 Windows   = / >%!"2/: 
  87. Information c : ISBN-10: 4798153222 ISBN-13: 978-4798153223 L L L

    Docker D ――Linux c : ISBN-10: 4774176540 ISBN-13: 978-4774176543 LD e LD
  88. Information kubernetes ?V@: &8$9*62; ISBN-10: 4873118409 ISBN-13: 978-4873118406 MUKubernetes Up

    & Running<=QXXAG !EM#I  <=F HS "!  / D ?V@: [W@ ISBN-10: 4798155373 ISBN-13: 978-4798155371 Google,RedHat,IBM, $;,1-/,Rancher#>J(;.0TC &'+/ :)7;#O! !NL 48-/3%5#YRDZP BK 
  89. Information The Kubernetes Book ,: Independently published ISBN-10: 1521823634 ISBN-13:

    978-1521823637 kubernetes )$(+ *% "'  Kubernetes in Action ,: Manning Pubns Co ISBN-10: 1617293725 ISBN-13: 978-1617293726 "'#-!   *&%  (+ 
  90. Information Docker/Kubernetes ! +8,:5674, ISBN-10: 4297100339 ISBN-13: 978-4297100339 Docker/Kubernetes1:)!"%'&).-2/ (9;

      )!".-#$(* 30       https://cyberblack28.hatenablog.com/
  91. Information Kubernetes 8L::7+5 ISBN-10: 4295004804 ISBN-13: 978-4295004806 C1M KubernetesFB 

    C2M KubernetesIH =KP C3M API3 kubectl C4M Workloads3 C5M Discovery & LB3 C6M Config & Storage3 C7M Cluster3 Metadata3 C8M 3G? $37 C9M ,4!" 7#& 2*4 C10M .7#&7 (% JD C11M ;< NO 137 C12M 13# C13M -'*$ Q@>9+7*$ C14M /' 37 C15M 7#&6 EA C16M CI/CDIH C17M -6) ServiceMesh C18M Kubernetes #!0 C19M Kubernetes 
  92. Information    Kubernetes Azure   :!:=<! ISBN-10:

    4798157848 ISBN-13: 978-4798157849 1  ,1; Kubernetes ,2; Kubernetes20/9 ,3; Kubernetes &  2  ,4; Kubernetes(+ ,5;  # ,6;  3  ,7; *-'7 ,8; .%)Availability ,9; 61)Scalability ,10; $3)Manageability ,11; "4Security ,12; .58)Observability
  93. Information Certified Kubernetes Administrator(CKA) ,/,. a C v kp l

    gh Y uc W ( kp l gh $$YZ W 1 no Ytcba Z rsN kp l gh 1 moei w U o l g 9CC 032EE      9CC FFF 6 4C: : 6 4 ) Certified Kubernetes Application Developer(CKAD)
  94. Information Kubernetes&Docker KD100  https://www.mirantis.co.jp/training/ Kubernetes&Docker KD200 MIRANTIS kubernetes 

    
  95. Information  https://www.katacoda.com/ Web # !   Docker,Kubernetes,Istio,Prometheus% &"$

     
  96. Information  https://labs.play-with-docker.com/ Play with Docker https://labs.play-with-k8s.com/ Play with Kubernetes

  97. Information  Kubernetes https://thinkit.co.jp/author/13290 cndjp   https://speakerdeck.com/hhiroshell  

  98. Information        http://ur0.work/NQqi 

  99. ! !