わりとゴツいKubernetesハンズオン そのあとに

ふくばねてす node-2

Transcript

1. ΘΓͱΰπ͍,VCFSOFUFTϋϯζΦϯ
   ͦͷ͋ͱʹ 
 ;͘͹Ͷͯ͢
   OPEF ໟར
   ܒଠ
2. ࣗݾ঺հ ‣ ໟར
   ܒଠ
    'VTJD
   $P -UE
   
   
   
   
   
   
   
   
   
   
   
   
   
   
   
   
   
   
   
   
   ൃҊऀ
   
   ϓϩμΫτΦʔφʔ LVCFSOFUFT࢖ͬͯͳ͍
    ,VCFSOFUFT͸׬શʹཧղ͍ͯ͠Δ 2
3. 3 ΘΓͱΰπ͍,VCFSOFUFTϋϯζΦϯ
4. ΘΓͱΰπ͍΍ͭ ‣ ΘΓͱόζΓ·ͨ͠ʂ͋Γ͕ͱ͏͍͟͝·͢ 4 https://qiita.com/Kta-M/items/ce475c0063d3d3f36d5d
5. ΘΓͱΰπ͍΍ͭ ‣ ͜Μͳ໨ඪͰ΍͍ͬͯ·ͨ͠ 5
6. ΘΓͱΰπ͍΍ͭ ‣ ओཁͳཁૉͷઆ໌͸͍ͯ͠Δ͸ͣ
    ,VCFSOFUFTͷ֓ཁ
    αϯϓϧΞϓϦͷσϓϩΠ
    ֤छϦιʔεͷઆ໌
    &,4 FLTDUM ͰͷΫϥελߏங
    μογϡϘʔυ
    ϩάऩू
    )FMN
    ؂ࢹ 1SPNFUIFVT
   (SBGBOB 6
7. ΘΓͱΰπ͍΍ͭ ‣ Ͱ΋ɺطଘͷ%PDLFSΠϝʔδ͔͠࢖ͬͯͳ͍ʜ Kubernetes Cluster Manifest File Container Registry ׬શʹཧղͨ͠ൣғ
8. ΘΓͱΰπ͍΍ͭ ‣ ͓෼͔Γ͍͚ͨͩͨͩΖ͏͔ʁ 8 Ͱ͖ΔΑ͏ʹͳΔͱ͸ݴͬͯͳ͍
9. ΦϦδφϧͷΞϓϦΛߏங͢Δʹ͸ ‣ ϩʔΧϧͷ։ൃϑϩʔΛ࡞Βͳ͖Ό 9 Kubernetes Cluster Manifest File Application Source Code Dockerfile Container Registry 1. ίʔυΛฤू 3.σϓϩΠ 2. Docker ImageΛBuild
10. ΦϦδφϧͷΞϓϦΛߏங͢Δʹ͸ ‣ $*$%؀ڥ΋੔͑ͳ͖Ό 10 Kubernetes Cluster Manifest File Application Source Code Dockerfile Container Registry 1. ίʔυΛฤू 2. Git Push 3. Docker ImageΛBuild 4. Docker ImageΛPush 5.σϓϩΠ
11. Ұؾʹ໘౗͘͘͞ͳ͖ͬͯͨ 11
12. ؆୯ʹͰ͖ΔΜͰ͢
    ͦ͏ɺ4LBGGPMEͳΒͶ 12
13. 13 4LBGGPMEͱ͸
14. 4LBGGPMEͱ͸ ‣ https://skaffold.dev/ ‣ (PPHMFۘ੡ͷ,VCFSOFUFTͷ։ൃࢧԉπʔϧ
    ‣ ΫϥελʹΞϓϦέʔγϣϯΛσϓϩΠ͢Δ·Ͱͷ
 Ұ࿈ͷ࡞ۀΛ ϩʔΧϧϦϞʔτ໰Θͣ ࣗಈԽͯ͘͠ΕΔ
     %PDLFSΠϝʔδͷCVJME
     %PDLFSΠϝʔδͷߏ଄ςετ
     %PDLFSΠϝʔδ΁ͷλά෇͚
     ίϯςφϨδετϦʹ%PDLFSΠϝʔδΛ1VTI
     ,VCFSOFUFTΫϥελʹ%FQMPZ 14
15. 4LBGGPMEͱ͸ ‣ ֤ϑΣʔζ͸ಠཱ͍ͯͯ͠ɺ͍Ζ͍Ζͳ૊Έ߹Θ͕ͤՄೳ 15
16. ͱΓ͋͑ͣ΍ͬͯΈΑ͏ 16
17. 17 4LBGGPMEΛ
 ϩʔΧϧ։ൃͰ࢖ͬͯΈΑ͏
18. ͜ΕΛ΍Γ͍ͨ 18 Kubernetes Cluster Manifest File Application Source Code Dockerfile Container Registry 1. ίʔυΛฤू 3.σϓϩΠ 2. Docker ImageΛBuild
19. ϩʔΧϧͷ
 ,VCFSOFUFT؀ڥΛ४උ͢Δ 19
20. ,VCFSOFUFT༗ޮԽ 20 1SFGFSFODFTը໘ 
    ,VCFSOFUFTλϒΛબ୒ 
    νΣοΫʂ 
    "QQMZ ॳճ͸༗ޮʹͳΔ·Ͱ
 ෼΄Ͳ͔͔Γ·͢
21. ,VCFSOFUFT༗ޮԽ 21 Ҏલ࢖ͬͨ͜ͱ͋Δਓ͸Ұ୴Ϧηοτ͓ͯ͘͠ͱ͍͍͔΋ 
    3FTFUλϒΛબ୒ Ϧηοτ
22. LVCFDUMͷDPOpHઃఆ 22 # kubectlͰૢ࡞͢ΔKubernetesΫϥελʔΛdocker-for-desktopʹ $ kubectl config use-context docker-for-desktop # ͪΌΜͱઃఆ͞Ε͍ͯΔ͔֬ೝ $ kubectl config current-context docker-for-desktop
23. ಈ࡞֬ೝͱԼ४උ 23 # ಈ࡞֬ೝɻͳΜ͔͍Ζ͍Ζಈ͍ͯΔ(આ໌͸͋ͱͰ) $ kubectl get pods --namespace=kube-system NAMESPACE NAME READY STATUS RESTARTS AGE kube-system etcd-docker-for-desktop 1/1 Running 0 1m kube-system kube-apiserver-docker-for-desktop 1/1 Running 0 1m kube-system kube-controller-manager-docker-for-desktop 1/1 Running 0 1m kube-system kube-dns-86f4d74b45-xb4qh 3/3 Running 0 2m kube-system kube-proxy-8r45p 1/1 Running 0 2m kube-system kube-scheduler-docker-for-desktop 1/1 Running 0 1m # docker-for-desktopͰingress͕࢖͑ΔΑ͏Լ४උɻ͋·Γؾʹ͠ͳͯ͘OK $ kubectl create namespace ingress-nginx $ cat << EOF > kustomization.yaml apiVersion: kustomize.config.k8s.io/v1beta1 kind: Kustomization namespace: ingress-nginx bases: - github.com/kubernetes/ingress-nginx/deploy/cluster-wide - github.com/kubernetes/ingress-nginx/deploy/cloud-generic EOF $ kubectl apply -k .
24. αϯϓϧΞϓϦέʔγϣϯΛ
 ४උ͢Δ 24
25. αϯϓϧΞϓϦέʔγϣϯऔಘ 25 # kubernetes/examples͔ΒguestbookΞϓϦΛऔͬͯ͘Δ $ git clone git@github.com:kubernetes/examples.git $ mkdir skaffold_example $ cp -R examples/guestbook/* skaffold_example $ cd skaffold_example # ෆཁͳϑΝΠϧΛ࡟আ $ rm -rf all-in-one $ rm -rf legacy # PCͷෛ୲ΛԼ͛ΔͨΊͪΐͬͱௐ੔ & ΠϝʔδͷऔಘઌΛมߋ $ vi examples/guestbook/frontend-deployment.yaml 10ߦ໨ replicas: 3 <- ͜ΕΛ1ʹมߋ 19ߦ໨ image: gcr.io/google-samples/gb-frontend:v4 <- ͜ΕΛ ${dockerhubͷϢʔβʔ໊}/skaffold- example-frontendʹมߋ $ vi examples/guestbook/redis-slave-deployment.yaml 11ߦ໨ replicas: 2 <- ͜ΕΛ1ʹมߋ 21ߦ໨ image: gcr.io/google_samples/gb-redisslave:v1 <- ͜ΕΛ ${dockerhubͷϢʔβʔ໊}/skaffold- example-redis-slaveʹมߋ
26. JOHSFTTͷϚχϑΣετΛ࡞੒ 26 $ cat << 'EOT' >./guestbook-ingress.yaml apiVersion: extensions/v1beta1 kind: Ingress metadata: name: guestbook-ingress spec: rules: - http: paths: - path: / backend: serviceName: frontend servicePort: 80 EOT
27. 4LBGGPME
    Πϯετʔϧ
    
    ηοτΞοϓ 27
28. 4LBGGPMEͷΠϯετʔϧ ‣ https://skaffold.dev/docs/getting-started/#installing-skaffold ‣ NBDͳΒCSFXͰೖΔ
     CSFX
    JOTUBMM
    TLBGGPME 28
29. 4LBGGPMEͷઃఆϑΝΠϧΛ࡞੒ 29 apiVersion: skaffold/v1beta11 kind: Config # DockerΠϝʔδͷϏϧυʹؔ͢Δઃఆ build: artifacts: # Ϗϧυ͢ΔΠϝʔδ - image: ${dockerhubͷϢʔβʔ໊}/skaffold-example-frontend context: php-redisɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹ # ιʔεϑΝΠϧ͕͋Δύεͷࢦఆ - image: ${dockerhubͷϢʔβʔ໊}/skaffold-example-redis-slave context: redis-slave local: # ϩʔΧϧͰΠϝʔδΛϏϧυ͢Δͱ͖ͷઃఆ push: false # Ϗϧυͨ͠ΠϝʔδΛpush͢Δ͔ # σϓϩΠʹؔ͢Δઃఆ deploy: kubectl: # σϓϩΠର৅ͷϚχϑΣετϑΝΠϧͷࢦఆ manifests: - frontend-*.yaml - redis-master-*.yaml - redis-slave-*.yaml - guestbook-ingress.yaml ͦͷଞύϥϝʔλʹ͍ͭͯ͸
    https://skaffold.dev/docs/references/yaml/
    Λࢀর skaffold.yaml
30. TLBGGPME
    EFW 30
31. TLBGGPME
    EFW 31 $ skaffold dev Generating tags... - mohri1219/skaffold-example-frontend -> mohri1219/skaffold-example-frontend:76a6cd0-dirty - mohri1219/skaffold-example-redis-slave -> mohri1219/skaffold-example-redis-slave:76a6cd0 Tags generated in 49.700092ms Starting build... Found [docker-for-desktop] context, using local docker daemon. Building [mohri1219/skaffold-example-frontend]... Sending build context to Docker daemon 7.68kB Step 1/8 : FROM php:5-apache ---> 24c791995c1e (தུ) Starting deploy... kubectl client version: 1.14 deployment.apps/frontend created service/frontend created ingress.extensions/guestbook-ingress created deployment.apps/redis-master created service/redis-master created deployment.apps/redis-slave created service/redis-slave created (தུ) Watching for changes every 1s... [frontend-7df8f8b7f8-mscxk php-redis] AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 10.1.1.154. Set the 'ServerName' directive globally to suppress this message ΠϝʔδͷCVJME σϓϩΠ ϑΝΠϧมߋ଴ػ
    
    ϩάग़ྗ
32. Ͱ͖ͨʂ 32 IUUQMPDBMIPTU
    1$ͷঢ়گʹΑͬͯ͸͏·͘ΞΫηεͰ͖ͳ͔ͬͨΓ͢Δ͔΋
    IUUQͩͱ͏·͍͘͘৔߹΋͋Δͱ͔
33. ͓΋ΉΖʹ1)1ͷϑΝΠϧΛม͑ͯΈΔ 33 # ద౰ʹλΠτϧΛม͑ͯΈΔ 11ߦ໨ <h2>Guestbook!!</h2> <- "!!"Λ௥Ճͯ͠ΈΔ Generating tags... - mohri1219/skaffold-example-frontend -> mohri1219/skaffold-example-frontend:76a6cd0-dirty Tags generated in 62.953504ms Starting build... Found [docker-for-desktop] context, using local docker daemon. Building [mohri1219/skaffold-example-frontend]... Sending build context to Docker daemon 7.68kB Step 1/8 : FROM php:5-apache ---> 24c791995c1e Step 2/8 : RUN pear channel-discover pear.nrk.io ---> Using cache ---> bd9823d8109d Step 3/8 : RUN pear install nrk/Predis ---> Using cache ---> 390dd33a211f ϑΝΠϧͷมߋΛࣗಈݕ஌ͯ͠ߋ৽ͯ͘͠ΕΔʂ php-redis/index.html
34. มΘͬͨʂ 34
35. $USM $Ͱऴྃ͢Δ 35 Pruning images... untagged image mohri1219/skaffold-example-frontend: 3eba4d84484d6648b4a0907db8f472e5df4439d9ba4ca012a5b96f95af4dd09b untagged image mohri1219/skaffold-example-frontend:c4971ec-dirty untagged image mohri1219/skaffold-example- frontend@sha256:c2f8eb06b6a9d0624c1f8ea6cba4dfaefa450f11d99c90a209617618b0687730 deleted image sha256:3eba4d84484d6648b4a0907db8f472e5df4439d9ba4ca012a5b96f95af4dd09b deleted image sha256:42794d3d5de90049ae108429e1536b747b145918b9972edd62e958b58b09c1ec WARN[0290] builder cleanup: pruning images: Error response from daemon: conflict: unable to delete 48d1ae3a57c7 (cannot be forced) - image is being used by running container 0b0107ee518b Cleaning up... deployment.apps "frontend" deleted service "frontend" deleted ingress.extensions "guestbook-ingress" deleted deployment.apps "redis-master" deleted service "redis-master" deleted deployment.apps "redis-slave" deleted service "redis-slave" deleted Cleanup complete in 3.70313407s ͖Ε͍ʹย෇͚ͯ͘ΕΔʂ Πϝʔδͷ࡟আ Ϋϥελ͔Β࡟আ
36. TLBGGPME
    SVO 36
37. TLBGGPME
    SVO 37 $ skaffold run Generating tags... - mohri1219/skaffold-example-frontend -> mohri1219/skaffold-example-frontend:76a6cd0-dirty - mohri1219/skaffold-example-redis-slave -> mohri1219/skaffold-example-redis-slave:76a6cd0 Tags generated in 48.802686ms Starting build... Found [docker-for-desktop] context, using local docker daemon. Building [mohri1219/skaffold-example-frontend]... Sending build context to Docker daemon 7.68kB Step 1/8 : FROM php:5-apache ---> 24c791995c1e (தུ) Starting deploy... kubectl client version: 1.14 deployment.apps/frontend created service/frontend created ingress.extensions/guestbook-ingress created deployment.apps/redis-master created service/redis-master created deployment.apps/redis-slave created service/redis-slave created Deploy complete in 1.763257066s ΠϝʔδͷCVJME σϓϩΠ ୯ൃͷσϓϩΠɻ׬ྃͯ͠΋Ϋϥελʹ࢒Γଓ͚Δɻ
38. TLBGGPMEͰςετ΋ͯ͠ΈΔ 38
39. DPOUBJOFSTUSVDUVSFUFTU ‣ https://github.com/GoogleContainerTools/container-structure-test ‣ ίϯςφΠϝʔδͷߏ଄Λςετ
     Πϝʔδ಺ͷίϚϯυͷग़ྗΛνΣοΫ
     ϑΝΠϧγεςϜͷϝλσʔλͱ಺༰Λݕূ
     FUD 39
40. ςετϑΝΠϧΛઃఆ 40 schemaVersion: 2.0.0 # ֤छϑΝΠϧͷଘࡏ֬ೝ fileExistenceTests: - name: 'php file' path: '/var/www/html/guestbook.php' shouldExist: true - name: 'js file' path: '/var/www/html/controllers.js' shouldExist: true - name: 'html file' path: '/var/www/html/index.html' shouldExist: true ͦͷଞͷςετ߲໨ʹ͍ͭͯ͸
    https://github.com/GoogleContainerTools/container-structure-test/blob/master/README.md
    Λࢀর structure-test/frontend.yaml
41. 4LBGGPMEͷઃఆϑΝΠϧΛߋ৽ 41 apiVersion: skaffold/v1beta11 kind: Config build: (ུ) # ςετͷઃఆ test: test: - image: ${dockerhubͷϢʔβʔ໊}/skaffold-example-frontend structureTests: - ./structure-test/frontend.yaml deploy: (ུ) skaffold.yaml
42. TLBGGPME
    SVO 42 $ skaffold run (தུ) Starting test... ====================================== ====== Test file: frontend.yaml ====== ====================================== === RUN: File Existence Test: php file --- PASS === RUN: File Existence Test: js file --- PASS === RUN: File Existence Test: html file --- PASS ======================================= =============== RESULTS =============== ======================================= Passes: 3 Failures: 0 Total tests: 3 PASS Test complete in 1.455478421s ΋ͪΖΜςετ͕௨Βͳ͔ͬͨΒσϓϩΠ͞Εͳ͍
43. ϩʔΧϧͰ༡Ϳͷ͸͜͜·Ͱ✋ 43
44. ,VCFSOFUFTແޮԽ 44 1SFGFSFODFTը໘ 
    ,VCFSOFUFTλϒΛબ୒ 
    νΣοΫΛ֎͢ 
    "QQMZ 
    3FTFUλϒΛબ୒ ҰԠϦηοτ
45. 45 4LBGGPMEΛ
 $*$%޲͚ʹ࢖ͬͯΈΑ͏ʂ
46. ͜ΕΛ΍Γ͍ͨ 46 Kubernetes Cluster Manifest File Application Source Code Dockerfile Container Registry 1. ίʔυΛฤू 2. Git Push 3. Docker ImageΛBuild 4. Docker ImageΛPush 5.σϓϩΠ ׬੒൛͸ͪ͜Β
    
    https://github.com/Kta-M/skaffold_example
47. ΍Δ͜ͱ ‣ (JU)VCͷϦϙδτϦΛ࡞੒
    লུ
    ‣ %PDLFS)VCͷϦϙδτϦΛ࡞੒
     \EPDLFSIVCͷϢʔβʔ໊^TLBGGPMEFYBNQMFGSPOUFOE
     \EPDLFSIVCͷϢʔβʔ໊^TLBGGPMEFYBNQMFSFEJTTMBWF
    ‣ &,4ͰΫϥελΛ࡞੒
    ‣ 4LBGGPMEͷઃఆϑΝΠϧΛௐ੔
    ‣ $JSDMF$*ͷઃఆ
    ‣ $JSDMF$*ͷDPOpHΛ࡞੒
    ‣ (JUIVCʹ1VTI 47
48. %PDLFS)VCͷϦϙδτϦΛ࡞੒ 48
49. &,4ͰΫϥελΛ࡞੒ ‣ ʮΘΓͱΰπ͍ϋϯζΦϯʯͰ΍ͬͨ΍ͭ 49 $ eksctl create cluster \ --name eksctl-handson \ --region ap-northeast-1 \ --nodes 3 \ --nodes-min 3 \ --nodes-max 3 \ --node-type t2.medium \ --ssh-public-key <ΩʔϖΞ໊> ˞ߏங׬ྃ·Ͱ෼͙Β͍͔͔Γ·͢ʜ
    ˞BQOPSUIFBTUC͕બ୒Ͱ͖Δݹ͍"84ΞΧ΢ϯτ͸ɺ";ࢦఆ΋ඞཁ
50. 4LBGGPMEͷઃఆϑΝΠϧΛௐ੔ ‣ ϩʔΧϧͱڍಈΛม͍͑ͨͱ͜Ζ
     ΠϝʔδΛίϯςφϨδετϦʹ1VTI͢Δ
     JOHSFTTͷϚχϑΣετϑΝΠϧ͸࢖Θͳ͍
    ‣ QSPpMFΛ࢖͓͏
     ༩͑ΒΕͨQSPpMF໊ʹԠͯ͡ઃఆΛΦʔόʔϥΠυͯ͘͠ΕΔػೳ 50
51. 4LBGGPMEͷઃఆϑΝΠϧΛௐ੔ 51 apiVersion: skaffold/v1beta11 kind: Config (தུ) # ҎԼΛ௥Ճ profiles: - name: prd build: local: push: true # Ϗϧυͨ͠ΠϝʔδΛίϯςφϨδετϦʹPush͢Δ deploy: kubectl: manifests: # ingressͷϑΝΠϧΛআ֎ - frontend-*.yaml - redis-master-*.yaml - redis-slave-*.yaml TLBGGPME
    SVO
    QSPpMF
    QSE
    ͱ͢Ε͹ɺQSEͷઃఆͰΦʔόʔϥΠυͯ͘͠ΕΔ skaffold.yml
52. $JSDMF$*ͷઃఆ ‣ (JUIVCͷϦϙδτϦΛొ࿥ 52
53. $JSDMF$*ͷઃఆ 53
54. $JSDMF$*ͷઃఆ ‣ ϓϩδΣΫτ؀ڥม਺Λઃఆ 54 "84@%&'"6-5@3&(*0/
    $645&3@/".&
    %0$,&3@)6#@148
    %0$,&3@)6#@643 
    BQOPSUIFBTU
    
    FLTDUMIBOETPO
    
    %PDLFS)VCͷύεϫʔυΛCBTFΤϯίʔυͨ͠΋ͷ
    
    %PDLFS)VCͷϢʔβʔ໊
55. $JSDMF$*ͷઃఆ ‣ "84ͷೝূ৘ใΛઃఆ 55 &,4ΫϥελΛཱͯΔͷʹ࢖ͬͨ*".Ϣʔβʔͱಉ͡΋ͷΛઃఆ͢Δ
 PSCΛ࢖͑ͱݴΘΕͯΔ͚ͲɺόʔδϣϯͰ͔͠࢖͑ͳ͍ʜɻ͸ϩʔΧϧͰͷಈ࡞֬ೝ͕Ͱ͖ͳ͍ͷͰͪ͜ΒͰ
56. $JSDMF$*ͷDPOpHΛ࡞੒ 56 version: 2 jobs: build: docker: - image: docker:18.09 steps: - run: name: set shell and install tools command: (ུ) - checkout - setup_remote_docker - run: name: install kubectl command: (ུ) - run: (kubectlͷcontextʹEKSΫϥελΛઃఆ͢ΔͨΊͷ΋Ζ΋Ζ) - run: name: install aws-iam-authenticator command: (ུ) - run: name: login to dockerhub command: echo $DOCKER_HUB_PSW | base64 -d | docker login -u $DOCKER_HUB_USR --password-stdin - run: name: install skaffold command: (ུ) - run: name: skaffold run command: skaffold run --profile prd workflows: version: 2 workflow: jobs: - build ԿΛ͍ͯ͠Δ͔ͷઆ໌ͷͨΊɺ֤छΠϯετʔϧ΋$JSDMF$*Ͱ΍͍ͬͯΔ
    ຊདྷ͸΋Ζ΋ΖΠϯετʔϧͨ͠%PDLFS*NBHFΛ࡞Δ΄͏͕͍͍ͱࢥ͏ .circleci/config.yml
57. (JUIVCʹ1VTI͢Δͱʜ 57
58. $ kubectl get svc NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE service/frontend LoadBalancer 10.100.61.11 xxxxxx.ap-northeast-1.elb.amazonaws.com 80:31673/TCP 8s service/kubernetes ClusterIP 10.100.0.1 <none> 443/TCP 23m service/redis-master ClusterIP 10.100.137.217 <none> 6379/TCP 7s service/redis-slave ClusterIP 10.100.217.57 <none> 6379/TCP 7s Ͱ͖ͨʂ 58 http://xxxxxx.ap-northeast-1.elb.amazonaws.com
    ʹΞΫηεʂ
59. 59 ·ͱΊ
60. ·ͱΊ ‣ ʮΘΓͱΰπ͍΍ͭʯ͔ΒҰาਐΜͰɺ
 ,VCFSOFUFTΛ࢖ͬͨ։ൃͷ۩ମతͳΠϝʔδ͕
 ௫ΊͨͷͰ͋Ε͹޾͍Ͱ͢
    ‣ ,VCFSOFUFTপʹ଍Λ౿ΈೖΕͨ͹͔Γ
     ࠓճ঺հ͖͠Εͳ͔ͬͨ༻ޠɺΤίγεςϜͳͲ·ͩ๲େʹʜ
    ‣ Զͨͪͷਅͷઓ͍͸͜Ε͔Βͩʂ 60