Cloud Native Storage Meetup #1 Longhorn

Transcript

1. © Copyright 2021 Rancher. All Rights Reserved. 1
   © Copyright 2021 Rancher. All Rights Reserved. 1
   Cloud Native Storage Meetup #1
   Longhorn
   17th Sep 2021
   Wenhan Shi
   Support Engineer
   

   View Slide

2. © Copyright 2021 Rancher. All Rights Reserved. 2
   ⾃⼰紹介
   • 施 ⽂翰 （シ ブンカン） Wenhan Shi
   • 経歴
   • ⽇⽴製作所 - Linux カーネルモジュールの保守サポート
   • Red Hat K.K. - GlusterFS/OpenShift サポート
   • Canonical Japan K.K. - Ubuntu/OpenStack/Kubernetes サポート
   • Rancher Lab -> SUSE - Rancher及び周辺製品のサポート
   @shi_wenhan
   wenhan.shi@suse.com
   

   View Slide

3. © Copyright 2021 Rancher. All Rights Reserved. 3
   Open Source
   Distributed Block Storage Software for Kubernetes
   https://longhorn.io/
   

   View Slide

4. © Copyright 2021 Rancher. All Rights Reserved. 4
   Longhorn Web UI
   

   View Slide

5. © Copyright 2021 Rancher. All Rights Reserved. 5 5
   • volume rebuildの仕組み
   • v1.2.0の紹介
   

   View Slide

6. © Copyright 2021 Rancher. All Rights Reserved. 6 6
   • volume rebuildの仕組み
   • v1.2.0の紹介
   

   View Slide

7. © Copyright 2021 Rancher. All Rights Reserved. 7
   Longhorn Web UI
   • Volumeの各コピーのStatusが確認できる
   • データは各ノードの /var/lib/longhorn に保存
   

   View Slide

8. © Copyright 2021 Rancher. All Rights Reserved. 8
   アーキテクチャー
   

   View Slide

9. © Copyright 2021 Rancher. All Rights Reserved. 9
   アーキテクチャー
   Engine
   (Data Plane)
   Manager
   (Control Plane)
   

   View Slide

10. © Copyright 2021 Rancher. All Rights Reserved. 10
    アーキテクチャー
    Worker
    node
    Longhorn
    node
    Longhorn
    node
    Pod
    Engine
    manager pod
    Pod
    Replica
    Manager Pod
    Replica
    Manager Pod
    Engine
    Process
    Replica
    Process
    Engine
    Process
    Replica
    Process
    Ext4/xfs Ext4/xfs
    Replica
    Process
    Ext4/xfs
    Replica
    Process
    Ext4/xfs
    PVC/PV
    Volume
    PVC/PV
    Volume
    Longhorn
    Manager
    (DS)
    Longhorn
    Manager
    (DS)
    Longhorn
    Manager
    (DS)
    

    View Slide

11. © Copyright 2021 Rancher. All Rights Reserved. 11
    エラーが突然やってくる
    msg="pvc-c2a86740-04cb-4982-9d41-f86e07570d5e-e-254bed26:
    time="2021-08-22T22:13:30Z" level=error msg="Setting replica
    tcp://10.42.6.168:10075 to ERR due to: r/w timeout""
    

    View Slide

12. © Copyright 2021 Rancher. All Rights Reserved. 12
    アーキテクチャー
    Worker
    node
    Longhorn
    node
    Longhorn
    node
    Pod
    Engine
    manager pod
    Pod
    Replica
    Manager Pod
    Replica
    Manager Pod
    Engine
    Process
    Replica
    Process
    Engine
    Process
    Replica
    Process
    Ext4/xfs Ext4/xfs
    Replica
    Process
    Ext4/xfs
    Replica
    Process
    Ext4/xfs
    PVC/PV
    Volume
    PVC/PV
    Volume
    Longhorn
    Manager
    (DS)
    Longhorn
    Manager
    (DS)
    Longhorn
    Manager
    (DS)
    msg="Prepare to delete
    instance pvc-c2a86740-
    04cb-4982-9d41-
    f86e07570d5e-r-cd767c21"
    

    View Slide

13. © Copyright 2021 Rancher. All Rights Reserved. 13
    その後は
    • ディスク上のDataはまだ利用可能
    • 新しいプロセスを作成
    • 差分だけコピー
    • dead -> online
    Longhorn
    node
    Replica
    Manager Pod
    Replica
    Process
    Replica
    Process
    Ext4/xfs Ext4/xfs
    Longhorn
    Manager
    (DS)
    Worker
    node
    Pod
    Engine
    manager pod
    Pod
    Engine
    Process
    Engine
    Process
    PVC/PV
    Volume
    PVC/PV
    Volume
    Longhorn
    Manager
    (DS)
    Replica
    Process
    

    View Slide

14. © Copyright 2021 Rancher. All Rights Reserved. 14
    その後は
    • replica-replenishment-wait-interval (600s)
    後でも回復不可
    • 新しいプロセスを作成
    • 健康状態のレプリカからフルコピー
    • dead -> online
    Longhorn
    node
    Replica
    Manager Pod
    Replica
    Process
    Replica
    Process
    Ext4/xfs Ext4/xfs
    Longhorn
    Manager
    (DS)
    Worker
    node
    Pod
    Engine
    manager pod
    Pod
    Engine
    Process
    Engine
    Process
    PVC/PV
    Volume
    PVC/PV
    Volume
    Longhorn
    Manager
    (DS)
    Replica
    Process
    Ext4/xfs
    

    View Slide

15. © Copyright 2021 Rancher. All Rights Reserved. 15 15
    • volume rebuildの仕組み
    • v1.2.0の紹介
    

    View Slide

16. © Copyright 2021 Rancher. All Rights Reserved. 16
    https://github.com/longhorn/longhorn/releases/tag/v1.2.0
    Latest release: Longhorn 1.2.0
    

    View Slide

17. © Copyright 2021 Rancher. All Rights Reserved. 17
    v1.2.0 Hightlight
    • #1815 - [FEATURE] Support CSI volume cloning
    • Cloning support for the longhorn CSI driver
    • #1859 Volume Encryption / #902 backup encryption
    • #587 automatic rebalance the replicas
    • automatically check if a new replica can be scheduled on the new node to restore the redundancy.
    • #2206 Enhance Longhorn data plane on low performance enviroment
    

    View Slide

18. © Copyright 2021 Rancher. All Rights Reserved. 18
    v1.2.0 knownissue
    • #2964 [BUG] Longhorn 1.2.0 - wrong volume permissions inside container / broken fsGroup
    • Issue: Will get a Permission denied when access the folder.
    • WA: Add a --default-fstype=ext4 to the csi-provisioner container
    • Will be fixed in v1.2.1, which is scheduled to be released at September 24, 2021
    ...
    containers:
    - args:
    - --v=2
    - --csi-address=$(ADDRESS)
    - --timeout=1m50s
    - --leader-election
    - --leader-election-namespace=$(POD_NAMESPACE)
    - --default-fstype=ext4
    env:
    ...
    

    View Slide

19. © Copyright 2021 Rancher. All Rights Reserved. 19
    Information
    • Homepage
    • https://longhorn.io/
    • Documentation
    • https://longhorn.io/docs/
    • Roadmap
    • https://github.com/longhorn/longhorn/wiki/Roadmap
    • Development updates
    • https://github.com/longhorn/longhorn/milestones/
    

    View Slide

20. © Copyright 2021 Rancher. All Rights Reserved. 20 20
    Thank you
    

    View Slide