JCConf 2016 zookeeper

Transcript

1. 使⽤用Zookeeper改造舊系統
   Matt @ JCConf 2016
   

   View Slide

2. About me
   ▸ 何適宇
   ▸ methodho@gmail.com
   ▸ https://github.com/methodho
   ▸ https://www.facebook.com/evermatt
   ▸ 松凌科技
   

   View Slide

3. ⼤大綱
   ▸ ZooKeeper 介紹
   ▸ 使⽤用 ZooKeeper 前的⽅方法
   ▸ Java programming to ZooKeeper
   ▸ o.a.z.ZooKeeper (3.4.9)
   ▸ o.a.c.f.r.l.LeaderLatch (2.11.0)
   

   View Slide

4. Because coordinating
   distributed systems is a Zoo
   Apache ZooKeeper wiki
   WHAT IS ZOOKEEPER
   

   View Slide

5. https://zookeeper.apache.org/doc/current/zookeeperOver.html
   WHAT IS ZOOKEEPER
   

   View Slide

6. ▸ 啟動server
   ▸ /bin/zkServer.sh start
   ▸ 連結到 zookeeper server
   ▸ /bin/zkCli.sh [-server localhost:2181]
   ▸ 關閉server
   ▸ /bin/zkServer.sh stop
   ZOOKEEPER CLI
   

   View Slide

7. ▸ 新增 znode
   ▸ create [-s] [-e] path data
   ▸ 刪除 znode
   ▸ delete path
   ▸ rmr path
   ZOOKEEPER CLI
   

   View Slide

8. ▸ 列列出 znode 下的 children
   ▸ ls path [watch]
   ▸ ls2 path [watch]
   ▸ 讀寫 znode data (byte[])
   ▸ set path data [watch]
   ▸ get path [watch]
   ZOOKEEPER CLI
   

   View Slide

9. CLIENT
   CLIENT
   CLIENT
   ZooKeeper Server
   ZOOKEEPER WATCH
   

   View Slide

10. ZOOKEEPER WATCH
    CLIENT
    CLIENT
    CLIENT
    ZooKeeper Server
    watch
    watch
    watch
    

    View Slide

11. ZOOKEEPER WATCH
    CLIENT
    CLIENT
    CLIENT
    非同步
    由 Server 端推送訊息
    ⼀一次性
    ZooKeeper Server
    watch
    watch
    watch
    Event
    Triggered
    

    View Slide

12. A MCV EXAMPLE (MODEL 2)
    Controller
    (Servlet/Filter)
    View
    (JSP)
    Model
    (Java Bean)
    ^_^
    

    View Slide

13. 故事是這樣⼦子的...
    ?
    

    View Slide

14. BEFORE ZOOKEEPER
    ▸ 由ap⾃自⼰己控制
    ▸ hardcode
    ▸ properties
    ▸ passes argument ...
    ▸ cons?
    AP1
    (run = false)
    AP3
    (run = false)
    AP2
    (run = true)
    

    View Slide

15. BEFORE ZOOKEEPER
    ▸ 將控制點抽離ap
    ▸ db table control
    ▸ db lock
    ▸ cons?
    AP1 AP3
    AP2
    DB
    run by AP2
    

    View Slide

16. BEFORE ZOOKEEPER
    有⼀一個⽅方法可以實作⾃自⼰己的邏輯:
    決定誰要執⾏行行
    我們需要的是...
    

    View Slide

17. https://zookeeper.apache.org/doc/trunk/recipes.html
    BEFORE ZOOKEEPER
    ZooKeeper Recipes
    !
    

    View Slide

18. https://zookeeper.apache.org/doc/trunk/recipes.html#sc_leaderElection
    LEADER ELECTION
    Zookeeper Server
    /
    /SOME-PATH
    /ZNODE_1
    CLIENT_1 /ZNODE_2
    CLIENT_2
    /ZNODE_3
    CLIENT_3
    A simple way:
    Use SEQUENCE &
    EPHEMERAL znode
    

    View Slide

19. https://zookeeper.apache.org/doc/trunk/recipes.html#sc_leaderElection
    LEADER ELECTION
    Zookeeper Server
    Advance solution
    /
    /SOME-PATH
    

    View Slide

20. https://zookeeper.apache.org/doc/trunk/recipes.html#sc_leaderElection
    LEADER ELECTION
    Zookeeper Server
    /
    /SOME-PATH
    /ZNODE_1
    CLIENT_1
    Advance solution
    

    View Slide

21. https://zookeeper.apache.org/doc/trunk/recipes.html#sc_leaderElection
    LEADER ELECTION
    Zookeeper Server
    /
    /SOME-PATH
    /ZNODE_1
    CLIENT_1 /ZNODE_2
    CLIENT_2
    Advance solution
    

    View Slide

22. https://zookeeper.apache.org/doc/trunk/recipes.html#sc_leaderElection
    LEADER ELECTION
    Zookeeper Server
    /
    /SOME-PATH
    /ZNODE_1
    CLIENT_1 /ZNODE_2
    CLIENT_2
    /ZNODE_3
    CLIENT_3
    Advance solution
    

    View Slide

23. IMPLEMENTING LEADER ELECTION
    Start
    (1)
    Connect to
    zookeeper server
    (2)
    Create parent if not
    exist
    (3)
    Create -e -s node z
    Node j
    found
    End
    (6)
    You are
    leader
    (7)
    Watch for
    node j
    Y
    N
    (5)
    Find node j , where
    j is the largest seq
    and j < z
    (4)
    Get and sort all
    children
    Node j
    deleted
    

    View Slide

24. IMPLEMENTING LEADER ELECTION
    Start
    (1)
    Connect to
    zookeeper server
    (2)
    Create parent if not
    exist
    (3)
    Create -e -s node z
    Node j
    found
    End
    (6)
    You are
    leader
    (7)
    Watch for
    node j
    Y
    N
    (5)
    Find node j , where
    j is the largest seq
    and j < z
    (4)
    Get and sort all
    children
    Node j
    deleted
    

    View Slide

25. PROGRAMMING TO ZOOKEEPER
    http://curator.apache.org/
    

    View Slide

26. PROGRAMMING TO ZOOKEEPER
    (1)
    Connect to
    zookeeper server
    

    View Slide

27. PROGRAMMING TO ZOOKEEPER
    (2)
    Create parent if not
    exist
    

    View Slide

28. PROGRAMMING TO ZOOKEEPER
    (3)
    Create -e -s node z
    

    View Slide

29. PROGRAMMING TO ZOOKEEPER
    (4)
    Get and sort all
    children
    (5)
    Find node j , where
    j is the largest seq
    and j < z
    

    View Slide

30. PROGRAMMING TO ZOOKEEPER
    (7)
    Watch for
    node j
    

    View Slide

31. PROGRAMMING TO ZOOKEEPER
    

    View Slide

32. PROGRAMMING TO ZOOKEEPER
    剛寫那麼多都是寫 ____ der…
    

    View Slide

33. 怎麼串串接到排程中..?
    ?
    

    View Slide

34. ADAPTING INTO SCHEDULE FRAMEWORK
    ADAPTING INTO SCHEDULE FRAMEWORK
    @Scheduled
    methods
    @EnableScheduling
    

    View Slide

35. ADAPTING INTO SCHEDULE FRAMEWORK
    ADAPTING INTO SCHEDULE FRAMEWORK
    @Scheduled
    methods
    @EnableScheduling ScheduledAnnotation
    BeanPostProcessor
    Find & register
    TaskScheduler
    [default implementation]
    ThreadPoolTaskScheduler
    To be invoked by
    java.util.concurrent.Scheduled
    ExecutorService
    ScheduledMethodRunnable(bean, method)
    Register
    

    View Slide

36. ADAPTING INTO SCHEDULE FRAMEWORK
    ADAPTING INTO SCHEDULE FRAMEWORK
    TaskScheduler decorator
    Original TaskScheduler
    [default implementation]
    ThreadPoolTaskScheduler
    ZooKeeper Server
    

    View Slide

37. PROGRAMMING TO SCHEDULE FRAMEWORK
    

    View Slide

38. PROGRAMMING TO SCHEDULE FRAMEWORK
    

    View Slide

39. PROGRAMMING TO SCHEDULE FRAMEWORK
    

    View Slide

40. PROGRAMMING TO ZOOKEEPER
    來來看看實際的程式碼吧...
    

    View Slide

41. PROGRAMMING TO ZOOKEEPER
    還可以做些什什麼?
    

    View Slide

42. LEADER ELECTION
    Zookeeper Server
    /
    /SOME-PATH
    /ZNODE_1
    CLIENT_1 /ZNODE_2
    CLIENT_2
    /ZNODE_3
    CLIENT_3
    

    View Slide

43. LEADER ELECTION
    Zookeeper Server
    /
    /SOME-PATH
    /ZNODE_1
    CLIENT_1
    80% /ZNODE_2
    CLIENT_2
    10%
    /ZNODE_3
    CLIENT_3
    55%
    看看誰的系統資源
    使⽤用率最低的就去
    執⾏行行吧
    

    View Slide

44. Please leave your comments ↓
    https://github.com/methodho/jcconf2016-zookeeper
    

    View Slide

45. Please leave your comments ↓
    https://github.com/methodho/jcconf2016-zookeeper
    And we’re hiring!
    http://www.softleader.com.tw/web/career.html
    

    View Slide