JCConf 2016 zookeeper - Speaker Deck

Slide 1

Slide 1 text

使⽤用Zookeeper改造舊系統 Matt @ JCConf 2016

Slide 2

Slide 2 text

About me ▸ 何適宇 ▸ [email protected] ▸ https://github.com/methodho ▸ https://www.facebook.com/evermatt ▸ 松凌科技

Slide 3

Slide 3 text

⼤大綱 ▸ ZooKeeper 介紹 ▸ 使⽤用 ZooKeeper 前的⽅方法 ▸ Java programming to ZooKeeper ▸ o.a.z.ZooKeeper (3.4.9) ▸ o.a.c.f.r.l.LeaderLatch (2.11.0)

Slide 4

Slide 4 text

Because coordinating distributed systems is a Zoo Apache ZooKeeper wiki WHAT IS ZOOKEEPER

Slide 5

Slide 5 text

https://zookeeper.apache.org/doc/current/zookeeperOver.html WHAT IS ZOOKEEPER

Slide 6

Slide 6 text

▸ 啟動server ▸ /bin/zkServer.sh start ▸ 連結到 zookeeper server ▸ /bin/zkCli.sh [-server localhost:2181] ▸ 關閉server ▸ /bin/zkServer.sh stop ZOOKEEPER CLI

Slide 7

Slide 7 text

▸ 新增 znode ▸ create [-s] [-e] path data ▸ 刪除 znode ▸ delete path ▸ rmr path ZOOKEEPER CLI

Slide 8

Slide 8 text

▸ 列列出 znode 下的 children ▸ ls path [watch] ▸ ls2 path [watch] ▸ 讀寫 znode data (byte[]) ▸ set path data [watch] ▸ get path [watch] ZOOKEEPER CLI

Slide 9

Slide 9 text

CLIENT CLIENT CLIENT ZooKeeper Server ZOOKEEPER WATCH

Slide 10

Slide 10 text

ZOOKEEPER WATCH CLIENT CLIENT CLIENT ZooKeeper Server watch watch watch

Slide 11

Slide 11 text

ZOOKEEPER WATCH CLIENT CLIENT CLIENT 非同步由 Server 端推送訊息⼀一次性 ZooKeeper Server watch watch watch Event Triggered

Slide 12

Slide 12 text

A MCV EXAMPLE (MODEL 2) Controller (Servlet/Filter) View (JSP) Model (Java Bean) ^_^

Slide 13

Slide 13 text

故事是這樣⼦子的... ?

Slide 14

Slide 14 text

BEFORE ZOOKEEPER ▸ 由ap⾃自⼰己控制 ▸ hardcode ▸ properties ▸ passes argument ... ▸ cons? AP1 (run = false) AP3 (run = false) AP2 (run = true)

Slide 15

Slide 15 text

BEFORE ZOOKEEPER ▸ 將控制點抽離ap ▸ db table control ▸ db lock ▸ cons? AP1 AP3 AP2 DB run by AP2

Slide 16

Slide 16 text

BEFORE ZOOKEEPER 有⼀一個⽅方法可以實作⾃自⼰己的邏輯: 決定誰要執⾏行行我們需要的是...

Slide 17

Slide 17 text

https://zookeeper.apache.org/doc/trunk/recipes.html BEFORE ZOOKEEPER ZooKeeper Recipes !

Slide 18

Slide 18 text

https://zookeeper.apache.org/doc/trunk/recipes.html#sc_leaderElection LEADER ELECTION Zookeeper Server / /SOME-PATH /ZNODE_1 CLIENT_1 /ZNODE_2 CLIENT_2 /ZNODE_3 CLIENT_3 A simple way: Use SEQUENCE & EPHEMERAL znode

Slide 19

Slide 19 text

https://zookeeper.apache.org/doc/trunk/recipes.html#sc_leaderElection LEADER ELECTION Zookeeper Server Advance solution / /SOME-PATH

Slide 20

Slide 20 text

https://zookeeper.apache.org/doc/trunk/recipes.html#sc_leaderElection LEADER ELECTION Zookeeper Server / /SOME-PATH /ZNODE_1 CLIENT_1 Advance solution

Slide 21

Slide 21 text

https://zookeeper.apache.org/doc/trunk/recipes.html#sc_leaderElection LEADER ELECTION Zookeeper Server / /SOME-PATH /ZNODE_1 CLIENT_1 /ZNODE_2 CLIENT_2 Advance solution

Slide 22

Slide 22 text

https://zookeeper.apache.org/doc/trunk/recipes.html#sc_leaderElection LEADER ELECTION Zookeeper Server / /SOME-PATH /ZNODE_1 CLIENT_1 /ZNODE_2 CLIENT_2 /ZNODE_3 CLIENT_3 Advance solution

Slide 23

Slide 23 text

IMPLEMENTING LEADER ELECTION Start (1) Connect to zookeeper server (2) Create parent if not exist (3) Create -e -s node z Node j found End (6) You are leader (7) Watch for node j Y N (5) Find node j , where j is the largest seq and j < z (4) Get and sort all children Node j deleted

Slide 24

Slide 24 text

Slide 25

Slide 25 text

PROGRAMMING TO ZOOKEEPER http://curator.apache.org/

Slide 26

Slide 26 text

PROGRAMMING TO ZOOKEEPER (1) Connect to zookeeper server

Slide 27

Slide 27 text

PROGRAMMING TO ZOOKEEPER (2) Create parent if not exist

Slide 28

Slide 28 text

PROGRAMMING TO ZOOKEEPER (3) Create -e -s node z

Slide 29

Slide 29 text

PROGRAMMING TO ZOOKEEPER (4) Get and sort all children (5) Find node j , where j is the largest seq and j < z

Slide 30

Slide 30 text

PROGRAMMING TO ZOOKEEPER (7) Watch for node j

Slide 31

Slide 31 text

PROGRAMMING TO ZOOKEEPER

Slide 32

Slide 32 text

PROGRAMMING TO ZOOKEEPER 剛寫那麼多都是寫 ____ der…

Slide 33

Slide 33 text

怎麼串串接到排程中..? ?

Slide 34

Slide 34 text

ADAPTING INTO SCHEDULE FRAMEWORK ADAPTING INTO SCHEDULE FRAMEWORK @Scheduled methods @EnableScheduling

Slide 35

Slide 35 text

ADAPTING INTO SCHEDULE FRAMEWORK ADAPTING INTO SCHEDULE FRAMEWORK @Scheduled methods @EnableScheduling ScheduledAnnotation BeanPostProcessor Find & register TaskScheduler [default implementation] ThreadPoolTaskScheduler To be invoked by java.util.concurrent.Scheduled ExecutorService ScheduledMethodRunnable(bean, method) Register

Slide 36

Slide 36 text

ADAPTING INTO SCHEDULE FRAMEWORK ADAPTING INTO SCHEDULE FRAMEWORK TaskScheduler decorator Original TaskScheduler [default implementation] ThreadPoolTaskScheduler ZooKeeper Server

Slide 37

Slide 37 text

PROGRAMMING TO SCHEDULE FRAMEWORK

Slide 38

Slide 38 text

PROGRAMMING TO SCHEDULE FRAMEWORK

Slide 39

Slide 39 text

PROGRAMMING TO SCHEDULE FRAMEWORK

Slide 40

Slide 40 text

PROGRAMMING TO ZOOKEEPER 來來看看實際的程式碼吧...

Slide 41

Slide 41 text

PROGRAMMING TO ZOOKEEPER 還可以做些什什麼?

Slide 42

Slide 42 text

LEADER ELECTION Zookeeper Server / /SOME-PATH /ZNODE_1 CLIENT_1 /ZNODE_2 CLIENT_2 /ZNODE_3 CLIENT_3

Slide 43

Slide 43 text

LEADER ELECTION Zookeeper Server / /SOME-PATH /ZNODE_1 CLIENT_1 80% /ZNODE_2 CLIENT_2 10% /ZNODE_3 CLIENT_3 55% 看看誰的系統資源使⽤用率最低的就去執⾏行行吧

Slide 44

Slide 44 text

Please leave your comments ↓ https://github.com/methodho/jcconf2016-zookeeper

Slide 45

Slide 45 text

Please leave your comments ↓ https://github.com/methodho/jcconf2016-zookeeper And we’re hiring! http://www.softleader.com.tw/web/career.html