Zookeeper CONFIGURATION PROVISIONING WITH... mariusz gil 

About me 

No content

No content

Coordination service for distributed applications and systems 

/app/feat_3 /app/f_2 /app/f_1 /servers /app In-memory file system model / 

clients zookeeper cluster leader 

7use cases name service configuration group membership leader election barriers queues locks 

many projects Apache Accumulo Apache Hadoop MapReduce Apache HBase Apache Hedwig Apache Kafka Apache S4 Apache Solr Apache Storm Mesos Neo4J 

ZooKeeper usage 

Installation wget http://ftp.ps.pl/pub/apache/zookeeper/stable/zookeeper-3.4.5.tar.gz touch conf/zoo.cfg vim conf/zoo.cfg # STANDALONE ENVIRONMENT # tickTime=2000 # dataDir=/var/zookeeper # clientPort=2181 # DISTRIBUTED ENVIRONMENT # tickTime=2000 # dataDir=/var/zookeeper # clientPort=2181 # initLimit=5 # syncLimit=2 # server.1=zoo1:2888:3888 # server.2=zoo2:2888:3888 # server.3=zoo3:2888:3888 bin/zkServer.sh start 

CLI mode bin/zkServer.sh start bin/zkCli.sh 127.0.0.1:2181 [zk: localhost:2181(CONNECTED) 31] help ZooKeeper -server host:port cmd args connect host:port get path [watch] ls path [watch] set path data [version] rmr path delquota [-n|-b] path quit printwatches on|off create [-s] [-e] path data acl stat path [watch] close ls2 path [watch] history listquota path setAcl path acl getAcl path sync path redo cmdno addauth scheme auth delete path [version] 

CLI mode bin/zkServer.sh start bin/zkCli.sh 127.0.0.1:2181 [zk: localhost:2181(CONNECTED) 8] ls / [zookeeper] ... [zk: localhost:2181(CONNECTED) 17] create /app/features/pymk '' Created /app/features/pymk [zk: localhost:2181(CONNECTED) 20] create /app/features/pymk/enabled 0 Created /app/features/pymk/enabled [zk: localhost:2181(CONNECTED) 24] set /app/features/pymk/enabled 1 [zk: localhost:2181(CONNECTED) 28] get /app/features/pymk/enabled 1 [zk: localhost:2181(CONNECTED) 26] ls /app/features [pymk, login, friends, messages] [zk: localhost:2181(CONNECTED) 29] rmr /app/features [zk: localhost:2181(CONNECTED) 30] ls /app [] bin/zkServer.sh stop 

PHP + ZooKeeper 

Edit config.prod.yml file Commit to repo Wait for CI report Build app package Deploy to production 

ZooKeeper cluster Web workers Tasks workers 

No content

$ cd $ git clone https://github.com/andreiz/php-zookeeper.git $ cd php-zookeeper $ phpize $ ./configure $ make $ sudo make install Installation of PHP extensions 

zookeeper->exists($path)) { $this->makePath($path); $this->makeNode($path, $value); } else { $this->zookeeper->set($path, $value); } } /** * Equivalent of "mkdir -p" on ZooKeeper */ public function makePath($path, $value = '') { $parts = explode('/', $path); $parts = array_filter($parts); $subpath = ''; while (count($parts) > 1) { $subpath .= '/' . array_shift($parts); if (!$this->zookeeper->exists($subpath)) { $this->makeNode($subpath, $value); } } } Example 1 Simple configuration client get('/')); var_dump($zk->getChildren('/')); var_dump($zk->set('/test123', 'abc')); var_dump($zk->get('/test123')); var_dump($zk->getChildren('/')); var_dump($zk->set('/foo/001', 'bar1')); var_dump($zk->set('/foo/002', 'bar2')); var_dump($zk->getChildren('/foo')); Tekst 

get( '/test', array($this, 'watcher' ) ) . "\n"; } } $zoo = new ZookeeperDemo('localhost:2181'); $zoo->get( '/test', array($zoo, 'watcher' ) ); while( true ) { echo '.'; sleep(2); } Example 2 Key change watcher 

Zookeeper::PERM_ALL, 'scheme' => 'world', 'id' => 'anyone' ) ); private $isLeader = false; private $znode; public function __construct( $host = '', $watcher_cb = null, $recv_timeout = 10000 ) { parent::__construct( $host, $watcher_cb, $recv_timeout ); } public function register() { if( ! $this->exists( self::CONTAINER ) ) { $this->create( self::CONTAINER, null, $this->acl ); } $this->znode = $this->create( self::CONTAINER . '/w-', null, $this->acl, Zookeeper::EPHEMERAL | Zookeeper::SEQUENCE ); $this->znode = str_replace( self::CONTAINER .'/', '', $this->znode ); Example 3 Multiple workers with automatic leader election # console 1 $ php example_worker.php I'm registred as: w-0000000001 Nobody here, I'm the leader Leading # console 2 $ php example_worker.php I'm registred as: w-0000000002 I'm watching w-0000000001 Working # console 3 $ php example_worker.php I'm registred as: w-0000000003 I'm watching w-0000000002 Working 

DevOp life? 

OMG! I have to disable feature X for a while and change database schema, but the rest of the app should works... set /app/feature/pymk/enabled 0 ALTER TABLE... set /app/feature/pymk/enabled 1 

OMG! I have to add more cache instances and distribute traffic across many servers set /servers/cache/c6/host 192... set /servers/cache/c6/port 11211 set /servers/cache/c7/host 192... set /servers/cache/c7/port 11211 ls /servers/cache 

leader zookeeper cluster service provider 192.168.12.201 client 192.168.12.201 

leader zookeeper cluster client service provider 192.168.12.201 service provider 192.168.12.123 192.168.12.123 

Your app should only know... ...where your ZooKeeper is 

Zookeeper CONFIGURATION PROVISIONING WITH... mariusz gil THANKS