Symfony y concurrencia: el componente Lock - Speaker Deck

Tweet

Tweet

Slide 1

Slide 1 text

deSymfony 30 junio - 1 julio 2017 Castellón SYMFONY Y CONCURRENCIA EL COMPONENTE LOCK César Suárez

Slide 2

Slide 2 text

No content

Slide 3

Slide 3 text

$ whoami • César Suárez • Backend developer @ https://mailtrack.io • One-man army @ https://suicidebystar.com • Producto típico de Extremadura • Twitter: @tharandur / GitHub: @csuarez

Slide 4

Slide 4 text

¿Por qué una charla de concurrencia?

Slide 5

Slide 5 text

No content

Slide 6

Slide 6 text

Concurrency is the tendency for things to happen at the same time in a system. http://sce.uhcl.edu/helm/rationaluniﬁedprocess/process/workﬂow/

Slide 7

Slide 7 text

Concurrent algorithm is the description of a set of sequential state machines that cooperate through a communication medium Concurrent Programming: Algorithms, Principles and Foundations Michel Raynal

Slide 8

Slide 8 text

No content

Slide 9

Slide 9 text

Mutual Exclusion Problem

Slide 10

Slide 10 text

Critical sections A C A C P1 P2 B B

Slide 11

Slide 11 text

Critical sections A C A C P1 P2 B B

Slide 12

Slide 12 text

Critical sections A C A C P1 P2 B B

Slide 13

Slide 13 text

Critical sections A C A C P1 P2 B B

Slide 14

Slide 14 text

Critical sections A C A C P1 P2 B B

Slide 15

Slide 15 text

acquire_mutex() release_mutex() critical_section()

Slide 16

Slide 16 text

Propiedades • Mutual exclusion • Sólo puede haber un proceso a la vez en la sección crítica • Starvation-freedom • Todas las llamadas a acquire_mutex() han de terminar eventualmente

Slide 17

Slide 17 text

Locks

Slide 18

Slide 18 text

free locked lock.acquire() lock.release()

Slide 19

Slide 19 text

Solución al problema de exclusión mutua

Slide 20

Slide 20 text

¿Implementación?

Slide 21

Slide 21 text

symfony/lock

Slide 22

Slide 22 text

symfony/lock • Symfony 3.3 3.4 • By Jérémy Derussé (@jderusse) • Múltiples stores • Evolución de LockHandler • Pensado para comandos

Slide 23

Slide 23 text

symfony/lock 101 $lock = $factory->createLock('some-id'); if ($lock->acquire()) { /** * critical section */ $lock->release(); }

Slide 24

Slide 24 text

Lock creation use Symfony\Component\Lock\Factory; use Symfony\Component\Lock\Store\SemaphoreStore; $store = new SemaphoreStore(); $factory = new Factory($store); $lock = $factory->createLock('some-id');

Slide 25

Slide 25 text

acquire() • Non-blocking acquire • Blocking acquire • Consultar si un lock está adquirido if ($lock->acquire()) { } $lock->acquire(true); $lock->isAcquired();

Slide 26

Slide 26 text

release() if ($lock->acquire()) { try { /** * critical section */ } finally { $lock->release(); } }

Slide 27

Slide 27 text

easy example

Slide 28

Slide 28 text

No lock $resource = new UnsafeSharedResource( 'very-important-thing' ); do { $counter = $resource->read(); $resource->write(++$counter); } while(true);

Slide 29

Slide 29 text

No content

Slide 30

Slide 30 text

No content

Slide 31

Slide 31 text

$resource = new UnsafeSharedResource( 'very-important-thing' ); do { $counter = $resource->read(); $resource->write(++$counter); } while(true); No lock

Slide 32

Slide 32 text

RE = 0 WR(1) P1 P2 RE = 0 WR(1) RE = 1 RE = 1

Slide 33

Slide 33 text

RE = 0 WR(1) P1 P2 RE = 0 WR(1) RE = 1 RE = 1 WR(2)

Slide 34

Slide 34 text

RE = 0 WR(1) P1 P2 RE = 1 WR(2) RE = 2 WR(3)

Slide 35

Slide 35 text

Simple lock $lock = $factory->createLock('simple:lock'); do { $lock->acquire(true); try { $counter = $resource->read(); $resource->write(++$counter); } finally { $lock->release(); } } while(true);

Slide 36

Slide 36 text

No content

Slide 37

Slide 37 text

Expiring locks • Creación • Incrementar Time To Live (TTL) $factory->createLock('expiring-lock', 30); $lock->refresh(); ¡¡ Recomendado para distlocks !!

Slide 38

Slide 38 text

Stores

Slide 39

Slide 39 text

SemaphoreStore • Usa las funciones semaphore de PHP • En memoria (System V IPC) • No funciona en Windows • No distribuido • Non-Expiring

Slide 40

Slide 40 text

SemaphoreStore use Symfony\Component\Lock\Factory; use Symfony\Component\Lock\Store\SemaphoreStore; $store = new SemaphoreStore(); $factory = new Factory($store); $lock = $factory->createLock('some-id');

Slide 41

Slide 41 text

FlockStore • Filesystem block • No distribuido • Non-expiring • +info: http://www.php.net/manual/en/ function.ﬂock.php

Slide 42

Slide 42 text

FlockStore • use Symfony\Component\Lock\Store\FlockStore; $store = new FlockStore(sys_get_temp_dir()); $factory = new Factory($store); $lock = $factory->createLock('file-lock');

Slide 43

Slide 43 text

MemcachedStore • Cache as lock • Lock distribuido • Expiring locks • Non-blocking locks (sin RetryTillSaveStore)

Slide 44

Slide 44 text

MemcachedStore use Symfony\Component\Lock\Store\MemcachedStore; $memcachedConn = new \Memcached; $memcachedConn->addServer('memcached', 11211); $store = new MemcachedStore($memcachedConn); $factory = new Factory($store); $lock = $factory->createLock('memory-lock');

Slide 45

Slide 45 text

WARNING!

Slide 46

Slide 46 text

WARNING!

Slide 47

Slide 47 text

No content

Slide 48

Slide 48 text

No content

Slide 49

Slide 49 text

El problema con memcached • Evictions! • Least Recently Used cache (LRU) • ¿Soluciones? • Máquinas con poca carga • Tunear memcached

Slide 50

Slide 50 text

RedisStore • Lock distribuido • Expiring locks • Non-blocking locks (sin RetryTillSaveStore) • Control sobre los evictions. • Persistencia en disco.

Slide 51

Slide 51 text

RedisStore use Symfony\Component\Lock\Store\RedisStore; $redisConn = new \Predis\Client( 'tcp://0.0.0.0:6379' ); $store = new RedisStore($redisConn); $factory = new Factory($store); $lock = $factory->createLock('memory-lock');

Slide 52

Slide 52 text

CombinedStore • Combinación de varios Stores • Varias estrategias • ConsensusStrategy • UnanimousStrategy • High Aviability

Slide 53

Slide 53 text

CombinedStore use Symfony\Component\Lock\Store\RedisStore; use Symfony\Component\Lock\Strategy\ConsensusStrategy; $store1 = new RedisStore($redisConn1); $store2 = new RedisStore($redisConn2); $store3 = new RedisStore($redisConn3); $factory = new Factory(new CombinedStore( [$store1, $store2, $store3], new ConsensusStrategy() )); $lock = $factory->createLock('ha-lock');

Slide 54

Slide 54 text

No content

Slide 55

Slide 55 text

No content

Slide 56

Slide 56 text

El problema con Redis • Distributed lock with Redis (redlock) • https://redis.io/topics/distlock • How to distributed locking • http://martin.kleppmann.com/2016/02/08/how-to-do- distributed-locking.html • Is Redlock safe? Reply to Redlock Analysis • http://antirez.com/news/101 • https://news.ycombinator.com/item?id=11065933

Slide 57

Slide 57 text

http://martin.kleppmann.com/2016/02/08/how-to-do-distributed-locking.html

Slide 58

Slide 58 text

No content

Slide 59

Slide 59 text

No content

Slide 60

Slide 60 text

¿Por qué usar un lock?

Slide 61

Slide 61 text

1. Exactitud

Slide 62

Slide 62 text

2. Rendimiento

Slide 63

Slide 63 text

No “big deal” scenarios

Slide 64

Slide 64 text

No content

Slide 65

Slide 65 text

Integración con Symfony framework: lock: users: ['memcached://m1.docker', 'memcached://m2.docker'] admin: 'flock' partners: 'redis://r1.docker' invoices: 'semaphore' $adminLock = $this->get('lock.admin')->acquire(); $invoicesLock = $this->get('lock.invoices')->acquire(true); • PHP • Conﬁguración

Slide 66

Slide 66 text

More examples https://github.com/csuarez/symfony-lock-playground

Slide 67

Slide 67 text

Read/Write lock

Slide 68

Slide 68 text

No content

Slide 69

Slide 69 text

// begin read $readLock->acquire(true); $readers = $readersStore ->increase(); if ($readers == 1) { $writeLock->acquire(true); } $readLock->release(); // read $read(); // end read $readLock->acquire(true); $readers = $readersStore ->decrease(); if ($readers == 0) { $writeLock->release(); } $readLock->release(); //begin write $writeLock->acquire(true); //write $write(); //end write $writeLock->release(); writer reader

Slide 70

Slide 70 text

//begin write $writeLock->acquire(true); //write $write(); //end write $writeLock->release(); writer

Slide 71

Slide 71 text

// begin read $readLock->acquire(true); $readers = $readersStore ->increase(); if ($readers == 1) { $writeLock->acquire(true); } $readLock->release(); // read $read(); // end read $readLock->acquire(true); $readers = $readersStore ->decrease(); if ($readers == 0) { $writeLock->release(); } $readLock->release(); //begin write $writeLock->acquire(true); //write $write(); //end write $writeLock->release(); writer reader

Slide 72

Slide 72 text

// begin read $readLock->acquire(true); $readers = $readersStore ->increase(); if ($readers == 1) { $writeLock->acquire(true); } $readLock->release(); reader

Slide 73

Slide 73 text

// begin read $readLock->acquire(true); $readers = $readersStore ->increase(); if ($readers == 1) { $writeLock->acquire(true); } $readLock->release(); // read $read(); // end read $readLock->acquire(true); $readers = $readersStore ->decrease(); if ($readers == 0) { $writeLock->release(); } $readLock->release(); //begin write $writeLock->acquire(true); //write $write(); //end write $writeLock->release(); writer reader

Slide 74

Slide 74 text

// end read $readLock->acquire(true); $readers = $readersStore ->decrease(); if ($readers == 0) { $writeLock->release(); } $readLock->release(); reader

Slide 75

Slide 75 text

Otros locks

Slide 76

Slide 76 text

distlock /w InnoDB • Shared mode lock • Reads as UPDATEs START TRANSACTION; SELECT * FROM Table WHERE name="foo" FOR UPDATE; ## Critical section COMMIT;

Slide 77

Slide 77 text

Dist consensus algorithms • Paxos made simple, Leslie Lamport • http://lamport.azurewebsites.net/pubs/paxos-simple.pdf • In Search of an Understandable Consensus Algorithmn (Extended Version), Diego Ontario and John Ousterhout • https://raft.github.io/raft.pdf • ZooKeeper’s atomic broadcast protocol: Theory and practice, André Medeiros • http://www.tcs.hut.ﬁ/Studies/T-79.5001/reports/2012- deSouzaMedeiros.pdf

Slide 78

Slide 78 text

No content

Slide 79

Slide 79 text

¡Gracias!

Slide 80

Slide 80 text

we are hiring!

Slide 81

Slide 81 text

¡Gracias! suarez.ortega.cesar@gmail.com