Symfony y concurrencia: el componente Lock

Los entornos concurrentes y distribuidos son una constante cuando queremos escalar un proyecto. Nos ayudan a la hora de construir sistemas capaces de procesar millones de peticiones, pero también dificultan nuestra manera de programar. Generan una incertidumbre en nuestras ejecuciones que debemos controlar, porque de nada vale un sistema con un gran rendimiento si no es fiable.

Los locks son una de las herramientas principales a la hora de sincronizar y ordenar distintas ejecuciones que deben acceder a un mismo recurso. En Symfony 3.4 se incluye un lock de serie en forma de componente. Éste nos servirá para ilustrar los problemas de los entornos concurrentes y las buenas prácticas que podemos adoptar a la hora de enfrentarnos a ellos.

Transcript

1. deSymfony 30 junio - 1 julio 2017 Castellón SYMFONY Y CONCURRENCIA EL COMPONENTE LOCK César Suárez
3. $ whoami • César Suárez • Backend developer @ https://mailtrack.io • One-man army @ https://suicidebystar.com • Producto típico de Extremadura • Twitter: @tharandur / GitHub: @csuarez
4. ¿Por qué una charla de concurrencia?
6. Concurrency is the tendency for things to happen at the same time in a system. http://sce.uhcl.edu/helm/rationalunifiedprocess/process/workflow/
7. Concurrent algorithm is the description of a set of sequential state machines that cooperate through a communication medium Concurrent Programming: Algorithms, Principles and Foundations Michel Raynal
9. Mutual Exclusion Problem
10. Critical sections A C A C P1 P2 B B
11. Critical sections A C A C P1 P2 B B
12. Critical sections A C A C P1 P2 B B
13. Critical sections A C A C P1 P2 B B
14. Critical sections A C A C P1 P2 B B
15. acquire_mutex() release_mutex() critical_section()
16. Propiedades • Mutual exclusion • Sólo puede haber un proceso a la vez en la sección crítica • Starvation-freedom • Todas las llamadas a acquire_mutex() han de terminar eventualmente
17. Locks
18. free locked lock.acquire() lock.release()
19. Solución al problema de exclusión mutua
20. ¿Implementación?
21. symfony/lock
22. symfony/lock • Symfony 3.3 3.4 • By Jérémy Derussé (@jderusse) • Múltiples stores • Evolución de LockHandler • Pensado para comandos
23. symfony/lock 101 $lock = $factory->createLock('some-id'); if ($lock->acquire()) { /** * critical section */ $lock->release(); }
24. Lock creation use Symfony\Component\Lock\Factory; use Symfony\Component\Lock\Store\SemaphoreStore; $store = new SemaphoreStore(); $factory = new Factory($store); $lock = $factory->createLock('some-id');
25. acquire() • Non-blocking acquire • Blocking acquire • Consultar si un lock está adquirido if ($lock->acquire()) { } $lock->acquire(true); $lock->isAcquired();
26. release() if ($lock->acquire()) { try { /** * critical section */ } finally { $lock->release(); } }
27. easy example
28. No lock $resource = new UnsafeSharedResource( 'very-important-thing' ); do { $counter = $resource->read(); $resource->write(++$counter); } while(true);
31. $resource = new UnsafeSharedResource( 'very-important-thing' ); do { $counter = $resource->read(); $resource->write(++$counter); } while(true); No lock
32. RE = 0 WR(1) P1 P2 RE = 0 WR(1) RE = 1 RE = 1
33. RE = 0 WR(1) P1 P2 RE = 0 WR(1) RE = 1 RE = 1 WR(2)
34. RE = 0 WR(1) P1 P2 RE = 1 WR(2) RE = 2 WR(3)
35. Simple lock $lock = $factory->createLock('simple:lock'); do { $lock->acquire(true); try { $counter = $resource->read(); $resource->write(++$counter); } finally { $lock->release(); } } while(true);
37. Expiring locks • Creación • Incrementar Time To Live (TTL) $factory->createLock('expiring-lock', 30); $lock->refresh(); ¡¡ Recomendado para distlocks !!
38. Stores
39. SemaphoreStore • Usa las funciones semaphore de PHP • En memoria (System V IPC) • No funciona en Windows • No distribuido • Non-Expiring
40. SemaphoreStore use Symfony\Component\Lock\Factory; use Symfony\Component\Lock\Store\SemaphoreStore; $store = new SemaphoreStore(); $factory = new Factory($store); $lock = $factory->createLock('some-id');
41. FlockStore • Filesystem block • No distribuido • Non-expiring • +info: http://www.php.net/manual/en/ function.flock.php
42. FlockStore • use Symfony\Component\Lock\Store\FlockStore; $store = new FlockStore(sys_get_temp_dir()); $factory = new Factory($store); $lock = $factory->createLock('file-lock');
43. MemcachedStore • Cache as lock • Lock distribuido • Expiring locks • Non-blocking locks (sin RetryTillSaveStore)
44. MemcachedStore use Symfony\Component\Lock\Store\MemcachedStore; $memcachedConn = new \Memcached; $memcachedConn->addServer('memcached', 11211); $store = new MemcachedStore($memcachedConn); $factory = new Factory($store); $lock = $factory->createLock('memory-lock');
45. WARNING!
46. WARNING!
49. El problema con memcached • Evictions! • Least Recently Used cache (LRU) • ¿Soluciones? • Máquinas con poca carga • Tunear memcached
50. RedisStore • Lock distribuido • Expiring locks • Non-blocking locks (sin RetryTillSaveStore) • Control sobre los evictions. • Persistencia en disco.
51. RedisStore use Symfony\Component\Lock\Store\RedisStore; $redisConn = new \Predis\Client( 'tcp://0.0.0.0:6379' ); $store = new RedisStore($redisConn); $factory = new Factory($store); $lock = $factory->createLock('memory-lock');
52. CombinedStore • Combinación de varios Stores • Varias estrategias • ConsensusStrategy • UnanimousStrategy • High Aviability
53. CombinedStore use Symfony\Component\Lock\Store\RedisStore; use Symfony\Component\Lock\Strategy\ConsensusStrategy; $store1 = new RedisStore($redisConn1); $store2 = new RedisStore($redisConn2); $store3 = new RedisStore($redisConn3); $factory = new Factory(new CombinedStore( [$store1, $store2, $store3], new ConsensusStrategy() )); $lock = $factory->createLock('ha-lock');
56. El problema con Redis • Distributed lock with Redis (redlock) • https://redis.io/topics/distlock • How to distributed locking • http://martin.kleppmann.com/2016/02/08/how-to-do- distributed-locking.html • Is Redlock safe? Reply to Redlock Analysis • http://antirez.com/news/101 • https://news.ycombinator.com/item?id=11065933
57. http://martin.kleppmann.com/2016/02/08/how-to-do-distributed-locking.html
60. ¿Por qué usar un lock?
61. 1. Exactitud
62. 2. Rendimiento
63. No “big deal” scenarios
65. Integración con Symfony framework: lock: users: ['memcached://m1.docker', 'memcached://m2.docker'] admin: 'flock' partners: 'redis://r1.docker' invoices: 'semaphore' $adminLock = $this->get('lock.admin')->acquire(); $invoicesLock = $this->get('lock.invoices')->acquire(true); • PHP • Configuración
66. More examples https://github.com/csuarez/symfony-lock-playground
67. Read/Write lock
69. // begin read $readLock->acquire(true); $readers = $readersStore ->increase(); if ($readers == 1) { $writeLock->acquire(true); } $readLock->release(); // read $read(); // end read $readLock->acquire(true); $readers = $readersStore ->decrease(); if ($readers == 0) { $writeLock->release(); } $readLock->release(); //begin write $writeLock->acquire(true); //write $write(); //end write $writeLock->release(); writer reader
70. //begin write $writeLock->acquire(true); //write $write(); //end write $writeLock->release(); writer
71. // begin read $readLock->acquire(true); $readers = $readersStore ->increase(); if ($readers == 1) { $writeLock->acquire(true); } $readLock->release(); // read $read(); // end read $readLock->acquire(true); $readers = $readersStore ->decrease(); if ($readers == 0) { $writeLock->release(); } $readLock->release(); //begin write $writeLock->acquire(true); //write $write(); //end write $writeLock->release(); writer reader
72. // begin read $readLock->acquire(true); $readers = $readersStore ->increase(); if ($readers == 1) { $writeLock->acquire(true); } $readLock->release(); reader
73. // begin read $readLock->acquire(true); $readers = $readersStore ->increase(); if ($readers == 1) { $writeLock->acquire(true); } $readLock->release(); // read $read(); // end read $readLock->acquire(true); $readers = $readersStore ->decrease(); if ($readers == 0) { $writeLock->release(); } $readLock->release(); //begin write $writeLock->acquire(true); //write $write(); //end write $writeLock->release(); writer reader
74. // end read $readLock->acquire(true); $readers = $readersStore ->decrease(); if ($readers == 0) { $writeLock->release(); } $readLock->release(); reader
75. Otros locks
76. distlock /w InnoDB • Shared mode lock • Reads as UPDATEs START TRANSACTION; SELECT * FROM Table WHERE name="foo" FOR UPDATE; ## Critical section COMMIT;
77. Dist consensus algorithms • Paxos made simple, Leslie Lamport • http://lamport.azurewebsites.net/pubs/paxos-simple.pdf • In Search of an Understandable Consensus Algorithmn (Extended Version), Diego Ontario and John Ousterhout • https://raft.github.io/raft.pdf • ZooKeeper’s atomic broadcast protocol: Theory and practice, André Medeiros • http://www.tcs.hut.fi/Studies/T-79.5001/reports/2012- deSouzaMedeiros.pdf
79. ¡Gracias!
80. we are hiring!
81. ¡Gracias! suarez.ortega.cesar@gmail.com