Save 37% off PRO during our Black Friday Sale! »

Symfony y concurrencia: el componente Lock

Symfony y concurrencia: el componente Lock

Los entornos concurrentes y distribuidos son una constante cuando queremos escalar un proyecto. Nos ayudan a la hora de construir sistemas capaces de procesar millones de peticiones, pero también dificultan nuestra manera de programar. Generan una incertidumbre en nuestras ejecuciones que debemos controlar, porque de nada vale un sistema con un gran rendimiento si no es fiable.

Los locks son una de las herramientas principales a la hora de sincronizar y ordenar distintas ejecuciones que deben acceder a un mismo recurso. En Symfony 3.4 se incluye un lock de serie en forma de componente. Éste nos servirá para ilustrar los problemas de los entornos concurrentes y las buenas prácticas que podemos adoptar a la hora de enfrentarnos a ellos.

267ee9b8bfeb0c72b5dbe643bbc4433b?s=128

César Suárez Ortega

July 01, 2017
Tweet

Transcript

  1. deSymfony 30 junio - 1 julio 2017 Castellón SYMFONY Y

    CONCURRENCIA EL COMPONENTE LOCK César Suárez
  2. None
  3. $ whoami • César Suárez • Backend developer @ https://mailtrack.io

    • One-man army @ https://suicidebystar.com • Producto típico de Extremadura • Twitter: @tharandur / GitHub: @csuarez
  4. ¿Por qué una charla de concurrencia?

  5. None
  6. Concurrency is the tendency for things to happen at the

    same time in a system. http://sce.uhcl.edu/helm/rationalunifiedprocess/process/workflow/
  7. Concurrent algorithm is the description of a set of sequential

    state machines that cooperate through a communication medium Concurrent Programming: Algorithms, Principles and Foundations Michel Raynal
  8. None
  9. Mutual Exclusion Problem

  10. Critical sections A C A C P1 P2 B B

  11. Critical sections A C A C P1 P2 B B

  12. Critical sections A C A C P1 P2 B B

  13. Critical sections A C A C P1 P2 B B

  14. Critical sections A C A C P1 P2 B B

  15. acquire_mutex() release_mutex() critical_section()

  16. Propiedades • Mutual exclusion • Sólo puede haber un proceso

    a la vez en la sección crítica • Starvation-freedom • Todas las llamadas a acquire_mutex() han de terminar eventualmente
  17. Locks

  18. free locked lock.acquire() lock.release()

  19. Solución al problema de exclusión mutua

  20. ¿Implementación?

  21. symfony/lock

  22. symfony/lock • Symfony 3.3 3.4 • By Jérémy Derussé (@jderusse)

    • Múltiples stores • Evolución de LockHandler • Pensado para comandos
  23. symfony/lock 101 $lock = $factory->createLock('some-id'); if ($lock->acquire()) { /** *

    critical section */ $lock->release(); }
  24. Lock creation use Symfony\Component\Lock\Factory; use Symfony\Component\Lock\Store\SemaphoreStore; $store = new SemaphoreStore();

    $factory = new Factory($store); $lock = $factory->createLock('some-id');
  25. acquire() • Non-blocking acquire • Blocking acquire • Consultar si

    un lock está adquirido if ($lock->acquire()) { } $lock->acquire(true); $lock->isAcquired();
  26. release() if ($lock->acquire()) { try { /** * critical section

    */ } finally { $lock->release(); } }
  27. easy example

  28. No lock $resource = new UnsafeSharedResource( 'very-important-thing' ); do {

    $counter = $resource->read(); $resource->write(++$counter); } while(true);
  29. None
  30. None
  31. $resource = new UnsafeSharedResource( 'very-important-thing' ); do { $counter =

    $resource->read(); $resource->write(++$counter); } while(true); No lock
  32. RE = 0 WR(1) P1 P2 RE = 0 WR(1)

    RE = 1 RE = 1
  33. RE = 0 WR(1) P1 P2 RE = 0 WR(1)

    RE = 1 RE = 1 WR(2)
  34. RE = 0 WR(1) P1 P2 RE = 1 WR(2)

    RE = 2 WR(3)
  35. Simple lock $lock = $factory->createLock('simple:lock'); do { $lock->acquire(true); try {

    $counter = $resource->read(); $resource->write(++$counter); } finally { $lock->release(); } } while(true);
  36. None
  37. Expiring locks • Creación • Incrementar Time To Live (TTL)

    $factory->createLock('expiring-lock', 30); $lock->refresh(); ¡¡ Recomendado para distlocks !!
  38. Stores

  39. SemaphoreStore • Usa las funciones semaphore de PHP • En

    memoria (System V IPC) • No funciona en Windows • No distribuido • Non-Expiring
  40. SemaphoreStore use Symfony\Component\Lock\Factory; use Symfony\Component\Lock\Store\SemaphoreStore; $store = new SemaphoreStore(); $factory

    = new Factory($store); $lock = $factory->createLock('some-id');
  41. FlockStore • Filesystem block • No distribuido • Non-expiring •

    +info: http://www.php.net/manual/en/ function.flock.php
  42. FlockStore • use Symfony\Component\Lock\Store\FlockStore; $store = new FlockStore(sys_get_temp_dir()); $factory =

    new Factory($store); $lock = $factory->createLock('file-lock');
  43. MemcachedStore • Cache as lock • Lock distribuido • Expiring

    locks • Non-blocking locks (sin RetryTillSaveStore)
  44. MemcachedStore use Symfony\Component\Lock\Store\MemcachedStore; $memcachedConn = new \Memcached; $memcachedConn->addServer('memcached', 11211); $store

    = new MemcachedStore($memcachedConn); $factory = new Factory($store); $lock = $factory->createLock('memory-lock');
  45. WARNING!

  46. WARNING!

  47. None
  48. None
  49. El problema con memcached • Evictions! • Least Recently Used

    cache (LRU) • ¿Soluciones? • Máquinas con poca carga • Tunear memcached
  50. RedisStore • Lock distribuido • Expiring locks • Non-blocking locks

    (sin RetryTillSaveStore) • Control sobre los evictions. • Persistencia en disco.
  51. RedisStore use Symfony\Component\Lock\Store\RedisStore; $redisConn = new \Predis\Client( 'tcp://0.0.0.0:6379' ); $store

    = new RedisStore($redisConn); $factory = new Factory($store); $lock = $factory->createLock('memory-lock');
  52. CombinedStore • Combinación de varios Stores • Varias estrategias •

    ConsensusStrategy • UnanimousStrategy • High Aviability
  53. CombinedStore use Symfony\Component\Lock\Store\RedisStore; use Symfony\Component\Lock\Strategy\ConsensusStrategy; $store1 = new RedisStore($redisConn1); $store2

    = new RedisStore($redisConn2); $store3 = new RedisStore($redisConn3); $factory = new Factory(new CombinedStore( [$store1, $store2, $store3], new ConsensusStrategy() )); $lock = $factory->createLock('ha-lock');
  54. None
  55. None
  56. El problema con Redis • Distributed lock with Redis (redlock)

    • https://redis.io/topics/distlock • How to distributed locking • http://martin.kleppmann.com/2016/02/08/how-to-do- distributed-locking.html • Is Redlock safe? Reply to Redlock Analysis • http://antirez.com/news/101 • https://news.ycombinator.com/item?id=11065933
  57. http://martin.kleppmann.com/2016/02/08/how-to-do-distributed-locking.html

  58. None
  59. None
  60. ¿Por qué usar un lock?

  61. 1. Exactitud

  62. 2. Rendimiento

  63. No “big deal” scenarios

  64. None
  65. Integración con Symfony framework: lock: users: ['memcached://m1.docker', 'memcached://m2.docker'] admin: 'flock'

    partners: 'redis://r1.docker' invoices: 'semaphore' $adminLock = $this->get('lock.admin')->acquire(); $invoicesLock = $this->get('lock.invoices')->acquire(true); • PHP • Configuración
  66. More examples https://github.com/csuarez/symfony-lock-playground

  67. Read/Write lock

  68. None
  69. // begin read $readLock->acquire(true); $readers = $readersStore ->increase(); if ($readers

    == 1) { $writeLock->acquire(true); } $readLock->release(); // read $read(); // end read $readLock->acquire(true); $readers = $readersStore ->decrease(); if ($readers == 0) { $writeLock->release(); } $readLock->release(); //begin write $writeLock->acquire(true); //write $write(); //end write $writeLock->release(); writer reader
  70. //begin write $writeLock->acquire(true); //write $write(); //end write $writeLock->release(); writer

  71. // begin read $readLock->acquire(true); $readers = $readersStore ->increase(); if ($readers

    == 1) { $writeLock->acquire(true); } $readLock->release(); // read $read(); // end read $readLock->acquire(true); $readers = $readersStore ->decrease(); if ($readers == 0) { $writeLock->release(); } $readLock->release(); //begin write $writeLock->acquire(true); //write $write(); //end write $writeLock->release(); writer reader
  72. // begin read $readLock->acquire(true); $readers = $readersStore ->increase(); if ($readers

    == 1) { $writeLock->acquire(true); } $readLock->release(); reader
  73. // begin read $readLock->acquire(true); $readers = $readersStore ->increase(); if ($readers

    == 1) { $writeLock->acquire(true); } $readLock->release(); // read $read(); // end read $readLock->acquire(true); $readers = $readersStore ->decrease(); if ($readers == 0) { $writeLock->release(); } $readLock->release(); //begin write $writeLock->acquire(true); //write $write(); //end write $writeLock->release(); writer reader
  74. // end read $readLock->acquire(true); $readers = $readersStore ->decrease(); if ($readers

    == 0) { $writeLock->release(); } $readLock->release(); reader
  75. Otros locks

  76. distlock /w InnoDB • Shared mode lock • Reads as

    UPDATEs START TRANSACTION; SELECT * FROM Table WHERE name="foo" FOR UPDATE; ## Critical section COMMIT;
  77. Dist consensus algorithms • Paxos made simple, Leslie Lamport •

    http://lamport.azurewebsites.net/pubs/paxos-simple.pdf • In Search of an Understandable Consensus Algorithmn (Extended Version), Diego Ontario and John Ousterhout • https://raft.github.io/raft.pdf • ZooKeeper’s atomic broadcast protocol: Theory and practice, André Medeiros • http://www.tcs.hut.fi/Studies/T-79.5001/reports/2012- deSouzaMedeiros.pdf
  78. None
  79. ¡Gracias!

  80. we are hiring!

  81. ¡Gracias! suarez.ortega.cesar@gmail.com