Symfony y concurrencia: el componente Lock

Los entornos concurrentes y distribuidos son una constante cuando queremos escalar un proyecto. Nos ayudan a la hora de construir sistemas capaces de procesar millones de peticiones, pero también dificultan nuestra manera de programar. Generan una incertidumbre en nuestras ejecuciones que debemos controlar, porque de nada vale un sistema con un gran rendimiento si no es fiable.

Los locks son una de las herramientas principales a la hora de sincronizar y ordenar distintas ejecuciones que deben acceder a un mismo recurso. En Symfony 3.4 se incluye un lock de serie en forma de componente. Éste nos servirá para ilustrar los problemas de los entornos concurrentes y las buenas prácticas que podemos adoptar a la hora de enfrentarnos a ellos.

Transcript

1. deSymfony 30 junio - 1 julio 2017
   Castellón
   SYMFONY Y CONCURRENCIA
   EL COMPONENTE LOCK
   César Suárez
   

   View Slide

2. View Slide

3. $ whoami
   • César Suárez
   • Backend developer @ https://mailtrack.io
   • One-man army @ https://suicidebystar.com
   • Producto típico de Extremadura
   • Twitter: @tharandur / GitHub: @csuarez
   

   View Slide

4. ¿Por qué una charla de
   concurrencia?
   

   View Slide

5. View Slide

6. Concurrency is the
   tendency for things to
   happen at the same time in
   a system.
   http://sce.uhcl.edu/helm/rationalunifiedprocess/process/workflow/
   

   View Slide

7. Concurrent algorithm is the
   description of a set of
   sequential state machines
   that cooperate through a
   communication medium
   Concurrent Programming: Algorithms, Principles and Foundations
   Michel Raynal
   

   View Slide

8. View Slide

9. Mutual
   Exclusion
   Problem
   

   View Slide

10. Critical sections
    A C
    A C
    P1
    P2
    B
    B
    

    View Slide

11. Critical sections
    A C
    A C
    P1
    P2
    B
    B
    

    View Slide

12. Critical sections
    A C
    A C
    P1
    P2
    B
    B
    

    View Slide

13. Critical sections
    A C
    A C
    P1
    P2
    B
    B
    

    View Slide

14. Critical sections
    A C
    A C
    P1
    P2
    B
    B
    

    View Slide

15. acquire_mutex()
    release_mutex()
    critical_section()
    

    View Slide

16. Propiedades
    • Mutual exclusion
    • Sólo puede haber un proceso a la vez en la
    sección crítica
    • Starvation-freedom
    • Todas las llamadas a acquire_mutex() han
    de terminar eventualmente
    

    View Slide

17. Locks
    

    View Slide

18. free locked
    lock.acquire()
    lock.release()
    

    View Slide

19. Solución al problema
    de exclusión mutua
    

    View Slide

20. ¿Implementación?
    

    View Slide

21. symfony/lock
    

    View Slide

22. symfony/lock
    • Symfony 3.3 3.4
    • By Jérémy Derussé (@jderusse)
    • Múltiples stores
    • Evolución de LockHandler
    • Pensado para comandos
    

    View Slide

23. symfony/lock 101
    $lock = $factory->createLock('some-id');
    if ($lock->acquire()) {
    /**
    * critical section
    */
    $lock->release();
    }
    

    View Slide

24. Lock creation
    use Symfony\Component\Lock\Factory;
    use Symfony\Component\Lock\Store\SemaphoreStore;
    $store = new SemaphoreStore();
    $factory = new Factory($store);
    $lock = $factory->createLock('some-id');
    

    View Slide

25. acquire()
    • Non-blocking acquire
    • Blocking acquire
    • Consultar si un lock está adquirido
    if ($lock->acquire()) { }
    $lock->acquire(true);
    $lock->isAcquired();
    

    View Slide

26. release()
    if ($lock->acquire()) {
    try {
    /**
    * critical section
    */
    } finally {
    $lock->release();
    }
    }
    

    View Slide

27. easy example
    

    View Slide

28. No lock
    $resource = new UnsafeSharedResource(
    'very-important-thing'
    );
    do {
    $counter = $resource->read();
    $resource->write(++$counter);
    } while(true);
    

    View Slide

29. View Slide

30. View Slide

31. $resource = new UnsafeSharedResource(
    'very-important-thing'
    );
    do {
    $counter = $resource->read();
    $resource->write(++$counter);
    } while(true);
    No lock
    

    View Slide

32. RE = 0 WR(1)
    P1
    P2 RE = 0 WR(1)
    RE = 1
    RE = 1
    

    View Slide

33. RE = 0 WR(1)
    P1
    P2 RE = 0 WR(1)
    RE = 1 RE = 1
    WR(2)
    

    View Slide

34. RE = 0 WR(1)
    P1
    P2 RE = 1 WR(2)
    RE = 2 WR(3)
    

    View Slide

35. Simple lock
    $lock = $factory->createLock('simple:lock');
    do {
    $lock->acquire(true);
    try {
    $counter = $resource->read();
    $resource->write(++$counter);
    } finally {
    $lock->release();
    }
    } while(true);
    

    View Slide

36. View Slide

37. Expiring locks
    • Creación
    • Incrementar Time To Live (TTL)
    $factory->createLock('expiring-lock', 30);
    $lock->refresh();
    ¡¡ Recomendado para distlocks !!
    

    View Slide

38. Stores
    

    View Slide

39. SemaphoreStore
    • Usa las funciones semaphore de PHP
    • En memoria (System V IPC)
    • No funciona en Windows
    • No distribuido
    • Non-Expiring
    

    View Slide

40. SemaphoreStore
    use Symfony\Component\Lock\Factory;
    use Symfony\Component\Lock\Store\SemaphoreStore;
    $store = new SemaphoreStore();
    $factory = new Factory($store);
    $lock = $factory->createLock('some-id');
    

    View Slide

41. FlockStore
    • Filesystem block
    • No distribuido
    • Non-expiring
    • +info: http://www.php.net/manual/en/
    function.flock.php
    

    View Slide

42. FlockStore
    •
    use Symfony\Component\Lock\Store\FlockStore;
    $store = new FlockStore(sys_get_temp_dir());
    $factory = new Factory($store);
    $lock = $factory->createLock('file-lock');
    

    View Slide

43. MemcachedStore
    • Cache as lock
    • Lock distribuido
    • Expiring locks
    • Non-blocking locks (sin RetryTillSaveStore)
    

    View Slide

44. MemcachedStore
    use Symfony\Component\Lock\Store\MemcachedStore;
    $memcachedConn = new \Memcached;
    $memcachedConn->addServer('memcached', 11211);
    $store = new MemcachedStore($memcachedConn);
    $factory = new Factory($store);
    $lock = $factory->createLock('memory-lock');
    

    View Slide

45. WARNING!
    

    View Slide

46. WARNING!
    

    View Slide

47. View Slide

48. View Slide

49. El problema con memcached
    • Evictions!
    • Least Recently Used cache (LRU)
    • ¿Soluciones?
    • Máquinas con poca carga
    • Tunear memcached
    

    View Slide

50. RedisStore
    • Lock distribuido
    • Expiring locks
    • Non-blocking locks (sin RetryTillSaveStore)
    • Control sobre los evictions.
    • Persistencia en disco.
    

    View Slide

51. RedisStore
    use Symfony\Component\Lock\Store\RedisStore;
    $redisConn = new \Predis\Client(
    'tcp://0.0.0.0:6379'
    );
    $store = new RedisStore($redisConn);
    $factory = new Factory($store);
    $lock = $factory->createLock('memory-lock');
    

    View Slide

52. CombinedStore
    • Combinación de varios Stores
    • Varias estrategias
    • ConsensusStrategy
    • UnanimousStrategy
    • High Aviability
    

    View Slide

53. CombinedStore
    use Symfony\Component\Lock\Store\RedisStore;
    use Symfony\Component\Lock\Strategy\ConsensusStrategy;
    $store1 = new RedisStore($redisConn1);
    $store2 = new RedisStore($redisConn2);
    $store3 = new RedisStore($redisConn3);
    $factory = new Factory(new CombinedStore(
    [$store1, $store2, $store3],
    new ConsensusStrategy()
    ));
    $lock = $factory->createLock('ha-lock');
    

    View Slide

54. View Slide

55. View Slide

56. El problema con Redis
    • Distributed lock with Redis (redlock)
    • https://redis.io/topics/distlock
    • How to distributed locking
    • http://martin.kleppmann.com/2016/02/08/how-to-do-
    distributed-locking.html
    • Is Redlock safe? Reply to Redlock Analysis
    • http://antirez.com/news/101
    • https://news.ycombinator.com/item?id=11065933
    

    View Slide

57. http://martin.kleppmann.com/2016/02/08/how-to-do-distributed-locking.html
    

    View Slide

58. View Slide

59. View Slide

60. ¿Por qué usar un lock?
    

    View Slide

61. 1. Exactitud
    

    View Slide

62. 2. Rendimiento
    

    View Slide

63. No “big deal” scenarios
    

    View Slide

64. View Slide

65. Integración con Symfony
    framework:
    lock:
    users: ['memcached://m1.docker', 'memcached://m2.docker']
    admin: 'flock'
    partners: 'redis://r1.docker'
    invoices: 'semaphore'
    $adminLock = $this->get('lock.admin')->acquire();
    $invoicesLock = $this->get('lock.invoices')->acquire(true);
    • PHP
    • Configuración
    

    View Slide

66. More examples
    https://github.com/csuarez/symfony-lock-playground
    

    View Slide

67. Read/Write lock
    

    View Slide

68. View Slide

69. // begin read
    $readLock->acquire(true);
    $readers = $readersStore
    ->increase();
    if ($readers == 1) {
    $writeLock->acquire(true);
    }
    $readLock->release();
    // read
    $read();
    // end read
    $readLock->acquire(true);
    $readers = $readersStore
    ->decrease();
    if ($readers == 0) {
    $writeLock->release();
    }
    $readLock->release();
    //begin write
    $writeLock->acquire(true);
    //write
    $write();
    //end write
    $writeLock->release();
    writer
    reader
    

    View Slide

70. //begin write
    $writeLock->acquire(true);
    //write
    $write();
    //end write
    $writeLock->release();
    writer
    

    View Slide

71. // begin read
    $readLock->acquire(true);
    $readers = $readersStore
    ->increase();
    if ($readers == 1) {
    $writeLock->acquire(true);
    }
    $readLock->release();
    // read
    $read();
    // end read
    $readLock->acquire(true);
    $readers = $readersStore
    ->decrease();
    if ($readers == 0) {
    $writeLock->release();
    }
    $readLock->release();
    //begin write
    $writeLock->acquire(true);
    //write
    $write();
    //end write
    $writeLock->release();
    writer
    reader
    

    View Slide

72. // begin read
    $readLock->acquire(true);
    $readers = $readersStore
    ->increase();
    if ($readers == 1) {
    $writeLock->acquire(true);
    }
    $readLock->release();
    reader
    

    View Slide

73. // begin read
    $readLock->acquire(true);
    $readers = $readersStore
    ->increase();
    if ($readers == 1) {
    $writeLock->acquire(true);
    }
    $readLock->release();
    // read
    $read();
    // end read
    $readLock->acquire(true);
    $readers = $readersStore
    ->decrease();
    if ($readers == 0) {
    $writeLock->release();
    }
    $readLock->release();
    //begin write
    $writeLock->acquire(true);
    //write
    $write();
    //end write
    $writeLock->release();
    writer
    reader
    

    View Slide

74. // end read
    $readLock->acquire(true);
    $readers = $readersStore
    ->decrease();
    if ($readers == 0) {
    $writeLock->release();
    }
    $readLock->release();
    reader
    

    View Slide

75. Otros locks
    

    View Slide

76. distlock /w InnoDB
    • Shared mode lock
    • Reads as UPDATEs
    START TRANSACTION;
    SELECT *
    FROM Table
    WHERE name="foo"
    FOR UPDATE;
    ## Critical section
    COMMIT;
    

    View Slide

77. Dist consensus algorithms
    • Paxos made simple, Leslie Lamport
    • http://lamport.azurewebsites.net/pubs/paxos-simple.pdf
    • In Search of an Understandable Consensus Algorithmn (Extended
    Version), Diego Ontario and John Ousterhout
    • https://raft.github.io/raft.pdf
    • ZooKeeper’s atomic broadcast protocol: Theory and practice, André
    Medeiros
    • http://www.tcs.hut.fi/Studies/T-79.5001/reports/2012-
    deSouzaMedeiros.pdf
    

    View Slide

78. View Slide

79. ¡Gracias!
    

    View Slide

80. we are hiring!
    

    View Slide

81. ¡Gracias!
    suarez.ortega.cesar@gmail.com
    

    View Slide