JCConf Taiwan 2023 - Let's Read the Source Code of Spring Distributed Lock with Redis!

In this talk, we embark on a journey to unravel the intricate mechanisms of Spring Distributed Lock using Redis. With a keen focus on source code analysis, we will unveil essential insights into the architecture of RedisLockRegistry. This talk also shares the experience of contributing to Spring Integration project.

The article related to the talk: https://hackmd.io/@GzkKWjbLTt6YExzxvjL9OA/H1fBMvJ63

JCConf Taiwan 2023: https://jcconf.tw/2023/

Transcript

1. Let's Read the Source Code of Spring Distributed Lock with Redis! Eddie Cho
2. 2 Agenda 1. Introduction to Distributed Lock 2. Let’s Read the Source Code of Spring Distributed Lock with Redis (6.1.2) 3. Contribution to Spring-Integration project
3. 3 1. Introduction to Distributed Lock
4. 4 1-1. Java Lock for Concurrency Control with Multiple Threads Thread 1 Thread 2 Lock lock() Critical Section Thread 2 blocked lock() unlock() time
5. 5 1-2. Distributed Lock for Concurrency Control with Multiple Processes Process 1 Process 2 Lock Status acquire Critical Section ok time acquire fail acquire fail release ok acquire ok
6. 6 1-3. Acquire a Spring Distributed Lock with Redis Process Redis Node Acquire the lock: If the lock is not owned by any process Insert the status of the lock ok time key: lockName value: processId TTL: 60000
7. 7 1-3. Acquire a Spring Distributed Lock with Redis Process 1 Redis Node ok time Process 2 Process 1 acquire TTL key: lockName value: process1 TTL: 60000 key: lockName value: process2 TTL: 60000 acquire ok
8. 8 1-4. Release a Spring Distributed Lock with Redis Process Redis Node Release the lock: If the lock is owned by the process Remove the status of the lock ok time key: lockName value: processId TTL: 60000
9. 9 1-5. Different Implementations of Distributed Lock Provide by Spring-Integration Project Reference: Spring Integration - Distributed Locks
10. 10 1-6. Implementation of Spring Distributed Lock with Redis
11. 11 2. Let’s Read the Source Code of Spring Distributed Lock with Redis(6.1.2)
12. 12 2-1. How to use Spring Distributed Lock? final Lock lock = lockRegistry.obtain(“key”); if (lock.tryLock(10, TimeUnit.MINUTES)) { try { // do something } catch (Exception e) { // log issue } finally { lock.unlock(); } } Reference: How To Implement a Spring Distributed Lock
13. 13 2-2. The Construction of RedisLockRegistry Bean(6.1.2) public final class RedisLockRegistry { //ID of the process private final String clientId = UUID.randomUUID().toString(); private final String registryKey; //prefix of the name of each key private final StringRedisTemplate redisTemplate; private final long expireAfter;.//TTL of each key public RedisLockRegistry(RedisConnectionFactory connectionFactory, String registryKey, long expireAfter) { this.redisTemplate = new StringRedisTemplate(connectionFactory); this.registryKey = registryKey; this.expireAfter = expireAfter; //... } }
14. 14 2-3. Obtaining Locks from RedisLockRegistry(6.1.2) final Lock lock = lockRegistry.obtain(“key”); if (lock.tryLock(10, TimeUnit.MINUTES)) { try { // do something } catch (Exception e) { // log issue } finally { lock.unlock(); } } Reference: How To Implement a Spring Distributed Lock
15. 15 2-3. Obtaining Locks from RedisLockRegistry(6.1.2) public final class RedisLockRegistry { private RedisLockType redisLockType = RedisLockType.SPIN_LOCK; private final Lock lock = new ReentrantLock(); private final Map<String, RedisLock> locks = new LinkedHashMap<>(); @Override public Lock obtain(Object lockKey) { String path = (String) lockKey; this.lock.lock(); try { return this.locks.computeIfAbsent( path,getRedisLockConstructor(this.redisLockType)); } finally { this.lock.unlock(); } } private Function<String, RedisLock> getRedisLockConstructor( RedisLockRegistry.RedisLockType redisLockType) { return switch (redisLockType) { case SPIN_LOCK -> RedisLockRegistry.RedisSpinLock::new; case PUB_SUB_LOCK -> RedisLockRegistry.RedisPubSubLock::new; }; }
16. 16 2-3. Obtaining Locks from RedisLockRegistry(6.1.2) public final class RedisLockRegistry { private final String registryKey; private abstract class RedisLock implements Lock { protected final String lockKey; private RedisLock(String path) { // e.g., registryKey: “spring” // path: “integration” // lockKey = “spring:integration” this.lockKey = constructLockKey(path); } private String constructLockKey(String path) { return RedisLockRegistry.this.registryKey + ':' + path; } } }
17. 17 2-4. Acquiring a Distributed Lock Through Invoking tryLock Method of RedisLock final Lock lock = lockRegistry.obtain(“key”); if (lock.tryLock(10, TimeUnit.MINUTES)) { try { // do something } catch (Exception e) { // log issue } finally { lock.unlock(); } } Reference: How To Implement a Spring Distributed Lock
18. 18 2-4. Acquiring a Distributed Lock Through Invoking tryLock Method of RedisLock(6.1.2) private abstract class RedisLock implements Lock { private final ReentrantLock localLock = new ReentrantLock(); @Override public final boolean tryLock(long time, TimeUnit unit) throws InterruptedException { if (!this.localLock.tryLock(time, unit)) { return false; } try { long waitTime = TimeUnit.MILLISECONDS.convert(time, unit); // Try to insert/update the status of lock into Redis boolean acquired = tryRedisLock(waitTime); if (!acquired) { this.localLock.unlock(); } return acquired; } catch (Exception e) { this.localLock.unlock(); rethrowAsLockException(e); } return false; } }
19. 19 2-4. Acquiring a Distributed Lock Through Invoking tryLock Method of RedisLock(6.1.2) public final class RedisSpinLock extends RedisLock { @Override protected boolean tryRedisLockInner(long time) throws InterruptedException { long now = System.currentTimeMillis(); if (time == -1L) { while (!obtainLock()) { Thread.sleep(100); } return true; } else { long expire = now + TimeUnit.MILLISECONDS.convert(time, TimeUnit.MILLISECONDS); boolean acquired; while (!(acquired = obtainLock()) && System.currentTimeMillis() < expire){ Thread.sleep(100); } return acquired; } } }
20. 20 2-4. Acquiring a Distributed Lock Through Invoking tryLock Method of RedisLock(6.1.2) private abstract class RedisLock implements Lock { protected final Boolean obtainLock() { return RedisLockRegistry.this.redisTemplate.execute( OBTAIN_LOCK_REDIS_SCRIPT, Collections.singletonList(this.lockKey),RedisLockRegistry.this.clientId, String.valueOf(RedisLockRegistry.this.expireAfter)); } // KEYS[1]: keyName (this.lockKey) // ARGV[1]: processId (RedisLockRegistry.this.clientId) // ARGV[2]: TTL (RedisLockRegistry.this.expireAfter) private static final String OBTAIN_LOCK_SCRIPT = """ local lockClientId = redis.call('GET', KEYS[1]) if lockClientId == ARGV[1] then redis.call('PEXPIRE', KEYS[1], ARGV[2]) return true elseif not lockClientId then redis.call('SET', KEYS[1], ARGV[1], 'PX', ARGV[2]) return true end return false"""; protected static final RedisScript<Boolean> OBTAIN_LOCK_REDIS_SCRIPT = … ; }
21. 21 2-4. Acquiring a Distributed Lock Through Invoking tryLock Method of RedisLock(6.1.2) Reference: Redis – Scripting with Lua
22. 22 2-5. Releasing a Distributed Lock by Invoking the unlock Method of RedisLock final Lock lock = lockRegistry.obtain(“key”); if (lock.tryLock(10, TimeUnit.MINUTES)) { try { // do something } catch (Exception e) { // log issue } finally { lock.unlock(); } } Reference: How To Implement a Spring Distributed Lock
23. 23 2-5. Releasing a Distributed Lock by Invoking the unlock Method of RedisLock(6.1.2) private abstract class RedisLock implements Lock { private final ReentrantLock localLock = new ReentrantLock(); @Override public final void unlock() { if (!this.localLock.isHeldByCurrentThread()) { throw new IllegalStateException(); } if (this.localLock.getHoldCount() > 1) { this.localLock.unlock(); return; } try { if (!isAcquiredInThisProcess()) { // GET operation and verify ownership throw new IllegalStateException(); } removeLockKey(); // UNLINK/DEL to remove the ownership } catch (Exception e) { ReflectionUtils.rethrowRuntimeException(e); } finally { this.localLock.unlock(); } }}
24. 24 2-5. Releasing a Distributed Lock by Invoking the unlock Method of RedisLock(6.1.2) private abstract class RedisLock implements Lock { private void removeLockKey() { if (RedisLockRegistry.this.unlinkAvailable) { try { removeLockKeyInnerUnlink(); return; } catch (Exception ex) { RedisLockRegistry.this.unlinkAvailable = false; //... } } removeLockKeyInnerDelete(); } } private final class RedisSpinLock extends RedisLockRegistry.RedisLock { protected void removeLockKeyInnerUnlink() { RedisLockRegistry.this.redisTemplate.unlink(this.lockKey); } protected void removeLockKeyInnerDelete() { RedisLockRegistry.this.redisTemplate.delete(this.lockKey); } }
25. 25 2-6. Summary of RedisLockRegistry. RedisLockRegistry: The registry where keeps the instances of RedisLock. RedisLock: The abstraction class of the distributed lock, it provides the general implementation of the lock, and unlock methods for different lock types. RedisSpinLock: One of the implementation of RedisLock, it provides the details implementation of the lock, and unlock methods e.g., tryRedisLockInner, removeLockKeyInnerUnlink, and removeLockKeyInnerDelete.
26. 26 3. Contribution to Spring-Integration project
27. 27 3-1. Issue of unlock Method of RedisLock(6.1.2) private abstract class RedisLock implements Lock { private final ReentrantLock localLock = new ReentrantLock(); @Override public final void unlock() { //... try { if (!isAcquiredInThisProcess()) { // GET operation and verify ownership throw new IllegalStateException(); } removeLockKey(); // UNLINK/DEL to remove the ownership } //... } }
28. 28 3-1. Issue of unlock Method of RedisLock(6.1.2) Process 1 Process 2 Redis Node GET lock:key return process1 lock:key expired SET lock:key process2 DEL lock:key ok time ok Lock status: owned by process1 Lock status: owned by process1 Lock status: owned by process2
29. 29 3-2. Enhance unlock() Method of RedisLock with an Atomic Operation Process Redis Node execute a Lua script to verify ownership of the lock and remove it. ok time key: lockName value: processId TTL: 60000
30. 30 3-2. Enhance unlock() Method of RedisLock with an Atomic Operation Process 1 Process 2 Redis Node lock:key expired execute unlock script ok time fail Lock status: owned by process2 acquire Lock status: owned by process1
31. 31 3-2. Enhance unlock() Method of RedisLock with an Atomic Operation Process 1 Process 2 Redis Node lock:key expired execute unlock script ok time fail acquire ok Lock status: owned by process1 Lock status: owned by process2 Lock status: empty execute unlock script
32. 32 3-2. Enhance unlock() Method of RedisLock with an Atomic Operation(RedisSpinLock) protected boolean removeLockKeyInnerUnlink() { return Boolean.TRUE.equals(RedisLockRegistry.this.redisTemplate.execute( UNLINK_UNLOCK_REDIS_SCRIPT, Collections.singletonList(this.lockKey), RedisLockRegistry.this.clientId)); } // KEYS[1]: keyName (this.lockKey) // ARGV[1]: processId (RedisLockRegistry.this.clientId) private static final String UNLINK_UNLOCK_SCRIPT = """ local lockClientId = redis.call('GET', KEYS[1]) if lockClientId == ARGV[1] then redis.call('UNLINK', KEYS[1]) return true end return false"""; private static final RedisScript<Boolean> UNLINK_UNLOCK_REDIS_SCRIPT = new DefaultRedisScript<>(UNLINK_UNLOCK_SCRIPT, Boolean.class); protected boolean removeLockKeyInnerDelete() { //... }
33. 33 3-2. Enhance unlock() Method of RedisLock with an Atomic Operation(RedisLock) private void removeLockKey() { if(RedisLockRegistry.this.unlinkAvailable) { try { removeLockKeyInnerUnlink(); return; } catch (Exception ex) { RedisLockRegistry.this.unlinkAvailable = false; //... } } removeLockKeyInnerDelete(); } private void removeLockKey() { if(RedisLockRegistry.this.unlinkAvailable) { Boolean unlinkResult = null; try { unlinkResult = removeLockKeyInnerUnlink(); } catch (Exception ex) { RedisLockRegistry.this.unlinkAvailable = false; //... } if (Boolean.TRUE.equals(unlinkResult)) { return; } else if(Boolean.FALSE.equals( unlinkResult)){ throw new IllegalStateException(); } } if (!removeLockKeyInnerDelete()) { throw new IllegalStateException(); } }
34. 34 3-2. Enhance unlock() Method of RedisLock with an Atomic Operation private abstract class RedisLock implements Lock { @Override public final void unlock() { //... try { // If current process owns the lock // Then remove the status of lock // from the Redis node removeLockKey(); } //... } } private abstract class RedisLock implements Lock { @Override public final void unlock() { //... try { //GET operation and verify ownership if (!isAcquiredInThisProcess()) { throw new IllegalStateException(); } //DEL/UNLINK to remove the ownership removeLockKey(); } //... } }
35. 35 3-3. Contribute to Spring-Integration project
36. 36 3-3. Contribute to Spring-Integration project
37. 37 3-3. Contribute to Spring-Integration project
38. 38 3-3. Contribute to Spring-Integration project
39. 39 3-3. Contribute to Spring-Integration project
40. 40 3-3. Contribute to Spring-Integration project
41. 41 References and Further Readings  RedisLockRegistry(Spring-Integration 6.1.2)  How To Implement a Spring Distributed Lock  Redis programmability  Distributed Locks with Redis  Pull Request - #8700  Pull Request - #8702  How Does Spring Distributed Lock Work? Exploring Source Code of RedisLockRegistry a nd RedisSpinLock  How to contribute to (Scala) open source & my experience  How to Contribute to Open Source
42. 42 Q&A