Go言語での排他制御/Mutual exclusion in Golang

Geeks Who Drink in Fukuoka -Go Go Golang Edition!-
https://nulab.connpass.com/event/57238/

Transcript

1. GoݴޠͰͷഉଞ੍ޚ monochromegane / Pepabo R&D Institute, GMO Pepabo, Inc. 2017.06.28 Geeks Who Drink in Fukuoka -Go Go Golang Edition!- Mutual exclusion in Golang.
2. ϓϦϯγύϧΤϯδχΞ ࡾ୐ ༔հ / @monochromegane 2 http://blog.monochromegane.com Yusuke Miyake ϖύϘݚڀॴ ݚڀһ
3. 1. Motivation 2. Mutual exclusion 3. Resource locking 3 Contents
4. 1. Motivation
5. Gannoy Approximate nearest neighbor search server and dynamic index written in Golang. https://github.com/monochromegane/gannoy
6. Similar images search system using Gannoy 6 Features Similar items Gannoy [2048]float64 query by http find similar features mapping similar features to items response Deep CNN index Features [2048]float64 Deep CNN register by http
7. Many goroutines access one resource concurrently.
8. 2. Mutual exclusion
9. Many goroutines access one variable concurrently.
10. • sync.Mutex • A Mutex is a mutual exclusion lock. • sync.RWMutex • An RWMutex is a reader/writer mutual exclusion lock. The lock can be held by an arbitrary number of readers or a single writer. 10 Mutual exclusion in Golang
12. 12 type Free struct { mu sync.Mutex free []int } func (f *Free) push(id int) { f.mu.Lock() defer f.mu.Unlock() f.free = append(f.free, id) } func (f *Free) pop() (int, error) { f.mu.Lock() defer f.mu.Unlock() if len(f.free) == 0 { return -1, fmt.Errorf("empty") } x, newFree := f.free[len(f.free)-1], f.free[:len(f.free)-1] f.free = newFree return x, nil } Free nodes stack using sync.Mutex Exclusive lock Exclusive lock
13. 13 type Maps struct { mu *sync.RWMutex keyToId map[int]int } func (m *Maps) add(id, key int) { m.mu.Lock() defer m.mu.Unlock() m.keyToId[key] = id } func (m Maps) getId(key int) (int, error) { m.mu.RLock() defer m.mu.RUnlock() if id, ok := m.keyToId[key]; !ok { return -1, fmt.Errorf("not found") } else { return id, nil } } Mapping id and key using sync.RWMutex Exclusive lock Shared lock
14. • Mutex and RWMutex must not be copied after first use. 14 Caution type Maps struct { mu sync.RWMutex keyToId map[int]int } func (m Maps) getId(key int) (int, error) { fmt.Printf("mu address in getId: %p\n", &m.mu) m.mu.RLock() defer m.mu.RUnlock() if id, ok := m.keyToId[key]; !ok { return -1, fmt.Errorf("not found") } else { return id, nil } } If type of member isn’t a pointer receiver isn’t a pointer $ go run -race main.go mu address in getId: 0xc420010110 mu address in getId: 0xc4200781c0 ================== WARNING: DATA RACE wg.Add(4) for i := 0; i < 2; i++ { go func(i int) { maps.add(i, i) wg.Done() }(i) go func(i int) { maps.getId(i) wg.Done() }(i) } wg.Wait() and
15. 3. Resource locking
16. Many goroutines access one file concurrently.
17. Data structure of Gannoy 17 Hyperplane Logical data structure Data structure on file system Bucket Bucket /PEF
    'FBUVSFT /PEF
    'FBUVSFT /PEF
    'FBUVSFT /PEF
    'FBUVSFT Node (Features) /PEF
    'FBUVSFT /PEF
    'FBUVSFT /PEF
    'FBUVSFT /PEF
    'FBUVSFT Node (Features) query features cosine similarity > 0 <= 0 B N H N free (1byte) number of descendants (4byte) key (4byte) parents (4byte * tree) children (4byte * K) or hyperplane / features (8byte* dim) …
18. • syscall.Flock • apply or remove an advisory lock on an open file. • syscall.FcntlFlock • file control. F_SETLK, F_SETLKW, and F_GETLK are used to acquire, release, and test for the existence of record locks. 18 Resource locking in Golang (using system call) locks associated with Flock file open file description Fcntl byte range process ID and i-node
19. 19 // Flock func flock(fd uintptr) { syscall.Flock(int(fd), syscall.LOCK_EX) // or LOCK_SH defer syscall.Flock(int(fd), syscall.LOCK_UN) } // Fcntl + F_SETLKW func fcntl(fd uintptr, start, len int64) { syscall.FcntlFlock(fd, syscall.F_SETLKW, &syscall.Flock_t{ Start: start, Len: len, Type: syscall.F_WRLCK, // or syscall.F_RDLCK Whence: io.SeekStart, }) defer syscall.FcntlFlock(fd, syscall.F_SETLKW, &syscall.Flock_t{ Start: start, Len: len, Type: syscall.F_UNLCK, Whence: io.SeekStart, }) } Flock Shared or Exclusive lock Fcntl + F_SETLKW Shared or Exclusive lock byte range
20. 20 file descriptor fd0 fd1 fd2 Open file description 0 1 2 10 20 i-node 1000 1001 1002 2000 3000 file descriptor fd0 fd1 fd2 fd10 dup fork Process System System Goroutine Goroutine OFD 0 2 ProcessID i-node 100 1000 101 1001 Flock Fcntl + F_SETLKW Locks that associated with same one don’t block each other.
21. 21 file1, _ := os.OpenFile("dummy.txt", os.O_RDWR, 0) file2, _ := os.OpenFile("dummy.txt", os.O_RDWR, 0) fd1 := file1.Fd() fd2 := file2.Fd() var wg sync.WaitGroup wg.Add(2) locker := gannoy.Flock{} go lockAndProcess("fd1", fd1, locker, &wg) go lockAndProcess("fd2", fd2, locker, &wg) wg.Wait() fd2 processed something (1sec). fd2 processed something (2sec). fd2 processed something (3sec). fd1 processed something (1sec). fd1 processed something (2sec). fd1 processed something (3sec). locker := gannoy.Fcntl{} fd2 processed something (1sec). fd1 processed something (1sec). fd2 processed something (2sec). fd1 processed something (2sec). fd2 processed something (3sec). fd1 processed something (3sec). No blocking Blocking Flock Fcntl + F_SETLKW
22. 22 func lockAndProcess(name string, fd uintptr, locker gannoy.Locker, wg *sync.WaitGroup) { locker.WriteLock(fd, 0, 4) defer locker.UnLock(fd, 0, 4) t := time.NewTicker(1 * time.Second) count := 1 loop: for { select { case <-t.C: fmt.Printf("%s processed something (%d sec).\n", name, count) count++ if count > 3 { break loop } } } t.Stop() wg.Done() }
23. Open file description lock open file description locks are associated with the open file description on which they are acquired, http://man7.org/linux/man-pages/man2/fcntl.2.html ❝ https://www.oreilly.co.jp/community/blog/2014/11/file-lock-and-new-ofd-lock.html
24. 24 type Fcntl struct { } const F_OFD_SETLKW = 38 func (f Fcntl) ReadLock(fd uintptr, start, len int64) error { return f.fcntl(syscall.F_RDLCK, fd, start, len) } func (f Fcntl) WriteLock(fd uintptr, start, len int64) error { return f.fcntl(syscall.F_WRLCK, fd, start, len) } func (f Fcntl) UnLock(fd uintptr, start, len int64) error { return f.fcntl(syscall.F_UNLCK, fd, start, len) } func (f Fcntl) fcntl(typ int16, fd uintptr, start, len int64) error { return syscall.FcntlFlock(fd, F_OFD_SETLKW, &syscall.Flock_t{ Start: start, Len: len, Type: typ, Whence: io.SeekStart, }) } Command for working with open file description locks Exclusive lock Shared lock
25. 25 file1, _ := os.OpenFile("dummy.txt", os.O_RDWR, 0) file2, _ := os.OpenFile("dummy.txt", os.O_RDWR, 0) fd1 := file1.Fd() fd2 := file2.Fd() var wg sync.WaitGroup wg.Add(2) locker := gannoy.Flock{} go lockAndProcess("fd1", fd1, locker, &wg) go lockAndProcess("fd2", fd2, locker, &wg) wg.Wait() fd2 processed something (1sec). fd2 processed something (2sec). fd2 processed something (3sec). fd1 processed something (1sec). fd1 processed something (2sec). fd1 processed something (3sec). locker := gannoy.Fcntl{} Blocking!! Blocking Flock Fcntl + F_ODF_SETLKW fd2 processed something (1sec). fd2 processed something (2sec). fd2 processed something (3sec). fd1 processed something (1sec). fd1 processed something (2sec). fd1 processed something (3sec).
26. 1. non-POSIX (Only Linux) 2. kernel >= 3.15.0 (CentOS7.2 is 3.10.0…) 26 Caution func newLocker() Locker { bytes, err := exec.Command("uname", "-sr").Output() if err != nil { return Flock{} } kernel := strings.Split(strings.TrimRight(string(bytes), "\n"), " ") if len(kernel) != 2 { return Flock{} } if kernel[0] == "Linux" && !semver.New(kernel[1]).LessThan(*semver.New("3.15.0")) { return Fcntl{} } return Flock{} } kernel >= 3.15.0
27. ݚڀһɺੵۃతʹืूதʂ http://rand.pepabo.com/