Automata representation of successful strategies for social dilemmas

Presentation for the following paper:
Yohsuke Murase and Seung Ki Baek "Automata representation of successful strategies for social dilemmas" Scientific reports 10, 13370 (2020)

Transcript

1. Automata representation of successful strategies for social dilemma Yohsuke Murase, Seung Ki Baek RIKEN Center for Computational Science, Pukyong National Univ. Scientific Reports 10, 13370 (2020)
2. iterated Prisoner’s Dilemma cooperation defection cooperation (3,3) (0,5) defection (5,0) (1,1) long-term payoff implementation error occurs with probability e <latexit sha1_base64="fSe004YKF6vMcUZEj33avSo9wuc=">AAACZHichVHLSsNAFD2Nr1ofrYogCFIsFVflRgTFVdGNS9taW1CRJE5raJqEJC3U4g/oVnHhSkFE/Aw3/oALf0AQlxXcuPA2DYgW9Q4zc+bMPXfOzKi2obse0VNI6unt6x8ID0aGhkdGo7Gx8S3XqjmayGuWYTlFVXGFoZsi7+meIYq2I5SqaoiCWllr7xfqwnF1y9z0GrbYrSplUy/pmuIxlRF7sQSlyI94N5ADkEAQG1bsBjvYhwUNNVQhYMJjbECBy20bMgg2c7toMucw0v19gSNEWFvjLMEZCrMVHsu82g5Yk9ftmq6v1vgUg7vDyjiS9Ei31KIHuqMX+vi1VtOv0fbS4FntaIW9Fz2eyr3/q6ry7OHgS/WnZw8lLPtedfZu+0z7FlpHXz88b+VWssnmHF3RK/u/pCe65xuY9TftOiOyF4jwB8g/n7sbbC2kZErJmcVEejX4ijCmMYt5fu8lpLGODeT5XIETnOIs9CwNSxPSZCdVCgWaCXwLaeYT0EyJ5Q==</latexit> <latexit sha1_base64="fSe004YKF6vMcUZEj33avSo9wuc=">AAACZHichVHLSsNAFD2Nr1ofrYogCFIsFVflRgTFVdGNS9taW1CRJE5raJqEJC3U4g/oVnHhSkFE/Aw3/oALf0AQlxXcuPA2DYgW9Q4zc+bMPXfOzKi2obse0VNI6unt6x8ID0aGhkdGo7Gx8S3XqjmayGuWYTlFVXGFoZsi7+meIYq2I5SqaoiCWllr7xfqwnF1y9z0GrbYrSplUy/pmuIxlRF7sQSlyI94N5ADkEAQG1bsBjvYhwUNNVQhYMJjbECBy20bMgg2c7toMucw0v19gSNEWFvjLMEZCrMVHsu82g5Yk9ftmq6v1vgUg7vDyjiS9Ei31KIHuqMX+vi1VtOv0fbS4FntaIW9Fz2eyr3/q6ry7OHgS/WnZw8lLPtedfZu+0z7FlpHXz88b+VWssnmHF3RK/u/pCe65xuY9TftOiOyF4jwB8g/n7sbbC2kZErJmcVEejX4ijCmMYt5fu8lpLGODeT5XIETnOIs9CwNSxPSZCdVCgWaCXwLaeYT0EyJ5Q==</latexit> <latexit sha1_base64="fSe004YKF6vMcUZEj33avSo9wuc=">AAACZHichVHLSsNAFD2Nr1ofrYogCFIsFVflRgTFVdGNS9taW1CRJE5raJqEJC3U4g/oVnHhSkFE/Aw3/oALf0AQlxXcuPA2DYgW9Q4zc+bMPXfOzKi2obse0VNI6unt6x8ID0aGhkdGo7Gx8S3XqjmayGuWYTlFVXGFoZsi7+meIYq2I5SqaoiCWllr7xfqwnF1y9z0GrbYrSplUy/pmuIxlRF7sQSlyI94N5ADkEAQG1bsBjvYhwUNNVQhYMJjbECBy20bMgg2c7toMucw0v19gSNEWFvjLMEZCrMVHsu82g5Yk9ftmq6v1vgUg7vDyjiS9Ei31KIHuqMX+vi1VtOv0fbS4FntaIW9Fz2eyr3/q6ry7OHgS/WnZw8lLPtedfZu+0z7FlpHXz88b+VWssnmHF3RK/u/pCe65xuY9TftOiOyF4jwB8g/n7sbbC2kZErJmcVEejX4ijCmMYt5fu8lpLGODeT5XIETnOIs9CwNSxPSZCdVCgWaCXwLaeYT0EyJ5Q==</latexit> <latexit sha1_base64="fSe004YKF6vMcUZEj33avSo9wuc=">AAACZHichVHLSsNAFD2Nr1ofrYogCFIsFVflRgTFVdGNS9taW1CRJE5raJqEJC3U4g/oVnHhSkFE/Aw3/oALf0AQlxXcuPA2DYgW9Q4zc+bMPXfOzKi2obse0VNI6unt6x8ID0aGhkdGo7Gx8S3XqjmayGuWYTlFVXGFoZsi7+meIYq2I5SqaoiCWllr7xfqwnF1y9z0GrbYrSplUy/pmuIxlRF7sQSlyI94N5ADkEAQG1bsBjvYhwUNNVQhYMJjbECBy20bMgg2c7toMucw0v19gSNEWFvjLMEZCrMVHsu82g5Yk9ftmq6v1vgUg7vDyjiS9Ei31KIHuqMX+vi1VtOv0fbS4FntaIW9Fz2eyr3/q6ry7OHgS/WnZw8lLPtedfZu+0z7FlpHXz88b+VWssnmHF3RK/u/pCe65xuY9TftOiOyF4jwB8g/n7sbbC2kZErJmcVEejX4ijCmMYt5fu8lpLGODeT5XIETnOIs9CwNSxPSZCdVCgWaCXwLaeYT0EyJ5Q==</latexit> fi ⌘ lim e!0 lim T !1 1 T T 1 X t=0 F(t) i <latexit sha1_base64="CfD1qyKg0ws1VkAs+U8zHbmbHOk=">AAACtXichVFNaxRBEH0Zv+IazUYvgpfBJSEeXGpCIEEQQgTxuEl2k0BmM8xMepIm8+VMz8JmmD/gH/DgSUFC8O4f8OIfyCGQq4HgMYIXD9bMDogGtZrurnpVr/p1txP7MlVEJ2PalavXrt8Yv9m4NXH7zmRz6u56GmWJK3pu5EfJpmOnwpeh6CmpfLEZJ8IOHF9sOPvPyvzGQCSpjMKuGsaiH9i7ofSkayuGrGbHs6RuipeZHOimLwMrF6aKdCrqqFtGpgw9NWTIS2w3N4q8W5hpxln1lIrtvPvYKJ5bcjufVY8Kq9miNlWmX3aM2mmhtk7UPISJHURwkSGAQAjFvg8bKY8tGCDEjPWRM5awJ6u8QIEGczOuElxhM7rP6y5HWzUaclz2TCu2y6f4PBNm6pimYzqiC/pMH+icfvy1V171KLUMeXdGXBFbk6/ur33/LyvgXWHvF+ufmhU8LFZaJWuPK6S8hTviDw5eX6w9WZ3OZ+gdfWX9b+mEPvENwsE39/2KWH2DBn+A8edzX3bW59oGtY2V+dbScv0V43iAh5jl917AEl6ggx6f+xGn+IIzbUHrazuaNyrVxmrOPfxmWvQTVAapmg==</latexit> <latexit sha1_base64="CfD1qyKg0ws1VkAs+U8zHbmbHOk=">AAACtXichVFNaxRBEH0Zv+IazUYvgpfBJSEeXGpCIEEQQgTxuEl2k0BmM8xMepIm8+VMz8JmmD/gH/DgSUFC8O4f8OIfyCGQq4HgMYIXD9bMDogGtZrurnpVr/p1txP7MlVEJ2PalavXrt8Yv9m4NXH7zmRz6u56GmWJK3pu5EfJpmOnwpeh6CmpfLEZJ8IOHF9sOPvPyvzGQCSpjMKuGsaiH9i7ofSkayuGrGbHs6RuipeZHOimLwMrF6aKdCrqqFtGpgw9NWTIS2w3N4q8W5hpxln1lIrtvPvYKJ5bcjufVY8Kq9miNlWmX3aM2mmhtk7UPISJHURwkSGAQAjFvg8bKY8tGCDEjPWRM5awJ6u8QIEGczOuElxhM7rP6y5HWzUaclz2TCu2y6f4PBNm6pimYzqiC/pMH+icfvy1V171KLUMeXdGXBFbk6/ur33/LyvgXWHvF+ufmhU8LFZaJWuPK6S8hTviDw5eX6w9WZ3OZ+gdfWX9b+mEPvENwsE39/2KWH2DBn+A8edzX3bW59oGtY2V+dbScv0V43iAh5jl917AEl6ggx6f+xGn+IIzbUHrazuaNyrVxmrOPfxmWvQTVAapmg==</latexit> <latexit sha1_base64="CfD1qyKg0ws1VkAs+U8zHbmbHOk=">AAACtXichVFNaxRBEH0Zv+IazUYvgpfBJSEeXGpCIEEQQgTxuEl2k0BmM8xMepIm8+VMz8JmmD/gH/DgSUFC8O4f8OIfyCGQq4HgMYIXD9bMDogGtZrurnpVr/p1txP7MlVEJ2PalavXrt8Yv9m4NXH7zmRz6u56GmWJK3pu5EfJpmOnwpeh6CmpfLEZJ8IOHF9sOPvPyvzGQCSpjMKuGsaiH9i7ofSkayuGrGbHs6RuipeZHOimLwMrF6aKdCrqqFtGpgw9NWTIS2w3N4q8W5hpxln1lIrtvPvYKJ5bcjufVY8Kq9miNlWmX3aM2mmhtk7UPISJHURwkSGAQAjFvg8bKY8tGCDEjPWRM5awJ6u8QIEGczOuElxhM7rP6y5HWzUaclz2TCu2y6f4PBNm6pimYzqiC/pMH+icfvy1V171KLUMeXdGXBFbk6/ur33/LyvgXWHvF+ufmhU8LFZaJWuPK6S8hTviDw5eX6w9WZ3OZ+gdfWX9b+mEPvENwsE39/2KWH2DBn+A8edzX3bW59oGtY2V+dbScv0V43iAh5jl917AEl6ggx6f+xGn+IIzbUHrazuaNyrVxmrOPfxmWvQTVAapmg==</latexit> <latexit sha1_base64="CfD1qyKg0ws1VkAs+U8zHbmbHOk=">AAACtXichVFNaxRBEH0Zv+IazUYvgpfBJSEeXGpCIEEQQgTxuEl2k0BmM8xMepIm8+VMz8JmmD/gH/DgSUFC8O4f8OIfyCGQq4HgMYIXD9bMDogGtZrurnpVr/p1txP7MlVEJ2PalavXrt8Yv9m4NXH7zmRz6u56GmWJK3pu5EfJpmOnwpeh6CmpfLEZJ8IOHF9sOPvPyvzGQCSpjMKuGsaiH9i7ofSkayuGrGbHs6RuipeZHOimLwMrF6aKdCrqqFtGpgw9NWTIS2w3N4q8W5hpxln1lIrtvPvYKJ5bcjufVY8Kq9miNlWmX3aM2mmhtk7UPISJHURwkSGAQAjFvg8bKY8tGCDEjPWRM5awJ6u8QIEGczOuElxhM7rP6y5HWzUaclz2TCu2y6f4PBNm6pimYzqiC/pMH+icfvy1V171KLUMeXdGXBFbk6/ur33/LyvgXWHvF+ufmhU8LFZaJWuPK6S8hTviDw5eX6w9WZ3OZ+gdfWX9b+mEPvENwsE39/2KWH2DBn+A8edzX3bW59oGtY2V+dbScv0V43iAh5jl917AEl6ggx6f+xGn+IIzbUHrazuaNyrVxmrOPfxmWvQTVAapmg==</latexit> player A player B
3. A well-known strategy: TFT c c c c d d c d c c d c It is guaranteed that your payoff is no less than the co-players'. Cooperation is fragile against an error. Copy the co-player's previous move. It cannot exploit naive cooperator. TFT TFT TFT TFT ALLC c c c c d d c d c c d c c c c c c c History Next move C,C C C,D D D,C C D,D D
4. A well-known strategy: WSLS c c c c d d c d c c d c The cooperation is tolerant against an error. It can exploit naive cooperators. Switch your action if you are unhappy. Weak against defectors. WSLS ALLC WSLS ALLD c c c c d d c d c c c c WSLS WSLS d d d d d d
5. Partners vs Rivals Efficiency ⌦ ⌦ ⌦ cooperation Defensibility ⌦ Strategies that work as a partner as well as a rival were recently found. friendly rival (cooperative Nash equilibrium with a guarantee that you'll never lose.) Hilbe et al., Nat.Hum.Behav. (2018)
6. Friendly rival strategy for PD • # of Memory-1 strategies: 16 • => no strategies satisfies the criteria • # of Memory-2 strategies: 2^16 = 65536 • 4 friendly rivals were found. Yi et al., J. Theor. Biol. (2017) TFT-ATFT strategy
7. Friendly rival strategy for three-player public goods game Y Murase & S.K. Baek J. Theor. Biol. (2018) • # of Memory-1 strategies: 256 • # of Memory-2 strategies: 2^40 • => No friendly rivals was found. • # of Memory-3 strategies: 2^288 • => There are at least 256 friendly rivals.
8. The strategy is defined by action table having 512 entries. (At 3At 2At 1, Bt 3Bt 2Bt 1, Ct 3Ct 2Ct 1) <latexit sha1_base64="XvomuL/8LiuroO7Gt1Sl5rSlFpw=">AAACyHichZHLSsNAFIZ/473eqm4EN2JRFLRMasHLSpuNuPJWFVRKEkcNpklIppVaunHpC7hwpeBCfAAfwI07V4J9BHGp4MaFp5mAqKgTknPmm/Ofyc8xPNsKBGPVOqW+obGpuaU11tbe0dkV7+5ZC9yCb/Ks6dquv2HoAbcth2eFJWy+4flczxs2XzcOtNr5epH7geU6q6Lk8e28vudYu5apC0K5uDYylyuL8YlKGFIyqJWxjKQZSTMR1STVJNUkHc3FEyzJwjXwM1GjJIFoLbrxG2xhBy5MFJAHhwNBuQ0dAT2bUMHgEdtGmZhPmRWec1QQI22BqjhV6EQP6LtHu82IOrSv9QxCtUm32PT6pBzAEHtgV+yF3bFr9sTef+1VDnvU/qVE0ZBa7uW6TvpW3v5V5SkK7H+q/lAYVP23J4FdTIVeLPLmhaTm0pT9i0enLyszy0PlYXbBnsnfOauyW3LoFF/NyyW+fIYYDUj9Po6fyVoqqaaT00vpxGwmGlUL+jGIEZrHJGYxj0Vk6d4r3OMRVWVB8ZRDpSRLlbpI04svSzn+ALWuqL0=</latexit> An intuitive interpretation is lacking.
9. # of players vs memory length m n <latexit sha1_base64="bTojWnh2XVCqg/Iuo89Dh7W5ZTw=">AAACi3ichVHNLgNRGD3GfynFRmLTaIhVc4sEjYUQiWV/tJogMjOuuun8mbltUo0XsLSxqA2JhXgAD2DjBSz6CGJZiY2Fr9NJBFHfZO4999zvfDMnR3MM4UnGGl1Kd09vX//AYGhoODwyGhkbz3t22dV5TrcN2y1oqscNYfGcFNLgBcflqqkZfEcrbbTudyrc9YRtbcuqw/dNtWiJI6GrkqiCGd0r8pOodRCJsTjzK/obJAIQQ1ApO/KAPRzCho4yTHBYkIQNqPDo2UUCDA5x+6gR5xIS/j3HGUKkLVMXpw6V2BKtRTrtBqxF59ZMz1fr9BWDXpeUUcywZ3bHmuyJ3bMX9vHnrJo/o/UvVdq1tpY7B6Pnk9n3f1Um7RLHX6oOCo26O3uSOMKy70WQN8dnWi719vzK6WUzm8zM1GbZDXslf9eswR7JoVV502/TPFNHiAJK/IzjN8jPxxOL8ZX0YmxtPYhqAFOYxhzlsYQ1bCGFnJ/DBeq4UsLKgpJUVtutSlegmcC3UjY/AZdykn0=</latexit> a necessary condition to be a friendly rival strategy theoretical lower bound No friendly rival strategy has been found yet for n>3. The size of action table increases exponentially as number of players and memory increases. 2nm <latexit sha1_base64="eIlR6RGVgrYj2OnaFSI9t7+cvFI=">AAACiXichVG7SgNBFD1Z3/GRqI1gIwbFKswGwZgqmMYyDxMFX+yuo47ui91NIC7+gJWdqJWChfgBfoCNP2DhJ4ilgo2FdzcLosF4h5k5c+aeO3O4qq0L12PsOSZ1dff09vUPxAeHhkcSydGxmmvVHY1XNUu3nDVVcbkuTF71hKfzNdvhiqHqfFU9LAT3qw3uuMIyV7ymzTcNZc8Uu0JTPKJqmS3fNI63kymWZmFMtQM5AilEUbSS99jADixoqMMAhwmPsA4FLo11yGCwiduET5xDSIT3HMeIk7ZOWZwyFGIPad2j03rEmnQOarqhWqNXdJoOKacww57YLXtjj+yOvbDPP2v5YY3gL03a1ZaW29uJk4nKx78qg3YP+9+qDgqVsjt78rCLbOhFkDc7ZAKXWqt+4+jsrZIrz/iz7Jq9kr8r9sweyKHZeNduSrx8iTg1SP7djnZQy6Tl+fRiaT6VX4pa1Y9JTGOO+rGAPJZRRJXePcApznEhDUqylJVyrVQpFmnG8SOkwhecBZIY</latexit>
10. In this work, we interpret the friendly rival strategies by representing them as automata. History Next move C,C C C,D D D,C C D,D D 0 1 cc cd dc dd "state-based" representation "history-based" representation
11. Core idea: DFA minimization i x y j cd cd cc cd k z cc cc i x y j cd cd cc cd k z cc cc Iteratively merge "equivalent states" into one state (Hopcroft's algorithm) History Next move C,C C C,D D D,C C D,D D action table cc cd dd dc cd cd cc cc dc dc dd dd 0 1 cc cd dc dd cooperation defection
12. Examples of DFA minimization 0 cccc 1 cccd 2 ccdc 3 ccdd 6 cddc 7 cddd 4 cdcc 8 dccc 9 dccd 5 cdcd 10 dcdc 11 dcdd 14 dddc 15 dddd 12 ddcc 13 ddcd 1 3 cc cd dc dd 0 cc cd 0 cccc 1 cccd 6 cddc 7 cddd 2 ccdc 4 cdcc 5 cdcd 3 ccdd 12 ddcc 13 ddcd 10 dcdc 11 dcdd 14 dddc 15 dddd 8 dccc 9 dccd 1 2 dc dd dd 0 cc cd dc TF2T (before) TF2T (after) AON2 (before) AON2 (after)
13. TFT-ATFT 11) TFT-c 7) TFT-d cccc cccd dccd dccc cddd dddd cddc dddc cdcc ddcc ddcd dcdc cdcd dcdd ccdc ccdd 1) TFT-d 4) ATFT-d 5) ATFT-c 0) TFT-c TFT-ATFT recurrent 0 1 cc cd dc dd 5 cc,cd 4 dc dd (cccc) ↓ (cdcc) represented by 4-state automaton
14. If there is no error, it behaves like TFT => rivalry recovery from error => partner c d c c ... ... ... ... d d c c c d c 0 0 1 1 0 0 0 4 5 0 0 1 cc cd dc dd 5 cc,cd 4 dc dd
15. partially successful strategy for 3-person game cdd 0 1 ccc 9 8 66 64 ddd ccc ccd,cdc, cdd ccc,cdc cdc dcc dcc ccd cdd 76 67 65 72 ccc,ccd cdd cdc ccd ddc dcd ddc dcd dcc dcc dcd,ddd ddc,ddd dcd ddc ddd ddd dcc dcc ddc dcd ddd ddc dcd α β δ δ γ γ
16. • error recovery patterns ccc,ccc,ccc (0,0,0) ccc,ccd,ccc (8,64,1) ccd,cdc,ccd (0,0,0) cdc,dcc,cdc (0,0,0) cdc,dcd,cdc (8,64,1) cdc,dcc,cdd (1,8,64) dcd,cdc,dcd (0,0,0) ccd,cdd,ccd (76,64,67) cdd,ddc,cdd (0,0,0) ccd,cdc,ccc (66,0,0) cdd,dcc,ccc (64,1,8) ddc,ccd,ccd (0,0,0) ccc,ccd,ccd (9,72,65) ccd,cdc,cdc (66,0,0) cdd,dcc,dcc (64,1,8) ddc,ccd,ccd (0,0,0) dcc,ccc,dcc (0,0,0) dcc,ccd,dcc (8,64,1) ccd,cdc,ccd (0,0,0) dcc,ccc,dcd (1,8,64) ccd,ccd,cdc (0,0,0) dcd,ccd,ddc (0,0,0) B B&C B C B C B C (α) (α) (α) (α) (α) (β) (γ) (δ) cdd 0 1 ccc 9 8 66 64 ddd ccc ccd,cdc, cdd ccc,cdc cdc dcc dcc ccd cdd 76 67 65 72 ccc,ccd cdd cdc ccd ddc dcd ddc dcd dcc dcc dcd,ddd ddc,ddd dcd ddc ddd ddd dcc dcc ddc dcd ddd ddc dcd α β δ δ γ γ recovery from 2-bit errors
17. Conclusion • We proposed a method to convert "history-based" representation into the "state-based" representation, which is effective for an intuitive interpretation of friendly rival strategies. • step toward future exploration of friendly rival strategies for general-n person game. cdd 0 1 ccc 9 8 66 64 ddd ccc ccd,cdc, cdd ccc,cdc cdc dcc dcc ccd cdd 76 67 65 72 ccc,ccd cdd cdc ccd ddc dcd ddc dcd dcc dcc dcd,ddd ddc,ddd dcd ddc ddd ddd dcc dcc ddc dcd ddd ddc dcd α β δ δ γ γ
18. https://www.nature.com/articles/s41598-020-70281-x