Tweet
Tweet

Slide 1

Slide 1 text

Automata representation of successful strategies for social dilemma Yohsuke Murase, Seung Ki Baek RIKEN Center for Computational Science, Pukyong National Univ. Scientific Reports 10, 13370 (2020)

Slide 2

Slide 2 text

iterated Prisoner’s Dilemma cooperation defection cooperation (3,3) (0,5) defection (5,0) (1,1) long-term payoff implementation error occurs with probability e AAACZHichVHLSsNAFD2Nr1ofrYogCFIsFVflRgTFVdGNS9taW1CRJE5raJqEJC3U4g/oVnHhSkFE/Aw3/oALf0AQlxXcuPA2DYgW9Q4zc+bMPXfOzKi2obse0VNI6unt6x8ID0aGhkdGo7Gx8S3XqjmayGuWYTlFVXGFoZsi7+meIYq2I5SqaoiCWllr7xfqwnF1y9z0GrbYrSplUy/pmuIxlRF7sQSlyI94N5ADkEAQG1bsBjvYhwUNNVQhYMJjbECBy20bMgg2c7toMucw0v19gSNEWFvjLMEZCrMVHsu82g5Yk9ftmq6v1vgUg7vDyjiS9Ei31KIHuqMX+vi1VtOv0fbS4FntaIW9Fz2eyr3/q6ry7OHgS/WnZw8lLPtedfZu+0z7FlpHXz88b+VWssnmHF3RK/u/pCe65xuY9TftOiOyF4jwB8g/n7sbbC2kZErJmcVEejX4ijCmMYt5fu8lpLGODeT5XIETnOIs9CwNSxPSZCdVCgWaCXwLaeYT0EyJ5Q== AAACZHichVHLSsNAFD2Nr1ofrYogCFIsFVflRgTFVdGNS9taW1CRJE5raJqEJC3U4g/oVnHhSkFE/Aw3/oALf0AQlxXcuPA2DYgW9Q4zc+bMPXfOzKi2obse0VNI6unt6x8ID0aGhkdGo7Gx8S3XqjmayGuWYTlFVXGFoZsi7+meIYq2I5SqaoiCWllr7xfqwnF1y9z0GrbYrSplUy/pmuIxlRF7sQSlyI94N5ADkEAQG1bsBjvYhwUNNVQhYMJjbECBy20bMgg2c7toMucw0v19gSNEWFvjLMEZCrMVHsu82g5Yk9ftmq6v1vgUg7vDyjiS9Ei31KIHuqMX+vi1VtOv0fbS4FntaIW9Fz2eyr3/q6ry7OHgS/WnZw8lLPtedfZu+0z7FlpHXz88b+VWssnmHF3RK/u/pCe65xuY9TftOiOyF4jwB8g/n7sbbC2kZErJmcVEejX4ijCmMYt5fu8lpLGODeT5XIETnOIs9CwNSxPSZCdVCgWaCXwLaeYT0EyJ5Q== AAACZHichVHLSsNAFD2Nr1ofrYogCFIsFVflRgTFVdGNS9taW1CRJE5raJqEJC3U4g/oVnHhSkFE/Aw3/oALf0AQlxXcuPA2DYgW9Q4zc+bMPXfOzKi2obse0VNI6unt6x8ID0aGhkdGo7Gx8S3XqjmayGuWYTlFVXGFoZsi7+meIYq2I5SqaoiCWllr7xfqwnF1y9z0GrbYrSplUy/pmuIxlRF7sQSlyI94N5ADkEAQG1bsBjvYhwUNNVQhYMJjbECBy20bMgg2c7toMucw0v19gSNEWFvjLMEZCrMVHsu82g5Yk9ftmq6v1vgUg7vDyjiS9Ei31KIHuqMX+vi1VtOv0fbS4FntaIW9Fz2eyr3/q6ry7OHgS/WnZw8lLPtedfZu+0z7FlpHXz88b+VWssnmHF3RK/u/pCe65xuY9TftOiOyF4jwB8g/n7sbbC2kZErJmcVEejX4ijCmMYt5fu8lpLGODeT5XIETnOIs9CwNSxPSZCdVCgWaCXwLaeYT0EyJ5Q== AAACZHichVHLSsNAFD2Nr1ofrYogCFIsFVflRgTFVdGNS9taW1CRJE5raJqEJC3U4g/oVnHhSkFE/Aw3/oALf0AQlxXcuPA2DYgW9Q4zc+bMPXfOzKi2obse0VNI6unt6x8ID0aGhkdGo7Gx8S3XqjmayGuWYTlFVXGFoZsi7+meIYq2I5SqaoiCWllr7xfqwnF1y9z0GrbYrSplUy/pmuIxlRF7sQSlyI94N5ADkEAQG1bsBjvYhwUNNVQhYMJjbECBy20bMgg2c7toMucw0v19gSNEWFvjLMEZCrMVHsu82g5Yk9ftmq6v1vgUg7vDyjiS9Ei31KIHuqMX+vi1VtOv0fbS4FntaIW9Fz2eyr3/q6ry7OHgS/WnZw8lLPtedfZu+0z7FlpHXz88b+VWssnmHF3RK/u/pCe65xuY9TftOiOyF4jwB8g/n7sbbC2kZErJmcVEejX4ijCmMYt5fu8lpLGODeT5XIETnOIs9CwNSxPSZCdVCgWaCXwLaeYT0EyJ5Q== fi ⌘ lim e!0 lim T !1 1 T T 1 X t=0 F(t) i AAACtXichVFNaxRBEH0Zv+IazUYvgpfBJSEeXGpCIEEQQgTxuEl2k0BmM8xMepIm8+VMz8JmmD/gH/DgSUFC8O4f8OIfyCGQq4HgMYIXD9bMDogGtZrurnpVr/p1txP7MlVEJ2PalavXrt8Yv9m4NXH7zmRz6u56GmWJK3pu5EfJpmOnwpeh6CmpfLEZJ8IOHF9sOPvPyvzGQCSpjMKuGsaiH9i7ofSkayuGrGbHs6RuipeZHOimLwMrF6aKdCrqqFtGpgw9NWTIS2w3N4q8W5hpxln1lIrtvPvYKJ5bcjufVY8Kq9miNlWmX3aM2mmhtk7UPISJHURwkSGAQAjFvg8bKY8tGCDEjPWRM5awJ6u8QIEGczOuElxhM7rP6y5HWzUaclz2TCu2y6f4PBNm6pimYzqiC/pMH+icfvy1V171KLUMeXdGXBFbk6/ur33/LyvgXWHvF+ufmhU8LFZaJWuPK6S8hTviDw5eX6w9WZ3OZ+gdfWX9b+mEPvENwsE39/2KWH2DBn+A8edzX3bW59oGtY2V+dbScv0V43iAh5jl917AEl6ggx6f+xGn+IIzbUHrazuaNyrVxmrOPfxmWvQTVAapmg== AAACtXichVFNaxRBEH0Zv+IazUYvgpfBJSEeXGpCIEEQQgTxuEl2k0BmM8xMepIm8+VMz8JmmD/gH/DgSUFC8O4f8OIfyCGQq4HgMYIXD9bMDogGtZrurnpVr/p1txP7MlVEJ2PalavXrt8Yv9m4NXH7zmRz6u56GmWJK3pu5EfJpmOnwpeh6CmpfLEZJ8IOHF9sOPvPyvzGQCSpjMKuGsaiH9i7ofSkayuGrGbHs6RuipeZHOimLwMrF6aKdCrqqFtGpgw9NWTIS2w3N4q8W5hpxln1lIrtvPvYKJ5bcjufVY8Kq9miNlWmX3aM2mmhtk7UPISJHURwkSGAQAjFvg8bKY8tGCDEjPWRM5awJ6u8QIEGczOuElxhM7rP6y5HWzUaclz2TCu2y6f4PBNm6pimYzqiC/pMH+icfvy1V171KLUMeXdGXBFbk6/ur33/LyvgXWHvF+ufmhU8LFZaJWuPK6S8hTviDw5eX6w9WZ3OZ+gdfWX9b+mEPvENwsE39/2KWH2DBn+A8edzX3bW59oGtY2V+dbScv0V43iAh5jl917AEl6ggx6f+xGn+IIzbUHrazuaNyrVxmrOPfxmWvQTVAapmg== AAACtXichVFNaxRBEH0Zv+IazUYvgpfBJSEeXGpCIEEQQgTxuEl2k0BmM8xMepIm8+VMz8JmmD/gH/DgSUFC8O4f8OIfyCGQq4HgMYIXD9bMDogGtZrurnpVr/p1txP7MlVEJ2PalavXrt8Yv9m4NXH7zmRz6u56GmWJK3pu5EfJpmOnwpeh6CmpfLEZJ8IOHF9sOPvPyvzGQCSpjMKuGsaiH9i7ofSkayuGrGbHs6RuipeZHOimLwMrF6aKdCrqqFtGpgw9NWTIS2w3N4q8W5hpxln1lIrtvPvYKJ5bcjufVY8Kq9miNlWmX3aM2mmhtk7UPISJHURwkSGAQAjFvg8bKY8tGCDEjPWRM5awJ6u8QIEGczOuElxhM7rP6y5HWzUaclz2TCu2y6f4PBNm6pimYzqiC/pMH+icfvy1V171KLUMeXdGXBFbk6/ur33/LyvgXWHvF+ufmhU8LFZaJWuPK6S8hTviDw5eX6w9WZ3OZ+gdfWX9b+mEPvENwsE39/2KWH2DBn+A8edzX3bW59oGtY2V+dbScv0V43iAh5jl917AEl6ggx6f+xGn+IIzbUHrazuaNyrVxmrOPfxmWvQTVAapmg== AAACtXichVFNaxRBEH0Zv+IazUYvgpfBJSEeXGpCIEEQQgTxuEl2k0BmM8xMepIm8+VMz8JmmD/gH/DgSUFC8O4f8OIfyCGQq4HgMYIXD9bMDogGtZrurnpVr/p1txP7MlVEJ2PalavXrt8Yv9m4NXH7zmRz6u56GmWJK3pu5EfJpmOnwpeh6CmpfLEZJ8IOHF9sOPvPyvzGQCSpjMKuGsaiH9i7ofSkayuGrGbHs6RuipeZHOimLwMrF6aKdCrqqFtGpgw9NWTIS2w3N4q8W5hpxln1lIrtvPvYKJ5bcjufVY8Kq9miNlWmX3aM2mmhtk7UPISJHURwkSGAQAjFvg8bKY8tGCDEjPWRM5awJ6u8QIEGczOuElxhM7rP6y5HWzUaclz2TCu2y6f4PBNm6pimYzqiC/pMH+icfvy1V171KLUMeXdGXBFbk6/ur33/LyvgXWHvF+ufmhU8LFZaJWuPK6S8hTviDw5eX6w9WZ3OZ+gdfWX9b+mEPvENwsE39/2KWH2DBn+A8edzX3bW59oGtY2V+dbScv0V43iAh5jl917AEl6ggx6f+xGn+IIzbUHrazuaNyrVxmrOPfxmWvQTVAapmg== player A player B

Slide 3

Slide 3 text

A well-known strategy: TFT c c c c d d c d c c d c It is guaranteed that your payoff is no less than the co-players'. Cooperation is fragile against an error. Copy the co-player's previous move. It cannot exploit naive cooperator. TFT TFT TFT TFT ALLC c c c c d d c d c c d c c c c c c c History Next move C,C C C,D D D,C C D,D D

Slide 4

Slide 4 text

A well-known strategy: WSLS c c c c d d c d c c d c The cooperation is tolerant against an error. It can exploit naive cooperators. Switch your action if you are unhappy. Weak against defectors. WSLS ALLC WSLS ALLD c c c c d d c d c c c c WSLS WSLS d d d d d d

Slide 5

Slide 5 text

Partners vs Rivals Efficiency ⌦ ⌦ ⌦ cooperation Defensibility ⌦ Strategies that work as a partner as well as a rival were recently found. friendly rival (cooperative Nash equilibrium with a guarantee that you'll never lose.) Hilbe et al., Nat.Hum.Behav. (2018)

Slide 6

Slide 6 text

Friendly rival strategy for PD • # of Memory-1 strategies: 16 • => no strategies satisfies the criteria • # of Memory-2 strategies: 2^16 = 65536 • 4 friendly rivals were found. Yi et al., J. Theor. Biol. (2017) TFT-ATFT strategy

Slide 7

Slide 7 text

Friendly rival strategy for three-player public goods game Y Murase & S.K. Baek J. Theor. Biol. (2018) • # of Memory-1 strategies: 256 • # of Memory-2 strategies: 2^40 • => No friendly rivals was found. • # of Memory-3 strategies: 2^288 • => There are at least 256 friendly rivals.

Slide 8

Slide 8 text

The strategy is defined by action table having 512 entries. (At 3At 2At 1, Bt 3Bt 2Bt 1, Ct 3Ct 2Ct 1) AAACyHichZHLSsNAFIZ/473eqm4EN2JRFLRMasHLSpuNuPJWFVRKEkcNpklIppVaunHpC7hwpeBCfAAfwI07V4J9BHGp4MaFp5mAqKgTknPmm/Ofyc8xPNsKBGPVOqW+obGpuaU11tbe0dkV7+5ZC9yCb/Ks6dquv2HoAbcth2eFJWy+4flczxs2XzcOtNr5epH7geU6q6Lk8e28vudYu5apC0K5uDYylyuL8YlKGFIyqJWxjKQZSTMR1STVJNUkHc3FEyzJwjXwM1GjJIFoLbrxG2xhBy5MFJAHhwNBuQ0dAT2bUMHgEdtGmZhPmRWec1QQI22BqjhV6EQP6LtHu82IOrSv9QxCtUm32PT6pBzAEHtgV+yF3bFr9sTef+1VDnvU/qVE0ZBa7uW6TvpW3v5V5SkK7H+q/lAYVP23J4FdTIVeLPLmhaTm0pT9i0enLyszy0PlYXbBnsnfOauyW3LoFF/NyyW+fIYYDUj9Po6fyVoqqaaT00vpxGwmGlUL+jGIEZrHJGYxj0Vk6d4r3OMRVWVB8ZRDpSRLlbpI04svSzn+ALWuqL0= An intuitive interpretation is lacking.

Slide 9

Slide 9 text

# of players vs memory length m n AAACi3ichVHNLgNRGD3GfynFRmLTaIhVc4sEjYUQiWV/tJogMjOuuun8mbltUo0XsLSxqA2JhXgAD2DjBSz6CGJZiY2Fr9NJBFHfZO4999zvfDMnR3MM4UnGGl1Kd09vX//AYGhoODwyGhkbz3t22dV5TrcN2y1oqscNYfGcFNLgBcflqqkZfEcrbbTudyrc9YRtbcuqw/dNtWiJI6GrkqiCGd0r8pOodRCJsTjzK/obJAIQQ1ApO/KAPRzCho4yTHBYkIQNqPDo2UUCDA5x+6gR5xIS/j3HGUKkLVMXpw6V2BKtRTrtBqxF59ZMz1fr9BWDXpeUUcywZ3bHmuyJ3bMX9vHnrJo/o/UvVdq1tpY7B6Pnk9n3f1Um7RLHX6oOCo26O3uSOMKy70WQN8dnWi719vzK6WUzm8zM1GbZDXslf9eswR7JoVV502/TPFNHiAJK/IzjN8jPxxOL8ZX0YmxtPYhqAFOYxhzlsYQ1bCGFnJ/DBeq4UsLKgpJUVtutSlegmcC3UjY/AZdykn0= a necessary condition to be a friendly rival strategy theoretical lower bound No friendly rival strategy has been found yet for n>3. The size of action table increases exponentially as number of players and memory increases. 2nm AAACiXichVG7SgNBFD1Z3/GRqI1gIwbFKswGwZgqmMYyDxMFX+yuo47ui91NIC7+gJWdqJWChfgBfoCNP2DhJ4ilgo2FdzcLosF4h5k5c+aeO3O4qq0L12PsOSZ1dff09vUPxAeHhkcSydGxmmvVHY1XNUu3nDVVcbkuTF71hKfzNdvhiqHqfFU9LAT3qw3uuMIyV7ymzTcNZc8Uu0JTPKJqmS3fNI63kymWZmFMtQM5AilEUbSS99jADixoqMMAhwmPsA4FLo11yGCwiduET5xDSIT3HMeIk7ZOWZwyFGIPad2j03rEmnQOarqhWqNXdJoOKacww57YLXtjj+yOvbDPP2v5YY3gL03a1ZaW29uJk4nKx78qg3YP+9+qDgqVsjt78rCLbOhFkDc7ZAKXWqt+4+jsrZIrz/iz7Jq9kr8r9sweyKHZeNduSrx8iTg1SP7djnZQy6Tl+fRiaT6VX4pa1Y9JTGOO+rGAPJZRRJXePcApznEhDUqylJVyrVQpFmnG8SOkwhecBZIY

Slide 10

Slide 10 text

In this work, we interpret the friendly rival strategies by representing them as automata. History Next move C,C C C,D D D,C C D,D D 0 1 cc cd dc dd "state-based" representation "history-based" representation

Slide 11

Slide 11 text

Core idea: DFA minimization i x y j cd cd cc cd k z cc cc i x y j cd cd cc cd k z cc cc Iteratively merge "equivalent states" into one state (Hopcroft's algorithm) History Next move C,C C C,D D D,C C D,D D action table cc cd dd dc cd cd cc cc dc dc dd dd 0 1 cc cd dc dd cooperation defection

Slide 12

Slide 12 text

Examples of DFA minimization 0 cccc 1 cccd 2 ccdc 3 ccdd 6 cddc 7 cddd 4 cdcc 8 dccc 9 dccd 5 cdcd 10 dcdc 11 dcdd 14 dddc 15 dddd 12 ddcc 13 ddcd 1 3 cc cd dc dd 0 cc cd 0 cccc 1 cccd 6 cddc 7 cddd 2 ccdc 4 cdcc 5 cdcd 3 ccdd 12 ddcc 13 ddcd 10 dcdc 11 dcdd 14 dddc 15 dddd 8 dccc 9 dccd 1 2 dc dd dd 0 cc cd dc TF2T (before) TF2T (after) AON2 (before) AON2 (after)

Slide 13

Slide 13 text

TFT-ATFT 11) TFT-c 7) TFT-d cccc cccd dccd dccc cddd dddd cddc dddc cdcc ddcc ddcd dcdc cdcd dcdd ccdc ccdd 1) TFT-d 4) ATFT-d 5) ATFT-c 0) TFT-c TFT-ATFT recurrent 0 1 cc cd dc dd 5 cc,cd 4 dc dd (cccc) ↓ (cdcc) represented by 4-state automaton

Slide 14

Slide 14 text

If there is no error, it behaves like TFT => rivalry recovery from error => partner c d c c ... ... ... ... d d c c c d c 0 0 1 1 0 0 0 4 5 0 0 1 cc cd dc dd 5 cc,cd 4 dc dd

Slide 15

Slide 15 text

partially successful strategy for 3-person game cdd 0 1 ccc 9 8 66 64 ddd ccc ccd,cdc, cdd ccc,cdc cdc dcc dcc ccd cdd 76 67 65 72 ccc,ccd cdd cdc ccd ddc dcd ddc dcd dcc dcc dcd,ddd ddc,ddd dcd ddc ddd ddd dcc dcc ddc dcd ddd ddc dcd α β δ δ γ γ

Slide 16

Slide 16 text

• error recovery patterns ccc,ccc,ccc (0,0,0) ccc,ccd,ccc (8,64,1) ccd,cdc,ccd (0,0,0) cdc,dcc,cdc (0,0,0) cdc,dcd,cdc (8,64,1) cdc,dcc,cdd (1,8,64) dcd,cdc,dcd (0,0,0) ccd,cdd,ccd (76,64,67) cdd,ddc,cdd (0,0,0) ccd,cdc,ccc (66,0,0) cdd,dcc,ccc (64,1,8) ddc,ccd,ccd (0,0,0) ccc,ccd,ccd (9,72,65) ccd,cdc,cdc (66,0,0) cdd,dcc,dcc (64,1,8) ddc,ccd,ccd (0,0,0) dcc,ccc,dcc (0,0,0) dcc,ccd,dcc (8,64,1) ccd,cdc,ccd (0,0,0) dcc,ccc,dcd (1,8,64) ccd,ccd,cdc (0,0,0) dcd,ccd,ddc (0,0,0) B B&C B C B C B C (α) (α) (α) (α) (α) (β) (γ) (δ) cdd 0 1 ccc 9 8 66 64 ddd ccc ccd,cdc, cdd ccc,cdc cdc dcc dcc ccd cdd 76 67 65 72 ccc,ccd cdd cdc ccd ddc dcd ddc dcd dcc dcc dcd,ddd ddc,ddd dcd ddc ddd ddd dcc dcc ddc dcd ddd ddc dcd α β δ δ γ γ recovery from 2-bit errors

Slide 17

Slide 17 text

Conclusion • We proposed a method to convert "history-based" representation into the "state-based" representation, which is effective for an intuitive interpretation of friendly rival strategies. • step toward future exploration of friendly rival strategies for general-n person game. cdd 0 1 ccc 9 8 66 64 ddd ccc ccd,cdc, cdd ccc,cdc cdc dcc dcc ccd cdd 76 67 65 72 ccc,ccd cdd cdc ccd ddc dcd ddc dcd dcc dcc dcd,ddd ddc,ddd dcd ddc ddd ddd dcc dcc ddc dcd ddd ddc dcd α β δ δ γ γ

Slide 18

Slide 18 text

https://www.nature.com/articles/s41598-020-70281-x