Friendly-rivalry strategies for general n-person public-goods game Yohsuke Murase, Seung Ki Baek RIKEN Center for Computational Science, Japan Pukyong National University, Korea YM and S.K. Baek "Friendly-rivalry solution to the iterated n-person public-goods game" Plos. Comp. Biol. (2021)

iterated Prisoner’s Dilemma cooperation defection cooperation (3,3) (0,5) defection (5,0) (1,1) long-term payoff in the repeated game implementation error occurs with probability e AAACZHichVHLSsNAFD2Nr1ofrYogCFIsFVflRgTFVdGNS9taW1CRJE5raJqEJC3U4g/oVnHhSkFE/Aw3/oALf0AQlxXcuPA2DYgW9Q4zc+bMPXfOzKi2obse0VNI6unt6x8ID0aGhkdGo7Gx8S3XqjmayGuWYTlFVXGFoZsi7+meIYq2I5SqaoiCWllr7xfqwnF1y9z0GrbYrSplUy/pmuIxlRF7sQSlyI94N5ADkEAQG1bsBjvYhwUNNVQhYMJjbECBy20bMgg2c7toMucw0v19gSNEWFvjLMEZCrMVHsu82g5Yk9ftmq6v1vgUg7vDyjiS9Ei31KIHuqMX+vi1VtOv0fbS4FntaIW9Fz2eyr3/q6ry7OHgS/WnZw8lLPtedfZu+0z7FlpHXz88b+VWssnmHF3RK/u/pCe65xuY9TftOiOyF4jwB8g/n7sbbC2kZErJmcVEejX4ijCmMYt5fu8lpLGODeT5XIETnOIs9CwNSxPSZCdVCgWaCXwLaeYT0EyJ5Q== AAACZHichVHLSsNAFD2Nr1ofrYogCFIsFVflRgTFVdGNS9taW1CRJE5raJqEJC3U4g/oVnHhSkFE/Aw3/oALf0AQlxXcuPA2DYgW9Q4zc+bMPXfOzKi2obse0VNI6unt6x8ID0aGhkdGo7Gx8S3XqjmayGuWYTlFVXGFoZsi7+meIYq2I5SqaoiCWllr7xfqwnF1y9z0GrbYrSplUy/pmuIxlRF7sQSlyI94N5ADkEAQG1bsBjvYhwUNNVQhYMJjbECBy20bMgg2c7toMucw0v19gSNEWFvjLMEZCrMVHsu82g5Yk9ftmq6v1vgUg7vDyjiS9Ei31KIHuqMX+vi1VtOv0fbS4FntaIW9Fz2eyr3/q6ry7OHgS/WnZw8lLPtedfZu+0z7FlpHXz88b+VWssnmHF3RK/u/pCe65xuY9TftOiOyF4jwB8g/n7sbbC2kZErJmcVEejX4ijCmMYt5fu8lpLGODeT5XIETnOIs9CwNSxPSZCdVCgWaCXwLaeYT0EyJ5Q== AAACZHichVHLSsNAFD2Nr1ofrYogCFIsFVflRgTFVdGNS9taW1CRJE5raJqEJC3U4g/oVnHhSkFE/Aw3/oALf0AQlxXcuPA2DYgW9Q4zc+bMPXfOzKi2obse0VNI6unt6x8ID0aGhkdGo7Gx8S3XqjmayGuWYTlFVXGFoZsi7+meIYq2I5SqaoiCWllr7xfqwnF1y9z0GrbYrSplUy/pmuIxlRF7sQSlyI94N5ADkEAQG1bsBjvYhwUNNVQhYMJjbECBy20bMgg2c7toMucw0v19gSNEWFvjLMEZCrMVHsu82g5Yk9ftmq6v1vgUg7vDyjiS9Ei31KIHuqMX+vi1VtOv0fbS4FntaIW9Fz2eyr3/q6ry7OHgS/WnZw8lLPtedfZu+0z7FlpHXz88b+VWssnmHF3RK/u/pCe65xuY9TftOiOyF4jwB8g/n7sbbC2kZErJmcVEejX4ijCmMYt5fu8lpLGODeT5XIETnOIs9CwNSxPSZCdVCgWaCXwLaeYT0EyJ5Q== AAACZHichVHLSsNAFD2Nr1ofrYogCFIsFVflRgTFVdGNS9taW1CRJE5raJqEJC3U4g/oVnHhSkFE/Aw3/oALf0AQlxXcuPA2DYgW9Q4zc+bMPXfOzKi2obse0VNI6unt6x8ID0aGhkdGo7Gx8S3XqjmayGuWYTlFVXGFoZsi7+meIYq2I5SqaoiCWllr7xfqwnF1y9z0GrbYrSplUy/pmuIxlRF7sQSlyI94N5ADkEAQG1bsBjvYhwUNNVQhYMJjbECBy20bMgg2c7toMucw0v19gSNEWFvjLMEZCrMVHsu82g5Yk9ftmq6v1vgUg7vDyjiS9Ei31KIHuqMX+vi1VtOv0fbS4FntaIW9Fz2eyr3/q6ry7OHgS/WnZw8lLPtedfZu+0z7FlpHXz88b+VWssnmHF3RK/u/pCe65xuY9TftOiOyF4jwB8g/n7sbbC2kZErJmcVEejX4ijCmMYt5fu8lpLGODeT5XIETnOIs9CwNSxPSZCdVCgWaCXwLaeYT0EyJ5Q== fi ⌘ lim e!0 lim T !1 1 T T 1 X t=0 F(t) i AAACtXichVFNaxRBEH0Zv+IazUYvgpfBJSEeXGpCIEEQQgTxuEl2k0BmM8xMepIm8+VMz8JmmD/gH/DgSUFC8O4f8OIfyCGQq4HgMYIXD9bMDogGtZrurnpVr/p1txP7MlVEJ2PalavXrt8Yv9m4NXH7zmRz6u56GmWJK3pu5EfJpmOnwpeh6CmpfLEZJ8IOHF9sOPvPyvzGQCSpjMKuGsaiH9i7ofSkayuGrGbHs6RuipeZHOimLwMrF6aKdCrqqFtGpgw9NWTIS2w3N4q8W5hpxln1lIrtvPvYKJ5bcjufVY8Kq9miNlWmX3aM2mmhtk7UPISJHURwkSGAQAjFvg8bKY8tGCDEjPWRM5awJ6u8QIEGczOuElxhM7rP6y5HWzUaclz2TCu2y6f4PBNm6pimYzqiC/pMH+icfvy1V171KLUMeXdGXBFbk6/ur33/LyvgXWHvF+ufmhU8LFZaJWuPK6S8hTviDw5eX6w9WZ3OZ+gdfWX9b+mEPvENwsE39/2KWH2DBn+A8edzX3bW59oGtY2V+dbScv0V43iAh5jl917AEl6ggx6f+xGn+IIzbUHrazuaNyrVxmrOPfxmWvQTVAapmg== AAACtXichVFNaxRBEH0Zv+IazUYvgpfBJSEeXGpCIEEQQgTxuEl2k0BmM8xMepIm8+VMz8JmmD/gH/DgSUFC8O4f8OIfyCGQq4HgMYIXD9bMDogGtZrurnpVr/p1txP7MlVEJ2PalavXrt8Yv9m4NXH7zmRz6u56GmWJK3pu5EfJpmOnwpeh6CmpfLEZJ8IOHF9sOPvPyvzGQCSpjMKuGsaiH9i7ofSkayuGrGbHs6RuipeZHOimLwMrF6aKdCrqqFtGpgw9NWTIS2w3N4q8W5hpxln1lIrtvPvYKJ5bcjufVY8Kq9miNlWmX3aM2mmhtk7UPISJHURwkSGAQAjFvg8bKY8tGCDEjPWRM5awJ6u8QIEGczOuElxhM7rP6y5HWzUaclz2TCu2y6f4PBNm6pimYzqiC/pMH+icfvy1V171KLUMeXdGXBFbk6/ur33/LyvgXWHvF+ufmhU8LFZaJWuPK6S8hTviDw5eX6w9WZ3OZ+gdfWX9b+mEPvENwsE39/2KWH2DBn+A8edzX3bW59oGtY2V+dbScv0V43iAh5jl917AEl6ggx6f+xGn+IIzbUHrazuaNyrVxmrOPfxmWvQTVAapmg== AAACtXichVFNaxRBEH0Zv+IazUYvgpfBJSEeXGpCIEEQQgTxuEl2k0BmM8xMepIm8+VMz8JmmD/gH/DgSUFC8O4f8OIfyCGQq4HgMYIXD9bMDogGtZrurnpVr/p1txP7MlVEJ2PalavXrt8Yv9m4NXH7zmRz6u56GmWJK3pu5EfJpmOnwpeh6CmpfLEZJ8IOHF9sOPvPyvzGQCSpjMKuGsaiH9i7ofSkayuGrGbHs6RuipeZHOimLwMrF6aKdCrqqFtGpgw9NWTIS2w3N4q8W5hpxln1lIrtvPvYKJ5bcjufVY8Kq9miNlWmX3aM2mmhtk7UPISJHURwkSGAQAjFvg8bKY8tGCDEjPWRM5awJ6u8QIEGczOuElxhM7rP6y5HWzUaclz2TCu2y6f4PBNm6pimYzqiC/pMH+icfvy1V171KLUMeXdGXBFbk6/ur33/LyvgXWHvF+ufmhU8LFZaJWuPK6S8hTviDw5eX6w9WZ3OZ+gdfWX9b+mEPvENwsE39/2KWH2DBn+A8edzX3bW59oGtY2V+dbScv0V43iAh5jl917AEl6ggx6f+xGn+IIzbUHrazuaNyrVxmrOPfxmWvQTVAapmg== AAACtXichVFNaxRBEH0Zv+IazUYvgpfBJSEeXGpCIEEQQgTxuEl2k0BmM8xMepIm8+VMz8JmmD/gH/DgSUFC8O4f8OIfyCGQq4HgMYIXD9bMDogGtZrurnpVr/p1txP7MlVEJ2PalavXrt8Yv9m4NXH7zmRz6u56GmWJK3pu5EfJpmOnwpeh6CmpfLEZJ8IOHF9sOPvPyvzGQCSpjMKuGsaiH9i7ofSkayuGrGbHs6RuipeZHOimLwMrF6aKdCrqqFtGpgw9NWTIS2w3N4q8W5hpxln1lIrtvPvYKJ5bcjufVY8Kq9miNlWmX3aM2mmhtk7UPISJHURwkSGAQAjFvg8bKY8tGCDEjPWRM5awJ6u8QIEGczOuElxhM7rP6y5HWzUaclz2TCu2y6f4PBNm6pimYzqiC/pMH+icfvy1V171KLUMeXdGXBFbk6/ur33/LyvgXWHvF+ufmhU8LFZaJWuPK6S8hTviDw5eX6w9WZ3OZ+gdfWX9b+mEPvENwsE39/2KWH2DBn+A8edzX3bW59oGtY2V+dbScv0V43iAh5jl917AEl6ggx6f+xGn+IIzbUHrazuaNyrVxmrOPfxmWvQTVAapmg== player A player B

well known strategies your payoff is no less than the other's fragile against an error It cannot exploit naive cooperator. History Next move C,C C C,D D D,C C D,D D Tit-for-Tat c c c c d d c c d d c c TFT c c c c d d c c d d c c TFT TFT c c c c c c TFT AllC c c c c c c Win-Stay-Lose-Shift History Next move C,C C C,D D D,C D D,D C tolerant against an error It can exploit naive cooperator. weak against defectors c c d d c d d WSLS AllD d d d d d c c c d d c c c WSLS AllC d d c c c c c c d c c d d c c WSLS WSLS c

partners or rivals It would be great if a single strategy works as a partner as well as a rival. friendly rivals Hilbe et al., Nat.Hum.Behav. (2018) ⌦ Rival ⌦ ⌦ cooperation Partner

friendly rivals A’s payoff B’s payoff (R,R) (T,S) (S,T) (P,P) Friendly rivals form a cooperative Nash equilibrium with a guarantee that you'll never lose. ⌦ Rival The long-term payoff is never outperformed by any kind of co-player. ⌦ ⌦ cooperation Partner When both players use the strategy, they maintain cooperation with probability 1 as e -> 0.

Does a friendly rival strategy really exist? • # of Memory-1 strategies: 16 • => no strategies satisfies the criteria • # of Memory-2 strategies: 2^16 = 65536 • A friendly rival TFT-ATFT was found. Yi et al., J. Theor. Biol. (2017) It is impossible to generalize TFT-ATFT to n-players game.

solutions to 3-player public goods game • # of Memory-1 strategies: 2^8 • # of Memory-2 strategies: 2^40 • => No friendly rivals was found. • # of Memory-3 strategies: 2^288 • => There are at least 256 friendly rivals. Y Murase & S.K. Baek J. Theor. Biol. (2018)

The strategy is defined by action table having 512 entries. m n AAACi3ichVHNLgNRGD3GfynFRmLTaIhVc4sEjYUQiWV/tJogMjOuuun8mbltUo0XsLSxqA2JhXgAD2DjBSz6CGJZiY2Fr9NJBFHfZO4999zvfDMnR3MM4UnGGl1Kd09vX//AYGhoODwyGhkbz3t22dV5TrcN2y1oqscNYfGcFNLgBcflqqkZfEcrbbTudyrc9YRtbcuqw/dNtWiJI6GrkqiCGd0r8pOodRCJsTjzK/obJAIQQ1ApO/KAPRzCho4yTHBYkIQNqPDo2UUCDA5x+6gR5xIS/j3HGUKkLVMXpw6V2BKtRTrtBqxF59ZMz1fr9BWDXpeUUcywZ3bHmuyJ3bMX9vHnrJo/o/UvVdq1tpY7B6Pnk9n3f1Um7RLHX6oOCo26O3uSOMKy70WQN8dnWi719vzK6WUzm8zM1GbZDXslf9eswR7JoVV502/TPFNHiAJK/IzjN8jPxxOL8ZX0YmxtPYhqAFOYxhzlsYQ1bCGFnJ/DBeq4UsLKgpJUVtutSlegmcC3UjY/AZdykn0= a necessary condition to be a friendly rival strategy theoretical lower bound cdd 0 1 ccc 9 8 66 64 ddd ccc ccd,cdc, cdd ccc,cdc cdc dcc dcc ccd cdd 76 67 65 72 ccc,ccd cdd cdc ccd ddc dcd ddc dcd dcc dcc dcd,ddd ddc,ddd dcd ddc ddd ddd dcc dcc ddc dcd ddd ddc dcd α β δ δ γ γ TFT-ATFT solution for n=3

exploration of m=3 strategy space for Prisoner's Dilemma theoretical lower bound TFT-ATFT solution for n=3 m=1 : 256 m=2 : 65,536 m=3 : 18,446,744,073,709,551,616 222m AAACjXichVHLSsNAFD2N7/po1Y3gRiyKqzIpFR+IFF3o0lar4pMkTmtoXiTTgob+gGvBhSgouBA/wA9w4w+48BPEpYIbF96kAVFRb2YyZ87cc2cOV3UM3ROMPcakpuaW1rb2jnhnV3dPItnbt+rZVVfjRc02bHddVTxu6BYvCl0YfN1xuWKqBl9TK/PB+VqNu55uWyviwOHbplK29JKuKYKojcyOHwyzXt9NpliahTH0E8gRSCGKJTt5iy3swYaGKkxwWBCEDSjw6NuEDAaHuG34xLmE9PCco444aauUxSlDIbZC/zLtNiPWon1Q0wvVGt1i0HRJOYQR9sCu2Qu7Zzfsib3/WssPawRvOaBVbWi5s5s4Glh++1dl0iqw/6n6Q6FS9t+eBEqYDL3o5M0JmcCl1qhfOzx5WZ4ujPij7JI9k78L9sjuyKFVe9Wu8rxwijg1SP7ejp9gNZOWs+mpfDaVm4ta1Y5BDGOM+jGBHBaxhCLda+EYZziXEtK4NCPNNlKlWKTpx5eQFj4A/aSTjA== Yi et al., JTB (2017) → 4 friendly rivals → 0 friendly rivals # of memory-m strategies: → 4,261,844,305,281 friendly rivals most of them are so complicated strategies

CAPRI: a simple friendly rival Cooperate at mutual cooperation. c c c c c c c c CAPRI CAPRI c c c c Recover cooperation when someone cooperated. c c c c CAPRI c c d d d d d c memory-3 strategy described by the five rules In all the other cases, defect. d d d d CAPRI d d d d d d d d Accept punishment when you made a mistake. c c c c d c c d c c CAPRI CAPRI c c accept punishment Punish co-player and then forgive him. conduct punishment

payoff of CAPRI B’s payoff payoff against randomly selected strategies CAPRI Partner Rival Other simulation with memory-1 species and CAPRI Evolutionary game

Friendly rivals for general n-player game CAPRI-n : an extension of CAPRI to general n-person game m = 2n + 1 c … c … c d … c … c c d … c … c d … … c d d

CAPRI-n v.s. a randomly constructed memory-n strategy CAPRI-n v.s. CAPRI-n Evolutionary game

Conclusion • We showed that friendly rival strategies exist for Prisoner's Dilemma and n-players' public goods game. • We found CAPRI-n that is described by five simple rules. In various respects, it is one of the best strategies ever. • cooperation is robust against error (partner) • never allows any co-player to outperform you (rivalry) • evolutionary robust for any environmental conditions • exploit a broad range of strategies (gTFT, WSLS) • not only for PD but for stag-hunt game and snow-drift game

References • Y. Murase, S.K. Baek “Seven rules to avoid the tragedy of the commons” Journal of Theoretical Biology, 449, 94-102 (2018) • Yohsuke Murase, Seung Ki Baek “Automata representation of successful strategies for social dilemmas” Scientific Reports, 10, 13370 (2020) • Yohsuke Murase, Seung Ki Baek “Five rules for friendly rivalry in direct reciprocity” Scientific Reports, 10, 16904 (2020) • Yohsuke Murase, Seung Ki Baek “Friendly-rivalry solution to the iterated n-person public- goods game” Plos Computational Biology, 17(1): e1008217 (2021) • Yohsuke Murase, Seung Ki Baek “Friendly-rivalry solutions for direct reciprocity” Proceedings of the ALIFE 2021: The 2021 Conference on Artificial Life. ALIFE 2021: The 2021 Conference on Artificial Life. Online. (pp. 68). ASME. (2021)