Five rules for friendly rivalry in direct reciprocity

Presentation for the following paper.
Yohsuke Murase and Seung Ki Baek "Five rules for friendly rivalry in direct reciprocity" arXiv:2004.00261

Transcript

1. Five rules for friendly rivalry in direct reciprocity Yohsuke Murase, Seung Ki Baek RIKEN Center for Computational Science, Japan Pukyong National University, Korea I have a solution! YM and S.K. Baek "Five rules for friendly rivalry in direct reciprocity" to appear in Sci.Rep. (arXiv:2004.00261)
2. iterated Prisoner’s Dilemma cooperation defection cooperation (3,3) (0,5) defection (5,0) (1,1) long-term payoff implementation error occurs with probability e <latexit sha1_base64="fSe004YKF6vMcUZEj33avSo9wuc=">AAACZHichVHLSsNAFD2Nr1ofrYogCFIsFVflRgTFVdGNS9taW1CRJE5raJqEJC3U4g/oVnHhSkFE/Aw3/oALf0AQlxXcuPA2DYgW9Q4zc+bMPXfOzKi2obse0VNI6unt6x8ID0aGhkdGo7Gx8S3XqjmayGuWYTlFVXGFoZsi7+meIYq2I5SqaoiCWllr7xfqwnF1y9z0GrbYrSplUy/pmuIxlRF7sQSlyI94N5ADkEAQG1bsBjvYhwUNNVQhYMJjbECBy20bMgg2c7toMucw0v19gSNEWFvjLMEZCrMVHsu82g5Yk9ftmq6v1vgUg7vDyjiS9Ei31KIHuqMX+vi1VtOv0fbS4FntaIW9Fz2eyr3/q6ry7OHgS/WnZw8lLPtedfZu+0z7FlpHXz88b+VWssnmHF3RK/u/pCe65xuY9TftOiOyF4jwB8g/n7sbbC2kZErJmcVEejX4ijCmMYt5fu8lpLGODeT5XIETnOIs9CwNSxPSZCdVCgWaCXwLaeYT0EyJ5Q==</latexit> <latexit sha1_base64="fSe004YKF6vMcUZEj33avSo9wuc=">AAACZHichVHLSsNAFD2Nr1ofrYogCFIsFVflRgTFVdGNS9taW1CRJE5raJqEJC3U4g/oVnHhSkFE/Aw3/oALf0AQlxXcuPA2DYgW9Q4zc+bMPXfOzKi2obse0VNI6unt6x8ID0aGhkdGo7Gx8S3XqjmayGuWYTlFVXGFoZsi7+meIYq2I5SqaoiCWllr7xfqwnF1y9z0GrbYrSplUy/pmuIxlRF7sQSlyI94N5ADkEAQG1bsBjvYhwUNNVQhYMJjbECBy20bMgg2c7toMucw0v19gSNEWFvjLMEZCrMVHsu82g5Yk9ftmq6v1vgUg7vDyjiS9Ei31KIHuqMX+vi1VtOv0fbS4FntaIW9Fz2eyr3/q6ry7OHgS/WnZw8lLPtedfZu+0z7FlpHXz88b+VWssnmHF3RK/u/pCe65xuY9TftOiOyF4jwB8g/n7sbbC2kZErJmcVEejX4ijCmMYt5fu8lpLGODeT5XIETnOIs9CwNSxPSZCdVCgWaCXwLaeYT0EyJ5Q==</latexit> <latexit sha1_base64="fSe004YKF6vMcUZEj33avSo9wuc=">AAACZHichVHLSsNAFD2Nr1ofrYogCFIsFVflRgTFVdGNS9taW1CRJE5raJqEJC3U4g/oVnHhSkFE/Aw3/oALf0AQlxXcuPA2DYgW9Q4zc+bMPXfOzKi2obse0VNI6unt6x8ID0aGhkdGo7Gx8S3XqjmayGuWYTlFVXGFoZsi7+meIYq2I5SqaoiCWllr7xfqwnF1y9z0GrbYrSplUy/pmuIxlRF7sQSlyI94N5ADkEAQG1bsBjvYhwUNNVQhYMJjbECBy20bMgg2c7toMucw0v19gSNEWFvjLMEZCrMVHsu82g5Yk9ftmq6v1vgUg7vDyjiS9Ei31KIHuqMX+vi1VtOv0fbS4FntaIW9Fz2eyr3/q6ry7OHgS/WnZw8lLPtedfZu+0z7FlpHXz88b+VWssnmHF3RK/u/pCe65xuY9TftOiOyF4jwB8g/n7sbbC2kZErJmcVEejX4ijCmMYt5fu8lpLGODeT5XIETnOIs9CwNSxPSZCdVCgWaCXwLaeYT0EyJ5Q==</latexit> <latexit sha1_base64="fSe004YKF6vMcUZEj33avSo9wuc=">AAACZHichVHLSsNAFD2Nr1ofrYogCFIsFVflRgTFVdGNS9taW1CRJE5raJqEJC3U4g/oVnHhSkFE/Aw3/oALf0AQlxXcuPA2DYgW9Q4zc+bMPXfOzKi2obse0VNI6unt6x8ID0aGhkdGo7Gx8S3XqjmayGuWYTlFVXGFoZsi7+meIYq2I5SqaoiCWllr7xfqwnF1y9z0GrbYrSplUy/pmuIxlRF7sQSlyI94N5ADkEAQG1bsBjvYhwUNNVQhYMJjbECBy20bMgg2c7toMucw0v19gSNEWFvjLMEZCrMVHsu82g5Yk9ftmq6v1vgUg7vDyjiS9Ei31KIHuqMX+vi1VtOv0fbS4FntaIW9Fz2eyr3/q6ry7OHgS/WnZw8lLPtedfZu+0z7FlpHXz88b+VWssnmHF3RK/u/pCe65xuY9TftOiOyF4jwB8g/n7sbbC2kZErJmcVEejX4ijCmMYt5fu8lpLGODeT5XIETnOIs9CwNSxPSZCdVCgWaCXwLaeYT0EyJ5Q==</latexit> fi ⌘ lim e!0 lim T !1 1 T T 1 X t=0 F(t) i <latexit sha1_base64="CfD1qyKg0ws1VkAs+U8zHbmbHOk=">AAACtXichVFNaxRBEH0Zv+IazUYvgpfBJSEeXGpCIEEQQgTxuEl2k0BmM8xMepIm8+VMz8JmmD/gH/DgSUFC8O4f8OIfyCGQq4HgMYIXD9bMDogGtZrurnpVr/p1txP7MlVEJ2PalavXrt8Yv9m4NXH7zmRz6u56GmWJK3pu5EfJpmOnwpeh6CmpfLEZJ8IOHF9sOPvPyvzGQCSpjMKuGsaiH9i7ofSkayuGrGbHs6RuipeZHOimLwMrF6aKdCrqqFtGpgw9NWTIS2w3N4q8W5hpxln1lIrtvPvYKJ5bcjufVY8Kq9miNlWmX3aM2mmhtk7UPISJHURwkSGAQAjFvg8bKY8tGCDEjPWRM5awJ6u8QIEGczOuElxhM7rP6y5HWzUaclz2TCu2y6f4PBNm6pimYzqiC/pMH+icfvy1V171KLUMeXdGXBFbk6/ur33/LyvgXWHvF+ufmhU8LFZaJWuPK6S8hTviDw5eX6w9WZ3OZ+gdfWX9b+mEPvENwsE39/2KWH2DBn+A8edzX3bW59oGtY2V+dbScv0V43iAh5jl917AEl6ggx6f+xGn+IIzbUHrazuaNyrVxmrOPfxmWvQTVAapmg==</latexit> <latexit sha1_base64="CfD1qyKg0ws1VkAs+U8zHbmbHOk=">AAACtXichVFNaxRBEH0Zv+IazUYvgpfBJSEeXGpCIEEQQgTxuEl2k0BmM8xMepIm8+VMz8JmmD/gH/DgSUFC8O4f8OIfyCGQq4HgMYIXD9bMDogGtZrurnpVr/p1txP7MlVEJ2PalavXrt8Yv9m4NXH7zmRz6u56GmWJK3pu5EfJpmOnwpeh6CmpfLEZJ8IOHF9sOPvPyvzGQCSpjMKuGsaiH9i7ofSkayuGrGbHs6RuipeZHOimLwMrF6aKdCrqqFtGpgw9NWTIS2w3N4q8W5hpxln1lIrtvPvYKJ5bcjufVY8Kq9miNlWmX3aM2mmhtk7UPISJHURwkSGAQAjFvg8bKY8tGCDEjPWRM5awJ6u8QIEGczOuElxhM7rP6y5HWzUaclz2TCu2y6f4PBNm6pimYzqiC/pMH+icfvy1V171KLUMeXdGXBFbk6/ur33/LyvgXWHvF+ufmhU8LFZaJWuPK6S8hTviDw5eX6w9WZ3OZ+gdfWX9b+mEPvENwsE39/2KWH2DBn+A8edzX3bW59oGtY2V+dbScv0V43iAh5jl917AEl6ggx6f+xGn+IIzbUHrazuaNyrVxmrOPfxmWvQTVAapmg==</latexit> <latexit sha1_base64="CfD1qyKg0ws1VkAs+U8zHbmbHOk=">AAACtXichVFNaxRBEH0Zv+IazUYvgpfBJSEeXGpCIEEQQgTxuEl2k0BmM8xMepIm8+VMz8JmmD/gH/DgSUFC8O4f8OIfyCGQq4HgMYIXD9bMDogGtZrurnpVr/p1txP7MlVEJ2PalavXrt8Yv9m4NXH7zmRz6u56GmWJK3pu5EfJpmOnwpeh6CmpfLEZJ8IOHF9sOPvPyvzGQCSpjMKuGsaiH9i7ofSkayuGrGbHs6RuipeZHOimLwMrF6aKdCrqqFtGpgw9NWTIS2w3N4q8W5hpxln1lIrtvPvYKJ5bcjufVY8Kq9miNlWmX3aM2mmhtk7UPISJHURwkSGAQAjFvg8bKY8tGCDEjPWRM5awJ6u8QIEGczOuElxhM7rP6y5HWzUaclz2TCu2y6f4PBNm6pimYzqiC/pMH+icfvy1V171KLUMeXdGXBFbk6/ur33/LyvgXWHvF+ufmhU8LFZaJWuPK6S8hTviDw5eX6w9WZ3OZ+gdfWX9b+mEPvENwsE39/2KWH2DBn+A8edzX3bW59oGtY2V+dbScv0V43iAh5jl917AEl6ggx6f+xGn+IIzbUHrazuaNyrVxmrOPfxmWvQTVAapmg==</latexit> <latexit sha1_base64="CfD1qyKg0ws1VkAs+U8zHbmbHOk=">AAACtXichVFNaxRBEH0Zv+IazUYvgpfBJSEeXGpCIEEQQgTxuEl2k0BmM8xMepIm8+VMz8JmmD/gH/DgSUFC8O4f8OIfyCGQq4HgMYIXD9bMDogGtZrurnpVr/p1txP7MlVEJ2PalavXrt8Yv9m4NXH7zmRz6u56GmWJK3pu5EfJpmOnwpeh6CmpfLEZJ8IOHF9sOPvPyvzGQCSpjMKuGsaiH9i7ofSkayuGrGbHs6RuipeZHOimLwMrF6aKdCrqqFtGpgw9NWTIS2w3N4q8W5hpxln1lIrtvPvYKJ5bcjufVY8Kq9miNlWmX3aM2mmhtk7UPISJHURwkSGAQAjFvg8bKY8tGCDEjPWRM5awJ6u8QIEGczOuElxhM7rP6y5HWzUaclz2TCu2y6f4PBNm6pimYzqiC/pMH+icfvy1V171KLUMeXdGXBFbk6/ur33/LyvgXWHvF+ufmhU8LFZaJWuPK6S8hTviDw5eX6w9WZ3OZ+gdfWX9b+mEPvENwsE39/2KWH2DBn+A8edzX3bW59oGtY2V+dbScv0V43iAh5jl917AEl6ggx6f+xGn+IIzbUHrazuaNyrVxmrOPfxmWvQTVAapmg==</latexit> player A player B
3. Strategies for IPD Tit-For-Tat (TFT) Win-Stay-Lose-Shift (WSLS) It is guaranteed that your payoff is no less than the co-players'. Cooperation is fragile against an error. Cooperation is tolerant against an error. Repeatedly exploited against AllD. c c c d c c c d d c c d d c d d c d d d Zero-determinant strategies (ZD) unilaterally control the co- players' payoff. They cannot achieve mutual cooperation. Press & Dyson, PNAS (2012)
4. partners or rivals? It would be great if a single strategy works as a partner as well as a rival. friendly rivalry (cooperative Nash equilibrium with a guarantee that you'll never lose.) Efficiency ⌦ ⌦ ⌦ cooperation Defensibility ⌦ Hilbe et al., Nat.Hum.Behav. (2018)
5. friendly rivals partners rivals A’s payoff B’s payoff (R,R) (T,S) (S,T) (P,P) Hilbe et al., Games Econ. Behav. (2015) rivals achieving mutual cooperation (the most strict partners)
6. previous research • # of Memory-1 strategies: 16 • => no strategies satisfies the criteria • # of Memory-2 strategies: 2^16 = 65536 • A friendly rival TFT-ATFT was found. Yi et al., J. Theor. Biol. (2017) Is TFT-ATFT the only possible solution for Prisoner's Dilemma? Yi et al. (2017) Murase&Baek (2018) We are going to study
7. exploration of a larger strategy space m=1 : 256 m=2 : 65,536 m=3 : 18,446,744,073,709,551,616 222m <latexit sha1_base64="SnIU+E5cPeQhQyyV4MGmqG9JhrI=">AAACjXichVHLSsNAFD2N7/po1Y3gRiyKqzIpFR+IFF3o0lar4pMkTmtoXiTTgob+gGvBhSgouBA/wA9w4w+48BPEpYIbF96kAVFRb2YyZ87cc2cOV3UM3ROMPcakpuaW1rb2jnhnV3dPItnbt+rZVVfjRc02bHddVTxu6BYvCl0YfN1xuWKqBl9TK/PB+VqNu55uWyviwOHbplK29JKuKYKojcyOHwyzXt9NpliahTH0E8gRSCGKJTt5iy3swYaGKkxwWBCEDSjw6NuEDAaHuG34xLmE9PCco444aauUxSlDIbZC/zLtNiPWon1Q0wvVGt1i0HRJOYQR9sCu2Qu7Zzfsib3/WssPawRvOaBVbWi5s5s4Glh++1dl0iqw/6n6Q6FS9t+eBEqYDL3o5M0JmcCl1qhfOzx5WZ4ujPij7JI9k78L9sjuyKFVe9Wu8rxwijg1SP7ejp9gNZOWs+mpfDaVm4ta1Y5BDGOM+jGBHBaxhCLda+EYZziXEtK4NCPNNlKlWKTpx5eQFj4A/aSTjA==</latexit> Yi et al., JTB (2017) We found 4,261,844,305,281 friendly rival strategies.
8. diverse recovery patterns There are indeed diverse friendly rival strategies other than TFT-ATFT. Alice Bob
9. 0 1 11 31 26 8 16 9 27 13 2 12 22 37 23 cc cc cd cd cc cd dc dd dd dc dd dc dd dc cd cc dc dd dc dd cd cc dd dc dd dc dc dd cc cd (ccc,ccc) ↓ (ccd,ccc) 0 1 10 14 32 8 17 35 13 59 19 2 21 16 12 25 56 3 dd dd cc cd dc dd dc cc cd dc dd dd dc dc dd dd dc dc cc cd cd cc dc dd dc dd cc cd cd cc cc cd dd dc cd cc (ccc,ccc)→(ccd,ccc) most of the strategies are not easy to interpret...
10. CAPRI: a simple friendly rival C A P R I ooperate at mutual cooperation. ccept punishment when you made a mistake. unish when your co-player defected. ecover cooperation when someone cooperated. n all the other cases, defect. CAPRI : A simple strategy described by five rules. 0 27 2 9 cc dd cc dd dc dd dc dc cd dd 10 1 cc 8 dc cd cd (ccc,ccc) ↓ (ccd,ccc)
11. YM and S.K. Baek, Sci.Rep. (2020) 0 27 2 9 cc dd cc dd dc dd dc dc cd dd 10 1 cc 8 dc cd cd (ccc,ccc) ↓ (ccd,ccc)
12. CAPRI is closer to Grim trigger (GT) rather than to TFT. GT : Cooperate in the first round. If defected once, switch to defect and never cooperate. 1. Error tolerant: CAPRI is ready to recover cooperation if Bob accepts punishment. (Rule A & P) 2. Recoverable: CAPRI can escape from mutual defection while GT is irreversible. (Rule R) c c c d d c c c ... ... ... ... c d c c c ... ... ... ... d d c c d d
13. Evolutionary simulation well-mixed population of N individuals updated by imitation process larger N, b/c → partners smaller N, b/c → rivals Hilbe et al., Nat.Hum.Behav. (2018) simulation with memory-1 species
14. when CAPRI is introduced friendly rivals are evolutionary robust for any N, b/c, and σ. ⇢  1 N <latexit sha1_base64="0EBTb+EBmBXDV2rQ3t+yE5oLH0c=">AAACmnichVHLSsNAFD3Gd31FRRB0USwVV2Uigo+V6EYRRKu1hVZKEqdtMC+TtKChP+APuHCl4EL9AD/AjT/gop8gLhXcuPAmDYgW6x1m5syZe+7M4Sq2rrkeY40OobOru6e3rz82MDg0PCKOjh24VtVReUa1dMvJKbLLdc3kGU/zdJ6zHS4bis6zyvF6cJ+tccfVLHPfO7X5oSGXTa2kqbJHVFGcKDgVK17Q+Um8UHJk1Zfq/na9KCZYioURbwVSBBKIYscSH1DAESyoqMIAhwmPsA4ZLo08JDDYxB3CJ84hpIX3HHXESFulLE4ZMrHHtJbplI9Yk85BTTdUq/SKTtMhZRxJ9sxu2Rt7YvfshX3+WcsPawR/OaVdaWq5XRw5n9z7+Fdl0O6h8q1qo1Aou70nDyUshV408maHTOBSbdavnV287a2kk/4su2av5O+KNdgjOTRr7+rNLk9fIkYNkn63oxUczKekhdTy7kJidS1qVR+mMIM56sciVrGBHWToXR9XuMO9MC2sCZvCVjNV6Ig04/gRwv4XYuCYZg==</latexit> evolutionary robust: for any mutant Plotkin & Stewart, PNAS (2014) CAPRI Partner Rival Other
15. CAPRI vs TFT-ATFT CAPRI is much better than TFT-ATFT. Distribution of payoffs against a randomly selected mixed strategy. CAPRI has strictly higher payoffs against a broad range of species. CAPRI Partner Rival Other TFT-ATFT
16. • is simple and plausible. • achieves mutual cooperation even under the noise. • recovers from erroneous defection in the shortest time. • never allows any co-player to outperform you. (rivalry) • gets a strictly higher payoff against a broad range of strategies. • is a deterministic strategy whose actions are independent of the values of benefit and cost of the game. • is effective not only for PD but for stag-hunt game and snow- drift game. • is evolutionary robust for any environmental conditions. • is generalizable to n-player public goods game. YM & S.K.Baek, "Friendly-rivalry solution to the iterated n-person public-goods game" arxiv:2008.00243 CAPRI is a superb strategy in various respects
17. Conclusion • We found (literally) trillions of friendly rival strategies. • TFT-ATFT is not the unique solution. • Among them, we found CAPRI that is described by five simple rules. In various respects, it is one of the best strategy ever. • Especially, it shows an excellent performance in evolutionary game, which may explain why the strategy looks plausible to us. YM and S.K. Baek "Five rules for friendly rivalry in direct reciprocity" to appear in Sci.Rep. (arXiv:2004.00261)