rinkou01_Cooperation Emergence under Resource-Constrained Peer Punishment

group learning with reading "Cooperation Emergence under Resource-Constrained Peer Punishment"

Transcript

1. Cooperation Emergence under
   Resource-Constrained Peer Punishment
   Samhar Mahmoud, Simon Miles, Michael Luck
   King’s College London, London, UK
   :7
   

   View Slide

2. ABOUT THIS PAPER
   • Publisher
   – AAMAS’16 Proceedings of the 2016
   International Conference on Autonomous
   Agents & Multiagent Systems, p900-908
   • Keywords
   – Metanorm, Emergence, Limited Enforcement Cost
   

   View Slide

3. ABSTRACT
   • bšIĤĦĬĵĆîëā
   ·”"(peer
   punishment)Ĉ‰ž‘(social norm)ė$øĔ
   ýĎĆ³
   • ôćķĭĽă÷āAxelrod[1]ćĶĩž‘ĠŁĵķĭ
   Ľðêēæûć[GĂêĔMahmoud[22,23]Žðê
   Ĕ
   • ÷ï÷æf=ćĶĩž‘ĠŁĵĂĈ”"Ć@øĔġ
   ĦĮė•U÷āîĒùæôć¦dĂĈ”"ĆďġĦĮ
   ė•U÷ýޑė$ĂñĔXvė_m÷āë
   Ĕ
   

   View Slide

4. INTRODUCTION
   • ġĿijĸŁĩĤĦĬĵĆîëāĈj·ć wð…
   i·Ć™:ą0c§ôēæ;»pć|Ĉ¼÷
   ë
   • ;»p3ĂąëĤĦĬĵĆîëā‰ž‘
   (Social Norm)ĈšDM‚ąĜŁĥěĿĮė“!
   øĔýĎĆ³ĂêĔ
   • ĜŁĥěĿĮė“!÷扞‘ė$øĔevă÷
   ā”"ėëĔôăðn¢öĕāñý
   

   View Slide

5. INTRODUCTION
   • ń P2PćĴĘęĽhĨĴĮ
   Data
   Data
   Data
   Data
   

   View Slide

6. INTRODUCTION
   • ń P2PćĴĘęĽhĨĴĮ
   Data
   Data
   Data
   Data
   Data
   ďĒìþół
   hĈ÷ąëł
   ŃĴļŁĻęĪŁń
   

   View Slide

7. INTRODUCTION
   • ń P2PćĴĘęĽhĨĴĮ
   Data
   Data
   Data
   Data
   Data
   ŃĴļŁĻęĪŁń
   ”
   ” ďĒìþół
   hĈ÷ąëł
   

   View Slide

8. INTRODUCTION
   • ń P2PćĴĘęĽhĨĴĮ
   Data
   Data
   Data
   Data
   Data
   h÷ąëăł
   ŃĴļŁĻęĪŁń
   ”
   ”
   

   View Slide

9. INTRODUCTION
   • ÷ï÷æôćè”"ėíĔéăëì›xĆġĦĮė•
   U÷āëĔ†‹ðąë
   – ħĿĢŁıīĮľŁĞąĄćļĨŁĦðBąë}8ĂĈę
   ĿĩĻĞĤĺĿė§ôøôăšðġĦĮĆąēìĔ
   • ôì÷ý}8ė{÷ýćŅĀćķĭĽð”"ćġĦĮ
   ė•U÷ý}8ĂĈž‘ėWŒĂñąëôăėˆ÷æôć/
   ½ėŸtøĔXvėˆøç
   Ä ĝļĥİĽćÊåÖÜáßÕÐÄÑ憋
   Å ÐÄÑė”"ð(‚Ć9)öúĒĕĔđìĆ[G÷ý
   ÎÒÙÝßäÕÐÅÅÁÅÆÑ憋
   

   View Slide

10. RELATED WORK
    • ĶĩIJĽĵĠŁĵ憋ĂĈËÖÙáŽĆđÿā”"ė
    ³òøĔôăĂĴļŁĻęĪŁðZ!öĕĔăëì†
    ‹ðêĔÐÄÃÁÄÄÑ
    • eĂĴļŁĻęĪŁð”"ė«øŃÔßäÞãÖá¿
    àäÞÚâÙÝÖÞãÀôăðĂñĔyuþăôć'kĈFČĔ
    ăÏÚÛÚ×ßáÒÛÚâĒðˆ÷āëĔÐÅÉÑ
    • ÌÖÜÓÚÞØĒĈôì÷ý'kėZíĔýĎĆ*¥÷ý6
    .ĆĈ5²ėíĔ’čė&íĔôăĂ*¥ė
    ®øĔôăĆW%÷āëĔÐÄÈÑ
    

    View Slide

11. RELATED WORK
    • ôì÷ý†‹ĂĈ”"Đ5²ćė1>Ć÷āîēæ
    ÎÒÙÝßäÕĒĈ(‚Ć”"ćė9)öúĔôăĂIJ
    Ľĵė~öúĔôăĆW%÷āëĔÐÅÄÑ
    • eĂÎÚÜÜÖáĒÐÅÇÁÅÈÑĐÍäáÔÒÐÄÉÑĒĈR5ė_ø
    ĔôăĆg‡ąaYëć’čė’čªčæq÷ë
    R5ė_öúĔôăė„]÷āëĔ
    • ôì÷ý†‹ĈQSćêĔĹŁģĆ@O÷đìă÷ý
    ďćþðæôĕĒĈā”"ėíĔôăšĆġ
    ĦĮė•U÷āëąë
    

    View Slide

12. PEER PUNISHMENT & LIMITED
    RESOURCES
    • P2PćĴĘęĽhĤĦĬĵĆîëāĈ
    ĴĘęĽėhøĔôă = *¥
    h÷ąëôă = œē
    ăøĔôăðĂñĔ
    • œĔĜŁĥěĿĮĆ@÷āĈ”"æíĉ>i
    ·ĴĘęĽėh÷ąëăëì›(ėăĔôăð•í
    ĒĕĔç
    • œÿýĜŁĥěĿĮăsĆĴĘęĽėh÷ąë
    ĖóĆĈëïąëýĎæĜŁĥěĿĮć›(ė•U
    ÷āæ°ąi·ėt>÷ûć·ĴĘęĽėh÷
    ąëăëìôăĆąĔ
    

    View Slide

13. Metanorm Model (Interaction Model)
    • ĜŁĥěĿĮõăĆæ
    *¥(cooperation)ïœē(defection)ėt>øĔ
    C%or%D
    C%or%D
    C%or%D
    C%or%D
    C%or%D
    

    View Slide

14. Metanorm Model (Interaction Model)
    • *¥Ĉš Lðąë
    C
    +0
    +0 +0
    +0
    +0
    

    View Slide

15. Metanorm Model (Interaction Model)
    • œē(temptationăď ì)Ĉ
    š©Ć ƒæ –Ć ƒ(hurt value)ėďýĒø
    C
    +3
    +1 +1
    +1
    +1
    D
    

    View Slide

16. Metanorm Model (Agent Model)
    • ĜŁĥěĿĮĈboldnessăvengefulnessė›(ć
    LïĒQ-learning÷āëĔ
    • ôć2ĀćïĒĜŁĥěĿĮć]µ(policy)ð
    t>öĕĔ
    :—ö(boldness)ņ œē(temptation)ė›ì‡z
    4P(vengefulness)%ņœēĜŁĥěĿĮė”øĔ‡z
    

    View Slide

17. Metanorm Model (Punishment Mechanism)
    • ”ėíĔo­ĆĈ2Š¾ðêĔ
    1. AxelrodćĝļĥİĽćķĭĽ(static model)
    – ”ć´Ĉ>
    2. MahmoudĒĆđĔ[GöĕýķĭĽ
    (adaptive model)
    – ”ć´ĈĜŁĥěĿĮć¯,ć›(ïĒ(‚Ćt>
    – ĜŁĥěĿĮĈûĕüĕćº^ĜŁĥěĿĮć›(ć
    Cr(image)ė\øĔ
    – imageĆKÿā”øĔėt>øĔ
    

    View Slide

18. EXPERIMENTAL EVALUATION
    • ĜŁĥěĿĮĈļĨŁĦė!¸öĕýyTĂæ-˜ą
    ›(ėpolicyĆđÿāt>›ì
    • ļĨŁĦĈ1ĻĚĿįŃāćĜŁĥěĿĮðęĿĩĻ
    ĞĤĺĿė›ìńõăĆļħīĮøĔ
    • Experimental EvaluationĂĈ£Ă¬ċý
    – Static model
    – Adaptive model
    ė¤øĔ
    • ıīĮľŁĞĈl<ıīĮľŁĞŀĦğŁĽĴļŁıī
    ĮľŁĞÛì
    

    View Slide

19. PARAMETERS SETUP
    

    View Slide

20. STATIC PUNISHMENT EXPERIMENTAL
    RESULTS
    Fig1%impact%of%limited%resources%with%punishment%on%final%B%and%V
    

    View Slide

21. ADAPTIVE PUNISHMENT EXPERIMENTAL
    RESULTS
    Fig2%impact%of%limited%resources%with%punishment%on%final%B%and%V
    

    View Slide

22. RESOURCE-AWARE PUNISHMENT MODEL
    • ôôČĂćXvĈĜŁĥěĿĮðš©ćļĨŁĦė•
    UøĔôăąò›(ė÷āëý
    • ûôĂĜŁĥěĿĮĈš©ćļĨŁĦăº^øĔĜŁ
    ĥěĿĮcæº^ĜŁĥěĿĮć¯,ć›((image)
    ïĒ”øĔ´ėt>øĔđìĆ[GøĔ
    

    View Slide

23. RESOURCE-AWARE PUNISHMENT MODEL
    • ĜŁĥěĿĮĈ¶önć£V¶ė\ĀďćăøĔ
    • êĔĜŁĥěĿĮ(agi)Ĉûćº^ĜŁĥěĿĮ(agj)õ
    ăƜēć#.(defection proportion): dpij
    ė¡
    øĔ
    • ôćdpij
    ėLocalDefImageăøĔ
    • öĒĆº^ĜŁĥěĿĮćdpij
    ćE2ė
    AvgDefImageăøĔ
    – LocalDefImageðAvgDefImageė¨íāëýĒHòæ¨
    íāëąóĕĉFò”øĔNðêĔ
    

    View Slide

24. RESOURCE-AWARE PUNISHMENT MODEL
    • ”øĔ#.(Deviation)ėt>øĔ
    • º^ĜŁĥěĿĮĊć+ļĨŁĦ´ė2ŽĆb÷
    UniformRes(agi
    )ăøĔ
    • ôĕĈġĦĮ汹ćĂæġĦĮĆ@øĔ”ć#.
    (enforcement cost percentage: ECP)Ôć#.
    (EquivPunish)Ć9`øĔ
    • ôĕĒïĒ?¹Ć”øĔ´Ĉ
    EquivPunish x DeviationăąĔ
    

    View Slide

25. Evaluation
    Fig3%impact%of%limited%resources%with%punishment%on%final%B%and%V
    

    View Slide

26. CONCLUSION
    • ôĕČĂćĶĩIJĽĵķĭĽėëý†‹ĂĈ”"
    ĆġĦĮė•U÷ýďćðąïÿý
    • ôôĂĈ”"ð>ćĝļĥİĽćķĭĽăġĦĮ
    ė(‚Ć9)öúýķĭĽćeĆæ”"ćġĦĮ
    ėA÷IJĽĵć~渀ėˆ÷ý
    • Čý暩ćļĨŁĦė•U÷ā”"ėîôąĔķĭ
    Ľė_m÷æđē*¥ĊćIJĽĵė~öúĒĕĔ
    ôăėˆ÷ý
    

    View Slide

27. FUTURE WORK
    • J憋ă÷āĈôćXvð ćo­( ćĠŁ
    ĵo­ńĆ°ĂñĔïŽðêĔ
    

    View Slide

28. REFERENCE
    [1] R. Axelrod. An evolutionary approach to norms. American Political Science Review,
    80(4):1095–1111, 1986.
    [10] E. Fehr and S. Ga%̈chter. Altruistic punishment in humans. Nature, 415(6868):137–140, Jan.
    2002.
    [11] E. Fehr and S. Ga%̈chter. Cooperation and punishment in public goods experiments. The
    American Economic Review, 90(4):pp. 980–994, 2000.
    [17] D. Helbing, A. Szolnoki, M. Perc, and G. SzabA%̃%̧s. Punish, but not too hard: how costly
    punishment spreads in the spatial public goods game. New Journal of Physics, 12(8):083005,
    2010.
    [18] R. Jurca and B. Faltings. An incentive compatible reputation mechanism. In Proceedings of
    the Second International Joint Conference on Autonomous Agents and Multiagent Systems,
    AAMAS ’03, pages 1026–1027. ACM, 2003.
    [21] S. Mahmoud, J. Keppens, N. Griffiths, and M. Luck. Efficient norm emergence through
    experiential dynamic punishment. In Proceedings of the 20th European Conference on Artificial
    Intelligence, pages 576–581. IOS Press, 2012.
    [22] S. Mahmoud, J. Keppens, M. Luck, and N. Griffiths. Norm establishment via metanorms in
    network topologies. In Proceedings of the 2011
    [23] S. Mahmoud, J. Keppens, M. Luck, and N. Griffiths. Overcoming omniscience in axelrod’s
    model. In Proceedings of the 2011 IEEE/WIC/ACM International Conferences on Web
    Intelligence and Intelligent Agent Technology - Volume 03, WI-IAT ’11, pages 29–32. IEEE
    Computer Society, 2011.
    [26] N. Miller, P. Resnick, and R. Zeckhauser. Eliciting honest feedback in electronic markets.
    KSG Working Paper Series RWP02-039, 2002.
    [27] N. Miller, P. Resnick, and R. Zeckhauser. Eliciting informative feedback: The peer-prediction
    method. Management Science, 51:2005, 2005.
    [28] N. Nikiforakis. Punishment and counter-punishment in public good games: Can we really
    govern ourselves? Journal of Public Economics, 92:91–112, 2008.
    

    View Slide