Optimisation of short memory strategies in the Iterated Prisoners Dilemma

Wales Mathematics Colloquium 2017.

Transcript

1. Optimisation of short memory strategies in the Iterated Prisoners Dilemma Nikoleta E. Glynatsi Supervised by: Dr. Vincent Knight Dr. Jonathan Gillard
2. (3, 3) (0, 5) (5, 0) (1, 1)
3. (3, 3) (0, 5) (5, 0) (1, 1) (R, P, S, T) = (3, 1, 0, 5)
4. 1950 1955 1960 1965 1970 1975 1980 1985 1990 1995 2000 2005 2010 2015 0 20 40 60 80 100 number of records Articles per Year (N=1145)
5. CC CD DC DD C D C D C D C D p1 1 − p1 p2 1 − p2 p3 1 − p3 p4 1 − p4 p = (p1 , p2 , p3 , p4 ) ∈ R4 [0,1]
6. Christopher Lee, Marc Harper, and Dashiell Fryer. The art of war: Beyond memory-one strategies in population games. 2015.
7. How good are memory one strategies ?
8. CC CD DC DD
9. M =     p1 q1 p1 (−q1 + 1) q1 (−p1 + 1) (−p1 + 1)(−q1 + 1) p2 q3 p2 (−q3 + 1) q3 (−p2 + 1) (−p2 + 1)(−q3 + 1) p3 q2 p3 (−q2 + 1) q2 (−p3 + 1) (−p3 + 1)(−q2 + 1) p4 q4 p4 (−q4 + 1) q4 (−p4 + 1) (−p4 + 1)(−q4 + 1)    
10. maxp uq (p) such that p ∈ R4 [0,1]
11. Lemma uq(p) = 1 2 pQpT + cT p + a 1 2 p ¯ QpT + ¯ cT p + ¯ a Q, ¯ Q ∈ R4×4 c, ¯ c ∈ R4×1 a, ¯ a ∈ R
12. maxp uq (p) such that p ∈ R4 [0,1]
13. maxp uq (p) such that p ∈ R4 [0,1] subject to p1 = p2 = p3 = p4 = p
14. Lemma uq(p) = n2p2 + n1p + n0 d1p + d0 n2 = −(q1 − q2 − 2q3 + 2q4) n1 = −q1 + 2q2 + 5q3 − 7q4 − 1 n0 = q2 − 5q4 − 1 d1 = q1 − q2 − q3 + q4 d0 = q2 − q4 − 1
15. q = 1, 1, 0, 2 3 0 1 p 0 1 2 3 4 5 theoretic simulated
16. q = 1, 1, 0, 2 3 0 1 p 0 1 2 3 4 5 theoretic simulated uq (p) = −4p2 3 + 14p 3 − 10 3 2p 3 − 2 3
17. q = 1, 1, 0, 2 3 0 1 p 0 1 2 3 4 5 theoretic simulated uq (p) = −4p2 3 + 14p 3 − 10 3 2p 3 − 2 3 = −2p + 5
18. q = 1, 0, 1, 1 3 0 1 p 0 1 2 3 4 5 theoretic simulated
19. q = 1, 0, 1, 1 3 0 1 p 0 1 2 3 4 5 theoretic simulated uq (p) = p2 3 + 8p 3 − 10 3 p 3 − 4 3
20. q = 1, 0, 1, 1 3 0 1 p 0 1 2 3 4 5 theoretic simulated uq (p) = p2 3 + 8p 3 − 10 3 p 3 − 4 3 = p + 2
21. q = 2 3 , 0, 2 3 , 1 3 0 1 p 0 1 2 3 4 5 theoretic simulated
22. q = 2 3 , 0, 2 3 , 1 3 0 1 p 0 1 2 3 4 5 theoretic simulated uq (p) = 2p 3 − 8 3 p 3 − 4 3
23. q = 2 3 , 0, 2 3 , 1 3 0 1 p 0 1 2 3 4 5 theoretic simulated uq (p) = 2p 3 − 8 3 p 3 − 4 3 = 2
24. q = 2 3 , 1 3 , 1 3 , 0 0 1 p 0 1 2 3 4 5 theoretic simulated
25. q = 2 3 , 1 3 , 1 3 , 0 0 1 p 0 1 2 3 4 5 theoretic simulated uq (p) = p2 3 − 2p 3 − 2 3 −2 3
26. q = 2 3 , 1 3 , 1 3 , 0 0 1 p 0 1 2 3 4 5 theoretic simulated uq (p) = p2 3 − 2p 3 − 2 3 −2 3 = − p2 2 + p + 1
27. Lemma (Indifferent) −q1 + q2 + 2q3 − 2q4 = 0 and (q2 − q4 − 1)(q1 − 2q2 − 5q3 + 7q4 + 1) − (q2 − 5q4 − 1)(q1 − q2 − q3 + q4 ) = 0. Proof. uq (p) = n2 p2 + n1 p + n0 d1 p + d0 = a0 n2 p2 + n1 p + n0 = a0 d1 p + a0 d0 n2 = 0 n1 d0 = d1 n0
28. Lemma (Linear) (q1 q4 − q2 q3 + q3 − q4 )(4q1 − 3q2 − 4q3 + 3q4 − 1) = 0 Proof. uq (p) = n2 p2 + n1 p + n0 d1 p + d0 = a1 p + a0 n2 p2 + n1 p + n0 = a1 d1 p2 + (d1 a0 + a1 d0 )p + a0 d0 n2 = d1 a1 n1 d0 = d1 n0 + a1 d0
29. Lemma (Quadratic) (q1 − q2 − q3 + q4 ) = 0, (q1 q4 − q2 q3 + q3 − q4 )(4q1 − 3q2 − 4q3 + 3q4 − 1) = 0 and q2 − q4 − 1 = 0 Proof. uq (p) = n2 p2 + n1 p + n0 d1 p + d0 = a2 p2 + a1 p + a0 n2 p2 + n1 p + n0 = d1 a2 p3 + (a1 d1 + d0 a2 )p2 + (d1 a0 + a1 d0 )p + a0 d0      a1 d1 = 0 n2 = d1 a1 + d0 an2 n1 d0 = d1 n0 + a1 d0
30. du dp = m2 p2 + m1 p + m0 (d1 p + d0 )2 p uq p− p+ p uq p− p+ p uq p− p+ p uq p− p+
31. Theorem (Optimization of purely random player) Sq = 0, p± , 1 0 < p± < 1, p± = −d0 d1 p∗ = argmax p∈Sq uq (p)
32. q = 7 8 , 7 16 , 3 8 , 0 0.0 0.2 0.4 0.6 0.8 1.0 p 0 1 2 3 4 5 theoretical p* simulated
33. q = 1 3 , 2 3 , 1, 0 0.0 0.2 0.4 0.6 0.8 1.0 p 0 1 2 3 4 5 theoretical p* simulated
34. q(1), q(2), q(3) . . . q(N) max p 1 N N i=1 uq (i)(p)
35. q(1), q(2), q(3) . . . q(N) max p 1 N N i=1 uq (i)(p) max p u 1 N N i=1 q(i) (p)
36. 0.0 0.2 0.4 0.6 0.8 1.0 0 1 2 3 4 5 Tournament size N=9 q u p* simulated
37. p∗ = argmaxS q(1),...,q(n) u(p) where, | Sq(1),...,q(n) |≤ 2N + 2
38. p∗ = argmaxS q(1),...,q(n) u(p) where, | Sq(1),...,q(n) |≤ 2N + 2 @NikoletaGlyn https://github.com/Nikoleta-v3