http://www.inter.ipc.i.u-tokyo.ac.jp/ Decision making using classical and quantum light Makoto Naruse1, Nicolas Chauvet1, Serge Huant2, Satoshi Sunada3, Atsushi Uchida4, and Hirokazu Hori5 1 The University of Tokyo, Tokyo, Japan 2 Université Grenoble Alpes, CNRS, Institut Néel, Grenoble, France 3 Kanazawa University, Saitama, Japan 4 Saitama University, Saitama, Japan 5 University of Yamanashi, Kofu, Japan 

http://www.inter.ipc.i.u-tokyo.ac.jp/ Decision making using classical and quantum light Outline 1. Introduction 2. Decision making based on single photons 3. Ultrafast decision making based on laser chaos 4. Social decision making by entangled photons 5. Application studies: Laser chaos WiFi 6. Summary Sci Rep 2015 Sci Rep 2017 Sci Rep 2018 Sci Rep 2019 Sci Rep 2019 Sci Rep 2020 

3 ■Decision making Conduct prompt and accurate action in dynamically changing uncertain environments Action Reward Decision-Making System Dynamically changing, uncertain environments 

http://www.inter.ipc.i.u-tokyo.ac.jp/ Sutton, et al. Reinforcement Learning, MIT Press Multi-Armed Bandit problem (MAB) - Simple but very difficult… a fundamental problem in reinforcement learning - A variety of computer algorithms (e.g., -greedy, UCB, etc.) 

http://www.inter.ipc.i.u-tokyo.ac.jp/ 5 Why? Decision Making (Reinforcement Learning) is foundation of important applications Decision Making by Photonics Single Photon Decision Maker (Sci. Rep. 2015) Decision making by Laser Chaos (Sci. Rep. 2017) Ultrafast 1GHz (100 Gsample/s) ●Robotics ●Fintech High Frequency Trading ●AI ●Data Center ●Arbitration ●Network ●Social optimization ●Everyday life Microsoft DeepMind (Unexpected) Impact.. Invited to Google 2016 Kimura award Docomo mobile science award 

http://www.inter.ipc.i.u-tokyo.ac.jp/ Single-photon decision maker Naruse, Huant, et al. Sci. Rep. 2015 6 Autonomous adaptation to environmental uncertainty Utilize wave-particle duality of light quanta Two-armed bandit problem 

http://www.inter.ipc.i.u-tokyo.ac.jp/ Decision making using classical and quantum light Outline 1. Introduction 2. Decision making based on single photons 3. Ultrafast decision making based on laser chaos 4. Social decision making by entangled photons 5. Application studies: Laser chaos WiFi 6. Summary Sci Rep 2015 Sci Rep 2017 Sci Rep 2018 Sci Rep 2019 Sci Rep 2020 

http://www.inter.ipc.i.u-tokyo.ac.jp/ 8 What is laser chaos? (1) J. Ohtsubo, Semiconductor lasers: stability, instability and chaos (Springer, 2012). (2) A. Uchida, Optical communication with chaotic lasers (Wiley-VCH, 2012). Random number generator: A. Uchida, A. et al. Nature Photonics 2008. Reservoir computing: L. Larger, L. et al. Optics Express 2012. Applications 

http://www.inter.ipc.i.u-tokyo.ac.jp/ Ultrafast Decision Making by Laser Chaos M. Naruse, A. Uchida et al. Sci. Rep. 2017 REWARD 2-armed bandit problem 0 

http://www.inter.ipc.i.u-tokyo.ac.jp/ ● LASER CHAOS achieved superior performance M. Naruse, A. Uchida et al. Sci. Rep. 2017 WHY ?? By CHAOS By pseudorandom numbers Ultrafast Decision Making by Laser Chaos (cont) Correct decision ratio ● Maximum performance is obtained when the sampling interval is 50 ps. 

http://www.inter.ipc.i.u-tokyo.ac.jp/ EXACT COINCIDENCE ! Decision making performance M. Naruse, A. Uchida et al. Sci. Rep. 2017 Time-domain property Time-domain dynamics in laser chaos provides certain positive effects!! Ultrafast Decision Making by Laser Chaos (cont) 

http://www.inter.ipc.i.u-tokyo.ac.jp/ MN, M. Berthel, et al. ACS Photonics 2016 Scalability of decision making 2-armed 4-armed 8-armed 

http://www.inter.ipc.i.u-tokyo.ac.jp/ Chaos-based scalable decision making 000 001 010 011 100 101 110 110 Most Significant Bit (MSB) Second Most Significant bit Least Significant Bit (LSB) TH1 TH2,0 TH2,1 TH3,0,0 TH3,0,1 TH3,1,0 TH3,1,1 Naruse, Mihana, Uchida et al. Sci Rep 2018 Time-domain multiplexing 

http://www.inter.ipc.i.u-tokyo.ac.jp/ 1.16 a N  By CHAOS By CHAOS By CHAOS By pseudorandom numbers By pseudorandom numbers By pseudorandom numbers Naruse, Mihana, Uchida et al. Sci Rep 2018 

http://www.inter.ipc.i.u-tokyo.ac.jp/ Why chaos works well?  2 1 1 ( ) ( ) ( ) T t ETMSD x t x t T            “Ensemble average of the Time-averaged Mean Square Displacement” (ETMSD) Random walker’s trajectory Diffusivity analysis Naruse, Uchida et al. Sci Rep 2018 

http://www.inter.ipc.i.u-tokyo.ac.jp/ Ring laser Waveguide GaAs/AlGaAs semiconductor ring laser Radius: 1 mm On-chip photonic decision making using a ring laser Ring laser - Integrated photonics technology - Usually, deterministic operations (e.g. optical switch) [Principle] Spontaneous & probabilistic switching between ＣＷ and CCW oscillation Solving two-armed bandit problem CW CCW 16 2019 Chaotic time series is not the only resource for photonic decision making… 

http://www.inter.ipc.i.u-tokyo.ac.jp/ Decision making using laser network The emerging and unique attributes in network of lasers Leader-laggard synchronization Spontaneous and probabilistic switching between the leader and the laggard Solving two-armed bandit problem 17 • Synchronization among geographically different locations • Spontaneous symmetry breaking • Ultrafast dynamics Mihana, Uchida, et al. Optics Express 2019 POC of the utilization of laser synchronizations for intelligent functions 

http://www.inter.ipc.i.u-tokyo.ac.jp/ Decision making using classical and quantum light Outline 1. Introduction 2. Decision making based on single photons 3. Ultrafast decision making based on laser chaos 4. Social decision making by entangled photons 5. Application studies: Laser chaos WiFi 6. Summary Sci Rep 2015 Sci Rep 2017 Sci Rep 2018 Sci Rep 2019 Sci Rep 2020 

http://www.inter.ipc.i.u-tokyo.ac.jp/ 7 7 7 7 7 7 SLOT MACHINE A SLOT MACHINE B REWARD PROBABILITY 0.2 0.8 PLAYER 1 CONFLICT OF INTERESTS!! PLAYER 2 Competitive Multi-Armed Bandit problem (Competitive MAB) 

http://www.inter.ipc.i.u-tokyo.ac.jp/ 20 

http://www.inter.ipc.i.u-tokyo.ac.jp/ Social optimization and Equality [Social welfare] Conflict ! OR MACHINE A MACHINE B PLAYER 1 PLAYER 2 

http://www.inter.ipc.i.u-tokyo.ac.jp/ Slot machine A Slot machine B |H> |V> PBS |H> |V> ENTANGLED PHOTONS +   1 , , 2 s i s i H V V H  Slot machine A Slot machine B PLAYER 1 PLAYER 2 ABSOLUTELY NO CONFLICT Architecture + Signal light Idler light 

http://www.inter.ipc.i.u-tokyo.ac.jp/ Slot machine A Slot machine B |H> |V> PBS |H> |V> CORRELATED PHOTON PAIR HV Slot machine A Slot machine B PLAYER 1 PLAYER 2 CONFLICTS CAN HAPPEN Polarization-orthogonal photon pair 

http://www.inter.ipc.i.u-tokyo.ac.jp/ Experimental setup SPDC Chuavet et al. Sci Rep 2019 

http://www.inter.ipc.i.u-tokyo.ac.jp/ CONFLICT ratio PLAYER 2 PLAYER 1 NO COLLABORATION Conflict ratio is VERY high! Accumulated Reward 7 7 7 Machine A 0.8 7 7 7 Machine B 0.2 7 7 7 Machine B 0.8 7 7 7 Machine A 0.2 Chuavet et al. Sci Rep 2019 

http://www.inter.ipc.i.u-tokyo.ac.jp/ PLAYER 2 PLAYER 1 ENTANGLED PHOTON PAIR Conflict ratio is very small! Total reward becomes very high! CONFLICT ratio Accumulated Reward Chuavet et al. Sci Rep 2019 

http://www.inter.ipc.i.u-tokyo.ac.jp/ PLAYER 2 PLAYER 1 CORRELATED PHOTON PAIR Conflict ratio still high! CONFLICT ratio Accumulated Reward Chuavet et al. Sci Rep 2019 

http://www.inter.ipc.i.u-tokyo.ac.jp/ PLAYER 2 PLAYER 1 28 DECEPTION 

http://www.inter.ipc.i.u-tokyo.ac.jp/ With correlated photons Chuavet et al. Sci Rep 2019 Deception, success 

http://www.inter.ipc.i.u-tokyo.ac.jp/ 30 Yes, this way I would earn more … With entangled photons Chuavet et al. Sci Rep 2019 Deception, fail 

http://www.inter.ipc.i.u-tokyo.ac.jp/ 31 Featured at Behavioural & Social Sciences in Nature Research 2019 

http://www.inter.ipc.i.u-tokyo.ac.jp/ Decision making using classical and quantum light Outline 1. Introduction 2. Decision making based on single photons 3. Ultrafast decision making based on laser chaos 4. Social decision making by entangled photons 5. Application studies: Laser chaos WiFi 6. Summary Sci Rep 2015 Sci Rep 2017 Sci Rep 2018 Sci Rep 2019 Sci Rep 2020 

http://www.inter.ipc.i.u-tokyo.ac.jp/ 33 IEEE802.11a Takeuchi et al Sci Rep 2020 Higher throughput = Reward 

http://www.inter.ipc.i.u-tokyo.ac.jp/ 34 Dynamic channel selection Takeuchi et al Sci Rep 2020 

http://www.inter.ipc.i.u-tokyo.ac.jp/ 35 Takeuchi et al Sci Rep 2020 Experimental results 

http://www.inter.ipc.i.u-tokyo.ac.jp/ 36 2019 Artificial data generation using chaos Via pseudorandom numbers Via laser chaos time series Chaos GAN 

http://www.inter.ipc.i.u-tokyo.ac.jp/ Category theory approach to photonic decision making MN, Hori, Huant, et al. Int. J. Information Technology and Decision Making, 2018 MN, Hori, et al. Philosophies 2017 Kubota, MN, Hori, et al. Philosophies 2017 MN, et al. PLOS ONE 2018 Casino Setting /2 Polarizer Setting /2 Decision RESULT （Win or Lose） Photon environment Machine environment P M F Q X Y Unveil complex interdependency involved in decision making Theoretical fundamentals 

Summary Decision making using classical and quantum light Thank you for your kind attention! 1. Decision making based on single photons 2. Ultrafast decision making based on laser chaos 3. Social decision making by entangled photons 4. Applications Sci Rep 2015 Sci Rep 2017, Sci Rep 2018 Sci Rep 2019 Sci Rep 2019 Sci Rep 2020