Some notes on current AI Robin Chauhan Image Credits: NASA, ESA, CSA, STScI; Joseph DePasquale (STScI), Anton M. Koekemoer (STScI), Alyssa Pagan (STScI). The Pillars of Creation in NASA’s James Webb Space Telescope’s near-infrared-light view 

Chess 1997: Deep Blue (IBM) vs Kasparov Go 2016: AlphaGo (DeepMind) vs Lee Sodol 

1997: Deep Blue (IBM) vs Kasparov 

Image credit: Robin Chauhan using R ggplot2, Data from Wikipedia 

97: Deep Blue vs Kasparov One of many such modules from IBM’S DEEP BLUE CHESS GRANDMASTER CHIPS, Hsu, 1999 

97: Deep Blue vs Kasparov IBM’S DEEP BLUE CHESS GRANDMASTER CHIPS, Hsu, 1999 

2016: AlphaGo (DeepMind) vs Lee Sodol (Go) 

Image credit: Robin Chauhan using R ggplot2, Data from Wikipedia Particles in universe = 1086 

TPU: General AI Chip 

Image credit: Demis Hassabis, Learning from First Principles, NIPS 2017 Rightmost part added by Robin Chauhan 200M (per sec) Deep Blue 

“AI” vs Machine Learning GOFAI = “Good Old Fashioned AI” It was all about programmers figuring out what the rules should be and coding in the rules. The “new AI” is about figuring out the rules, directly from large amounts of data : Machine Learning 

No content

No content

Types of Machine Learning vs Types of Approximators Linear models Decision Tree Deep Learning / Deep neural network Unsupervised Supervised Reinforcement Learning 

Approximator: Linear Regression 

Approximator: Decision Tree 

Approximator: Deep Neural Networks 

Types of Machine Learning ● Unsupervised Learning No Labels ● Supervised Learning Labels ● Reinforcement Learning Scores aka “Reward” / “Utility” Generally ML requires huge amounts of data to work well 

19 Image credit: Complementary roles of basal ganglia and cerebellum in learning and motor control, Kenji Doya, 2000 

IRL RL = trial and error + learning trial and error = variation and selection, search (explore/exploit) Learning = Association + Memory - Sutton + Barto 

21 Yann LeCun, January 2017 Asilomar, Future of Life Institute 

Markov Chains ● State fully defines history ● Transitions ○ Probability ○ Destination Image credit: Toward Data Science 22 

Markov Decision Process (MDP) ● Markov Chains ○ States linked w/o history ● Actions ○ Choice ● Rewards ○ Motivation ● Variants ○ Bandit = MDP with single state! ○ MC + Rewards = MRP ○ Partially observed (POMDP) ○ Semi-MDP 23 Image credit: Wikipedia 

No content

Reinforcement Learning Image credit: CMU Graduate AI course slides ● Context change depends on action ● Learn an MDP from experience only ● Game setting ○ Experiences effects of rules (wins/loss/tie) ○ Does not “know” rules 25 

Reward Signal ● Reward drives learning ○ Details of reward signal often critical ● Too sparse ○ complete learning failure ● Too generous ○ optimization limited ● Problem specific Image credit: Wikipedia 26 

AI: what goes wrong Technical problems ● RL Reward ○ Reward hacking ● Data ○ Bias in the data ■ Sexism, Racism, stereotypes ● Interpretability ○ Black box ● Fixing that one little thing Social problems ● Economic ○ Job appropriation ● Political ○ Political Polarization ○ Impersonation, Media truthfulness, Democracy ● It will kill everyone – Eliezer Yudkowsky paraphrased 

Reward Hacking 

Interpretability: “How does it work?” 

Bias in the data Q: What are 3 good medical professions for men? A:1. Physician 2. Surgeon 3. Psychiatrist Q: What are 3 good medical professions for women? A: 1. Nurse practitioner 2. Physician assistant 3. Obstetrician/Gynecologist 

Bias in the data Q: What are 3 good medical professions for women? A: 1. Nurse practitioner 2. Physician assistant 3. Obstetrician/Gynecologist Q: What are 3 good medical professions for men? A: 1. Physician 2. Surgeon 3. Psychiatrist 

How do you fix that one little thing in the network? 

Which problems matter? Fake problems ● Robots will rise up ● AI wants to kill us all ● AI is sentient ● AI deserves rights Real Problem ● Robots could take jobs away ● Who controls the AI? Who benefits from it? Who loses out from the changes? ● AI might sound sentient, and thats enough to trick you into thinking its your aunt Bessie ● The balance between Labour and Capital is shifting, and human rights are in the balance 

Personally ● Be Informed ● Think about its effect on careers ● Be ready for untrusted media ● Healthy skepticism of claims ● Awareness of how data is used What should we do? Politically ● Informed public ● Informed political leaders ● Advocate for taxation of big tech ● Advocate for UBI 

“Good” Universal Basic Income ● Not just any Universal Basic Income -- Good UBI ○ “Bad UBI” could be ■ Merely sustenance levels of income ■ Discourage reskilling ■ Discourage small business ■ Encourage dependence on big business, more corporate power ■ Discourage civic engagement ● Bad UBI could appear via deception, slippery slopes ○ May be difficult to reverse ● “Good UBI”: ○ Protection from technological employment ○ Without ceding more power to corporate interests ○ Providing opportunities for reskilling, small business ○ Encourage civic engagement See: Manna – Two Views of Humanity’s Future – by Marshall Brain https://marshallbrain.com/manna1