How Deep is Deep Learning? Amar Lalwani Lead Engineer, R & D, funtoot Ph.D. Candidate, IIIT-Bangalore 

In a Classroom .. 

Homogeneous Teaching 

Two Sigma Problem 

Two sigma problem 

Funtoot: Intelligent Tutoring System • Every child is unique • Personalised (One-on-One) Tutoring • Mastery Learning 

Funtoot: Journey so far .. 

Student Data 1. Q1 => solved 2. Q2 => unsolved 3. Q3 => solved 4. Q1 => unsolved 5. Q3 => solved 6. Q4 => unsolved 7. Q1 => solved 8. Q2 => unsolved 

Student Data (Contd..) • How much does the student know? • Ask the student! 

Knowledge Tracing (KT) • For some skill K • Given student’s response sequence 1 to n, predict n+1 0 0 0 1 1 1 ? 1 ………..……… n n+1 Chronological response sequence for student Y [ 0 = Incorrect response 1 = Correct response] 

How do we approach this? • Modelling learner’s knowledge acquisition process • Fairly complex • Need a • General model • Flexible model • Powerful model 

Play the best card: Deep Learning 

Deep Knowledge Tracing (DKT) • Recurrent Neural Networks (RNNs) 

Deep Knowledge Tracing (DKT) • RNN or LSTM Model 0.9,0.3,0.2 0.8,0.2,0.1 0.8,0.5,0.3 Q1 Q2 Q3 …. Skill A Skill B Skill C 1.0,0.3,0.7 pCorrect(Skill A), pCorrect(Skill B), pCorrect(Skill C) 

Inter-skill Relationships 

Dataset • 6th Grade Math CBSE Curriculum • 22 topics, 69 sub-topics, 119 sub-sub-topics • 442 skills (LGs), 1523 problems • 7780 students, 176 schools • 2.4 million problem attempts • 5.6 million data-points • 76% avoidances (positive class:1) 

Results: Accuracy 0.75 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 Deep 

Results: Accuracy 0.75 0.65 0.6 0.62 0.64 0.66 0.68 0.7 0.72 0.74 0.76 Deep Shallow 

Results: Accuracy 

Bayesian Knowledge Tracing (BKT) Learned (know) UnLearned (Does not know) Incorrect Correct P(L0 ) 1-P(L0 ) P(T) 1-P(G) 1-P(S) P(G) P(S) 

BKT: Parameters • BKT: 2-state Hidden Markov Model (HMM) • P(L0 ): Probability of Initial Knowledge • P(T): Probability of Learning • P(S): Probability of Slip • P(G): Probability of Guess 

Shallow* Vs Deep Shallow* Deep Shallow* = Deep Performance Parameters 4 x # skills pInit, pLearn, pGuess, pSlip Few hundred thousand parameters Interpretability 

Deep Model: Advantages • Intelligent Curriculum Design • Finding best sequence of tasks • Discovery of structure • Instead of skill labels, question labels can be used as input • Complex representations and features 

is Deep Learning really DEEP? 

References • Knowledge Tracing: Modelling the acquisition of procedural knowledge (Corbett et. Al., 1995) • Bloom’s Two Sigma Problem (1984) • Deep Knowledge Tracing (Peich et. Al., 2015) • How deep is Knowledge Tracing? (Khajah et. Al., 2016) • Few hundred parameters outperform few hundred thousand? (Lalwani et. Al.), 2017 

Thank You! Questions??