and everything is too slow à Panic • Simplify model à Go back to basics: bag of vectors + nnet • Make a smaller network and dataset for debugging • Once no bugs: increase model size • Make sure you can overfit to your dataset • Plot your training and dev errors over training iterations • Then regularize with L2 and Dropout • Then do hyperparameter search • Come to OH! ( 3/6/18 Richard Socher Lecture 1, Slide 2 খ͍͞؆୯ͳϞσϧɾػೳͷ࣮͔Β࢝ΊΑ͏ খ͍࣍͞ݩɾσʔληοτͰࢼͯ͠ɼόάΛચ͍ग़ͦ͏ όά͕ແ͘ͳͬͨΒϞσϧͷαΠζͱσʔλΛ૿ͦ͏ 5SBJOJOHσʔλʹରͯ͠աֶशͰ͖Δ͜ͱΛ֬ೝ͠Α͏ 5SBJOJOHͱ7BMJEͷ-PTTΛಉ͡άϥϑͰ֬ೝ͠Α͏ աֶशΛ֬ೝ͔ͯ͠Βਖ਼ଇԽख๏ʢ-%SPQPVUʣΛࢼͦ͏ ࠷ޙʹϋΠύʔύϥϝʔλͷ୳ࡧΛ͠Α͏ https://web.stanford.edu/class/cs224n/lectures/lecture15.pdf ্͔Βॱ൪ʹΔ Լ͔Βࢼͯ͠͏·͍͔͘ͳ͍ à ্ʹͬͯΓ͠ʢखΓʣ͕ൃੜ ࣌ؒͷແବ
ࣗͷٳΈ࣌ؒʹܭࢉػΛ͍͡Ίൈ͘Πϝʔδ "Your computer should always be running experiments, since there's no reason that you should be working harder than your computer... On the other hand, just because your computer is running, it doesn't mean that actual progress is being made on the research, if your code/scripts have bugs!"