Tweet
Tweet

Slide 1

Slide 1 text

ML: recurrent neural networks April 2017

Slide 2

Slide 2 text

Recap

Slide 3

Slide 3 text

Dropout

Slide 4

Slide 4 text

Fully connected vs Convolution

Slide 5

Slide 5 text

“Deep” layers

Slide 6

Slide 6 text

Convolutional architecture

Slide 7

Slide 7 text

Convolutional network -> fully connected

Slide 8

Slide 8 text

Sliding window with different scales

Slide 9

Slide 9 text

Transforming a panda into a vulture

Slide 10

Slide 10 text

RNN

Slide 11

Slide 11 text

Limits of DNN/CNN - can only handle fixed-size inputs - unable to handle temporal data - no memory

Slide 12

Slide 12 text

New brick

Slide 13

Slide 13 text

Unrolling an RNN

Slide 14

Slide 14 text

Predicting the next letter

Slide 15

Slide 15 text

Example: translation

Slide 16

Slide 16 text

Example: speech-to-text

Slide 17

Slide 17 text

Multiple layers

Slide 18

Slide 18 text

Different architectures

Slide 19

Slide 19 text

Training RNN: Backpropagation through time

Slide 20

Slide 20 text

Problem: long term dependencies

Slide 21

Slide 21 text

Problem: long term dependencies

Slide 22

Slide 22 text

LSTM

Slide 23

Slide 23 text

Inside a standard RNN cell

Slide 24

Slide 24 text

Inside an LSTM cell

Slide 25

Slide 25 text

Decomposition: cell inner state

Slide 26

Slide 26 text

Forget gate

Slide 27

Slide 27 text

Update gate

Slide 28

Slide 28 text

Updating the state

Slide 29

Slide 29 text

Calculating an output

Slide 30

Slide 30 text

GRU

Slide 31

Slide 31 text

GRU (Gated Recurrent Unit)

Slide 32

Slide 32 text

Demo !

Slide 33

Slide 33 text

Example : Wikipedia Naturalism and decision for the majority of Arab countries' capitalide was grounded by the Irish language by [[John Clair]], [[An Imperial Japanese Revolt]], associated with Guangzham's sovereignty. His generals were the powerful ruler of the Portugal in the [[Protestant Immineners]], which could be said to be directly in Cantonese Communication, which followed a ceremony and set inspired prison, training. The emperor travelled back to [[Antioch, Perth, October 25|21]] to note, the Kingdom of Costa Rica, unsuccessful fashioned the [[Thrales]], [[Cynth's Dajoard]], known in western [[Scotland]], near Italy to the conquest of India with the conflict. Copyright was the succession of independence in the slop of Syrian influence that was a famous German movement based on a more popular servicious, non-doctrinal and sexual power post. Many governments recognize the military housing of the [[Civil Liberalization and Infantry Resolution 265 National Party in Hungary]], that is sympathetic to be to the [[Punjab Resolution]] (PJS)[http://www.humah.yahoo.com/guardian. cfm/7754800786d17551963s89.htm Official economics Adjoint for the Nazism, Montgomery was swear to advance to the resources for those Socialism's rule, was starting to signing a major tripad of aid exile.]]

Slide 34

Slide 34 text

Example : LATEX

Slide 35

Slide 35 text

Word2Vec

Slide 36

Slide 36 text

Word embedding?

Slide 37

Slide 37 text

Word2Vec

Slide 38

Slide 38 text

Implementing word2vec

Slide 39

Slide 39 text

Implementing word2vec

Slide 40

Slide 40 text

Yelp dataset

Slide 41

Slide 41 text

Yelp dataset

Slide 42

Slide 42 text

Next Unsupervised Learning Reinforcement Learning April 2017