Tweet
Tweet

Slide 1

Slide 1 text

D H L /2 2 3 1 3 2 / / @himkt or @_makoh_ @himkt

Slide 2

Slide 2 text

6 A2 A :C6 C A6 A 2 6 ) C:CF 06 :C: 2 ( 6A6 6 C 6 AC 6A: 2 ( 2 C6A C 6 :2C: A ( C2C: 2 : : C: ( : 2 6 2 6 6C 2 (2A 6 :6 .6 1 :E6A :CF G 2 6 06 62A 2 : https://github.com/glample/tagger

Slide 3

Slide 3 text

• ec a ) 02 2 ( 1 0 2 10 ( E B • N ( E • d R 6 ;6 ; 3

Slide 4

Slide 4 text

• ( ) ) ( )& • #.,"* '-%+* • '-)&!$ ./ https://explosion.ai/demos/displacy-ent 4

Slide 5

Slide 5 text

• E g • i Br Bi s E d g : • ol s S In e I O :E ) () () ( ,) George B-PER Washington E-PER Visited O England S-LOC PERSON: George Washington, LOCATION: England 5

Slide 6

Slide 6 text

• 2 0 3 • #'/1= 8B;765< 3ENEAD • ,0: Reuters Corpustrain, dev: 10?C/1, test: 1?C/1 • "0: ECI Multilingual Text Corpustrain, dev, test: 14-C/1 • 1. *%@2 • http://trec.nist.gov/data/reuters/reuters.html 2. NE$(+ @2 • https://www.clips.uantwerpen.be/conll2003/ner/ 3. NEAD%:9 • 2@2 >:9)&! . 6 https://www.clips.uantwerpen.be/conll2003/ner/

Slide 7

Slide 7 text

• ) ) 1S B N L • NE I prefixO L • 0NE L 0 B prefixO L • 2NER IC B prefix :E 3 • ) ) • NE L N 0 NE B prefixO L • ) ( B 1 • NE N E prefixO L • NE ELC S prefixO L 7 I-PER E-PER S-PER O S-LOC O B-PER I-PER I-PER B-PER O I-LOC O B-PER I-PER I-PER B-PER O I-LOC O I-PER IOB BIO BIOES

Slide 8

Slide 8 text

• R T 0 / 3 LC2 • N -) 3 M S EB • (- LF 2 E 3 8

Slide 9

Slide 9 text

X = (x1, x2, x3, x4), Y = (y1, y2, y3, y4), AAADa3ichVFNS9xQFD2Z2KpjW6e6EXUROsxgQeTFCopQEN0IdeFHR6c4MiTxjUbzRfJmaBzmD/gHuuhKwUVx7x9w486VC3+CiCuFbrroTSbSpuL0huTdc3LPue9ydc8yA8HYtZSRu1687O7pzfa9ev2mP/d2YD1w677BS4ZruX5Z1wJumQ4vCVNYvOz5XLN1i2/o+wvR/40G9wPTdT6L0ONbtrbjmDXT0ARR1dxZxdbErl5rlltK8aMy9gi/tqrquPIXmkyhDyk09X5cSUdRqVSyjxVf0tZhyjpMWYcp6/BZ62ouzyZYHMrTRE2SPJJYdnOXqGAbLgzUYYPDgaDcgoaAnk2oYPCI20ITe8Ro8AmZcQ1HC1nS14nnVKURu0/fHUKbCesQjnyD2MGgTha9PikVFNgV+8Hu2QU7ZTfs17Nezdgjuk9Ip97Wcq/afzi09vO/KptOgd0/qg4Kne7mEhbE6FTReT6BGmbiuUya04uZaGKj3atx8O1+bXa10CyyY3ZLsx6xa3ZO0zqNB+Nkha9+73gTO+pOC1X/Xd/TZH1yQqV8ZSo/N5+stgcjeIcx2t805rCIZZRgSKPSvPRJWsrcyYPysDzaLs1IiWYQqZALvwEsOdII AAADa3ichVFNS9xQFD2Z2KpjW6e6EXUROsxgQeTFCopQEN0IdeFHR6c4MiTxjUbzRfJmaBzmD/gHuuhKwUVx7x9w486VC3+CiCuFbrroTSbSpuL0huTdc3LPue9ydc8yA8HYtZSRu1687O7pzfa9ev2mP/d2YD1w677BS4ZruX5Z1wJumQ4vCVNYvOz5XLN1i2/o+wvR/40G9wPTdT6L0ONbtrbjmDXT0ARR1dxZxdbErl5rlltK8aMy9gi/tqrquPIXmkyhDyk09X5cSUdRqVSyjxVf0tZhyjpMWYcp6/BZ62ouzyZYHMrTRE2SPJJYdnOXqGAbLgzUYYPDgaDcgoaAnk2oYPCI20ITe8Ro8AmZcQ1HC1nS14nnVKURu0/fHUKbCesQjnyD2MGgTha9PikVFNgV+8Hu2QU7ZTfs17Nezdgjuk9Ip97Wcq/afzi09vO/KptOgd0/qg4Kne7mEhbE6FTReT6BGmbiuUya04uZaGKj3atx8O1+bXa10CyyY3ZLsx6xa3ZO0zqNB+Nkha9+73gTO+pOC1X/Xd/TZH1yQqV8ZSo/N5+stgcjeIcx2t805rCIZZRgSKPSvPRJWsrcyYPysDzaLs1IiWYQqZALvwEsOdII AAADa3ichVFNS9xQFD2Z2KpjW6e6EXUROsxgQeTFCopQEN0IdeFHR6c4MiTxjUbzRfJmaBzmD/gHuuhKwUVx7x9w486VC3+CiCuFbrroTSbSpuL0huTdc3LPue9ydc8yA8HYtZSRu1687O7pzfa9ev2mP/d2YD1w677BS4ZruX5Z1wJumQ4vCVNYvOz5XLN1i2/o+wvR/40G9wPTdT6L0ONbtrbjmDXT0ARR1dxZxdbErl5rlltK8aMy9gi/tqrquPIXmkyhDyk09X5cSUdRqVSyjxVf0tZhyjpMWYcp6/BZ62ouzyZYHMrTRE2SPJJYdnOXqGAbLgzUYYPDgaDcgoaAnk2oYPCI20ITe8Ro8AmZcQ1HC1nS14nnVKURu0/fHUKbCesQjnyD2MGgTha9PikVFNgV+8Hu2QU7ZTfs17Nezdgjuk9Ip97Wcq/afzi09vO/KptOgd0/qg4Kne7mEhbE6FTReT6BGmbiuUya04uZaGKj3atx8O1+bXa10CyyY3ZLsx6xa3ZO0zqNB+Nkha9+73gTO+pOC1X/Xd/TZH1yQqV8ZSo/N5+stgcjeIcx2t805rCIZZRgSKPSvPRJWsrcyYPysDzaLs1IiWYQqZALvwEsOdII AAADa3ichVFNS9xQFD2Z2KpjW6e6EXUROsxgQeTFCopQEN0IdeFHR6c4MiTxjUbzRfJmaBzmD/gHuuhKwUVx7x9w486VC3+CiCuFbrroTSbSpuL0huTdc3LPue9ydc8yA8HYtZSRu1687O7pzfa9ev2mP/d2YD1w677BS4ZruX5Z1wJumQ4vCVNYvOz5XLN1i2/o+wvR/40G9wPTdT6L0ONbtrbjmDXT0ARR1dxZxdbErl5rlltK8aMy9gi/tqrquPIXmkyhDyk09X5cSUdRqVSyjxVf0tZhyjpMWYcp6/BZ62ouzyZYHMrTRE2SPJJYdnOXqGAbLgzUYYPDgaDcgoaAnk2oYPCI20ITe8Ro8AmZcQ1HC1nS14nnVKURu0/fHUKbCesQjnyD2MGgTha9PikVFNgV+8Hu2QU7ZTfs17Nezdgjuk9Ip97Wcq/afzi09vO/KptOgd0/qg4Kne7mEhbE6FTReT6BGmbiuUya04uZaGKj3atx8O1+bXa10CyyY3ZLsx6xa3ZO0zqNB+Nkha9+73gTO+pOC1X/Xd/TZH1yQqV8ZSo/N5+stgcjeIcx2t805rCIZZRgSKPSvPRJWsrcyYPysDzaLs1IiWYQqZALvwEsOdII : : h = Bi-LSTM(X), Y = CRF(h), where h = (h1, h2, h3, h4). AAADxniclVHPS9xAFH4xrdVt1bVeCr2ELi4KdplYoaUgiELxUEF3XV0xsmTSWTM1v0hmd7Vhpef+Az2UHipoEf+MXrx56sE/ofRowYsHX7IRs0pXfCEz733zvm/m41HP4oEg5FTqkR887H3U1595/GRgcCg7/HQlcOu+wcqGa7l+heoBs7jDyoILi1U8n+k2tdgq3ZqLzlcbzA+46yyLHY9t2Pqmw2vc0AVC1WEpq9m6MGktNFtKflrRbOpuh7P85fvS8kJr7Oqw0hqfUO4TeUXTMlfstZT0XPHdtax5X9mUdCTWNJnPdlE9ZWJaSclX1QklVU12VK86qqnxQixdzeZIgcSh3E7UJMlBEotu9gQ0+AAuGFAHGxg4IDC3QIcAv3VQgYCH2AaE8BERHXyseNzDoAUZ5NcRZ9ilI7qF6yZW6wnqYB3pBrGCgTdZ+PvIVGCU/CaH5IwckyPyh1z8VyuMNaL37OBO21zmVYe+PCud38mycRdgXrO6MCi+zcVaIEKxo7s/ATV4E/vi6NOLkcix0b6r8enrWeltcTTMkz3yF73+IKfkF7p1Gv+M/SVW/Nb1JXZ0Ow5UvTm+28nKZEHFfGkqNzObjLYPnsMLGMP5vYYZmIdFKIMhNaXv0oH0U56XHbkuN9utPVLCGYGOkD9fAlv15MY= AAADxniclVHPS9xAFH4xrdVt1bVeCr2ELi4KdplYoaUgiELxUEF3XV0xsmTSWTM1v0hmd7Vhpef+Az2UHipoEf+MXrx56sE/ofRowYsHX7IRs0pXfCEz733zvm/m41HP4oEg5FTqkR887H3U1595/GRgcCg7/HQlcOu+wcqGa7l+heoBs7jDyoILi1U8n+k2tdgq3ZqLzlcbzA+46yyLHY9t2Pqmw2vc0AVC1WEpq9m6MGktNFtKflrRbOpuh7P85fvS8kJr7Oqw0hqfUO4TeUXTMlfstZT0XPHdtax5X9mUdCTWNJnPdlE9ZWJaSclX1QklVU12VK86qqnxQixdzeZIgcSh3E7UJMlBEotu9gQ0+AAuGFAHGxg4IDC3QIcAv3VQgYCH2AaE8BERHXyseNzDoAUZ5NcRZ9ilI7qF6yZW6wnqYB3pBrGCgTdZ+PvIVGCU/CaH5IwckyPyh1z8VyuMNaL37OBO21zmVYe+PCud38mycRdgXrO6MCi+zcVaIEKxo7s/ATV4E/vi6NOLkcix0b6r8enrWeltcTTMkz3yF73+IKfkF7p1Gv+M/SVW/Nb1JXZ0Ow5UvTm+28nKZEHFfGkqNzObjLYPnsMLGMP5vYYZmIdFKIMhNaXv0oH0U56XHbkuN9utPVLCGYGOkD9fAlv15MY= AAADxniclVHPS9xAFH4xrdVt1bVeCr2ELi4KdplYoaUgiELxUEF3XV0xsmTSWTM1v0hmd7Vhpef+Az2UHipoEf+MXrx56sE/ofRowYsHX7IRs0pXfCEz733zvm/m41HP4oEg5FTqkR887H3U1595/GRgcCg7/HQlcOu+wcqGa7l+heoBs7jDyoILi1U8n+k2tdgq3ZqLzlcbzA+46yyLHY9t2Pqmw2vc0AVC1WEpq9m6MGktNFtKflrRbOpuh7P85fvS8kJr7Oqw0hqfUO4TeUXTMlfstZT0XPHdtax5X9mUdCTWNJnPdlE9ZWJaSclX1QklVU12VK86qqnxQixdzeZIgcSh3E7UJMlBEotu9gQ0+AAuGFAHGxg4IDC3QIcAv3VQgYCH2AaE8BERHXyseNzDoAUZ5NcRZ9ilI7qF6yZW6wnqYB3pBrGCgTdZ+PvIVGCU/CaH5IwckyPyh1z8VyuMNaL37OBO21zmVYe+PCud38mycRdgXrO6MCi+zcVaIEKxo7s/ATV4E/vi6NOLkcix0b6r8enrWeltcTTMkz3yF73+IKfkF7p1Gv+M/SVW/Nb1JXZ0Ow5UvTm+28nKZEHFfGkqNzObjLYPnsMLGMP5vYYZmIdFKIMhNaXv0oH0U56XHbkuN9utPVLCGYGOkD9fAlv15MY= AAADxniclVHPS9xAFH4xrdVt1bVeCr2ELi4KdplYoaUgiELxUEF3XV0xsmTSWTM1v0hmd7Vhpef+Az2UHipoEf+MXrx56sE/ofRowYsHX7IRs0pXfCEz733zvm/m41HP4oEg5FTqkR887H3U1595/GRgcCg7/HQlcOu+wcqGa7l+heoBs7jDyoILi1U8n+k2tdgq3ZqLzlcbzA+46yyLHY9t2Pqmw2vc0AVC1WEpq9m6MGktNFtKflrRbOpuh7P85fvS8kJr7Oqw0hqfUO4TeUXTMlfstZT0XPHdtax5X9mUdCTWNJnPdlE9ZWJaSclX1QklVU12VK86qqnxQixdzeZIgcSh3E7UJMlBEotu9gQ0+AAuGFAHGxg4IDC3QIcAv3VQgYCH2AaE8BERHXyseNzDoAUZ5NcRZ9ilI7qF6yZW6wnqYB3pBrGCgTdZ+PvIVGCU/CaH5IwckyPyh1z8VyuMNaL37OBO21zmVYe+PCud38mycRdgXrO6MCi+zcVaIEKxo7s/ATV4E/vi6NOLkcix0b6r8enrWeltcTTMkz3yF73+IKfkF7p1Gv+M/SVW/Nb1JXZ0Ow5UvTm+28nKZEHFfGkqNzObjLYPnsMLGMP5vYYZmIdFKIMhNaXv0oH0U56XHbkuN9utPVLCGYGOkD9fAlv15MY= 9

Slide 10

Slide 10 text

• • • Bi-LSTM 10

Slide 11

Slide 11 text

Ct = (c1, c2, . . . , cN ), ct = Bi-LSTM(Ct), xt = wt ct, ht = Bi-LSTM(xt) AAAD1XichVHPT9RAFH6lorj+YNGLiQcbN2wgATIlJhoTEsJePKABlmVJKGk6wyw70HaadnYBm94MBy8ePXDSxIPRq/4DXrx58sCfYDxiwsWDr92Ku7Ku07Tzvq/v+957eTRwRaQIOdaG9AvDFy+NXC5cuXrt+mhx7MZaJFsh4zUmXRmuUyfirvB5TQnl8vUg5I5HXV6nu5X0f73Nw0hIf1UdBHzTc7Z90RDMUUjZY9ody3NUkzbiSmIrozxnTPwmWGKbU0YXmkW0JVXUQz6ZnDLKhmUVurjMx/Ko3I8XxPRidfVxcuZasVWCkr6nx2f/zKeD91JsycBtRd31VR+vHp/m4H7SMpP928ms7GKJzJDsGOcDMw9KkJ8lWfwCFmyBBAYt8ICDDwpjFxyI8NkAEwgEyG1CDDvIOBAiElkOhwQKqG8hzzHLQXYXv9uINnLWR5z6RpkDw0ouviEqDRgnX8lbckI+k3fkG/n5T68480j7OcCbdrQ8sEef36qe/lfl4a2g+Uc1QEGxN4lYIUMxY/B8ChrwIJtL4JxBxqQTs06t9tOXJ9WHK+Nxmbwm33HWV+SYfMJp/fYP9maZrxwN7MRLq+NCzb/Xdz5Ym50xMV6+V5pfyFc7ArfhLkzg/u7DPDyCJagB0w6199oH7aNe1xP9mX7YSR3Scs1N6Dn6i1/YL/Fz AAAD1XichVHPT9RAFH6lorj+YNGLiQcbN2wgATIlJhoTEsJePKABlmVJKGk6wyw70HaadnYBm94MBy8ePXDSxIPRq/4DXrx58sCfYDxiwsWDr92Ku7Ku07Tzvq/v+957eTRwRaQIOdaG9AvDFy+NXC5cuXrt+mhx7MZaJFsh4zUmXRmuUyfirvB5TQnl8vUg5I5HXV6nu5X0f73Nw0hIf1UdBHzTc7Z90RDMUUjZY9ody3NUkzbiSmIrozxnTPwmWGKbU0YXmkW0JVXUQz6ZnDLKhmUVurjMx/Ko3I8XxPRidfVxcuZasVWCkr6nx2f/zKeD91JsycBtRd31VR+vHp/m4H7SMpP928ms7GKJzJDsGOcDMw9KkJ8lWfwCFmyBBAYt8ICDDwpjFxyI8NkAEwgEyG1CDDvIOBAiElkOhwQKqG8hzzHLQXYXv9uINnLWR5z6RpkDw0ouviEqDRgnX8lbckI+k3fkG/n5T68480j7OcCbdrQ8sEef36qe/lfl4a2g+Uc1QEGxN4lYIUMxY/B8ChrwIJtL4JxBxqQTs06t9tOXJ9WHK+Nxmbwm33HWV+SYfMJp/fYP9maZrxwN7MRLq+NCzb/Xdz5Ym50xMV6+V5pfyFc7ArfhLkzg/u7DPDyCJagB0w6199oH7aNe1xP9mX7YSR3Scs1N6Dn6i1/YL/Fz AAAD1XichVHPT9RAFH6lorj+YNGLiQcbN2wgATIlJhoTEsJePKABlmVJKGk6wyw70HaadnYBm94MBy8ePXDSxIPRq/4DXrx58sCfYDxiwsWDr92Ku7Ku07Tzvq/v+957eTRwRaQIOdaG9AvDFy+NXC5cuXrt+mhx7MZaJFsh4zUmXRmuUyfirvB5TQnl8vUg5I5HXV6nu5X0f73Nw0hIf1UdBHzTc7Z90RDMUUjZY9ody3NUkzbiSmIrozxnTPwmWGKbU0YXmkW0JVXUQz6ZnDLKhmUVurjMx/Ko3I8XxPRidfVxcuZasVWCkr6nx2f/zKeD91JsycBtRd31VR+vHp/m4H7SMpP928ms7GKJzJDsGOcDMw9KkJ8lWfwCFmyBBAYt8ICDDwpjFxyI8NkAEwgEyG1CDDvIOBAiElkOhwQKqG8hzzHLQXYXv9uINnLWR5z6RpkDw0ouviEqDRgnX8lbckI+k3fkG/n5T68480j7OcCbdrQ8sEef36qe/lfl4a2g+Uc1QEGxN4lYIUMxY/B8ChrwIJtL4JxBxqQTs06t9tOXJ9WHK+Nxmbwm33HWV+SYfMJp/fYP9maZrxwN7MRLq+NCzb/Xdz5Ym50xMV6+V5pfyFc7ArfhLkzg/u7DPDyCJagB0w6199oH7aNe1xP9mX7YSR3Scs1N6Dn6i1/YL/Fz AAAD1XichVHPT9RAFH6lorj+YNGLiQcbN2wgATIlJhoTEsJePKABlmVJKGk6wyw70HaadnYBm94MBy8ePXDSxIPRq/4DXrx58sCfYDxiwsWDr92Ku7Ku07Tzvq/v+957eTRwRaQIOdaG9AvDFy+NXC5cuXrt+mhx7MZaJFsh4zUmXRmuUyfirvB5TQnl8vUg5I5HXV6nu5X0f73Nw0hIf1UdBHzTc7Z90RDMUUjZY9ody3NUkzbiSmIrozxnTPwmWGKbU0YXmkW0JVXUQz6ZnDLKhmUVurjMx/Ko3I8XxPRidfVxcuZasVWCkr6nx2f/zKeD91JsycBtRd31VR+vHp/m4H7SMpP928ms7GKJzJDsGOcDMw9KkJ8lWfwCFmyBBAYt8ICDDwpjFxyI8NkAEwgEyG1CDDvIOBAiElkOhwQKqG8hzzHLQXYXv9uINnLWR5z6RpkDw0ouviEqDRgnX8lbckI+k3fkG/n5T68480j7OcCbdrQ8sEef36qe/lfl4a2g+Uc1QEGxN4lYIUMxY/B8ChrwIJtL4JxBxqQTs06t9tOXJ9WHK+Nxmbwm33HWV+SYfMJp/fYP9maZrxwN7MRLq+NCzb/Xdz5Ym50xMV6+V5pfyFc7ArfhLkzg/u7DPDyCJagB0w6199oH7aNe1xP9mX7YSR3Scs1N6Dn6i1/YL/Fz 11

Slide 12

Slide 12 text

ct = Bi-LSTM(Ct), AAAC/HichVFNSxxBEH2OmhiNcWMugpchi4uCSm8IRARB9BAPBtR1VXBkmB57teN8MdO7aIb1B+QP5JBTQjwE8eofyCW3HMSDoD9APCrk4iE1sxNETdYaprvqdb2qehQPHBkpxk5atNa29kePO550dj3tftaTe967FPnV0BZl23f8cIVbkXCkJ8pKKkesBKGwXO6IZb41nbwv10QYSd9bVDuBWHOtDU9WpG0pgszcW8O11CavxHbdVHphQjdc7m/HU3JktrT4rj7493naVPWhYf2fVtANw8zl2ShLTb/vFDMnj8zm/NwvGFiHDxtVuBDwoMh3YCGibxVFMASErSHGe0IshBTJNEegjk7iVwkXlGURukXnBkWrGepRnNSN0go2dXLoD4mpY4Ads+/skv1k++ycXf+3VpzWSObZoZs3uCIwez72lX4/yHLpVti8YTVhcJrNp1gRwimjuT6FCsZSXZJ0BimSKLYbvWofPl2WxhcG4gL7yi5I6xd2wn6QWq92Ze/Ni4XPTSdxk+600OLd9d13ll6NFsmff52fnMpW24F+vMQg7e8NJjGDOZSp/gGOcIozbVf7pu1rB41UrSXjvMAt0w7/AH7MsO0= AAAC/HichVFNSxxBEH2OmhiNcWMugpchi4uCSm8IRARB9BAPBtR1VXBkmB57teN8MdO7aIb1B+QP5JBTQjwE8eofyCW3HMSDoD9APCrk4iE1sxNETdYaprvqdb2qehQPHBkpxk5atNa29kePO550dj3tftaTe967FPnV0BZl23f8cIVbkXCkJ8pKKkesBKGwXO6IZb41nbwv10QYSd9bVDuBWHOtDU9WpG0pgszcW8O11CavxHbdVHphQjdc7m/HU3JktrT4rj7493naVPWhYf2fVtANw8zl2ShLTb/vFDMnj8zm/NwvGFiHDxtVuBDwoMh3YCGibxVFMASErSHGe0IshBTJNEegjk7iVwkXlGURukXnBkWrGepRnNSN0go2dXLoD4mpY4Ads+/skv1k++ycXf+3VpzWSObZoZs3uCIwez72lX4/yHLpVti8YTVhcJrNp1gRwimjuT6FCsZSXZJ0BimSKLYbvWofPl2WxhcG4gL7yi5I6xd2wn6QWq92Ze/Ni4XPTSdxk+600OLd9d13ll6NFsmff52fnMpW24F+vMQg7e8NJjGDOZSp/gGOcIozbVf7pu1rB41UrSXjvMAt0w7/AH7MsO0= AAAC/HichVFNSxxBEH2OmhiNcWMugpchi4uCSm8IRARB9BAPBtR1VXBkmB57teN8MdO7aIb1B+QP5JBTQjwE8eofyCW3HMSDoD9APCrk4iE1sxNETdYaprvqdb2qehQPHBkpxk5atNa29kePO550dj3tftaTe967FPnV0BZl23f8cIVbkXCkJ8pKKkesBKGwXO6IZb41nbwv10QYSd9bVDuBWHOtDU9WpG0pgszcW8O11CavxHbdVHphQjdc7m/HU3JktrT4rj7493naVPWhYf2fVtANw8zl2ShLTb/vFDMnj8zm/NwvGFiHDxtVuBDwoMh3YCGibxVFMASErSHGe0IshBTJNEegjk7iVwkXlGURukXnBkWrGepRnNSN0go2dXLoD4mpY4Ads+/skv1k++ycXf+3VpzWSObZoZs3uCIwez72lX4/yHLpVti8YTVhcJrNp1gRwimjuT6FCsZSXZJ0BimSKLYbvWofPl2WxhcG4gL7yi5I6xd2wn6QWq92Ze/Ni4XPTSdxk+600OLd9d13ll6NFsmff52fnMpW24F+vMQg7e8NJjGDOZSp/gGOcIozbVf7pu1rB41UrSXjvMAt0w7/AH7MsO0= AAAC/HichVFNSxxBEH2OmhiNcWMugpchi4uCSm8IRARB9BAPBtR1VXBkmB57teN8MdO7aIb1B+QP5JBTQjwE8eofyCW3HMSDoD9APCrk4iE1sxNETdYaprvqdb2qehQPHBkpxk5atNa29kePO550dj3tftaTe967FPnV0BZl23f8cIVbkXCkJ8pKKkesBKGwXO6IZb41nbwv10QYSd9bVDuBWHOtDU9WpG0pgszcW8O11CavxHbdVHphQjdc7m/HU3JktrT4rj7493naVPWhYf2fVtANw8zl2ShLTb/vFDMnj8zm/NwvGFiHDxtVuBDwoMh3YCGibxVFMASErSHGe0IshBTJNEegjk7iVwkXlGURukXnBkWrGepRnNSN0go2dXLoD4mpY4Ads+/skv1k++ycXf+3VpzWSObZoZs3uCIwez72lX4/yHLpVti8YTVhcJrNp1gRwimjuT6FCsZSXZJ0BimSKLYbvWofPl2WxhcG4gL7yi5I6xd2wn6QWq92Ze/Ni4XPTSdxk+600OLd9d13ll6NFsmff52fnMpW24F+vMQg7e8NJjGDOZSp/gGOcIozbVf7pu1rB41UrSXjvMAt0w7/AH7MsO0= - - 12 ht = ~ ht ~ ht ht = ~ h1 ~ hN AAAECHiclVI9axRBGH43qzGumlwSC8Fm8chhFWYlEBWEYBoryYeXBLLHsTvO5dbsF7tza+Jy+QH5AxZWChZiYye22vgHLNLbiGUEGwufmVuJMeSCs+zM+z7v5zPv+GkY5JKxA2PEPHd+9MLYRevS5SvjE7XJqbU86WVcNHkSJtmG7+UiDGLRlIEMxUaaCS/yQ7Huby8q+3ohsjxI4kdyNxWtyNuKg07APQmoXdtzI092/U7Z7bel3bhnu4Xg5RGoUDdJw15uu1wUxy0N23WtMxM4pyZ42GjX6myW6WWfFJxKqFO1lpJJ4w659JgS4tSjiATFJCGH5FGOb5McYpQCa1FJT4B4lEELtI+gPlmI7wEX8PKAbmPfgrZZoTF0lTfXGTgqhfgzRNo0w76wN+yQfWZv2Tf269Rcpc6h+tnF6Q9iRdqe2L+2+vPMqAinpO5R1JAIH70l0CUQHx7D+Unq0G3NKwDPVCOKMR/UKp49P1y9uzJTNtgr9h1cX7ID9gls4+IHf70sVl4M7SQaUr3EnkFOwUvNYtCp8o5hfarvOdLMY0y3BM6BF/BSE21pJAPS0VPk6Fsx39G4YpCjqrL/uTuFF9rz/yPrqNj/67PwSp1/3+RJYe3WrAN5ea6+cL96r2N0nW7QTeSbpwV6QEvURAdfDcuYNq6a++Y78735YeA6YlQx03RsmR9/A9iE9a0= AAAECHiclVI9axRBGH43qzGumlwSC8Fm8chhFWYlEBWEYBoryYeXBLLHsTvO5dbsF7tza+Jy+QH5AxZWChZiYye22vgHLNLbiGUEGwufmVuJMeSCs+zM+z7v5zPv+GkY5JKxA2PEPHd+9MLYRevS5SvjE7XJqbU86WVcNHkSJtmG7+UiDGLRlIEMxUaaCS/yQ7Huby8q+3ohsjxI4kdyNxWtyNuKg07APQmoXdtzI092/U7Z7bel3bhnu4Xg5RGoUDdJw15uu1wUxy0N23WtMxM4pyZ42GjX6myW6WWfFJxKqFO1lpJJ4w659JgS4tSjiATFJCGH5FGOb5McYpQCa1FJT4B4lEELtI+gPlmI7wEX8PKAbmPfgrZZoTF0lTfXGTgqhfgzRNo0w76wN+yQfWZv2Tf269Rcpc6h+tnF6Q9iRdqe2L+2+vPMqAinpO5R1JAIH70l0CUQHx7D+Unq0G3NKwDPVCOKMR/UKp49P1y9uzJTNtgr9h1cX7ID9gls4+IHf70sVl4M7SQaUr3EnkFOwUvNYtCp8o5hfarvOdLMY0y3BM6BF/BSE21pJAPS0VPk6Fsx39G4YpCjqrL/uTuFF9rz/yPrqNj/67PwSp1/3+RJYe3WrAN5ea6+cL96r2N0nW7QTeSbpwV6QEvURAdfDcuYNq6a++Y78735YeA6YlQx03RsmR9/A9iE9a0= AAAECHiclVI9axRBGH43qzGumlwSC8Fm8chhFWYlEBWEYBoryYeXBLLHsTvO5dbsF7tza+Jy+QH5AxZWChZiYye22vgHLNLbiGUEGwufmVuJMeSCs+zM+z7v5zPv+GkY5JKxA2PEPHd+9MLYRevS5SvjE7XJqbU86WVcNHkSJtmG7+UiDGLRlIEMxUaaCS/yQ7Huby8q+3ohsjxI4kdyNxWtyNuKg07APQmoXdtzI092/U7Z7bel3bhnu4Xg5RGoUDdJw15uu1wUxy0N23WtMxM4pyZ42GjX6myW6WWfFJxKqFO1lpJJ4w659JgS4tSjiATFJCGH5FGOb5McYpQCa1FJT4B4lEELtI+gPlmI7wEX8PKAbmPfgrZZoTF0lTfXGTgqhfgzRNo0w76wN+yQfWZv2Tf269Rcpc6h+tnF6Q9iRdqe2L+2+vPMqAinpO5R1JAIH70l0CUQHx7D+Unq0G3NKwDPVCOKMR/UKp49P1y9uzJTNtgr9h1cX7ID9gls4+IHf70sVl4M7SQaUr3EnkFOwUvNYtCp8o5hfarvOdLMY0y3BM6BF/BSE21pJAPS0VPk6Fsx39G4YpCjqrL/uTuFF9rz/yPrqNj/67PwSp1/3+RJYe3WrAN5ea6+cL96r2N0nW7QTeSbpwV6QEvURAdfDcuYNq6a++Y78735YeA6YlQx03RsmR9/A9iE9a0= AAAECHiclVI9axRBGH43qzGumlwSC8Fm8chhFWYlEBWEYBoryYeXBLLHsTvO5dbsF7tza+Jy+QH5AxZWChZiYye22vgHLNLbiGUEGwufmVuJMeSCs+zM+z7v5zPv+GkY5JKxA2PEPHd+9MLYRevS5SvjE7XJqbU86WVcNHkSJtmG7+UiDGLRlIEMxUaaCS/yQ7Huby8q+3ohsjxI4kdyNxWtyNuKg07APQmoXdtzI092/U7Z7bel3bhnu4Xg5RGoUDdJw15uu1wUxy0N23WtMxM4pyZ42GjX6myW6WWfFJxKqFO1lpJJ4w659JgS4tSjiATFJCGH5FGOb5McYpQCa1FJT4B4lEELtI+gPlmI7wEX8PKAbmPfgrZZoTF0lTfXGTgqhfgzRNo0w76wN+yQfWZv2Tf269Rcpc6h+tnF6Q9iRdqe2L+2+vPMqAinpO5R1JAIH70l0CUQHx7D+Unq0G3NKwDPVCOKMR/UKp49P1y9uzJTNtgr9h1cX7ID9gls4+IHf70sVl4M7SQaUr3EnkFOwUvNYtCp8o5hfarvOdLMY0y3BM6BF/BSE21pJAPS0VPk6Fsx39G4YpCjqrL/uTuFF9rz/yPrqNj/67PwSp1/3+RJYe3WrAN5ea6+cL96r2N0nW7QTeSbpwV6QEvURAdfDcuYNq6a++Y78735YeA6YlQx03RsmR9/A9iE9a0=

Slide 13

Slide 13 text

13 0 0 2 0- • ... TagLM [3]: 91.9, ELMo [4]: 92.2, BERT: 92.8... Lample10 6 ( )

Slide 14

Slide 14 text

14

Slide 15

Slide 15 text

15

Slide 16

Slide 16 text

16 https://www.preferred-networks.jp/ja/pfn-logo https://twitter.com/chakki_works https://github.com/RaRe-Technologies/gensim • / / • E : / - / • -

Slide 17

Slide 17 text

Trainer Updater Iterator Extension Optimizer Dataset Model see also: https://qiita.com/mitmul/items/1e35fba085eb07a92560#trainer 17

Slide 18

Slide 18 text

Trainer Updater Iterator Extension Optimizer Dataset Transformer Model 18

Slide 19

Slide 19 text

• • SequenceLabelingDataset, SequenceLabelingIterator: # %H' • NamedEntityEvaluator: BIO(ES)NEC- -> seqevalB+ • LearningRateDecay: Ma+, 2016,8G"$& • DatasetTransformer: ?0 <-> ?0id =/C-2 • tag2idx idx2tagAF (!) • convertericonverter35;@ • &&C<@ )I • transformer: https://github.com/himkt/pyner/blob/master/pyner/named_entity/dataset.py#L23 • dataset: https://github.com/himkt/pyner/blob/master/pyner/named_entity/dataset.py#L58 • XavierInitializer: Lample+, 2016167>4C<9.*DE • ChainerGlorot Uniform(: ... 19

Slide 20

Slide 20 text

• -03 p • S p • t : k e: 100 / 50 • t : [ BiLSTM nd: 2x100 / 2x25 • ivL ] : 5.0 • ratio: 0.5 • N s ma • o G k e 05 -2 , r C 20

Slide 21

Slide 21 text

• • : • 21

Slide 22

Slide 22 text

22 Setting Fscore Improvement Word only 69.76 – +char 82.62 +12.86 +pretrain 87.39 +17.63 +pretrain +char 87.76 +18.00 +pretrain +dropout 89.65 +19.89 +pretrain +char +dropout 90.81 +21.05 AAAEuHichZG/b9NQEMevxkAxP5rCgsRiEbVCimo9u6qTdKpAQrC1DW0q1VWwnZf0UdvPen6OFKL8AyyMgJhAYkD8GSxsTAz9ExBjkVgYODsJKQ1Jz7J973yf7935vDhgiSTkeE65oF68dHn+inb12vUbC4XFm7sJT4VPd3wecLHnuQkNWER3JJMB3YsFdUMvoHXv6EH2vd6hImE8eiK7MT0I3XbEWsx3JYYai4rieLTNop50vTRwRb8XCNHXHMljkQZU06dbjUrJora+rOsPE58Lit7jMBa8Q0MaSd1xNCdkzXNk6lw0dR4F3UzHrhplO3NyW1nJNGZ0gFbyD12RERXLsK0RWjIto2LPpkv4n6RwWZTTZWO1OqbLhr06jR5z49rlU22XzIpByH/pU2hT8JinMqerhr02pqtGpXqG7jlea6LuSKKfoXlGlRgVsz9SGkCWaZC1fr4Lj0vJw3wdDo2afzeuNQpFgi1npk865tApwtA2eeErONAEDj6kEAKFCCT6AbiQ4LUPJhCIMXYAPXiGERcEnlieQ6EPGvIpxilmuRg9wmcbT/vDaITnTDfJFXysFOAtkNRhiXwjH8kJ+UI+ke/k91StXq6R9dPFtzdgadxYeHG79utcKsS3hMMxNYPwsDeOZ4kRDzNmzyehBZV8LoZzxnkkm9gf1Oo8f3VSW99e6i2T9+QHzvqOHJPPOG3U+el/2KLbb2d2EmbVcaHm2fVNOruWYaK/ZRU37g9XOw934C7cw/2VYQMewSbsgK8w5aXyWnmjrqtP1bbKBqnK3JC5Bf+YKv4A6kUTIQ== AAAEuHichZG/b9NQEMevxkAxP5rCgsRiEbVCimo9u6qTdKpAQrC1DW0q1VWwnZf0UdvPen6OFKL8AyyMgJhAYkD8GSxsTAz9ExBjkVgYODsJKQ1Jz7J973yf7935vDhgiSTkeE65oF68dHn+inb12vUbC4XFm7sJT4VPd3wecLHnuQkNWER3JJMB3YsFdUMvoHXv6EH2vd6hImE8eiK7MT0I3XbEWsx3JYYai4rieLTNop50vTRwRb8XCNHXHMljkQZU06dbjUrJora+rOsPE58Lit7jMBa8Q0MaSd1xNCdkzXNk6lw0dR4F3UzHrhplO3NyW1nJNGZ0gFbyD12RERXLsK0RWjIto2LPpkv4n6RwWZTTZWO1OqbLhr06jR5z49rlU22XzIpByH/pU2hT8JinMqerhr02pqtGpXqG7jlea6LuSKKfoXlGlRgVsz9SGkCWaZC1fr4Lj0vJw3wdDo2afzeuNQpFgi1npk865tApwtA2eeErONAEDj6kEAKFCCT6AbiQ4LUPJhCIMXYAPXiGERcEnlieQ6EPGvIpxilmuRg9wmcbT/vDaITnTDfJFXysFOAtkNRhiXwjH8kJ+UI+ke/k91StXq6R9dPFtzdgadxYeHG79utcKsS3hMMxNYPwsDeOZ4kRDzNmzyehBZV8LoZzxnkkm9gf1Oo8f3VSW99e6i2T9+QHzvqOHJPPOG3U+el/2KLbb2d2EmbVcaHm2fVNOruWYaK/ZRU37g9XOw934C7cw/2VYQMewSbsgK8w5aXyWnmjrqtP1bbKBqnK3JC5Bf+YKv4A6kUTIQ== AAAEuHichZG/b9NQEMevxkAxP5rCgsRiEbVCimo9u6qTdKpAQrC1DW0q1VWwnZf0UdvPen6OFKL8AyyMgJhAYkD8GSxsTAz9ExBjkVgYODsJKQ1Jz7J973yf7935vDhgiSTkeE65oF68dHn+inb12vUbC4XFm7sJT4VPd3wecLHnuQkNWER3JJMB3YsFdUMvoHXv6EH2vd6hImE8eiK7MT0I3XbEWsx3JYYai4rieLTNop50vTRwRb8XCNHXHMljkQZU06dbjUrJora+rOsPE58Lit7jMBa8Q0MaSd1xNCdkzXNk6lw0dR4F3UzHrhplO3NyW1nJNGZ0gFbyD12RERXLsK0RWjIto2LPpkv4n6RwWZTTZWO1OqbLhr06jR5z49rlU22XzIpByH/pU2hT8JinMqerhr02pqtGpXqG7jlea6LuSKKfoXlGlRgVsz9SGkCWaZC1fr4Lj0vJw3wdDo2afzeuNQpFgi1npk865tApwtA2eeErONAEDj6kEAKFCCT6AbiQ4LUPJhCIMXYAPXiGERcEnlieQ6EPGvIpxilmuRg9wmcbT/vDaITnTDfJFXysFOAtkNRhiXwjH8kJ+UI+ke/k91StXq6R9dPFtzdgadxYeHG79utcKsS3hMMxNYPwsDeOZ4kRDzNmzyehBZV8LoZzxnkkm9gf1Oo8f3VSW99e6i2T9+QHzvqOHJPPOG3U+el/2KLbb2d2EmbVcaHm2fVNOruWYaK/ZRU37g9XOw934C7cw/2VYQMewSbsgK8w5aXyWnmjrqtP1bbKBqnK3JC5Bf+YKv4A6kUTIQ== AAAEuHichZG/b9NQEMevxkAxP5rCgsRiEbVCimo9u6qTdKpAQrC1DW0q1VWwnZf0UdvPen6OFKL8AyyMgJhAYkD8GSxsTAz9ExBjkVgYODsJKQ1Jz7J973yf7935vDhgiSTkeE65oF68dHn+inb12vUbC4XFm7sJT4VPd3wecLHnuQkNWER3JJMB3YsFdUMvoHXv6EH2vd6hImE8eiK7MT0I3XbEWsx3JYYai4rieLTNop50vTRwRb8XCNHXHMljkQZU06dbjUrJora+rOsPE58Lit7jMBa8Q0MaSd1xNCdkzXNk6lw0dR4F3UzHrhplO3NyW1nJNGZ0gFbyD12RERXLsK0RWjIto2LPpkv4n6RwWZTTZWO1OqbLhr06jR5z49rlU22XzIpByH/pU2hT8JinMqerhr02pqtGpXqG7jlea6LuSKKfoXlGlRgVsz9SGkCWaZC1fr4Lj0vJw3wdDo2afzeuNQpFgi1npk865tApwtA2eeErONAEDj6kEAKFCCT6AbiQ4LUPJhCIMXYAPXiGERcEnlieQ6EPGvIpxilmuRg9wmcbT/vDaITnTDfJFXysFOAtkNRhiXwjH8kJ+UI+ke/k91StXq6R9dPFtzdgadxYeHG79utcKsS3hMMxNYPwsDeOZ4kRDzNmzyehBZV8LoZzxnkkm9gf1Oo8f3VSW99e6i2T9+QHzvqOHJPPOG3U+el/2KLbb2d2EmbVcaHm2fVNOruWYaK/ZRU37g9XOw934C7cw/2VYQMewSbsgK8w5aXyWnmjrqtP1bbKBqnK3JC5Bf+YKv4A6kUTIQ==

Slide 23

Slide 23 text

23 Setting Fscore Improvement Word only 69.76 – +char 82.62 +12.86 +pretrain 87.39 +17.63 +pretrain +char 87.76 +18.00 +pretrain +dropout 89.65 +19.89 +pretrain +char +dropout 90.81 +21.05 AAAEuHichZG/b9NQEMevxkAxP5rCgsRiEbVCimo9u6qTdKpAQrC1DW0q1VWwnZf0UdvPen6OFKL8AyyMgJhAYkD8GSxsTAz9ExBjkVgYODsJKQ1Jz7J973yf7935vDhgiSTkeE65oF68dHn+inb12vUbC4XFm7sJT4VPd3wecLHnuQkNWER3JJMB3YsFdUMvoHXv6EH2vd6hImE8eiK7MT0I3XbEWsx3JYYai4rieLTNop50vTRwRb8XCNHXHMljkQZU06dbjUrJora+rOsPE58Lit7jMBa8Q0MaSd1xNCdkzXNk6lw0dR4F3UzHrhplO3NyW1nJNGZ0gFbyD12RERXLsK0RWjIto2LPpkv4n6RwWZTTZWO1OqbLhr06jR5z49rlU22XzIpByH/pU2hT8JinMqerhr02pqtGpXqG7jlea6LuSKKfoXlGlRgVsz9SGkCWaZC1fr4Lj0vJw3wdDo2afzeuNQpFgi1npk865tApwtA2eeErONAEDj6kEAKFCCT6AbiQ4LUPJhCIMXYAPXiGERcEnlieQ6EPGvIpxilmuRg9wmcbT/vDaITnTDfJFXysFOAtkNRhiXwjH8kJ+UI+ke/k91StXq6R9dPFtzdgadxYeHG79utcKsS3hMMxNYPwsDeOZ4kRDzNmzyehBZV8LoZzxnkkm9gf1Oo8f3VSW99e6i2T9+QHzvqOHJPPOG3U+el/2KLbb2d2EmbVcaHm2fVNOruWYaK/ZRU37g9XOw934C7cw/2VYQMewSbsgK8w5aXyWnmjrqtP1bbKBqnK3JC5Bf+YKv4A6kUTIQ== AAAEuHichZG/b9NQEMevxkAxP5rCgsRiEbVCimo9u6qTdKpAQrC1DW0q1VWwnZf0UdvPen6OFKL8AyyMgJhAYkD8GSxsTAz9ExBjkVgYODsJKQ1Jz7J973yf7935vDhgiSTkeE65oF68dHn+inb12vUbC4XFm7sJT4VPd3wecLHnuQkNWER3JJMB3YsFdUMvoHXv6EH2vd6hImE8eiK7MT0I3XbEWsx3JYYai4rieLTNop50vTRwRb8XCNHXHMljkQZU06dbjUrJora+rOsPE58Lit7jMBa8Q0MaSd1xNCdkzXNk6lw0dR4F3UzHrhplO3NyW1nJNGZ0gFbyD12RERXLsK0RWjIto2LPpkv4n6RwWZTTZWO1OqbLhr06jR5z49rlU22XzIpByH/pU2hT8JinMqerhr02pqtGpXqG7jlea6LuSKKfoXlGlRgVsz9SGkCWaZC1fr4Lj0vJw3wdDo2afzeuNQpFgi1npk865tApwtA2eeErONAEDj6kEAKFCCT6AbiQ4LUPJhCIMXYAPXiGERcEnlieQ6EPGvIpxilmuRg9wmcbT/vDaITnTDfJFXysFOAtkNRhiXwjH8kJ+UI+ke/k91StXq6R9dPFtzdgadxYeHG79utcKsS3hMMxNYPwsDeOZ4kRDzNmzyehBZV8LoZzxnkkm9gf1Oo8f3VSW99e6i2T9+QHzvqOHJPPOG3U+el/2KLbb2d2EmbVcaHm2fVNOruWYaK/ZRU37g9XOw934C7cw/2VYQMewSbsgK8w5aXyWnmjrqtP1bbKBqnK3JC5Bf+YKv4A6kUTIQ== AAAEuHichZG/b9NQEMevxkAxP5rCgsRiEbVCimo9u6qTdKpAQrC1DW0q1VWwnZf0UdvPen6OFKL8AyyMgJhAYkD8GSxsTAz9ExBjkVgYODsJKQ1Jz7J973yf7935vDhgiSTkeE65oF68dHn+inb12vUbC4XFm7sJT4VPd3wecLHnuQkNWER3JJMB3YsFdUMvoHXv6EH2vd6hImE8eiK7MT0I3XbEWsx3JYYai4rieLTNop50vTRwRb8XCNHXHMljkQZU06dbjUrJora+rOsPE58Lit7jMBa8Q0MaSd1xNCdkzXNk6lw0dR4F3UzHrhplO3NyW1nJNGZ0gFbyD12RERXLsK0RWjIto2LPpkv4n6RwWZTTZWO1OqbLhr06jR5z49rlU22XzIpByH/pU2hT8JinMqerhr02pqtGpXqG7jlea6LuSKKfoXlGlRgVsz9SGkCWaZC1fr4Lj0vJw3wdDo2afzeuNQpFgi1npk865tApwtA2eeErONAEDj6kEAKFCCT6AbiQ4LUPJhCIMXYAPXiGERcEnlieQ6EPGvIpxilmuRg9wmcbT/vDaITnTDfJFXysFOAtkNRhiXwjH8kJ+UI+ke/k91StXq6R9dPFtzdgadxYeHG79utcKsS3hMMxNYPwsDeOZ4kRDzNmzyehBZV8LoZzxnkkm9gf1Oo8f3VSW99e6i2T9+QHzvqOHJPPOG3U+el/2KLbb2d2EmbVcaHm2fVNOruWYaK/ZRU37g9XOw934C7cw/2VYQMewSbsgK8w5aXyWnmjrqtP1bbKBqnK3JC5Bf+YKv4A6kUTIQ== AAAEuHichZG/b9NQEMevxkAxP5rCgsRiEbVCimo9u6qTdKpAQrC1DW0q1VWwnZf0UdvPen6OFKL8AyyMgJhAYkD8GSxsTAz9ExBjkVgYODsJKQ1Jz7J973yf7935vDhgiSTkeE65oF68dHn+inb12vUbC4XFm7sJT4VPd3wecLHnuQkNWER3JJMB3YsFdUMvoHXv6EH2vd6hImE8eiK7MT0I3XbEWsx3JYYai4rieLTNop50vTRwRb8XCNHXHMljkQZU06dbjUrJora+rOsPE58Lit7jMBa8Q0MaSd1xNCdkzXNk6lw0dR4F3UzHrhplO3NyW1nJNGZ0gFbyD12RERXLsK0RWjIto2LPpkv4n6RwWZTTZWO1OqbLhr06jR5z49rlU22XzIpByH/pU2hT8JinMqerhr02pqtGpXqG7jlea6LuSKKfoXlGlRgVsz9SGkCWaZC1fr4Lj0vJw3wdDo2afzeuNQpFgi1npk865tApwtA2eeErONAEDj6kEAKFCCT6AbiQ4LUPJhCIMXYAPXiGERcEnlieQ6EPGvIpxilmuRg9wmcbT/vDaITnTDfJFXysFOAtkNRhiXwjH8kJ+UI+ke/k91StXq6R9dPFtzdgadxYeHG79utcKsS3hMMxNYPwsDeOZ4kRDzNmzyehBZV8LoZzxnkkm9gf1Oo8f3VSW99e6i2T9+QHzvqOHJPPOG3U+el/2KLbb2d2EmbVcaHm2fVNOruWYaK/ZRU37g9XOw934C7cw/2VYQMewSbsgK8w5aXyWnmjrqtP1bbKBqnK3JC5Bf+YKv4A6kUTIQ==

Slide 24

Slide 24 text

24 Setting Fscore Improvement Word only 69.76 – +char 82.62 +12.86 +pretrain 87.39 +17.63 +pretrain +char 87.76 +18.00 +pretrain +dropout 89.65 +19.89 +pretrain +char +dropout 90.81 +21.05 AAAEuHichZG/b9NQEMevxkAxP5rCgsRiEbVCimo9u6qTdKpAQrC1DW0q1VWwnZf0UdvPen6OFKL8AyyMgJhAYkD8GSxsTAz9ExBjkVgYODsJKQ1Jz7J973yf7935vDhgiSTkeE65oF68dHn+inb12vUbC4XFm7sJT4VPd3wecLHnuQkNWER3JJMB3YsFdUMvoHXv6EH2vd6hImE8eiK7MT0I3XbEWsx3JYYai4rieLTNop50vTRwRb8XCNHXHMljkQZU06dbjUrJora+rOsPE58Lit7jMBa8Q0MaSd1xNCdkzXNk6lw0dR4F3UzHrhplO3NyW1nJNGZ0gFbyD12RERXLsK0RWjIto2LPpkv4n6RwWZTTZWO1OqbLhr06jR5z49rlU22XzIpByH/pU2hT8JinMqerhr02pqtGpXqG7jlea6LuSKKfoXlGlRgVsz9SGkCWaZC1fr4Lj0vJw3wdDo2afzeuNQpFgi1npk865tApwtA2eeErONAEDj6kEAKFCCT6AbiQ4LUPJhCIMXYAPXiGERcEnlieQ6EPGvIpxilmuRg9wmcbT/vDaITnTDfJFXysFOAtkNRhiXwjH8kJ+UI+ke/k91StXq6R9dPFtzdgadxYeHG79utcKsS3hMMxNYPwsDeOZ4kRDzNmzyehBZV8LoZzxnkkm9gf1Oo8f3VSW99e6i2T9+QHzvqOHJPPOG3U+el/2KLbb2d2EmbVcaHm2fVNOruWYaK/ZRU37g9XOw934C7cw/2VYQMewSbsgK8w5aXyWnmjrqtP1bbKBqnK3JC5Bf+YKv4A6kUTIQ== AAAEuHichZG/b9NQEMevxkAxP5rCgsRiEbVCimo9u6qTdKpAQrC1DW0q1VWwnZf0UdvPen6OFKL8AyyMgJhAYkD8GSxsTAz9ExBjkVgYODsJKQ1Jz7J973yf7935vDhgiSTkeE65oF68dHn+inb12vUbC4XFm7sJT4VPd3wecLHnuQkNWER3JJMB3YsFdUMvoHXv6EH2vd6hImE8eiK7MT0I3XbEWsx3JYYai4rieLTNop50vTRwRb8XCNHXHMljkQZU06dbjUrJora+rOsPE58Lit7jMBa8Q0MaSd1xNCdkzXNk6lw0dR4F3UzHrhplO3NyW1nJNGZ0gFbyD12RERXLsK0RWjIto2LPpkv4n6RwWZTTZWO1OqbLhr06jR5z49rlU22XzIpByH/pU2hT8JinMqerhr02pqtGpXqG7jlea6LuSKKfoXlGlRgVsz9SGkCWaZC1fr4Lj0vJw3wdDo2afzeuNQpFgi1npk865tApwtA2eeErONAEDj6kEAKFCCT6AbiQ4LUPJhCIMXYAPXiGERcEnlieQ6EPGvIpxilmuRg9wmcbT/vDaITnTDfJFXysFOAtkNRhiXwjH8kJ+UI+ke/k91StXq6R9dPFtzdgadxYeHG79utcKsS3hMMxNYPwsDeOZ4kRDzNmzyehBZV8LoZzxnkkm9gf1Oo8f3VSW99e6i2T9+QHzvqOHJPPOG3U+el/2KLbb2d2EmbVcaHm2fVNOruWYaK/ZRU37g9XOw934C7cw/2VYQMewSbsgK8w5aXyWnmjrqtP1bbKBqnK3JC5Bf+YKv4A6kUTIQ== AAAEuHichZG/b9NQEMevxkAxP5rCgsRiEbVCimo9u6qTdKpAQrC1DW0q1VWwnZf0UdvPen6OFKL8AyyMgJhAYkD8GSxsTAz9ExBjkVgYODsJKQ1Jz7J973yf7935vDhgiSTkeE65oF68dHn+inb12vUbC4XFm7sJT4VPd3wecLHnuQkNWER3JJMB3YsFdUMvoHXv6EH2vd6hImE8eiK7MT0I3XbEWsx3JYYai4rieLTNop50vTRwRb8XCNHXHMljkQZU06dbjUrJora+rOsPE58Lit7jMBa8Q0MaSd1xNCdkzXNk6lw0dR4F3UzHrhplO3NyW1nJNGZ0gFbyD12RERXLsK0RWjIto2LPpkv4n6RwWZTTZWO1OqbLhr06jR5z49rlU22XzIpByH/pU2hT8JinMqerhr02pqtGpXqG7jlea6LuSKKfoXlGlRgVsz9SGkCWaZC1fr4Lj0vJw3wdDo2afzeuNQpFgi1npk865tApwtA2eeErONAEDj6kEAKFCCT6AbiQ4LUPJhCIMXYAPXiGERcEnlieQ6EPGvIpxilmuRg9wmcbT/vDaITnTDfJFXysFOAtkNRhiXwjH8kJ+UI+ke/k91StXq6R9dPFtzdgadxYeHG79utcKsS3hMMxNYPwsDeOZ4kRDzNmzyehBZV8LoZzxnkkm9gf1Oo8f3VSW99e6i2T9+QHzvqOHJPPOG3U+el/2KLbb2d2EmbVcaHm2fVNOruWYaK/ZRU37g9XOw934C7cw/2VYQMewSbsgK8w5aXyWnmjrqtP1bbKBqnK3JC5Bf+YKv4A6kUTIQ== AAAEuHichZG/b9NQEMevxkAxP5rCgsRiEbVCimo9u6qTdKpAQrC1DW0q1VWwnZf0UdvPen6OFKL8AyyMgJhAYkD8GSxsTAz9ExBjkVgYODsJKQ1Jz7J973yf7935vDhgiSTkeE65oF68dHn+inb12vUbC4XFm7sJT4VPd3wecLHnuQkNWER3JJMB3YsFdUMvoHXv6EH2vd6hImE8eiK7MT0I3XbEWsx3JYYai4rieLTNop50vTRwRb8XCNHXHMljkQZU06dbjUrJora+rOsPE58Lit7jMBa8Q0MaSd1xNCdkzXNk6lw0dR4F3UzHrhplO3NyW1nJNGZ0gFbyD12RERXLsK0RWjIto2LPpkv4n6RwWZTTZWO1OqbLhr06jR5z49rlU22XzIpByH/pU2hT8JinMqerhr02pqtGpXqG7jlea6LuSKKfoXlGlRgVsz9SGkCWaZC1fr4Lj0vJw3wdDo2afzeuNQpFgi1npk865tApwtA2eeErONAEDj6kEAKFCCT6AbiQ4LUPJhCIMXYAPXiGERcEnlieQ6EPGvIpxilmuRg9wmcbT/vDaITnTDfJFXysFOAtkNRhiXwjH8kJ+UI+ke/k91StXq6R9dPFtzdgadxYeHG79utcKsS3hMMxNYPwsDeOZ4kRDzNmzyehBZV8LoZzxnkkm9gf1Oo8f3VSW99e6i2T9+QHzvqOHJPPOG3U+el/2KLbb2d2EmbVcaHm2fVNOruWYaK/ZRU37g9XOw934C7cw/2VYQMewSbsgK8w5aXyWnmjrqtP1bbKBqnK3JC5Bf+YKv4A6kUTIQ==

Slide 25

Slide 25 text

25 Setting Fscore Improvement Word only 69.76 – +char 82.62 +12.86 +pretrain 87.39 +17.63 +pretrain +char 87.76 +18.00 +pretrain +dropout 89.65 +19.89 +pretrain +char +dropout 90.81 +21.05 AAAEuHichZG/b9NQEMevxkAxP5rCgsRiEbVCimo9u6qTdKpAQrC1DW0q1VWwnZf0UdvPen6OFKL8AyyMgJhAYkD8GSxsTAz9ExBjkVgYODsJKQ1Jz7J973yf7935vDhgiSTkeE65oF68dHn+inb12vUbC4XFm7sJT4VPd3wecLHnuQkNWER3JJMB3YsFdUMvoHXv6EH2vd6hImE8eiK7MT0I3XbEWsx3JYYai4rieLTNop50vTRwRb8XCNHXHMljkQZU06dbjUrJora+rOsPE58Lit7jMBa8Q0MaSd1xNCdkzXNk6lw0dR4F3UzHrhplO3NyW1nJNGZ0gFbyD12RERXLsK0RWjIto2LPpkv4n6RwWZTTZWO1OqbLhr06jR5z49rlU22XzIpByH/pU2hT8JinMqerhr02pqtGpXqG7jlea6LuSKKfoXlGlRgVsz9SGkCWaZC1fr4Lj0vJw3wdDo2afzeuNQpFgi1npk865tApwtA2eeErONAEDj6kEAKFCCT6AbiQ4LUPJhCIMXYAPXiGERcEnlieQ6EPGvIpxilmuRg9wmcbT/vDaITnTDfJFXysFOAtkNRhiXwjH8kJ+UI+ke/k91StXq6R9dPFtzdgadxYeHG79utcKsS3hMMxNYPwsDeOZ4kRDzNmzyehBZV8LoZzxnkkm9gf1Oo8f3VSW99e6i2T9+QHzvqOHJPPOG3U+el/2KLbb2d2EmbVcaHm2fVNOruWYaK/ZRU37g9XOw934C7cw/2VYQMewSbsgK8w5aXyWnmjrqtP1bbKBqnK3JC5Bf+YKv4A6kUTIQ== AAAEuHichZG/b9NQEMevxkAxP5rCgsRiEbVCimo9u6qTdKpAQrC1DW0q1VWwnZf0UdvPen6OFKL8AyyMgJhAYkD8GSxsTAz9ExBjkVgYODsJKQ1Jz7J973yf7935vDhgiSTkeE65oF68dHn+inb12vUbC4XFm7sJT4VPd3wecLHnuQkNWER3JJMB3YsFdUMvoHXv6EH2vd6hImE8eiK7MT0I3XbEWsx3JYYai4rieLTNop50vTRwRb8XCNHXHMljkQZU06dbjUrJora+rOsPE58Lit7jMBa8Q0MaSd1xNCdkzXNk6lw0dR4F3UzHrhplO3NyW1nJNGZ0gFbyD12RERXLsK0RWjIto2LPpkv4n6RwWZTTZWO1OqbLhr06jR5z49rlU22XzIpByH/pU2hT8JinMqerhr02pqtGpXqG7jlea6LuSKKfoXlGlRgVsz9SGkCWaZC1fr4Lj0vJw3wdDo2afzeuNQpFgi1npk865tApwtA2eeErONAEDj6kEAKFCCT6AbiQ4LUPJhCIMXYAPXiGERcEnlieQ6EPGvIpxilmuRg9wmcbT/vDaITnTDfJFXysFOAtkNRhiXwjH8kJ+UI+ke/k91StXq6R9dPFtzdgadxYeHG79utcKsS3hMMxNYPwsDeOZ4kRDzNmzyehBZV8LoZzxnkkm9gf1Oo8f3VSW99e6i2T9+QHzvqOHJPPOG3U+el/2KLbb2d2EmbVcaHm2fVNOruWYaK/ZRU37g9XOw934C7cw/2VYQMewSbsgK8w5aXyWnmjrqtP1bbKBqnK3JC5Bf+YKv4A6kUTIQ== AAAEuHichZG/b9NQEMevxkAxP5rCgsRiEbVCimo9u6qTdKpAQrC1DW0q1VWwnZf0UdvPen6OFKL8AyyMgJhAYkD8GSxsTAz9ExBjkVgYODsJKQ1Jz7J973yf7935vDhgiSTkeE65oF68dHn+inb12vUbC4XFm7sJT4VPd3wecLHnuQkNWER3JJMB3YsFdUMvoHXv6EH2vd6hImE8eiK7MT0I3XbEWsx3JYYai4rieLTNop50vTRwRb8XCNHXHMljkQZU06dbjUrJora+rOsPE58Lit7jMBa8Q0MaSd1xNCdkzXNk6lw0dR4F3UzHrhplO3NyW1nJNGZ0gFbyD12RERXLsK0RWjIto2LPpkv4n6RwWZTTZWO1OqbLhr06jR5z49rlU22XzIpByH/pU2hT8JinMqerhr02pqtGpXqG7jlea6LuSKKfoXlGlRgVsz9SGkCWaZC1fr4Lj0vJw3wdDo2afzeuNQpFgi1npk865tApwtA2eeErONAEDj6kEAKFCCT6AbiQ4LUPJhCIMXYAPXiGERcEnlieQ6EPGvIpxilmuRg9wmcbT/vDaITnTDfJFXysFOAtkNRhiXwjH8kJ+UI+ke/k91StXq6R9dPFtzdgadxYeHG79utcKsS3hMMxNYPwsDeOZ4kRDzNmzyehBZV8LoZzxnkkm9gf1Oo8f3VSW99e6i2T9+QHzvqOHJPPOG3U+el/2KLbb2d2EmbVcaHm2fVNOruWYaK/ZRU37g9XOw934C7cw/2VYQMewSbsgK8w5aXyWnmjrqtP1bbKBqnK3JC5Bf+YKv4A6kUTIQ== AAAEuHichZG/b9NQEMevxkAxP5rCgsRiEbVCimo9u6qTdKpAQrC1DW0q1VWwnZf0UdvPen6OFKL8AyyMgJhAYkD8GSxsTAz9ExBjkVgYODsJKQ1Jz7J973yf7935vDhgiSTkeE65oF68dHn+inb12vUbC4XFm7sJT4VPd3wecLHnuQkNWER3JJMB3YsFdUMvoHXv6EH2vd6hImE8eiK7MT0I3XbEWsx3JYYai4rieLTNop50vTRwRb8XCNHXHMljkQZU06dbjUrJora+rOsPE58Lit7jMBa8Q0MaSd1xNCdkzXNk6lw0dR4F3UzHrhplO3NyW1nJNGZ0gFbyD12RERXLsK0RWjIto2LPpkv4n6RwWZTTZWO1OqbLhr06jR5z49rlU22XzIpByH/pU2hT8JinMqerhr02pqtGpXqG7jlea6LuSKKfoXlGlRgVsz9SGkCWaZC1fr4Lj0vJw3wdDo2afzeuNQpFgi1npk865tApwtA2eeErONAEDj6kEAKFCCT6AbiQ4LUPJhCIMXYAPXiGERcEnlieQ6EPGvIpxilmuRg9wmcbT/vDaITnTDfJFXysFOAtkNRhiXwjH8kJ+UI+ke/k91StXq6R9dPFtzdgadxYeHG79utcKsS3hMMxNYPwsDeOZ4kRDzNmzyehBZV8LoZzxnkkm9gf1Oo8f3VSW99e6i2T9+QHzvqOHJPPOG3U+el/2KLbb2d2EmbVcaHm2fVNOruWYaK/ZRU37g9XOw934C7cw/2VYQMewSbsgK8w5aXyWnmjrqtP1bbKBqnK3JC5Bf+YKv4A6kUTIQ==

Slide 26

Slide 26 text

26 Setting Fscore Improvement Word only 69.76 – +char 82.62 +12.86 +pretrain 87.39 +17.63 +pretrain +char 87.76 +18.00 +pretrain +dropout 89.65 +19.89 +pretrain +char +dropout 90.81 +21.05 AAAEuHichZG/b9NQEMevxkAxP5rCgsRiEbVCimo9u6qTdKpAQrC1DW0q1VWwnZf0UdvPen6OFKL8AyyMgJhAYkD8GSxsTAz9ExBjkVgYODsJKQ1Jz7J973yf7935vDhgiSTkeE65oF68dHn+inb12vUbC4XFm7sJT4VPd3wecLHnuQkNWER3JJMB3YsFdUMvoHXv6EH2vd6hImE8eiK7MT0I3XbEWsx3JYYai4rieLTNop50vTRwRb8XCNHXHMljkQZU06dbjUrJora+rOsPE58Lit7jMBa8Q0MaSd1xNCdkzXNk6lw0dR4F3UzHrhplO3NyW1nJNGZ0gFbyD12RERXLsK0RWjIto2LPpkv4n6RwWZTTZWO1OqbLhr06jR5z49rlU22XzIpByH/pU2hT8JinMqerhr02pqtGpXqG7jlea6LuSKKfoXlGlRgVsz9SGkCWaZC1fr4Lj0vJw3wdDo2afzeuNQpFgi1npk865tApwtA2eeErONAEDj6kEAKFCCT6AbiQ4LUPJhCIMXYAPXiGERcEnlieQ6EPGvIpxilmuRg9wmcbT/vDaITnTDfJFXysFOAtkNRhiXwjH8kJ+UI+ke/k91StXq6R9dPFtzdgadxYeHG79utcKsS3hMMxNYPwsDeOZ4kRDzNmzyehBZV8LoZzxnkkm9gf1Oo8f3VSW99e6i2T9+QHzvqOHJPPOG3U+el/2KLbb2d2EmbVcaHm2fVNOruWYaK/ZRU37g9XOw934C7cw/2VYQMewSbsgK8w5aXyWnmjrqtP1bbKBqnK3JC5Bf+YKv4A6kUTIQ== AAAEuHichZG/b9NQEMevxkAxP5rCgsRiEbVCimo9u6qTdKpAQrC1DW0q1VWwnZf0UdvPen6OFKL8AyyMgJhAYkD8GSxsTAz9ExBjkVgYODsJKQ1Jz7J973yf7935vDhgiSTkeE65oF68dHn+inb12vUbC4XFm7sJT4VPd3wecLHnuQkNWER3JJMB3YsFdUMvoHXv6EH2vd6hImE8eiK7MT0I3XbEWsx3JYYai4rieLTNop50vTRwRb8XCNHXHMljkQZU06dbjUrJora+rOsPE58Lit7jMBa8Q0MaSd1xNCdkzXNk6lw0dR4F3UzHrhplO3NyW1nJNGZ0gFbyD12RERXLsK0RWjIto2LPpkv4n6RwWZTTZWO1OqbLhr06jR5z49rlU22XzIpByH/pU2hT8JinMqerhr02pqtGpXqG7jlea6LuSKKfoXlGlRgVsz9SGkCWaZC1fr4Lj0vJw3wdDo2afzeuNQpFgi1npk865tApwtA2eeErONAEDj6kEAKFCCT6AbiQ4LUPJhCIMXYAPXiGERcEnlieQ6EPGvIpxilmuRg9wmcbT/vDaITnTDfJFXysFOAtkNRhiXwjH8kJ+UI+ke/k91StXq6R9dPFtzdgadxYeHG79utcKsS3hMMxNYPwsDeOZ4kRDzNmzyehBZV8LoZzxnkkm9gf1Oo8f3VSW99e6i2T9+QHzvqOHJPPOG3U+el/2KLbb2d2EmbVcaHm2fVNOruWYaK/ZRU37g9XOw934C7cw/2VYQMewSbsgK8w5aXyWnmjrqtP1bbKBqnK3JC5Bf+YKv4A6kUTIQ== AAAEuHichZG/b9NQEMevxkAxP5rCgsRiEbVCimo9u6qTdKpAQrC1DW0q1VWwnZf0UdvPen6OFKL8AyyMgJhAYkD8GSxsTAz9ExBjkVgYODsJKQ1Jz7J973yf7935vDhgiSTkeE65oF68dHn+inb12vUbC4XFm7sJT4VPd3wecLHnuQkNWER3JJMB3YsFdUMvoHXv6EH2vd6hImE8eiK7MT0I3XbEWsx3JYYai4rieLTNop50vTRwRb8XCNHXHMljkQZU06dbjUrJora+rOsPE58Lit7jMBa8Q0MaSd1xNCdkzXNk6lw0dR4F3UzHrhplO3NyW1nJNGZ0gFbyD12RERXLsK0RWjIto2LPpkv4n6RwWZTTZWO1OqbLhr06jR5z49rlU22XzIpByH/pU2hT8JinMqerhr02pqtGpXqG7jlea6LuSKKfoXlGlRgVsz9SGkCWaZC1fr4Lj0vJw3wdDo2afzeuNQpFgi1npk865tApwtA2eeErONAEDj6kEAKFCCT6AbiQ4LUPJhCIMXYAPXiGERcEnlieQ6EPGvIpxilmuRg9wmcbT/vDaITnTDfJFXysFOAtkNRhiXwjH8kJ+UI+ke/k91StXq6R9dPFtzdgadxYeHG79utcKsS3hMMxNYPwsDeOZ4kRDzNmzyehBZV8LoZzxnkkm9gf1Oo8f3VSW99e6i2T9+QHzvqOHJPPOG3U+el/2KLbb2d2EmbVcaHm2fVNOruWYaK/ZRU37g9XOw934C7cw/2VYQMewSbsgK8w5aXyWnmjrqtP1bbKBqnK3JC5Bf+YKv4A6kUTIQ== AAAEuHichZG/b9NQEMevxkAxP5rCgsRiEbVCimo9u6qTdKpAQrC1DW0q1VWwnZf0UdvPen6OFKL8AyyMgJhAYkD8GSxsTAz9ExBjkVgYODsJKQ1Jz7J973yf7935vDhgiSTkeE65oF68dHn+inb12vUbC4XFm7sJT4VPd3wecLHnuQkNWER3JJMB3YsFdUMvoHXv6EH2vd6hImE8eiK7MT0I3XbEWsx3JYYai4rieLTNop50vTRwRb8XCNHXHMljkQZU06dbjUrJora+rOsPE58Lit7jMBa8Q0MaSd1xNCdkzXNk6lw0dR4F3UzHrhplO3NyW1nJNGZ0gFbyD12RERXLsK0RWjIto2LPpkv4n6RwWZTTZWO1OqbLhr06jR5z49rlU22XzIpByH/pU2hT8JinMqerhr02pqtGpXqG7jlea6LuSKKfoXlGlRgVsz9SGkCWaZC1fr4Lj0vJw3wdDo2afzeuNQpFgi1npk865tApwtA2eeErONAEDj6kEAKFCCT6AbiQ4LUPJhCIMXYAPXiGERcEnlieQ6EPGvIpxilmuRg9wmcbT/vDaITnTDfJFXysFOAtkNRhiXwjH8kJ+UI+ke/k91StXq6R9dPFtzdgadxYeHG79utcKsS3hMMxNYPwsDeOZ4kRDzNmzyehBZV8LoZzxnkkm9gf1Oo8f3VSW99e6i2T9+QHzvqOHJPPOG3U+el/2KLbb2d2EmbVcaHm2fVNOruWYaK/ZRU37g9XOw934C7cw/2VYQMewSbsgK8w5aXyWnmjrqtP1bbKBqnK3JC5Bf+YKv4A6kUTIQ==

Slide 27

Slide 27 text

27 Setting Fscore Improvement Word only 69.76 – +char 82.62 +12.86 +pretrain 87.39 +17.63 +pretrain +char 87.76 +18.00 +pretrain +dropout 89.65 +19.89 +pretrain +char +dropout 90.81 +21.05 AAAEuHichZG/b9NQEMevxkAxP5rCgsRiEbVCimo9u6qTdKpAQrC1DW0q1VWwnZf0UdvPen6OFKL8AyyMgJhAYkD8GSxsTAz9ExBjkVgYODsJKQ1Jz7J973yf7935vDhgiSTkeE65oF68dHn+inb12vUbC4XFm7sJT4VPd3wecLHnuQkNWER3JJMB3YsFdUMvoHXv6EH2vd6hImE8eiK7MT0I3XbEWsx3JYYai4rieLTNop50vTRwRb8XCNHXHMljkQZU06dbjUrJora+rOsPE58Lit7jMBa8Q0MaSd1xNCdkzXNk6lw0dR4F3UzHrhplO3NyW1nJNGZ0gFbyD12RERXLsK0RWjIto2LPpkv4n6RwWZTTZWO1OqbLhr06jR5z49rlU22XzIpByH/pU2hT8JinMqerhr02pqtGpXqG7jlea6LuSKKfoXlGlRgVsz9SGkCWaZC1fr4Lj0vJw3wdDo2afzeuNQpFgi1npk865tApwtA2eeErONAEDj6kEAKFCCT6AbiQ4LUPJhCIMXYAPXiGERcEnlieQ6EPGvIpxilmuRg9wmcbT/vDaITnTDfJFXysFOAtkNRhiXwjH8kJ+UI+ke/k91StXq6R9dPFtzdgadxYeHG79utcKsS3hMMxNYPwsDeOZ4kRDzNmzyehBZV8LoZzxnkkm9gf1Oo8f3VSW99e6i2T9+QHzvqOHJPPOG3U+el/2KLbb2d2EmbVcaHm2fVNOruWYaK/ZRU37g9XOw934C7cw/2VYQMewSbsgK8w5aXyWnmjrqtP1bbKBqnK3JC5Bf+YKv4A6kUTIQ== AAAEuHichZG/b9NQEMevxkAxP5rCgsRiEbVCimo9u6qTdKpAQrC1DW0q1VWwnZf0UdvPen6OFKL8AyyMgJhAYkD8GSxsTAz9ExBjkVgYODsJKQ1Jz7J973yf7935vDhgiSTkeE65oF68dHn+inb12vUbC4XFm7sJT4VPd3wecLHnuQkNWER3JJMB3YsFdUMvoHXv6EH2vd6hImE8eiK7MT0I3XbEWsx3JYYai4rieLTNop50vTRwRb8XCNHXHMljkQZU06dbjUrJora+rOsPE58Lit7jMBa8Q0MaSd1xNCdkzXNk6lw0dR4F3UzHrhplO3NyW1nJNGZ0gFbyD12RERXLsK0RWjIto2LPpkv4n6RwWZTTZWO1OqbLhr06jR5z49rlU22XzIpByH/pU2hT8JinMqerhr02pqtGpXqG7jlea6LuSKKfoXlGlRgVsz9SGkCWaZC1fr4Lj0vJw3wdDo2afzeuNQpFgi1npk865tApwtA2eeErONAEDj6kEAKFCCT6AbiQ4LUPJhCIMXYAPXiGERcEnlieQ6EPGvIpxilmuRg9wmcbT/vDaITnTDfJFXysFOAtkNRhiXwjH8kJ+UI+ke/k91StXq6R9dPFtzdgadxYeHG79utcKsS3hMMxNYPwsDeOZ4kRDzNmzyehBZV8LoZzxnkkm9gf1Oo8f3VSW99e6i2T9+QHzvqOHJPPOG3U+el/2KLbb2d2EmbVcaHm2fVNOruWYaK/ZRU37g9XOw934C7cw/2VYQMewSbsgK8w5aXyWnmjrqtP1bbKBqnK3JC5Bf+YKv4A6kUTIQ== AAAEuHichZG/b9NQEMevxkAxP5rCgsRiEbVCimo9u6qTdKpAQrC1DW0q1VWwnZf0UdvPen6OFKL8AyyMgJhAYkD8GSxsTAz9ExBjkVgYODsJKQ1Jz7J973yf7935vDhgiSTkeE65oF68dHn+inb12vUbC4XFm7sJT4VPd3wecLHnuQkNWER3JJMB3YsFdUMvoHXv6EH2vd6hImE8eiK7MT0I3XbEWsx3JYYai4rieLTNop50vTRwRb8XCNHXHMljkQZU06dbjUrJora+rOsPE58Lit7jMBa8Q0MaSd1xNCdkzXNk6lw0dR4F3UzHrhplO3NyW1nJNGZ0gFbyD12RERXLsK0RWjIto2LPpkv4n6RwWZTTZWO1OqbLhr06jR5z49rlU22XzIpByH/pU2hT8JinMqerhr02pqtGpXqG7jlea6LuSKKfoXlGlRgVsz9SGkCWaZC1fr4Lj0vJw3wdDo2afzeuNQpFgi1npk865tApwtA2eeErONAEDj6kEAKFCCT6AbiQ4LUPJhCIMXYAPXiGERcEnlieQ6EPGvIpxilmuRg9wmcbT/vDaITnTDfJFXysFOAtkNRhiXwjH8kJ+UI+ke/k91StXq6R9dPFtzdgadxYeHG79utcKsS3hMMxNYPwsDeOZ4kRDzNmzyehBZV8LoZzxnkkm9gf1Oo8f3VSW99e6i2T9+QHzvqOHJPPOG3U+el/2KLbb2d2EmbVcaHm2fVNOruWYaK/ZRU37g9XOw934C7cw/2VYQMewSbsgK8w5aXyWnmjrqtP1bbKBqnK3JC5Bf+YKv4A6kUTIQ== AAAEuHichZG/b9NQEMevxkAxP5rCgsRiEbVCimo9u6qTdKpAQrC1DW0q1VWwnZf0UdvPen6OFKL8AyyMgJhAYkD8GSxsTAz9ExBjkVgYODsJKQ1Jz7J973yf7935vDhgiSTkeE65oF68dHn+inb12vUbC4XFm7sJT4VPd3wecLHnuQkNWER3JJMB3YsFdUMvoHXv6EH2vd6hImE8eiK7MT0I3XbEWsx3JYYai4rieLTNop50vTRwRb8XCNHXHMljkQZU06dbjUrJora+rOsPE58Lit7jMBa8Q0MaSd1xNCdkzXNk6lw0dR4F3UzHrhplO3NyW1nJNGZ0gFbyD12RERXLsK0RWjIto2LPpkv4n6RwWZTTZWO1OqbLhr06jR5z49rlU22XzIpByH/pU2hT8JinMqerhr02pqtGpXqG7jlea6LuSKKfoXlGlRgVsz9SGkCWaZC1fr4Lj0vJw3wdDo2afzeuNQpFgi1npk865tApwtA2eeErONAEDj6kEAKFCCT6AbiQ4LUPJhCIMXYAPXiGERcEnlieQ6EPGvIpxilmuRg9wmcbT/vDaITnTDfJFXysFOAtkNRhiXwjH8kJ+UI+ke/k91StXq6R9dPFtzdgadxYeHG79utcKsS3hMMxNYPwsDeOZ4kRDzNmzyehBZV8LoZzxnkkm9gf1Oo8f3VSW99e6i2T9+QHzvqOHJPPOG3U+el/2KLbb2d2EmbVcaHm2fVNOruWYaK/ZRU37g9XOw934C7cw/2VYQMewSbsgK8w5aXyWnmjrqtP1bbKBqnK3JC5Bf+YKv4A6kUTIQ==

Slide 28

Slide 28 text

• ) ) ) )( ) • s fE N g E er • BiLSTM-CRF ho f • cR Nm t • di E • Chainer di • u R N ATrainer E P l • Eho a n 28

Slide 29

Slide 29 text

29 himkt/pyner https://speakerdeck.com/himkt/implement-neural-named-entity-tagger

Slide 30

Slide 30 text

• • CoNLL2003 0 • • : 246,679 -> 210,023 30 https://github.com/glample/tagger/blob/master/utils.py#L82

Slide 31

Slide 31 text

31 Setting Fscore Improvement +pretrain +char +dropout 90.81 – +pretrain +char +dropout +BIO 89.61 -1.20 +pretrain +char +dropout -clipping 90.42 -0.39 +pretrain +char +dropout +lower 90.50 -0.31 +pretrain +char +dropout -zero 90.78 -0.03 AAAEb3ichZHNbtNAEIAnboBifprCASQktCJqBYoSrVOgCaeqSBU9NW1IW6muItvZpEttr7VeB7VRXoAX4MAJJA6IJ+DMhRsnDn0ExAkViQsHxo4JpZXDWPbOjOebnx07cHmoKD3KaVP5c+cvTF/UL12+cnWmMHttMxSRdFjLEa6Q27YVMpf7rKW4ctl2IJnl2S7bsvcfx/+3+kyGXPhP1UHAdj2r5/MudyyFrvZs7oNpsx73B8qyI9eSw4Er5VA3lQhk5DKdZEuTKcX9HpknZCV0hGSorXqBFH3mMV8R09RNj3dOpxmYdpeUsEklLe6TkrNnSVLqSBGISA3jbElEnVZqRmLGUi7H6UZ6FktKy6trMVCrVx4aY9KoVGkCZ3Jlx+VBkI6Cde9XxzCtLNT/VM6u64rnTKbwA3oSNsZtZ1c/ZFKk8GLtBEwXkiu0hVLCS27RZH5nvCi9XShiUCLkrGKkShFSaYjCZzChAwIciMADBj4o1F2wIMRnBwygEKBvFwbwDD0WSLR4EsNgCDryEfoZRlno3cdvD62d1OujHecNkwwOVnLxlUgSmKNf6Dt6TD/R9/Qr/ZWZa5DkiPs5wNMesSxoz7y42fz5X8rDU8HeX2oCYWNvAm2FHhsjJs+noAu1ZC6OcwaJJ57YGdXqH748bj7amBvM0zf0G876mh7Rjzit3//hvF1nG68mduLF1XGhxun1nVU2qxUD9fVqcWk5Xe003II7cBf3twhL8AQa0AJHI9qKtqY1pr7nb+Rv58koVMulzHX4R/L3fgPquBTF AAAEb3ichZHNbtNAEIAnboBifprCASQktCJqBYoSrVOgCaeqSBU9NW1IW6muItvZpEttr7VeB7VRXoAX4MAJJA6IJ+DMhRsnDn0ExAkViQsHxo4JpZXDWPbOjOebnx07cHmoKD3KaVP5c+cvTF/UL12+cnWmMHttMxSRdFjLEa6Q27YVMpf7rKW4ctl2IJnl2S7bsvcfx/+3+kyGXPhP1UHAdj2r5/MudyyFrvZs7oNpsx73B8qyI9eSw4Er5VA3lQhk5DKdZEuTKcX9HpknZCV0hGSorXqBFH3mMV8R09RNj3dOpxmYdpeUsEklLe6TkrNnSVLqSBGISA3jbElEnVZqRmLGUi7H6UZ6FktKy6trMVCrVx4aY9KoVGkCZ3Jlx+VBkI6Cde9XxzCtLNT/VM6u64rnTKbwA3oSNsZtZ1c/ZFKk8GLtBEwXkiu0hVLCS27RZH5nvCi9XShiUCLkrGKkShFSaYjCZzChAwIciMADBj4o1F2wIMRnBwygEKBvFwbwDD0WSLR4EsNgCDryEfoZRlno3cdvD62d1OujHecNkwwOVnLxlUgSmKNf6Dt6TD/R9/Qr/ZWZa5DkiPs5wNMesSxoz7y42fz5X8rDU8HeX2oCYWNvAm2FHhsjJs+noAu1ZC6OcwaJJ57YGdXqH748bj7amBvM0zf0G876mh7Rjzit3//hvF1nG68mduLF1XGhxun1nVU2qxUD9fVqcWk5Xe003II7cBf3twhL8AQa0AJHI9qKtqY1pr7nb+Rv58koVMulzHX4R/L3fgPquBTF AAAEb3ichZHNbtNAEIAnboBifprCASQktCJqBYoSrVOgCaeqSBU9NW1IW6muItvZpEttr7VeB7VRXoAX4MAJJA6IJ+DMhRsnDn0ExAkViQsHxo4JpZXDWPbOjOebnx07cHmoKD3KaVP5c+cvTF/UL12+cnWmMHttMxSRdFjLEa6Q27YVMpf7rKW4ctl2IJnl2S7bsvcfx/+3+kyGXPhP1UHAdj2r5/MudyyFrvZs7oNpsx73B8qyI9eSw4Er5VA3lQhk5DKdZEuTKcX9HpknZCV0hGSorXqBFH3mMV8R09RNj3dOpxmYdpeUsEklLe6TkrNnSVLqSBGISA3jbElEnVZqRmLGUi7H6UZ6FktKy6trMVCrVx4aY9KoVGkCZ3Jlx+VBkI6Cde9XxzCtLNT/VM6u64rnTKbwA3oSNsZtZ1c/ZFKk8GLtBEwXkiu0hVLCS27RZH5nvCi9XShiUCLkrGKkShFSaYjCZzChAwIciMADBj4o1F2wIMRnBwygEKBvFwbwDD0WSLR4EsNgCDryEfoZRlno3cdvD62d1OujHecNkwwOVnLxlUgSmKNf6Dt6TD/R9/Qr/ZWZa5DkiPs5wNMesSxoz7y42fz5X8rDU8HeX2oCYWNvAm2FHhsjJs+noAu1ZC6OcwaJJ57YGdXqH748bj7amBvM0zf0G876mh7Rjzit3//hvF1nG68mduLF1XGhxun1nVU2qxUD9fVqcWk5Xe003II7cBf3twhL8AQa0AJHI9qKtqY1pr7nb+Rv58koVMulzHX4R/L3fgPquBTF AAAEb3ichZHNbtNAEIAnboBifprCASQktCJqBYoSrVOgCaeqSBU9NW1IW6muItvZpEttr7VeB7VRXoAX4MAJJA6IJ+DMhRsnDn0ExAkViQsHxo4JpZXDWPbOjOebnx07cHmoKD3KaVP5c+cvTF/UL12+cnWmMHttMxSRdFjLEa6Q27YVMpf7rKW4ctl2IJnl2S7bsvcfx/+3+kyGXPhP1UHAdj2r5/MudyyFrvZs7oNpsx73B8qyI9eSw4Er5VA3lQhk5DKdZEuTKcX9HpknZCV0hGSorXqBFH3mMV8R09RNj3dOpxmYdpeUsEklLe6TkrNnSVLqSBGISA3jbElEnVZqRmLGUi7H6UZ6FktKy6trMVCrVx4aY9KoVGkCZ3Jlx+VBkI6Cde9XxzCtLNT/VM6u64rnTKbwA3oSNsZtZ1c/ZFKk8GLtBEwXkiu0hVLCS27RZH5nvCi9XShiUCLkrGKkShFSaYjCZzChAwIciMADBj4o1F2wIMRnBwygEKBvFwbwDD0WSLR4EsNgCDryEfoZRlno3cdvD62d1OujHecNkwwOVnLxlUgSmKNf6Dt6TD/R9/Qr/ZWZa5DkiPs5wNMesSxoz7y42fz5X8rDU8HeX2oCYWNvAm2FHhsjJs+noAu1ZC6OcwaJJ57YGdXqH748bj7amBvM0zf0G876mh7Rjzit3//hvF1nG68mduLF1XGhxun1nVU2qxUD9fVqcWk5Xe003II7cBf3twhL8AQa0AJHI9qKtqY1pr7nb+Rv58koVMulzHX4R/L3fgPquBTF

Slide 32

Slide 32 text

32

Slide 33

Slide 33 text

https://github.com/glample/tagger/blob/master/utils.py#L44 33

Slide 34

Slide 34 text

https://github.com/glample/tagger/blob/master/utils.py#L44 Xavier initialization [7] 34

Slide 35

Slide 35 text

https://github.com/glample/tagger/blob/master/model.py#L281 https://github.com/chainer/chainer/blob/master/chainer/links/loss/crf1d.py https://github.com/chainer/chainer/pull/5807 35

Slide 36

Slide 36 text

36 • 7 • [-sqrt(6/(in+out)), sqrt(6/(in+out))] (Xavier initialization [7]) • Lample+ [1]: https://github.com/glample/tagger/blob/master/utils.py#L52 • [-sqrt(3/dim), sqrt(3/dim)] (LeCun initialization [8]) • Ma+ [2]: https://github.com/XuezheMax/NeuroNLP/blob/master/sequence_labeling.py#L161 • Liu+ [6]: https://github.com/LiyuanLucasLiu/LM-LSTM-CRF/blob/master/model/utils.py#L793 • ( :Glorot ) • word2vec/word2vec: [-(0.5/dim), (0.5/dim)] • https://github.com/tmikolov/word2vec/blob/master/word2vec.c#L365 • facebookresearch/fastText: [-(1/dim), (1/dim)] • https://github.com/facebookresearch/fastText/blob/master/src/fasttext.cc#L734 • 7 : C

Slide 37

Slide 37 text

37 Setting Fscore Improvement Word only 78.05 +8.29 +char 84.04 +14.28 +pretrain 86.93 +17.17 +pretrain +char 88.51 +18.75 +pretrain +dropout 89.91 +20.15 +pretrain +char +dropout 90.99 +21.23 +pretrain +char +dropout +BIO 89.99 +20.23 +pretrain +char +dropout -clipping 90.60 +20.84 +pretrain +char +dropout +lower 90.52 +20.76 +pretrain +char +dropout -zero 90.25 +20.49 AAAFrnichZK/b9NAFMdfGwIl/GgLCxKLRdUKKeJ0dpPGZqqKhOhEf9AfUl1FtnNNTc8+63wJaqOMLAwwMjCBxID4M1jYmBj6JyDGIsHAwLPj1IGS9Fm23z2/z/e95zs34n6sKD0eGy9cKF68NHG5dOXqteuTU9M3NmPRkh7b8AQXctt1Ysb9kG0oX3G2HUnmBC5nW+7Bg+T7VpvJ2BfhE3UYsd3AaYb+nu85CkP16fFftsuafthRjtvijux2uOTdkq1EJFuclbQRts6U8sOmNqdpD2NPSIbechBJ0WYBC5Vm2yU78Bvn6GwJ2dBEyA8TnZpJaDVxUiubxLASmVFdYJq378gEMiuEVvp0Wa8QwxxNl/FfKen4YUovEGs+p2tErw2jcy6vbZKqntMmqVX/Sw+gDSki0VIpbRErpw1K9Jzu2O7emYp9uJtAaYZFiWV1+xo9yNCJMd/N+ximopWXlh/3+7AG+zDmU3ooeM/jfhRlpwBbWKCDtFnp1x5emYtnTGZ01RikawunnQ+vf8SkyGijOkhX0pNju0IpEaRn0GZh4/Scl+pTM5TQ1LSzjp45M5DZipj6AjY0QIAHLQiAQQgKfQ4OxHjtgA4UIoztQgeeYsQBiSs/zWHQhRLyLYwzzHIweoDPJq52smiI60Q3ThU8rMTxlkhqMEu/0g/0hH6mH+k3+nuoVifVSPo5xLfbY1lUn3xxa/3nuVSAbwX7OTWCcLE3gWuFERczRs+nYA/MdC4f54zSSDKx16vVPnp9sn5/bbYzR9/R7zjrW3pMP+G0YfuH936Vrb0Z2UmQVMcN1f/dvrPOpkF09FeNmcWlbGsn4Dbcgbu4fzVYhEewAhvgFeqF54WXhVdFWtws7hbrvdTxsYy5CX9Zcf8PZKxUlQ== AAAFrnichZK/b9NAFMdfGwIl/GgLCxKLRdUKKeJ0dpPGZqqKhOhEf9AfUl1FtnNNTc8+63wJaqOMLAwwMjCBxID4M1jYmBj6JyDGIsHAwLPj1IGS9Fm23z2/z/e95zs34n6sKD0eGy9cKF68NHG5dOXqteuTU9M3NmPRkh7b8AQXctt1Ysb9kG0oX3G2HUnmBC5nW+7Bg+T7VpvJ2BfhE3UYsd3AaYb+nu85CkP16fFftsuafthRjtvijux2uOTdkq1EJFuclbQRts6U8sOmNqdpD2NPSIbechBJ0WYBC5Vm2yU78Bvn6GwJ2dBEyA8TnZpJaDVxUiubxLASmVFdYJq378gEMiuEVvp0Wa8QwxxNl/FfKen4YUovEGs+p2tErw2jcy6vbZKqntMmqVX/Sw+gDSki0VIpbRErpw1K9Jzu2O7emYp9uJtAaYZFiWV1+xo9yNCJMd/N+ximopWXlh/3+7AG+zDmU3ooeM/jfhRlpwBbWKCDtFnp1x5emYtnTGZ01RikawunnQ+vf8SkyGijOkhX0pNju0IpEaRn0GZh4/Scl+pTM5TQ1LSzjp45M5DZipj6AjY0QIAHLQiAQQgKfQ4OxHjtgA4UIoztQgeeYsQBiSs/zWHQhRLyLYwzzHIweoDPJq52smiI60Q3ThU8rMTxlkhqMEu/0g/0hH6mH+k3+nuoVifVSPo5xLfbY1lUn3xxa/3nuVSAbwX7OTWCcLE3gWuFERczRs+nYA/MdC4f54zSSDKx16vVPnp9sn5/bbYzR9/R7zjrW3pMP+G0YfuH936Vrb0Z2UmQVMcN1f/dvrPOpkF09FeNmcWlbGsn4Dbcgbu4fzVYhEewAhvgFeqF54WXhVdFWtws7hbrvdTxsYy5CX9Zcf8PZKxUlQ== AAAFrnichZK/b9NAFMdfGwIl/GgLCxKLRdUKKeJ0dpPGZqqKhOhEf9AfUl1FtnNNTc8+63wJaqOMLAwwMjCBxID4M1jYmBj6JyDGIsHAwLPj1IGS9Fm23z2/z/e95zs34n6sKD0eGy9cKF68NHG5dOXqteuTU9M3NmPRkh7b8AQXctt1Ysb9kG0oX3G2HUnmBC5nW+7Bg+T7VpvJ2BfhE3UYsd3AaYb+nu85CkP16fFftsuafthRjtvijux2uOTdkq1EJFuclbQRts6U8sOmNqdpD2NPSIbechBJ0WYBC5Vm2yU78Bvn6GwJ2dBEyA8TnZpJaDVxUiubxLASmVFdYJq378gEMiuEVvp0Wa8QwxxNl/FfKen4YUovEGs+p2tErw2jcy6vbZKqntMmqVX/Sw+gDSki0VIpbRErpw1K9Jzu2O7emYp9uJtAaYZFiWV1+xo9yNCJMd/N+ximopWXlh/3+7AG+zDmU3ooeM/jfhRlpwBbWKCDtFnp1x5emYtnTGZ01RikawunnQ+vf8SkyGijOkhX0pNju0IpEaRn0GZh4/Scl+pTM5TQ1LSzjp45M5DZipj6AjY0QIAHLQiAQQgKfQ4OxHjtgA4UIoztQgeeYsQBiSs/zWHQhRLyLYwzzHIweoDPJq52smiI60Q3ThU8rMTxlkhqMEu/0g/0hH6mH+k3+nuoVifVSPo5xLfbY1lUn3xxa/3nuVSAbwX7OTWCcLE3gWuFERczRs+nYA/MdC4f54zSSDKx16vVPnp9sn5/bbYzR9/R7zjrW3pMP+G0YfuH936Vrb0Z2UmQVMcN1f/dvrPOpkF09FeNmcWlbGsn4Dbcgbu4fzVYhEewAhvgFeqF54WXhVdFWtws7hbrvdTxsYy5CX9Zcf8PZKxUlQ== AAAFrnichZK/b9NAFMdfGwIl/GgLCxKLRdUKKeJ0dpPGZqqKhOhEf9AfUl1FtnNNTc8+63wJaqOMLAwwMjCBxID4M1jYmBj6JyDGIsHAwLPj1IGS9Fm23z2/z/e95zs34n6sKD0eGy9cKF68NHG5dOXqteuTU9M3NmPRkh7b8AQXctt1Ysb9kG0oX3G2HUnmBC5nW+7Bg+T7VpvJ2BfhE3UYsd3AaYb+nu85CkP16fFftsuafthRjtvijux2uOTdkq1EJFuclbQRts6U8sOmNqdpD2NPSIbechBJ0WYBC5Vm2yU78Bvn6GwJ2dBEyA8TnZpJaDVxUiubxLASmVFdYJq378gEMiuEVvp0Wa8QwxxNl/FfKen4YUovEGs+p2tErw2jcy6vbZKqntMmqVX/Sw+gDSki0VIpbRErpw1K9Jzu2O7emYp9uJtAaYZFiWV1+xo9yNCJMd/N+ximopWXlh/3+7AG+zDmU3ooeM/jfhRlpwBbWKCDtFnp1x5emYtnTGZ01RikawunnQ+vf8SkyGijOkhX0pNju0IpEaRn0GZh4/Scl+pTM5TQ1LSzjp45M5DZipj6AjY0QIAHLQiAQQgKfQ4OxHjtgA4UIoztQgeeYsQBiSs/zWHQhRLyLYwzzHIweoDPJq52smiI60Q3ThU8rMTxlkhqMEu/0g/0hH6mH+k3+nuoVifVSPo5xLfbY1lUn3xxa/3nuVSAbwX7OTWCcLE3gWuFERczRs+nYA/MdC4f54zSSDKx16vVPnp9sn5/bbYzR9/R7zjrW3pMP+G0YfuH936Vrb0Z2UmQVMcN1f/dvrPOpkF09FeNmcWlbGsn4Dbcgbu4fzVYhEewAhvgFeqF54WXhVdFWtws7hbrvdTxsYy5CX9Zcf8PZKxUlQ==

Slide 38

Slide 38 text

5APM G M DE A PMAN B M 5 A I E T 7A CIE E I 3 GA A G 5 -3 I AI 8ALPAI A 3 AGEIC E E EMA E I G 38 4 -55N -70 4 A G -3 8A E NP AM ENA NALPAI A CCEIC RE D E EMA E I G G ICP CA AGN 6A AMN A G -3 .AA I AS P GEUA R M MA MANAI E IN 6A AMN A G 5 -3 RAM 8ALPAI A 3 AGEIC RE D NF R MA 5APM G 3 ICP CA 4 AG 3EP A G 2 5 GG I AS N MA MA A ALP G A AM R M MA MANAI E IN RE D ME GA AI E I 3EIC A G 4536 :I AMN I EIC DA EBBE PG T B M EIEIC AA BAA B MR M IAPM G IA R MFN 1G M A G 28 8 BBE EAI F M 3A-PI A G 5APM G 5A R MF ME FN B DA M A 38