Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
MeCabとKerasを使ったテキスト分類
Search
masa-ita
February 23, 2019
Technology
1
410
MeCabとKerasを使ったテキスト分類
masa-ita
February 23, 2019
Tweet
Share
More Decks by masa-ita
See All by masa-ita
Run Instant NeRF on Docker
itagakim
1
1.9k
3D Clustering and Metric Learning
itagakim
0
230
Cloud TPUの使い方〜BigBirdの日本語学習済みモデルを作る〜
itagakim
0
520
多言語学習済みモデルmT5とは?
itagakim
1
440
AWSのGPUを安く使ってTensorFlowモデルを訓練する方法
itagakim
0
260
最近の自然言語処理モデルの動向
itagakim
1
480
ディープラーニングで芸術はできるか?〜生成系ネットワークの進展〜
itagakim
0
260
AWSとTerraform初心者がやってみたこと
itagakim
1
360
IntroductionToTensorFlow2_0.pdf
itagakim
1
280
Other Decks in Technology
See All in Technology
NgRx Signal Store
rainerhahnekamp
0
120
Data and AI Governance: Existing Challenges and Emerging Trends
scotthsieh825
0
160
コンテナセキュリティの基本と脅威への対策
kyohmizu
3
700
ユーザーストーリーのレビューを自動化したみたの
bun913
1
330
Delivering Millions of Messages within seconds @ Duolingo
pelelgrino
0
340
TransitGatewayの基礎
toru_kubota
0
230
Hands-on / Kaname Frusawa / Cloud Compare Users Meetup 2024 at University of Tokyo on April 17
paraworld
2
470
"好き"との生活/Regularly update profile with GitHub Actions
judeeeee
0
150
VS CodeでAWSを操作しよう
smt7174
1
110
Garoon 開発チーム / Garoon development team
cybozuinsideout
PRO
2
2.9k
コンパウンドスタートアップのためのスケーラブルでセキュアなInfrastructure as Codeパイプラインを考える / Scalable and Secure Infrastructure as Code Pipeline for a Compound Startup
yuyatakeyama
3
3.5k
Databricks におけるデータエンジニアリング
databricksjapan
0
380
Featured
See All Featured
Automating Front-end Workflow
addyosmani
1355
200k
The Invisible Side of Design
smashingmag
294
49k
Learning to Love Humans: Emotional Interface Design
aarron
266
39k
Why You Should Never Use an ORM
jnunemaker
PRO
50
8.6k
Designing with Data
zakiwarfel
95
4.8k
Rebuilding a faster, lazier Slack
samanthasiow
72
8.2k
Fight the Zombie Pattern Library - RWD Summit 2016
marcelosomers
226
16k
Intergalactic Javascript Robots from Outer Space
tanoku
266
26k
Templates, Plugins, & Blocks: Oh My! Creating the theme that thinks of everything
marktimemedia
18
1.7k
A Modern Web Designer's Workflow
chriscoyier
688
190k
How To Stay Up To Date on Web Technology
chriscoyier
782
250k
Bootstrapping a Software Product
garrettdimon
PRO
301
110k
Transcript
MeCabKeras 2019/2/23 @Python in
3F-*"% Q:<+/M@3F-*8L )9 3F O8L$?.
IDP6S E<6S >16S KFREG6S /M6S C4-*"% 3F-*8L)9 <JNF '0=A#&H ! 5 72; B, ("%
!!$A<7> 7>-=N-Gram .C(2 !$,@ 7>A<A1
0 # $?/<"A<85 3B!$, %&<*'9)+:. %&<*'D46 =;C2E6 0 Ex. MeCab
'!, ",*+$J8 AOIQH=
FORBFO"( E9 RLRB20N16AOIQ H= RLAAG>U &$ CV .@W73 RL?K MS 16E -D16/5:TH= /5:T;=46 )%#+P 46<
livedoor NHN Japan58+- 42 livedoor $' ) #%&* (!*
=. $'1,79 :6;HTML"/<30 https://www.rondhuit.com/download.html#ldcc
livedoor
MeCab
MeCab HN7GSMGegi−69PKPLW`8:%/0-$ &25iGQoegI _@eg1-*,.4'",BC? !.5)(
fdkRm 5'5 V;T[nUJaGoogle Inc. ^p\Ffh]cX +.3-5#><jl = Y ,"5DAbEZ O
MeCab MeCab C++ '& # !*(
Windows %$ https://taku910.github.io/mecab/#download #"+) 32 64 , https://github.com/ikegami-yukino/mecab/releases/tag/v0.996 #"+) Mac %$ Homebrew mecab, mecab-ipadic #!+) Ubuntu %$ apt mecab, mecab-ipadic #!+)
Keras
keras.preprocessing.text.Tokenizer /-.2 /- !%"(8$&5 * #31)76 0)% +4
', fit &5tokenize !%0) %
keras.preprocessing.sequence.pad_sequences ! ( " # $'%
&
BoW: Bag of Words # %EC* G DEC?
- J;/ F<+EC,8=@1/0&%) 58 ()! '"%*$* ,8I209&%) 58 /1 TF-IDF: Term Frequency Inverse Document Frequency EHI2 ><,8 EC:67B4A .1&% )3
Word Embedding a]!.$*2C<@ fTY=!UD :9RPJG5 a]J ?Z10,000 20,000K6
Ni '3&, &.$*2 7<a]![RP7dJ`RPe.$*2 F S< Word Embeddinga]gO Google A; Xb!LWord2vec^V \B W^Ec!80)2H_!LRP IM Word2vec&#(-%1/Qh@Ec!8 )"-1 +4%0)27> Ec!8<@
RNN: Recurrent Neural Network *-H,+.=8 G "!%AB !*DF
@162 ,'/5?)/ G#$&!:(8 RNN> C;79304E LSTMLong Short Term MemoryGRU Gated Recurrent Unit<
BoW DNN
Word EmbeddingGlobalAveragePooling1D
Word EmbeddingRNNLSTM DNN
BoWDNN 0.5E #9("%$)CBoW+/ DNN4: * DBG6GlobalAveragePooling1D1 !$=2F
A LSTM7H2F,- <4: ' ; 7I ?3>8)CLSTM 4: & @:4
NLP,B8?=4-1$!&)%+"C5>@.A 7EFDQ&A-1Sequence-to-Sequence($* Attention :($*.A;3 OpenAIGoogle
Transformer '#Allen Institute 2.ELMo Google G5($*3BERTOpenAI .6GPT-204 <($* 9/