Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
MeCabとKerasを使ったテキスト分類
Search
masa-ita
February 23, 2019
Technology
1
470
MeCabとKerasを使ったテキスト分類
masa-ita
February 23, 2019
Tweet
Share
More Decks by masa-ita
See All by masa-ita
Ollamaを使ったLocal Language Model活用法
itagakim
1
71
Run Instant NeRF on Docker
itagakim
1
2.2k
3D Clustering and Metric Learning
itagakim
0
300
Cloud TPUの使い方〜BigBirdの日本語学習済みモデルを作る〜
itagakim
0
630
多言語学習済みモデルmT5とは?
itagakim
1
630
AWSのGPUを安く使ってTensorFlowモデルを訓練する方法
itagakim
0
340
最近の自然言語処理モデルの動向
itagakim
1
540
ディープラーニングで芸術はできるか?〜生成系ネットワークの進展〜
itagakim
0
300
AWSとTerraform初心者がやってみたこと
itagakim
1
430
Other Decks in Technology
See All in Technology
開発者のための FinOps/FinOps for Engineers
oracle4engineer
PRO
2
180
AIエージェント入門
minorun365
PRO
32
19k
Introduction to OpenSearch Project - Search Engineering Tech Talk 2025 Winter
tkykenmt
2
140
Potential EM 制度を始めた理由、そして2年後にやめた理由 - EMConf JP 2025
hoyo
2
2.8k
OCI Success Journey OCIの何が評価されてる?疑問に答える事例セミナー(2025年2月実施)
oracle4engineer
PRO
2
170
クラウド食堂とは?
hiyanger
0
120
役員・マネージャー・著者・エンジニアそれぞれの立場から見たAWS認定資格
nrinetcom
PRO
4
6.4k
JavaにおけるNull非許容性
skrb
2
2.7k
What's new in Go 1.24?
ciarana
1
110
EDRの検知の仕組みと検知回避について
chayakonanaika
12
5.1k
大規模アジャイルフレームワークから学ぶエンジニアマネジメントの本質
staka121
PRO
3
1.3k
あなたが人生で成功するための5つの普遍的法則 #jawsug #jawsdays2025 / 20250301 HEROZ
yoshidashingo
2
320
Featured
See All Featured
Understanding Cognitive Biases in Performance Measurement
bluesmoon
27
1.6k
Cheating the UX When There Is Nothing More to Optimize - PixelPioneers
stephaniewalter
280
13k
It's Worth the Effort
3n
184
28k
Performance Is Good for Brains [We Love Speed 2024]
tammyeverts
7
650
Bash Introduction
62gerente
611
210k
Principles of Awesome APIs and How to Build Them.
keavy
126
17k
What’s in a name? Adding method to the madness
productmarketing
PRO
22
3.3k
Improving Core Web Vitals using Speculation Rules API
sergeychernyshev
10
520
CoffeeScript is Beautiful & I Never Want to Write Plain JavaScript Again
sstephenson
160
15k
No one is an island. Learnings from fostering a developers community.
thoeni
21
3.2k
Rebuilding a faster, lazier Slack
samanthasiow
80
8.9k
How STYLIGHT went responsive
nonsquared
98
5.4k
Transcript
MeCabKeras 2019/2/23 @Python in
3F-*"% Q:<+/M@3F-*8L )9 3F O8L$?.
IDP6S E<6S >16S KFREG6S /M6S C4-*"% 3F-*8L)9 <JNF '0=A#&H ! 5 72; B, ("%
!!$A<7> 7>-=N-Gram .C(2 !$,@ 7>A<A1
0 # $?/<"A<85 3B!$, %&<*'9)+:. %&<*'D46 =;C2E6 0 Ex. MeCab
'!, ",*+$J8 AOIQH=
FORBFO"( E9 RLRB20N16AOIQ H= RLAAG>U &$ CV .@W73 RL?K MS 16E -D16/5:TH= /5:T;=46 )%#+P 46<
livedoor NHN Japan58+- 42 livedoor $' ) #%&* (!*
=. $'1,79 :6;HTML"/<30 https://www.rondhuit.com/download.html#ldcc
livedoor
MeCab
MeCab HN7GSMGegi−69PKPLW`8:%/0-$ &25iGQoegI _@eg1-*,.4'",BC? !.5)(
fdkRm 5'5 V;T[nUJaGoogle Inc. ^p\Ffh]cX +.3-5#><jl = Y ,"5DAbEZ O
MeCab MeCab C++ '& # !*(
Windows %$ https://taku910.github.io/mecab/#download #"+) 32 64 , https://github.com/ikegami-yukino/mecab/releases/tag/v0.996 #"+) Mac %$ Homebrew mecab, mecab-ipadic #!+) Ubuntu %$ apt mecab, mecab-ipadic #!+)
Keras
keras.preprocessing.text.Tokenizer /-.2 /- !%"(8$&5 * #31)76 0)% +4
', fit &5tokenize !%0) %
keras.preprocessing.sequence.pad_sequences ! ( " # $'%
&
BoW: Bag of Words # %EC* G DEC?
- J;/ F<+EC,8=@1/0&%) 58 ()! '"%*$* ,8I209&%) 58 /1 TF-IDF: Term Frequency Inverse Document Frequency EHI2 ><,8 EC:67B4A .1&% )3
Word Embedding a]!.$*2C<@ fTY=!UD :9RPJG5 a]J ?Z10,000 20,000K6
Ni '3&, &.$*2 7<a]![RP7dJ`RPe.$*2 F S< Word Embeddinga]gO Google A; Xb!LWord2vec^V \B W^Ec!80)2H_!LRP IM Word2vec&#(-%1/Qh@Ec!8 )"-1 +4%0)27> Ec!8<@
RNN: Recurrent Neural Network *-H,+.=8 G "!%AB !*DF
@162 ,'/5?)/ G#$&!:(8 RNN> C;79304E LSTMLong Short Term MemoryGRU Gated Recurrent Unit<
BoW DNN
Word EmbeddingGlobalAveragePooling1D
Word EmbeddingRNNLSTM DNN
BoWDNN 0.5E #9("%$)CBoW+/ DNN4: * DBG6GlobalAveragePooling1D1 !$=2F
A LSTM7H2F,- <4: ' ; 7I ?3>8)CLSTM 4: & @:4
NLP,B8?=4-1$!&)%+"C5>@.A 7EFDQ&A-1Sequence-to-Sequence($* Attention :($*.A;3 OpenAIGoogle
Transformer '#Allen Institute 2.ELMo Google G5($*3BERTOpenAI .6GPT-204 <($* 9/