Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Sign up for free
Menu
Search
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Pricing
Search
Sign in
Sign up for free
Text Mining: Exploratory Data Analysis to Machi...
Search
Julia Silge
March 04, 2019
Technology
280
1
Share
Embed
Copy iframe code
Copy JS code
Copy link
Start on current slide
Text Mining: Exploratory Data Analysis to Machine Learning
March 2019 talk at WiDS Salt Lake City regional event
Julia Silge
March 04, 2019
Transcript
38 slides
T E X T M I N I N G
EXPLORATORY DATA ANALYSIS TO MACHINE LEARNING
HELLO T I D Y T E X T Data
Scientist at Stack Overflow @juliasilge https://juliasilge.com/ I’m Julia Silge
T I D Y T E X T TEXT DATA
IS INCREASINGLY IMPORTANT
T I D Y T E X T TEXT DATA
IS INCREASINGLY IMPORTANT NLP TRAINING IS SCARCE ON THE GROUND
TIDY DATA PRINCIPLES + COUNT-BASED METHODS = T I D
Y T E X T
https://github.com/juliasilge/tidytext
https://github.com/juliasilge/tidytext
http://tidytextmining.com/
T I D Y T E X T EXPLORATORY DATA
ANALYSIS N-GRAMS AND MORE WORDS MACHINE LEARNING
EXPLORATORY DATA ANALYSIS T I D Y T E X
T
from the Washington Post’s Wonkblog
from the Washington Post’s Wonkblog
D3 visualization on Glitch
WHAT IS A DOCUMENT ABOUT? T I D Y T
E X T TERM FREQUENCY INVERSE DOCUMENT FREQUENCY
None
None
• As part of the NASA Datanauts program, I worked
on a project to understand NASA datasets • Metadata includes title, description, keywords, etc
None
T A K I N G T I D Y
T E X T T O T H E N E X T L E V E L N-GRAMS, NETWORKS, & NEGATION
None
None
None
None
None
T A K I N G T I D Y
T E X T T O T H E N E X T L E V E L TOPIC MODELING
TOPIC MODELING T I D Y T E X T
•Each DOCUMENT = mixture of topics •Each TOPIC = mixture of words
None
None
None
None
T A K I N G T I D Y
T E X T T O T H E N E X T L E V E L TEXT CLASSIFICATION
TRAIN A GLMNET MODEL T I D Y T E
X T
TEXT CLASSIFICATION T I D Y T E X T
> library(glmnet) > library(doMC) > registerDoMC(cores = 8) > > is_jane <- books_joined$title == "Pride and Prejudice" > > model <- cv.glmnet(sparse_words, is_jane, family = "binomial", + parallel = TRUE, keep = TRUE)
None
None
THANK YOU T I D Y T E X T
@juliasilge https://juliasilge.com JULIA SILGE
THANK YOU T I D Y T E X T
@juliasilge https://juliasilge.com Author portraits from Wikimedia Photos by Glen Noble and Kimberly Farmer on Unsplash JULIA SILGE
More Decks by Julia Silge
See All by Julia Silge
Introducing Positron
juliasilge
1
440
The right tool for the job
juliasilge
0
120
Good practices for applied machine learning
juliasilge
0
290
Applied machine learning with tidymodels
juliasilge
0
200
Maintaining an R Package
juliasilge
0
480
Publishing the Stack Overflow Developer Survey
juliasilge
2
120
Text Mining Using Tidy Data Principles
juliasilge
0
220
North American Developer Hiring Landscape
juliasilge
0
110
Understanding Principal Component Analysis Using Stack Overflow Data
juliasilge
13
4.6k
Other Decks in Technology
See All in Technology
OWASP AI Agentic Top10に準拠するクラウドネイティブなガードレールの実践(CNDW 2026)
keitah
1
390
E2E Test というハーネス — AI時代の自動テスト民主化
daishukojima
10
5.9k
おうち環境でも「なるべく止めたくない!」 ― おうち環境の機器冗長化の理想と現実
yuutosi
1
630
New Relicのアラート対応をClaude Codeで自動修正するワークフローを作った話
manabuseki
0
130
AWS Well-Arhitected Frameworkセキュリティの柱 について
miu_crescent
PRO
2
130
Platform Engineering をベースとした Agentic Software Factory の実践
110y
2
760
メルカリの小規模チーム「Pod」で、 エンジニアがデータアナリストなしで、 AIを使ってデータ分析の自走を目指す取り組み
jackojacko_
0
330
社内アプリ基盤「Quartz」による、 非エンジニアとAIに本番デプロイを 任せても壊れないプラットフォームの設計
staka121
PRO
1
140
20261113_JAWS_Osaka
takuyay0ne
1
600
Kubernetes, Agones, WebRTC, GPU で作る クラウドゲーミング基盤を本番運用に乗せるまで
vivion
0
240
Test Impact Analysis を武器に立ち向かう、AI 時代のテスト増加に対するヌーラボのアプローチ / getting-started-datadog-test-impact-analysis
k6s4i53rx
1
290
HTTP通信に使おうとすると実はめんどくさいfile_get_contents
maguroalternative
0
240
Featured
See All Featured
Design in an AI World
tapps
1
390
Code Reviewing Like a Champion
maltzj
529
40k
Java REST API Framework Comparison - PWX 2021
mraible
34
9.8k
Skip the Path - Find Your Career Trail
mkilby
1
300
Organizational Design Perspectives: An Ontology of Organizational Design Elements
kimpetersen
PRO
1
880
How to make the Groovebox
asonas
2
2.6k
コードの90%をAIが書く世界で何が待っているのか / What awaits us in a world where 90% of the code is written by AI
rkaga
63
48k
ラッコキーワード サービス紹介資料
rakko
5
6M
Faster Mobile Websites
deanohume
310
32k
Utilizing Notion as your number one productivity tool
mfonobong
4
650
Principles of Awesome APIs and How to Build Them.
keavy
128
18k
How to Align SEO within the Product Triangle To Get Buy-In & Support - #RIMC
aleyda
2
1.9k