Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
The Great Language Game
Search
Lars Yencken
September 02, 2013
Programming
0
310
The Great Language Game
A brief introduction to the Great Language Game, given to the Melbourne Python User Group.
Lars Yencken
September 02, 2013
Tweet
Share
More Decks by Lars Yencken
See All by Lars Yencken
Linguistics, a whirlwind tour!
larsyencken
0
53
Pycon 2014 Recap
larsyencken
0
62
Nine months of food
larsyencken
0
270
Automation for web development
larsyencken
0
150
Scaling a web stack
larsyencken
4
200
Similarity metrics for Japanese kanji
larsyencken
0
81
Other Decks in Programming
See All in Programming
Let's Write a Train Tracking Algorithm
twocentstudios
0
220
XP, Testing and ninja testing ZOZ5
m_seki
2
240
Model Pollution
hschwentner
1
180
Build your own WebP codec in Swift
kishikawakatsumi
2
890
CSC305 Lecture 01
javiergs
PRO
1
380
CSC305 Lecture 02
javiergs
PRO
1
260
非同期jobをtransaction内で 呼ぶなよ!絶対に呼ぶなよ!
alstrocrack
0
470
Local Peer-to-Peer APIはどのように使われていくのか?
hal_spidernight
2
440
AccessorySetupKitで実現するシームレスなペアリング体験 / Seamless pairing with AccessorySetupKit
nekowen
0
210
パフォーマンスチューニングで Web 技術を深掘り直す
progfay
18
4.9k
ИИ-Агенты в каждый дом – Алексей Порядин, PythoNN
sobolevn
0
150
あなたの知らない「動画広告」の世界 - iOSDC Japan 2025
ukitaka
0
350
Featured
See All Featured
Optimizing for Happiness
mojombo
379
70k
Typedesign – Prime Four
hannesfritz
42
2.8k
A designer walks into a library…
pauljervisheath
208
24k
The MySQL Ecosystem @ GitHub 2015
samlambert
251
13k
How to Ace a Technical Interview
jacobian
280
23k
I Don’t Have Time: Getting Over the Fear to Launch Your Podcast
jcasabona
33
2.4k
Cheating the UX When There Is Nothing More to Optimize - PixelPioneers
stephaniewalter
285
14k
Bootstrapping a Software Product
garrettdimon
PRO
307
110k
GraphQLとの向き合い方2022年版
quramy
49
14k
GraphQLの誤解/rethinking-graphql
sonatard
72
11k
Keith and Marios Guide to Fast Websites
keithpitt
411
22k
The Art of Delivering Value - GDevCon NA Keynote
reverentgeek
15
1.7k
Transcript
. . . .. . . . .. . .
. .. . . . . .. . . . .. . . . .. . . . .. . . . . .. . . . .. . . . .. . . . .. . . . . .. . . . .. . . . .. . . . .. . . . . .. . . . .. . . . . .. . . . .. . . . .. . The Great Language Game Lars Yencken Melbourne Python User Group September 2, 2013
. . . .. . . . .. . .
. .. . . . . .. . . . .. . . . .. . . . .. . . . . .. . . . .. . . . .. . . . .. . . . . .. . . . .. . . . .. . . . .. . . . . .. . . . .. . . . . .. . . . .. . . . .. . I’m a language geek
. . . .. . . . .. . .
. .. . . . . .. . . . .. . . . .. . . . .. . . . . .. . . . .. . . . .. . . . .. . . . . .. . . . .. . . . .. . . . .. . . . . .. . . . .. . . . . .. . . . .. . . . .. . I’m a human language geek
. . . .. . . . .. . .
. .. . . . . .. . . . .. . . . .. . . . .. . . . . .. . . . .. . . . .. . . . .. . . . . .. . . . .. . . . .. . . . .. . . . . .. . . . .. . . . . .. . . . .. . . . .. . The world has something like 7,000 languages
. . . .. . . . .. . .
. .. . . . . .. . . . .. . . . .. . . . .. . . . . .. . . . .. . . . .. . . . .. . . . . .. . . . .. . . . .. . . . .. . . . . .. . . . .. . . . . .. . . . .. . . . .. . The world has something like 7,000 languages So many!
. . . .. . . . .. . .
. .. . . . . .. . . . .. . . . .. . . . .. . . . . .. . . . .. . . . .. . . . .. . . . . .. . . . .. . . . .. . . . .. . . . . .. . . . .. . . . . .. . . . .. . . . .. . The world has something like 7,000 languages Too many to learn!
. . . .. . . . .. . .
. .. . . . . .. . . . .. . . . .. . . . .. . . . . .. . . . .. . . . .. . . . .. . . . . .. . . . .. . . . .. . . . .. . . . . .. . . . .. . . . . .. . . . .. . . . .. . But... with the help of a lil Python we can at least learn to tell the difference between languages
. . . .. . . . .. . .
. .. . . . . .. . . . .. . . . .. . . . .. . . . . .. . . . .. . . . .. . . . .. . . . . .. . . . .. . . . .. . . . .. . . . . .. . . . .. . . . . .. . . . .. . . . .. . Aside: langid.py Distinguish between languages in text form
. . . .. . . . .. . .
. .. . . . . .. . . . .. . . . .. . . . .. . . . . .. . . . .. . . . .. . . . .. . . . . .. . . . .. . . . .. . . . .. . . . . .. . . . .. . . . . .. . . . .. . . . .. . Aside: langid.py Distinguish between languages in text form ผู้สื่อข่าวไทยวิเคราะห์นโยบายผู้ขอลี้ภัยพรรคต่างๆ >>> import langid >>> langid.classify(l.encode(’utf8’)) (’th’, 1.0) >>> langid.classify(’¡Venga hombre!’) (’es’, 0.5726778160604622)
. . . .. . . . .. . .
. .. . . . . .. . . . .. . . . .. . . . .. . . . . .. . . . .. . . . .. . . . .. . . . . .. . . . .. . . . .. . . . .. . . . . .. . . . .. . . . . .. . . . .. . . . .. . First attempt: streaming radio ▶ There’s lots of internet radio out there! ▶ But it’s all in shitty old formats ▶ And Python support for decoding them all is not great ▶ Solution: sh module and mplayer ▶ Still too hard!
. . . .. . . . .. . .
. .. . . . . .. . . . .. . . . .. . . . .. . . . . .. . . . .. . . . .. . . . .. . . . . .. . . . .. . . . .. . . . .. . . . . .. . . . .. . . . . .. . . . .. . . . .. . Second attempt: scrape SBS ▶ Podcasts News podcasts in about 70 languages ▶ Good quality recordings! ▶ (Sometimes) daggy Australian accents ▶ Fetching: pyquery, requests and parse ▶ Processing audio: wave + sh wrapping avconv and mp3gain ▶ Success!
. . . .. . . . .. . .
. .. . . . . .. . . . .. . . . .. . . . .. . . . . .. . . . .. . . . .. . . . .. . . . . .. . . . .. . . . .. . . . .. . . . . .. . . . .. . . . . .. . . . .. . . . .. . Aside: sh Wraps shell calls like a boss! >>> from sh import ffmpeg >>> ffmpeg(’-i’, input_file, output_file) >>> from sh import mp3gain >>> mp3gain(’-r’, ’-k’, ’-t’, ’-s’, ’r’, sound_file)
. . . .. . . . .. . .
. .. . . . . .. . . . .. . . . .. . . . .. . . . . .. . . . .. . . . .. . . . .. . . . . .. . . . .. . . . .. . . . .. . . . . .. . . . .. . . . . .. . . . .. . . . .. . More about languages ▶ Wikipedia: manual data entry ▶ Freebase API: via requests
. . . .. . . . .. . .
. .. . . . . .. . . . .. . . . .. . . . .. . . . . .. . . . .. . . . .. . . . .. . . . . .. . . . .. . . . .. . . . .. . . . . .. . . . .. . . . . .. . . . .. . . . .. . End result: demo time!
. . . .. . . . .. . .
. .. . . . . .. . . . .. . . . .. . . . .. . . . . .. . . . .. . . . .. . . . .. . . . . .. . . . .. . . . .. . . . .. . . . . .. . . . .. . . . . .. . . . .. . . . .. . Thanks http://greatlanguagegame.com/