Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
The state of NLP in production 🥽
Search
Abdur-Rahmaan Janhangeer
August 27, 2023
Programming
0
130
The state of NLP in production 🥽
NLP in production vs real life
Abdur-Rahmaan Janhangeer
August 27, 2023
Tweet
Share
More Decks by Abdur-Rahmaan Janhangeer
See All by Abdur-Rahmaan Janhangeer
Building AI Agents with Python: A Deep Dive
osdotsystem
0
22
Extending Flask using the Flask Plugins API
osdotsystem
0
110
PEPs that hit the front page
osdotsystem
0
88
libSQL: Taking Sqlite To The Moon
osdotsystem
0
200
Boosting Python With Rust 🚀
osdotsystem
0
190
Flet: Flutter in Python
osdotsystem
0
420
SQLite Internals: How The World's Most Used Database Works
osdotsystem
2
3.7k
Fast Flask Dev For Big Codebases
osdotsystem
0
220
Python Bytecode or How Python Operates
osdotsystem
0
300
Other Decks in Programming
See All in Programming
Startups on Rails in Past, Present and Future–Irina Nazarova, RailsConf 2025
irinanazarova
0
290
CDK引数設計道場100本ノック
badmintoncryer
2
560
可変変数との向き合い方 $$変数名が踊り出す$$ / php conference Variable variables
gunji
0
220
ISUCON研修おかわり会 講義スライド
arfes0e2b3c
1
480
ソフトウェア設計とAI技術の活用
masuda220
PRO
25
6.7k
はじめてのWeb API体験 ー 飲食店検索アプリを作ろうー
akinko_0915
0
160
Git Sync を超える!OSS で実現する CDK Pull 型デプロイ / Deploying CDK with PipeCD in Pull-style
tkikuc
4
450
PHPカンファレンス関西2025 基調講演
sugimotokei
5
930
코딩 에이전트 체크리스트: Claude Code ver.
nacyot
0
990
階層化自動テストで開発に機動力を
ickx
1
410
PHPUnitの限界をPlaywrightで補完するテストアプローチ
yuzneri
0
270
MCPを使ってイベントソーシングのAIコーディングを効率化する / Streamlining Event Sourcing AI Coding with MCP
tomohisa
0
180
Featured
See All Featured
Stop Working from a Prison Cell
hatefulcrawdad
271
21k
Refactoring Trust on Your Teams (GOTO; Chicago 2020)
rmw
34
3.1k
How to Create Impact in a Changing Tech Landscape [PerfNow 2023]
tammyeverts
53
2.9k
Producing Creativity
orderedlist
PRO
346
40k
Reflections from 52 weeks, 52 projects
jeffersonlam
351
21k
Put a Button on it: Removing Barriers to Going Fast.
kastner
60
3.9k
Product Roadmaps are Hard
iamctodd
PRO
54
11k
Building an army of robots
kneath
306
45k
Adopting Sorbet at Scale
ufuk
77
9.5k
実際に使うSQLの書き方 徹底解説 / pgcon21j-tutorial
soudai
PRO
181
54k
Responsive Adventures: Dirty Tricks From The Dark Corners of Front-End
smashingmag
251
21k
Scaling GitHub
holman
461
140k
Transcript
The state of NLP in production
None
Python Mauritius Usergroup site fb linkedin mailing list 3
url pymug.com site 4
About me compileralchemy.com 5
slides 6
The state of NLP in production 7
Hardest part of a real-world project 8
? 9
Is it cooking up an awesome model? 10
No, the world is more complex than this 11
Elements of an NLP project 12
NLP project gather data clean store train use model retrain
model 13
gather data 14
Toy project use curated data set quick extraction 15
Real project a lot of data needed data corresponds to
business case. data probably does not exist speed of data gathering find ingenious / better ways of getting data automate collection 16
clean/preprocess data 17
Toy project use an existing parser / curator e.g. NLTK
existing options 18
Real project use a parser intended for it, several custom
steps parallel processing of data 19
store data 20
Toy project laptop 21
Real project cloud database hot / cold data TTL 22
training 23
Toy project use laptop / external GPU 24
Real project on cloud training on cloud knowledge cross-cloud skills
fault tolerance 25
use model 26
Toy project local website / code 27
Real project continuation of pipeline web service architecture devops /
deploy 28
retraining 29
Toy project euhh this even exists???? 30
Real project learn cloud offerings for continuous learning ways to
retrain / fine tune 31
It's more than serving a model 32
Operation model 33
[ pipeline ] data collection --- process --- train -<-
| | --------------------------- model ^ | | | | --->--- V web service [pod] [pod] --- happy user | -> users service [pod] [pod] | -> db service [pod] 34
skills chart 35
skills --------------- --------------- | | | | | backend |
| devops | | | | | --------------- --------------- --------------- --------------- | | | | | backend | | data eng | | | | | --------------- --------------- 36
skills --------------- --------------- | | | | | backend |
| devops | | | | | --------------- --------------- web service deploy --------------- --------------- | | | | | ml | | data eng | | | | | --------------- --------------- models pipelining 37
code blueprint [ architecture repos ] [ pipeline repos ]
[ ml repos ] [ backend repos ] 38
Tools 39
Pandas Good queries Much resources Read SQL 40
Dask Good for it's purpose: Parallelize tasks Poor docs 41
Polars Awesome parallelizations Great docs 42
NLTK use spacy if possible 43
Notebooks great for cloud used in production on the cloud
44
Advice to research / scientists folks keep everything clean people
will come after you always in hurry / messy / i'll clean it later mood good practices? is this phrase in the korean dictionary? 45
General advices have great docs good onboarding have great standards
46
Keep learning! 47