Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Command line & Data Science
Search
Yatish Mehta
October 28, 2014
0
59
Command line & Data Science
Yatish Mehta
October 28, 2014
Tweet
Share
More Decks by Yatish Mehta
See All by Yatish Mehta
Shore | A modern Ruby on Rails template to start your next project
yatish27
0
120
Taming The Rails Monolith Mammoth
yatish27
0
33
ActionCable and ReactJS tie the knot
yatish27
1
270
Featured
See All Featured
RailsConf 2023
tenderlove
30
1.2k
Why Our Code Smells
bkeepers
PRO
340
57k
[Rails World 2023 - Day 1 Closing Keynote] - The Magic of Rails
eileencodes
37
2.6k
Building Adaptive Systems
keathley
44
2.8k
Learning to Love Humans: Emotional Interface Design
aarron
274
41k
The Invisible Side of Design
smashingmag
302
51k
Thoughts on Productivity
jonyablonski
70
4.9k
Distributed Sagas: A Protocol for Coordinating Microservices
caitiem20
333
22k
Build your cross-platform service in a week with App Engine
jlugia
232
18k
Typedesign – Prime Four
hannesfritz
42
2.8k
Designing Dashboards & Data Visualisations in Web Apps
destraynor
231
53k
For a Future-Friendly Web
brad_frost
180
10k
Transcript
Yatish Mehta @yatish27
Command line & Data Science
• pip install csvkit • cat leads.csv | csvlook •
csvstat leads.csv • csvgrep -c 6 -m samplecompany.com | csvlook 1.csvkit
2. grep,sed,sort,uniq • cat wiki.txt | grep -oE '\w+' |
tee words • < words grep '^a' | sort | uniq -c | sort -r • sed ’s/data/tata/g’ wiki.txt > wiki2.txt
• brew install jq • < data.json jq ‘.[]’ •
< data.json jq ‘.[] | select(.age>22)’ • cat data.json | jq '.[] | {isActive: ._id, name: .name}' 3. jq JSON processor
4. qstats • qstats one_hundred_milion.dat Min.
44.947 1st Qu. 93.2553 Median 100.001 Mean 100.001 3rd Qu. 106.747 Max. 156.997 Range 112.05 Std Dev. 10.0002 Length 100000000 • Faster than awk, sort, R
5. parallel • iterative • shell parallel.sh , each action
as a job • parallel keyword
Thank You