Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Command line & Data Science
Search
Yatish Mehta
October 28, 2014
0
58
Command line & Data Science
Yatish Mehta
October 28, 2014
Tweet
Share
More Decks by Yatish Mehta
See All by Yatish Mehta
Shore | A modern Ruby on Rails template to start your next project
yatish27
0
110
Taming The Rails Monolith Mammoth
yatish27
0
32
ActionCable and ReactJS tie the knot
yatish27
1
270
Featured
See All Featured
How to Create Impact in a Changing Tech Landscape [PerfNow 2023]
tammyeverts
53
2.8k
Designing for humans not robots
tammielis
253
25k
Design and Strategy: How to Deal with People Who Don’t "Get" Design
morganepeng
130
19k
Optimising Largest Contentful Paint
csswizardry
37
3.3k
Why Our Code Smells
bkeepers
PRO
337
57k
StorybookのUI Testing Handbookを読んだ
zakiyama
30
5.9k
A Modern Web Designer's Workflow
chriscoyier
694
190k
Intergalactic Javascript Robots from Outer Space
tanoku
271
27k
A Tale of Four Properties
chriscoyier
160
23k
ReactJS: Keep Simple. Everything can be a component!
pedronauck
667
120k
Being A Developer After 40
akosma
90
590k
Building a Modern Day E-commerce SEO Strategy
aleyda
42
7.4k
Transcript
Yatish Mehta @yatish27
Command line & Data Science
• pip install csvkit • cat leads.csv | csvlook •
csvstat leads.csv • csvgrep -c 6 -m samplecompany.com | csvlook 1.csvkit
2. grep,sed,sort,uniq • cat wiki.txt | grep -oE '\w+' |
tee words • < words grep '^a' | sort | uniq -c | sort -r • sed ’s/data/tata/g’ wiki.txt > wiki2.txt
• brew install jq • < data.json jq ‘.[]’ •
< data.json jq ‘.[] | select(.age>22)’ • cat data.json | jq '.[] | {isActive: ._id, name: .name}' 3. jq JSON processor
4. qstats • qstats one_hundred_milion.dat Min.
44.947 1st Qu. 93.2553 Median 100.001 Mean 100.001 3rd Qu. 106.747 Max. 156.997 Range 112.05 Std Dev. 10.0002 Length 100000000 • Faster than awk, sort, R
5. parallel • iterative • shell parallel.sh , each action
as a job • parallel keyword
Thank You