Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Command line & Data Science
Search
Yatish Mehta
October 28, 2014
0
59
Command line & Data Science
Yatish Mehta
October 28, 2014
Tweet
Share
More Decks by Yatish Mehta
See All by Yatish Mehta
Shore | A modern Ruby on Rails template to start your next project
yatish27
0
110
Taming The Rails Monolith Mammoth
yatish27
0
32
ActionCable and ReactJS tie the knot
yatish27
1
270
Featured
See All Featured
Cheating the UX When There Is Nothing More to Optimize - PixelPioneers
stephaniewalter
282
13k
Raft: Consensus for Rubyists
vanstee
140
7k
Agile that works and the tools we love
rasmusluckow
329
21k
Improving Core Web Vitals using Speculation Rules API
sergeychernyshev
18
990
How to train your dragon (web standard)
notwaldorf
96
6.1k
The Cult of Friendly URLs
andyhume
79
6.5k
Bash Introduction
62gerente
613
210k
Chrome DevTools: State of the Union 2024 - Debugging React & Beyond
addyosmani
7
750
Designing Experiences People Love
moore
142
24k
Optimising Largest Contentful Paint
csswizardry
37
3.3k
No one is an island. Learnings from fostering a developers community.
thoeni
21
3.4k
The Success of Rails: Ensuring Growth for the Next 100 Years
eileencodes
45
7.5k
Transcript
Yatish Mehta @yatish27
Command line & Data Science
• pip install csvkit • cat leads.csv | csvlook •
csvstat leads.csv • csvgrep -c 6 -m samplecompany.com | csvlook 1.csvkit
2. grep,sed,sort,uniq • cat wiki.txt | grep -oE '\w+' |
tee words • < words grep '^a' | sort | uniq -c | sort -r • sed ’s/data/tata/g’ wiki.txt > wiki2.txt
• brew install jq • < data.json jq ‘.[]’ •
< data.json jq ‘.[] | select(.age>22)’ • cat data.json | jq '.[] | {isActive: ._id, name: .name}' 3. jq JSON processor
4. qstats • qstats one_hundred_milion.dat Min.
44.947 1st Qu. 93.2553 Median 100.001 Mean 100.001 3rd Qu. 106.747 Max. 156.997 Range 112.05 Std Dev. 10.0002 Length 100000000 • Faster than awk, sort, R
5. parallel • iterative • shell parallel.sh , each action
as a job • parallel keyword
Thank You