Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Sign up for free
Menu
Search
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Pricing
Search
Sign in
Sign up for free
Processando textos enormes com ferramentas "Unix"
Search
Sponsored
·
Your Podcast. Everywhere. Effortlessly.
Share. Educate. Inspire. Entertain. You do you. We'll handle the rest.
→
Luiz Menezes
February 11, 2017
77
0
Share
Embed
Copy iframe code
Copy JS code
Copy link
Start on current slide
Processando textos enormes com ferramentas "Unix"
Palestra dada no evento "Linux em Prosa" do sancaLUG.
Luiz Menezes
February 11, 2017
Transcript
21 slides
processando textos enormes com ferramentas "unix"
oi! eu sou o luiz me encontre em @luiz_amf github.com/lamenezes
abrir/ler e processar textões arquivos de texto com mais de
8 mil 10 milhões de linhas (>4 GB) o problema
None
“ Write programs that do one thing and do it
well Write programs to work together. Write programs to handle text streams, because that is a universal interface. "Unix Philosophy" por Peter Salus
fazer uma coisa e fazer bem a caixa de ferramentas
cat concatenate files and print ▷ visualizar arquivos texto ▷
exemplo $ cat foo.txt
less ▷ visualizar arquivos texto ▷ permite navegação ▷ lê
arquivo enquanto executa ▷ exemplo $ less foo.txt
cp copy ▷ sempre tenha um backup de seus dados
▷ exemplo $ cp foo.txt backup.txt
head & tail ▷ imprime X linhas do arquivo ▷
dividir e conquistar ▷ exemplos $ head foo.txt -n 20 $ tail foo.txt -n 50
▷ qual o tamanho da bronca? ▷ calcula do arquivo
◦ linhas ◦ caracteres/bytes ◦ palavras ▷ exemplo $ wc foo.txt wc word count
▷ remove partes de cada linha de um arquivo ▷
exemplo $ cut -f2,3-5 foo.txt cut
▷ editor "completo" ◦ substituição/remoção de caracteres ◦ duplica linhas
◦ remoção de linhas ◦ busca ▷ exemplo $ sed 's/foo/bar/' foo.txt sed stream editor
▷ busca ▷ exemplo $ grep agulha palheiro.txt grep global
search a regular expression and print
trabalhar bem em conjunto a interface universal de stream de
textos
pipes ▷ encadeamento de comandos
cat <arquivo> | tr <de> <para> ▷ traduz caracteres ▷
deleta ▷ "aperta" tr translate or delete characters
▷ busca com filtros múltiplos cat random.csv | grep AC
| grep "Sr\." cat random.csv | grep João | grep AL ▷ busca + remoção de palavras cat random.csv | grep AC | sed "s/Dr. //" cat random.csv | grep GO | sed "s/Sr. //" pipes
▷ visualizar consumo de memória dos programas ps aux |
sed "s/ \+/\t/g" | cut -f 4,11- | less pipes
obrigado! @luiz_amf github.com/lamenezes
Credits Special thanks to all the people who made and
released these awesome resources for free: ▷ Presentation template by SlidesCarnival
More Decks by Luiz Menezes
See All by Luiz Menezes
async é bom, async eu gosto
luizmenezes
0
64
Testando aplicações web com py.test e selenium
luizmenezes
0
160
Python 3 Orientado a Objetos
luizmenezes
3
340
Expondo o Raspberry Pi via servidor web
luizmenezes
0
110
Workshop Django
luizmenezes
2
270
Bottle Admin
luizmenezes
0
120
IoT, Raspberry Pi e Python
luizmenezes
1
95
Coding Dojo
luizmenezes
0
82
Featured
See All Featured
Designing for Timeless Needs
cassininazir
1
560
Building a Modern Day E-commerce SEO Strategy
aleyda
45
9.3k
The Organizational Zoo: Understanding Human Behavior Agility Through Metaphoric Constructive Conversations (based on the works of Arthur Shelley, Ph.D)
kimpetersen
PRO
0
540
Understanding Cognitive Biases in Performance Measurement
bluesmoon
32
3.1k
Automating Front-end Workflow
addyosmani
1370
210k
SEO Brein meetup: CTRL+C is not how to scale international SEO
lindahogenes
2
3k
Rails Girls Zürich Keynote
gr2m
96
14k
Scaling GitHub
holman
464
140k
Fireside Chat
paigeccino
43
4.1k
The Psychology of Web Performance [Beyond Tellerrand 2023]
tammyeverts
49
3.6k
[SF Ruby Conf 2025] Rails X
palkan
3
1.5k
The Language of Interfaces
destraynor
162
27k