Unix Command Line Productivity Tips

Unix Command
Line Productivity
Tips
Keith Bennett
Bennett Business Solutions, Inc.
http://www.bbsinc.biz
1

View Slide

Unix Command Line Productivity Tips http://www.bbsinc.biz Copyright 2008, Bennett Business Solutions, Inc.
Shell Goodness
• The shell enables a huge amount of ﬂexibility, and is only
as complex as you want it to be (i.e. it is simple to use it
simply).
• Available natively on all Unix platforms, including Linux
and Mac OS X.
• Available on Windows by installing Cygwin
(cygwin.com).
• Bash is great, but I like Zshell (zsh) even better. There are
other shells as well.
2

View Slide

Using Shell Commands in Scripting
Languages
In addition to the obvious command line use case, shell
commands can be executed from within programs.
Shell commands can be executed from Ruby scripts, for
example, as a powerful sysadmin tool:
def num_empty_subdirs_of_current_dir
`find . -type d -empty | wc -l`.to_i
end
Certainly it is preferable to use tools in your language for
accomplishing these tasks, if you have them. Nevertheless,
when the target environment is known to have standard
Unix commands, this approach is sensible.
3

View Slide

Chaining and Scripting
Unlike performing tasks in a GUI, using the command line
enables easy chaining and scripting:
Chaining:
> echo $PATH
> echo $PATH | tr : \\n
> echo $PATH | tr : \\n | grep /bin
Scripting:
> for f in **/*; do cp $f $f.bak; done
> for f in **/*.jar; do md5sum $f > $f.md5; done
4

View Slide

Redirection
The output of any command can be redirected to a ﬁle or to
another command, any number of times:
> ls **/*jpg > jpg-dir.txt
> ls | sort -r | less
> echo c:b:b:a | tr : \\n | sort -u > abc.txt
Stderr can also be redirected by using 2> and 2|:
>a-nonexistent-command
zsh: command not found: a-nonexistent-command
>a-nonexistent-command 2> /dev/null
# (produces no output)
5

View Slide

Recursive Directory Listings
Can’t find “foo.txt” in your directory tree? Do:
> ls **/foo.txt
Want to view it? No need to specify its directory. Do:
> less **/foo.txt # or mate, or emacs, or vi...
Want to delete all those *~ files? Do:
> rm **/*~
(Use expansion, tab key in zsh, to view files before
deleting.)
6

View Slide

Recalling Previous Commands
The history command will show you a list of the most
recently used commands, with command sequence
numbers. To specify a previously issued command:
• use the up and down arrow keys to scroll through the
history
• use ”!” and the command number, or “!!” for the most
recent command.
• use [ctrl-r]: recall a previous command containing a
desired string by typing [ctrl-r] and the string. Then
continue to press [ctrl-r] to cycle through the matching
commands. They can be edited.
7

View Slide

Changing Directories
> cd /etc # change to absolute path
> cd mypath # change to relative path
> cd ~ # change to home directory
> cd # same as cd ~ (unlike DOS/Windows)
> cd - # toggles current and previous directory
> pushd/popd # directory stack
If the shell ﬁnds a CDPATH environment variable, then all
directories in that path will be searched for the subdirectory
when a relative cd is executed:
> export CDPATH=~/docs
> cd a_doc_subdir # will work even when not in ~/docs
8

View Slide

mkdir -p
Creates a subdirectory with all necessary intermediate
directories in a single command:
> mkdir -p a/really/deep/directory/tree
A shortcut for capturing the last parameter in the previously
executed command is !$, so you can do this to change to
that new directory:
> cd !$
As stated before, to jump back up, you can do:
> cd -
9

View Slide

man and info
• For more information about a command, run man and/or
info on it:
> man ls
> info ls
• For commands built into the shell, query the documentation
of the shell itself:
> man bash
> man zsh
• In KDE you can get a nicely formatted web page with the
man or info help by typing man:ls, info:ls, etc. in
Konqueror or the Alt-F2 command prompt.
10

View Slide

grep
The grep command ﬁlters text lines to show only those that
match your target string. Common options are:
• -i case insensitive
• -v reverse the ﬁlter (show only nonmatches)
• -C show context lines before and after matching line
• -r recurse directories
> grep localhost /etc/hosts
> echo $PATH | tr : \\n | grep /bin
> alias lsd=”ls -l | grep ^d”
> grep -i i18n *.txt
> grep -ir -C4 install .
11

View Slide

find
Lists all directories and files recursively, with many options:
• -name # name filtering, including regex
• -type # directories, files, links, sockets, and more
• -depth, -mindepth, -maxdepth # directory depth
constraints
• -empty # file or directory is empty
• -exec, -execdir, -ok, -okdir, -delete # run a command on
each entry found
• (others) - file date/time filtering, link count, group
12

View Slide

find
Count all empty directories:
> find . -type d -empty | wc -l
Count and delete them:
> find . -type d -empty -delete -print | wc -l
List info on all files with size > 1 MB (2048 * 512):
> find . -size +2048 -ls
Find files changed <= 30 minutes ago:
> find . -cmin -30
13

View Slide

less
less is a simple text mode file pager. It’s really fast, and
handy for simple text file viewing. less has more features
than more, though on some systems more might be an alias
for less.
> less a-text-file.txt
• g,G - go to start/end of file
• [space], [PgDn], b, [PgUp] - page down/up
• / - find text in a file
• n - find next match in a file
• h - get help
14

View Slide

diff
The diff command outputs the difference, if any, between
two text ﬁles.
> diff before.txt after.txt
See info and man pages for more options such as
whitespace and blank line handling, case insensitivity,
inclusion of context lines, and output format.
15

View Slide

lsof
lsof lists all open files.
• -c option lists all files opened by the command whose
name beings with the specified argument:
> lsof -c thunderbir | wc -l
105
> lsof | wc -l
866
> sudo !!
sudo lsof | wc -l
1817
16

View Slide

du
The du command displays directory usage storage for the
current directory and its subdirectories, listed individually.
• default unit of measure is blocks (typically 1024 bytes).
• -h option displays numbers in more human readable
format (e.g. “6.8 G”).
• -s option displays a single number that is the sum of the
storage of all directories in the tree.
17

View Slide

df
The df command displays the disk usage of the ﬁlesystem.
• -h option displays the sizes in human readable format,
where the units of measure are multiples of 1024.
• -H option displays the sizes in human readable format,
where the units of measure are multiples of 1000.
> df -h | grep disk
/dev/disk0s2 149Gi 137Gi 11Gi 93% /
> df -H | grep disk
/dev/disk0s2 160G 147G 12G 93% /
18

View Slide

tee
tee takes input and sends it both to standard output and to a
file. This allows you to both monitor the output and save it
to a file for later viewing or processing.
> find / | tee ~/allfiles.out
• -a option appends to the output file rather than
overwriting it.
> find dir1 | tee dirs1and2.out
> find dir2 | tee -a dirs1and2.out
19

View Slide

locate
locate searches a data base containing a snapshot of all the
files on your filesystem for a file name or name fragment.
• Because it searches a data base rather than scanning
the file system realtime:
• It is extremely fast.
• It will not reflect changes that occurred to the
filesystem since the filesystem snapshot was made.
• The snapshot is created by the updatedb command
(/usr/libexec/locate.updatedb on OS X).
• On some Linux distributions, the snapshot is
regenerated once a day by default.
20

View Slide

zip / unzip
In addition to tar, the zip and unzip command line tools
can be very handy. To see a zip file's content:
> unzip -v myzipfile.zip
Combine with grep to find files whose names end in “html”:
> unzip -v myzipfile.zip | grep html$
To create a zip file in a backup directory, containing the
current directory and all its subdirectories, do:
> zip -r ~/bu/docs-YYYYMMDD-HHMM.zip *
21

View Slide

Aliases
Aliases provide abbreviations for your commonly used
commands. They can be created on the command line or
in your .bashrc or .zshrc ﬁle.
> alias countlines=”wc -l”
> alias show-empty-dirs="find . -type d -empty"
> alias \
count-empty-dirs="show-empty-dirs | countlines"
> alias showpath='echo $PATH | tr : \\\n'
> alias lsd='ls -l | grep ^d' # list directories
22

View Slide

Environment Variable Overrides
To override the value of an environment variable for a
single command, just specify the desired value before the
command:
> echo "puts ENV['FOO']" > test.rb
> chmod +x test.rb
> ruby test.rb # FOO not defined
nil
> FOO=xyz ruby test.rb # FOO defined on cmd line
xyz
23

View Slide

Symbolic Links
Find yourself often navigating several levels deep, many
times? Create a symbolic link:
> ln -s work/yadameter/biz/bbsinc/yadameter y
When you don’t need it anymore, just remove the link:
> rm y
The directory pointed to will remain untouched; only the
link will be deleted.
24

View Slide

Symbolic Links
Symbolic links can also be used to have a constant name
that always points to the most recent software version. Let’s
say your JRuby versions are installed in /opt. Your most
recent version has been 1.1.1. You have created a symbolic
link, so that your /opt directory contains:
lrwxr-xr-x 1 root staff 11 Apr 26 20:59 jruby
-> jruby-1.1.1
drwxr-xr-x 12 root staff 408 Apr 26 20:56
jruby-1.1.1
Then you download and install 1.1.2, and update the link:
>sudo rm jruby ; sudo ln -s jruby-1.1.2 jruby
25

View Slide

Text Mode Terminals in Linux
To get a text mode terminal, use [Ctrl-Alt-F1]...[Ctrl-Alt-Fn].
This can be useful if you have X Windows problems and
cannot start up your desktop environment.
26

View Slide

Other Handy Tools
• [ctrl-L] or clear – clears the screen
• ncftp – a full featured text mode ftp client with command
recall and other nice features.
• mc – Midnight Commander – if you ever find yourself
having to work on a server without a graphical desktop
environment, this will make file management easier. Old
technology, but can be more productive than raw
commands.
• nc - netcat; like cat, but sends output to, or reads input
from, a TCP/IP port. Combined with tar, can be used to
copy an entire filesystem over a network more efficiently
than a file by file copy.
27

View Slide

Unix Command Line Productivity Tips

Unix Command Line Productivity Tips

keithrbennett

More Decks by keithrbennett

Other Decks in Technology

Featured

Transcript