Unix Command Line Productivity Tips Keith Bennett Bennett Business Solutions, Inc. http://www.bbsinc.biz 1 

Unix Command Line Productivity Tips http://www.bbsinc.biz Copyright 2008, Bennett Business Solutions, Inc. Shell Goodness • The shell enables a huge amount of flexibility, and is only as complex as you want it to be (i.e. it is simple to use it simply). • Available natively on all Unix platforms, including Linux and Mac OS X. • Available on Windows by installing Cygwin (cygwin.com). • Bash is great, but I like Zshell (zsh) even better. There are other shells as well. 2 

Unix Command Line Productivity Tips http://www.bbsinc.biz Copyright 2008, Bennett Business Solutions, Inc. Using Shell Commands in Scripting Languages In addition to the obvious command line use case, shell commands can be executed from within programs. Shell commands can be executed from Ruby scripts, for example, as a powerful sysadmin tool: def num_empty_subdirs_of_current_dir `find . -type d -empty | wc -l`.to_i end Certainly it is preferable to use tools in your language for accomplishing these tasks, if you have them. Nevertheless, when the target environment is known to have standard Unix commands, this approach is sensible. 3 

Unix Command Line Productivity Tips http://www.bbsinc.biz Copyright 2008, Bennett Business Solutions, Inc. Chaining and Scripting Unlike performing tasks in a GUI, using the command line enables easy chaining and scripting: Chaining: > echo $PATH > echo $PATH | tr : \\n > echo $PATH | tr : \\n | grep /bin Scripting: > for f in **/*; do cp $f $f.bak; done > for f in **/*.jar; do md5sum $f > $f.md5; done 4 

Unix Command Line Productivity Tips http://www.bbsinc.biz Copyright 2008, Bennett Business Solutions, Inc. Redirection The output of any command can be redirected to a file or to another command, any number of times: > ls **/*jpg > jpg-dir.txt > ls | sort -r | less > echo c:b:b:a | tr : \\n | sort -u > abc.txt Stderr can also be redirected by using 2> and 2|: >a-nonexistent-command zsh: command not found: a-nonexistent-command >a-nonexistent-command 2> /dev/null # (produces no output) 5 

Unix Command Line Productivity Tips http://www.bbsinc.biz Copyright 2008, Bennett Business Solutions, Inc. Recursive Directory Listings Can’t find “foo.txt” in your directory tree? Do: > ls **/foo.txt Want to view it? No need to specify its directory. Do: > less **/foo.txt # or mate, or emacs, or vi... Want to delete all those *~ files? Do: > rm **/*~ (Use expansion, tab key in zsh, to view files before deleting.) 6 

Unix Command Line Productivity Tips http://www.bbsinc.biz Copyright 2008, Bennett Business Solutions, Inc. Recalling Previous Commands The history command will show you a list of the most recently used commands, with command sequence numbers. To specify a previously issued command: • use the up and down arrow keys to scroll through the history • use ”!” and the command number, or “!!” for the most recent command. • use [ctrl-r]: recall a previous command containing a desired string by typing [ctrl-r] and the string. Then continue to press [ctrl-r] to cycle through the matching commands. They can be edited. 7 

Unix Command Line Productivity Tips http://www.bbsinc.biz Copyright 2008, Bennett Business Solutions, Inc. Changing Directories > cd /etc # change to absolute path > cd mypath # change to relative path > cd ~ # change to home directory > cd # same as cd ~ (unlike DOS/Windows) > cd - # toggles current and previous directory > pushd/popd # directory stack If the shell finds a CDPATH environment variable, then all directories in that path will be searched for the subdirectory when a relative cd is executed: > export CDPATH=~/docs > cd a_doc_subdir # will work even when not in ~/docs 8 

Unix Command Line Productivity Tips http://www.bbsinc.biz Copyright 2008, Bennett Business Solutions, Inc. mkdir -p Creates a subdirectory with all necessary intermediate directories in a single command: > mkdir -p a/really/deep/directory/tree A shortcut for capturing the last parameter in the previously executed command is !$, so you can do this to change to that new directory: > cd !$ As stated before, to jump back up, you can do: > cd - 9 

Unix Command Line Productivity Tips http://www.bbsinc.biz Copyright 2008, Bennett Business Solutions, Inc. man and info • For more information about a command, run man and/or info on it: > man ls > info ls • For commands built into the shell, query the documentation of the shell itself: > man bash > man zsh • In KDE you can get a nicely formatted web page with the man or info help by typing man:ls, info:ls, etc. in Konqueror or the Alt-F2 command prompt. 10 

Unix Command Line Productivity Tips http://www.bbsinc.biz Copyright 2008, Bennett Business Solutions, Inc. grep The grep command filters text lines to show only those that match your target string. Common options are: • -i case insensitive • -v reverse the filter (show only nonmatches) • -C show context lines before and after matching line • -r recurse directories > grep localhost /etc/hosts > echo $PATH | tr : \\n | grep /bin > alias lsd=”ls -l | grep ^d” > grep -i i18n *.txt > grep -ir -C4 install . 11 

Unix Command Line Productivity Tips http://www.bbsinc.biz Copyright 2008, Bennett Business Solutions, Inc. find Lists all directories and files recursively, with many options: • -name # name filtering, including regex • -type # directories, files, links, sockets, and more • -depth, -mindepth, -maxdepth # directory depth constraints • -empty # file or directory is empty • -exec, -execdir, -ok, -okdir, -delete # run a command on each entry found • (others) - file date/time filtering, link count, group 12 

Unix Command Line Productivity Tips http://www.bbsinc.biz Copyright 2008, Bennett Business Solutions, Inc. find Count all empty directories: > find . -type d -empty | wc -l Count and delete them: > find . -type d -empty -delete -print | wc -l List info on all files with size > 1 MB (2048 * 512): > find . -size +2048 -ls Find files changed <= 30 minutes ago: > find . -cmin -30 13 

Unix Command Line Productivity Tips http://www.bbsinc.biz Copyright 2008, Bennett Business Solutions, Inc. less less is a simple text mode file pager. It’s really fast, and handy for simple text file viewing. less has more features than more, though on some systems more might be an alias for less. > less a-text-file.txt • g,G - go to start/end of file • [space], [PgDn], b, [PgUp] - page down/up • / - find text in a file • n - find next match in a file • h - get help 14 

Unix Command Line Productivity Tips http://www.bbsinc.biz Copyright 2008, Bennett Business Solutions, Inc. diff The diff command outputs the difference, if any, between two text files. > diff before.txt after.txt See info and man pages for more options such as whitespace and blank line handling, case insensitivity, inclusion of context lines, and output format. 15 

Unix Command Line Productivity Tips http://www.bbsinc.biz Copyright 2008, Bennett Business Solutions, Inc. lsof lsof lists all open files. • -c option lists all files opened by the command whose name beings with the specified argument: > lsof -c thunderbir | wc -l 105 > lsof | wc -l 866 > sudo !! sudo lsof | wc -l 1817 16 

Unix Command Line Productivity Tips http://www.bbsinc.biz Copyright 2008, Bennett Business Solutions, Inc. du The du command displays directory usage storage for the current directory and its subdirectories, listed individually. • default unit of measure is blocks (typically 1024 bytes). • -h option displays numbers in more human readable format (e.g. “6.8 G”). • -s option displays a single number that is the sum of the storage of all directories in the tree. 17 

Unix Command Line Productivity Tips http://www.bbsinc.biz Copyright 2008, Bennett Business Solutions, Inc. df The df command displays the disk usage of the filesystem. • -h option displays the sizes in human readable format, where the units of measure are multiples of 1024. • -H option displays the sizes in human readable format, where the units of measure are multiples of 1000. > df -h | grep disk /dev/disk0s2 149Gi 137Gi 11Gi 93% / > df -H | grep disk /dev/disk0s2 160G 147G 12G 93% / 18 

Unix Command Line Productivity Tips http://www.bbsinc.biz Copyright 2008, Bennett Business Solutions, Inc. tee tee takes input and sends it both to standard output and to a file. This allows you to both monitor the output and save it to a file for later viewing or processing. > find / | tee ~/allfiles.out • -a option appends to the output file rather than overwriting it. > find dir1 | tee dirs1and2.out > find dir2 | tee -a dirs1and2.out 19 

Unix Command Line Productivity Tips http://www.bbsinc.biz Copyright 2008, Bennett Business Solutions, Inc. locate locate searches a data base containing a snapshot of all the files on your filesystem for a file name or name fragment. • Because it searches a data base rather than scanning the file system realtime: • It is extremely fast. • It will not reflect changes that occurred to the filesystem since the filesystem snapshot was made. • The snapshot is created by the updatedb command (/usr/libexec/locate.updatedb on OS X). • On some Linux distributions, the snapshot is regenerated once a day by default. 20 

Unix Command Line Productivity Tips http://www.bbsinc.biz Copyright 2008, Bennett Business Solutions, Inc. zip / unzip In addition to tar, the zip and unzip command line tools can be very handy. To see a zip file's content: > unzip -v myzipfile.zip Combine with grep to find files whose names end in “html”: > unzip -v myzipfile.zip | grep html$ To create a zip file in a backup directory, containing the current directory and all its subdirectories, do: > zip -r ~/bu/docs-YYYYMMDD-HHMM.zip * 21 

Unix Command Line Productivity Tips http://www.bbsinc.biz Copyright 2008, Bennett Business Solutions, Inc. Aliases Aliases provide abbreviations for your commonly used commands. They can be created on the command line or in your .bashrc or .zshrc file. > alias countlines=”wc -l” > alias show-empty-dirs="find . -type d -empty" > alias \ count-empty-dirs="show-empty-dirs | countlines" > alias showpath='echo $PATH | tr : \\\n' > alias lsd='ls -l | grep ^d' # list directories 22 

Unix Command Line Productivity Tips http://www.bbsinc.biz Copyright 2008, Bennett Business Solutions, Inc. Environment Variable Overrides To override the value of an environment variable for a single command, just specify the desired value before the command: > echo "puts ENV['FOO']" > test.rb > chmod +x test.rb > ruby test.rb # FOO not defined nil > FOO=xyz ruby test.rb # FOO defined on cmd line xyz 23 

Unix Command Line Productivity Tips http://www.bbsinc.biz Copyright 2008, Bennett Business Solutions, Inc. Symbolic Links Find yourself often navigating several levels deep, many times? Create a symbolic link: > ln -s work/yadameter/biz/bbsinc/yadameter y When you don’t need it anymore, just remove the link: > rm y The directory pointed to will remain untouched; only the link will be deleted. 24 

Unix Command Line Productivity Tips http://www.bbsinc.biz Copyright 2008, Bennett Business Solutions, Inc. Symbolic Links Symbolic links can also be used to have a constant name that always points to the most recent software version. Let’s say your JRuby versions are installed in /opt. Your most recent version has been 1.1.1. You have created a symbolic link, so that your /opt directory contains: lrwxr-xr-x 1 root staff 11 Apr 26 20:59 jruby -> jruby-1.1.1 drwxr-xr-x 12 root staff 408 Apr 26 20:56 jruby-1.1.1 Then you download and install 1.1.2, and update the link: >sudo rm jruby ; sudo ln -s jruby-1.1.2 jruby 25 

Unix Command Line Productivity Tips http://www.bbsinc.biz Copyright 2008, Bennett Business Solutions, Inc. Text Mode Terminals in Linux To get a text mode terminal, use [Ctrl-Alt-F1]...[Ctrl-Alt-Fn]. This can be useful if you have X Windows problems and cannot start up your desktop environment. 26 

Unix Command Line Productivity Tips http://www.bbsinc.biz Copyright 2008, Bennett Business Solutions, Inc. Other Handy Tools • [ctrl-L] or clear – clears the screen • ncftp – a full featured text mode ftp client with command recall and other nice features. • mc – Midnight Commander – if you ever find yourself having to work on a server without a graphical desktop environment, this will make file management easier. Old technology, but can be more productive than raw commands. • nc - netcat; like cat, but sends output to, or reads input from, a TCP/IP port. Combined with tar, can be used to copy an entire filesystem over a network more efficiently than a file by file copy. 27