UNLOCKING THE POWER OF THE UNIX SHELL Renan Ranelli

Milhouse (@renanranelli)

Software Engineer @ Milhouse (@renanranelli)

(former) Software Engineer @ Milhouse (@renanranelli)

Milhouse (@renanranelli)

AGENDA • Why should we understand shell scripting. • A little bit of what I've done with shell. • The main elements of Bash. • Building a very limited shell with Ruby. • Breaking down some complex shell commands.

WHY SHOULD I UNDERSTAND SHELL SCRIPTING?

WHY SHOULD I UNDERSTAND THE SHELL? • You can always count on the shell to be there. • We use it every day. And it is great to “glue” things together • System administrators use it for too many things. You will someday need to understand something they done. • It's a gateway drug to many other interesting things about programming and operating systems. • Its fun.

WHAT CAN BE DONE WITH SHELL?

THINGS I'VE DONE WITH BASH • A Redis client. • A “thread pool”. • A script that interacts with github's api and `git clone`'s all repositories under a person or organization. • A Password manager. • Parallel deployment of our frontend @ Xerpa. • A “framework” for build CI jobs. • Monitoring plugins for `Sensu`.

AND I DOCUMENTED THE EXPERIENCE: • Redis client: – http://milhouseonsoftware.com/2015/07/27/writing-a-redis-client-in-pure-bash-part-1/ – http://milhouseonsoftware.com/2015/08/11/writing-a-redis-client-in-pure-bash-part-2/ • Repository cloner – http://milhouseonsoftware.com/2015/11/20/writing-a-process-pool-in-bash/ – http://milhouseonsoftware.com/2015/12/27/building-a-repository-cloner-with-bash/ • Password manager – http://milhouseonsoftware.com/2015/10/26/write-your-own-password-manager/

This was written in March 2005

THE MAIN ELEMENTS OF BASH

THE MAIN ELEMENTS OF BASH • Shell (interpreter) TTY, Terminal (device) ~ • Bash is untyped. (like assembly, unlike C) • Every value is a string. • Strings are interpreted as commands. • (the best way to really grok Bash is to read its man page. `man bash` is all you need. I'm serious) • `explainshell.com` is pretty good too.

THE MAIN ELEMENTS OF BASH • Bash is untyped. (like assembly, unlike C) • Every value is a string. • Strings are interpreted as commands.

THE MAIN ELEMENTS OF BASH • Programs are composed of commands. Everything in the Shell is a command. • This idea was described by Antirez (Salvatore Sanfilippo) when explaining the semantics of the TCL language: http://antirez.com/articoli/tclmisunderstood.html

THE MAIN ELEMENTS OF BASH

THE MAIN ELEMENTS OF BASH • A given command is subjected to “parameter expansion” and “substitution” prior to being “executed”. • Substitution is very close to what happens in Ruby when you pass “variables” as “arguments” to a method. Its value is substituted where we see the variable “name”. • Expansion is hard and tricky. I won't talk about it now.

THE MAIN ELEMENTS OF BASH • Variable Expansion: – Is exactly what you expect (like in Ruby):

THE MAIN ELEMENTS OF BASH

MEÇA SUAS QUOTES PARÇA

MEÇA SUAS QUOTES PARÇA NOT WHAT YOU WANT

THE “EXECUTABLE” IS NOT SPECIAL

THE MAIN ELEMENTS OF BASH: IO

THE MAIN ELEMENTS OF BASH: IO

IO REDIRECTION

IO REDIRECTION

STRING & PROCESS SUBSTITUTION

THE MAGIC OF $( … )

THE MAGIC OF <( … )

ENVIRONMENT VARIABLES • Every process in your OS has a group of “environment variables”. (You can inspect them reading the file `/proc/$pid/environ`). • When you “fork” a process, the child process “inherits” the values of its parent's environment variables.

I WILL STOP HERE. • There are other aspects to Bash more akin to what we do with “programming languages”, like loops, conditionals and things like that. • I won't talk about those because you already know enough to explore them by yourself. • When in doubt, check the `man page` or the Bash manual. `explainshell.com` is also your friend.

BUILDING OUR OWN SHELL (THAT'S WHERE THE FUN STARTS)

READING SOME (COMPLICATED) BASH PROGRAMS

OBRIGADO !

@renanranelli /rranelli Renan Ranelli (Milhouse) milhouseonsofware.com