Introduction To Bash Shell Scripting

References

Learning the Bash Shell, Cameron Newham & Bill Rosenblatt (O'Reilly & Associates, Inc)
Linux in a Nutshell, Jessica Perry Hekman (O'Reilly & Associates, Inc)
The UNIX Programming Environment, Brian W Kernigham & Rob Pike (Prenice Hall)
Unix Power Tools, Jerry Peek, Tim O'Relly, & Mike Lookides (O'Reilly & Associates,Inc)
Advance Bash Scripting Guide
Appendix B has some good Reference Cards

Review

Getting Help

man                Get off information about commands
man -k             Get off information via keyword
info               Read info documents
help               Show help on shell built-in commands
command --help     Help on command .i.e.  cp --help   Will give help on cp
bash -c "help"     Short help on bash
bash -c "help set" Short help on bash options

Special Characters

Space             Argument separator
\                 Quote a single character                  A \ followed by a carriage return extend the current line
'                 Quote rext with spaces in it i.e. 'Hello world'
$'                Quote allows string expansion, backslash-escaped characters
"                 Quote rext, expands variables and command substitution
`                 Command substitution i.e. echo "The date is `date`"
$                 Denote a shell variable
#                 Comments the rest of the line
;                 Commands separator
(commands)        Run multiple commands in a subshell
Control-C         Interrupt a command
Control-D         Sends End of File from terminal
Control-U         Erases the entire command line
Control-\         Is a stronger terminate than Control-c
Control-Z          Suspend process

Redirection, Pipes and Filters

-          Standard In for some commands
0          Standard In	
1          Standard Out
2          Standard Error
>2&1       Redirect standard error to standard out
&>         Redirect standard error & standard out
<          Redirect input
>          Redirect output
>>         Concatenate output to file
<<         Here Is File for scripts
|          Pipe
tee        Command write to file and standard out
tee -a FILE #Allow appending to FILE
xargs      Build command from standardin

Wildcards

*          Wildcard for any character(s)
?          Wildcard for single character
[set]      Wildcard for character in set
[^set ]    Wildcard for not the character  in the set
[!set ]    Wildcard for not the character  in the set
{ab,dc}    Wildcard for alternate between commas
All wildcard work with existing files
Only {} alternate work to create files

Process Control

(command1; command)  Run command1, then command2 in subshell
command1&&command    Run command1, then command2 if command1 successes
command1||command    Run command1, then command2 if command1 fails

Regular Expressions

Definition: Text pattern of text character and meta characters
Some Meta charactersEscape character  \
Single Character Meta characters.         Matches any one character
[...]     Matches any one character in a set
[^...]    Matches any one character not in the set
Quantifiers>*       Matches the previous character zero or more times
\{n\}   Matches the previous character n times
\{n,m\} Matches the previous character at least n & at most m
\{n,\}  Matches the previous character n or more times
Anchors^       Matching at the start the line
$       Matching at the end of line
Grouping       \( \)

Commands that use regular expressions

awk        Pattern scanning and text processing language
ed         Line-oriented text editor
egrep      extended grep
emacs      Emacs full screen text editor
ex         Line-oriented text editor
expr       Command evaluates an expression
fgrep      Grep from patterns in a file
gawk       GNU pattern scanning and processing language
grep       Searches file for pattern (also see fgrep & egrep)grep [OPTIONS] PATTERN [INPUT-FILE...]-E        same as egrep
-c        Count
-e        pattern (for multiple pattern on line)
-f        same as fgrep
-i        Ignore case
-l        Only list files containing pattern
-q        Quit (No output, only Return Code)
-v        Invert sense mode
perl      Perl scripting
python    Python scripting
sed        Applies a set of user-specified editing command to a filesed [OPTIONS] 'sed_command' [INPUT_FILE...]-n        Suppress automatic printing
-e        expression - sed_command
substitute other_text for some_textsed 's/some_text/other_text/g' FILE > NEWFILE 
multiple changessed -e 's@abc@def@g' -e 's@xyz@mno@g' FILE
print out line with faq in themsed -n '/faq/p' FILE
change Page ### to (Page ###) at end of linesed 's/Page [0-9]+$/(&)/' file      # & replace the match
delete blank linessed '/^[ \t]*$/d
tcl        Tool command language
vi         Full screen text editor

Shell Variables

Built-in Shell Variables

CDPATH     Path of shortcuts for cd (like PATH)
COLUMNS   Numbers of columns on the display
EDITOR     Path for editor
HISTSIZE   Number of commands in command history (default 500)
IFS        Input Field Separator
LINES      Numbers of lines on the display
OFS        Output Field Separator
SECONDS    Seconds that this shell is running
SHELLOPT   Colon separate list of shell options

Environment Variables

export var Will make a variable an environment variable
HOME       User's home directory
LOGNAME    User's name
MAIL       Name of user's mailbox
PATH       List of directories to be search by the shell
           to find programs whose names are type as commands
PS1        String that is used by the shell prompt
PWD        Name of current directory
SHELL      Name of current shell
TERM       The kind of terminal being used
Environment variable are global to shell and subshells

User Variables

Can be either upper or lower case
var=value  Define a variable
var=""     Define a variable as null
local var  Define a variable local to its scope

Positional Variables

$0        Name of function or script being called
$1 ... $9 Replace by arguments to shell or function
${n}      Replace by n-th arguments to shell or function          required if number of argument is over 9

Special Variables

$?        Exit status or return code of last command
$#        Number of arguments
$@        Argument 1 thru n with Input Field Separator
$*        "$1" $2" ... $n 
$!        Process id of last background process
$$        Process id of shell running this script
$-        The current shell flags

Some Command Useful With Scripts

basename   Strip directory and option suffix
declare    Built-in command declares variable
dirname    Strip non-directory part
echo       Built-in command display message to standard out
echo -n    Built-in command display message to standard out without newline
echo -e    Builtin Command display message to standout with escape sequences
enable     Built-in command to enable/disable (-n) built-in commands
env        Built-in command displays environment variables
eval       Built-in command evaluate arguments before executing results
exec       Built-in command runs command as the shell process
exit       Built-in command exits shell
expr       Evaluates an expression and output its value
false      Built-in command always return false condition
local      Built-in command make variable local
read       Built-in command reads data into variable from standard in
seq        Print a sequence of numbers       seq [OPTION]... LAST
       seq [OPTION]... FIRST LAST
       seq [OPTION]... FIRST INCREMENT LAST
       -w   equalize width by padding with leading zeroes
sleep      Command causes execution to stop for a           specified number of seconds
test       Built-in command tests for various conditions, such as           existence of a file, useful for controlling           conditional script execution
time       Built-in command times commands
times      Print accumulated user and system times.
true       Built-in command always return true condition
type       Built-in command show what word is
wait       wait for the completion of background           processing; is used to ensure that critical           processing is complete before proceeding in a script

Making Scripts Colorful

tput sgr0     #Reset text attributes to normal without clearing screen
Escape sequence to change colors   #\e[${forground};${background}m  example white on black sequence is \e[37;40m  #Forground colors  Backgrounfd Color  black=30;          bgblace=40  red=31;            bgred=41  green=32;          bggreen=42  yellow=33;         bgyellow=43   blue=34;           bgblue=44  magenta=35;        bgmagenta=45  cyan=36;           bgcyan=46  white=37;          bgwhite=47
Turn text attribute off \e[0m
Change text attribute bold \e[1m
Change text attribute underline \e[4m
Change text attributer everse \e[7m

Shell Functions and Scripts

Functions

Function must be sourced just like .bashrc
type function will list the function
function functionname {                        shell commands}
functionname () {                  shell commands}
Example - a helpful functionfunction givehelp { exec $1 --help | more; }

Scripts

#!/bin/bash        Script name run script is run if execute bit is set
#!/bin/bash -x     As above, but script lines are displayed
bash -x script     As above
/usr/bin/env bash  Use the environment bash for the script

Script Debugging options

set -o OPTION  Command line  Action     set -o noexec  sh -n        Don't run command, just check for syntax
set -o verbose sh -v        Echo commands before running them
set -o xtrace  sh -x        Echo commands after running them
set +o OPTION               Turns OPTION off

Operation With Variables

Variable Usage

$var      Value of variable var
"$var"    Is null if variable is undefined          avoids some shell syntax errors if variable is undefined
${var}    Value of variable var          avoids confusion when concatenating with text
${#var}   Gives the length of the string contained in var
${var:FIRST:N} Extract string from var starting at FIRST position           and continuing for N-1 characters.  Note FIRST starts at 0.

Passing an variable to a program or script

Arrays

declare -a name         Declare an array
name[index]=value       Just assigning a value defines it
Index starts at zero
No maximum limit
Need not be contiguous

Setting Variable From Execution of Command

var=`command`           var is set to output of command
var=$(command)          same as above

Arithmetic Operation

$(( expression ))       Almost the same as 'expr expression'
$[ expression ]         Same as above
( expression )          Groups expression within $(( ... ))
expr expression         Note must quote *

Arithmetic operator

+     add
-     substract
*     multiply
/     divide
%     remainder
#     number conversion (Only in bash not expr)

Test Operation

[ -option arg ]         Same as 'test -option arg
[[ -option arg ]]       Same as 'test -option arg
[ arg1 -option arg2 ]   Same as 'test arg1 -option arg2'
[[ arg1 -option arg2 ]] Same as 'test arg1 -option arg2'
See test command for test complete list of optionsstring1 = string2       String 1 equals string 2
string1 !=string2       String 1 not equals string 2
-n string               String is not zero length
-z string               String is zero length
-d FILE                 File is a directory
-e FILE                 File exists
-f FILE                 File exists and is a regular file
-r FILE                 File exists and is readable
-s FILE                 File exists and has length greater than zero
-w FILE                 File exists and is writable
-x FILE                 File exists and is executable
num1 -eq num2           Number 1 equals number 2
num1 -ne num2           Number 1 not equals number 2
num1 -lt num2           Number 1 less than number 2
num1 -le num2           Number 1 less than or equals number 2
num1 -gt num2           Number 1 greater than number 2
num1 -ge num2           Number 1 greater than or equals number 2

String Operation

${varname:-word}        Return var if exists and is not null, else word
${varname:+word}        Return word if var exists and is not null, else null
${varname:?mess}        Return var if exists and is not null, else                        display mess and return from script with error

Pattern Operations

${variable#pattern}     If the pattern matches the beginning of the                        variable value, delete the shortest part  that                        matches and return the rest
${variable##pattern}    If the pattern matches the beginning the of                        variable value, delete the longest part  that                         matches and return the rest
${variable%pattern}     If the pattern matches the end of the variable                        value, delete the shortest part  that matches and                        return the rest
${variable%%pattern}    If the pattern matches the end of the variable                        value, delete the longest part  that matches and                        return the rest

The handy 'expr'

expr string : regrep          Anchored pattern match
expr match string pattern     Same as above
expr substr string pos {len]  Substring beginning at pos of length len
expr length string            Length of string

Flow Control

if - general information

if command               Test return code of commandif command1 && command2  Test return code of command1 and command2if command1 || command2  Test return code of command1 or command2command can be condition i.e. [ $# -eq 0 ]

if/then/else or if/then/elif..

if condition  then statements...  [else statements...]fiif condition  then statements...  [elif condition    then statements...]    [else] statements...]fi

return from Function

return [ numeric expression or variable ]  This is the return code

for

for name [ in list ] do  statementsdonefor  variable = start to end do  statementsdone

while/until

while condition do  statementsdoneuntil condition do  statementsdone

break/continue

break [n]      Break out of loop & select
continue [n]   Continue next iteration of loop

case

case expression in  pattern1[|pattern11] } statements ;;  pattern2[|pattern21] } statements ;;  ...esac

User Interfaces

select

select name [in list] do  statements that can use $namedoneGenerate a menu of each item in the list formatted witha number for each choice
Prompt the user for the number
Store the selected choice in name.  The number is stored in REPLY
execute the statements in the do
Repeat the process again

Command Line Options

shift      Shift 1st argument form the argument list
getopts    Used to process command line options
OPTIND     Variable contains number of options
OPTARG     Default variable for option
Example  while getopts ":ab:c" opt; do    case $opt in      a) process_option_a;;      b) process_option_b         $OPTARG is the option's argument;;      c) process_option_c;;    esac  done  shift $(($OPTIND - 1))

I/O in Scripts

printf        Built-in command for formatted output
read          Built-in command to read one line into variable(s)  read            #Everything entered goes to REPLY  read var        #Everything is read into var  read a b        #Read 1st word in to a and rest into b  read -t 300 var #Read with 300 second timeout
Example how to use read from $file  while read line; do    do_something_with_line  done <$file

Process Handling

Signals

Name  Number  Control CharacterEXIT     0                      Used to trap exiting script
HUP      1                      Logout
INT      2        Control-C
QUIT     3        Control-\
KILL     9  can not be ignored or trapped
TERM    15                      Default kill
TSTP    24        Control-Z

Traps

trap "" signal-list      Ignore signal
trap "cmds" signal-list  Execute commands if signal is caught
trap signal-list         Reset signal to original condition
trap : signal-list       (undocumented) ignore signal, pass to child                         Signal are normally not passed to subprocesses
Examples  trap 'rm tmpfile; exit' 0 1 2 #remove tmpfile on exit, logout, interrupt  trap "echo 'You hit Control-C'" INT  while true ; do    sleep 60  done
Example parent child process#!/bin/bash                   #parentecho parent runningtrap 'echo parent exiting; exit' 0trap :2childsleep 1000#!/bin/bash                    #childecho child started. pid is $$trap 'echo child got signal 2; exit' INTsleep 1000

Examples Useful Scripts or Function

#  Function top5   Example how to set defaults#  Usage   top5 {n}   #list n processesfunction top5 {  ps -ef | head -${1:-5}}
#  Function hereis    Example of HERE IS FILE and handling arguments#  Usage  hereis word1 word2 ...function hereis {  for name in "$@"    do    cat <<MSG        This is an example of an HERE IS FILE.        One argument is ${name}.        The date is `date`.MSG  done}
#  Function pick    Return selected items by user#  Usage:           .e.g  var=`pick *`function pick {  for name in $@ ; do                  #for each item in argument list    echo -n "$name (y/n/q)?" >/dev/tty #ask user to select    read ans                           #read answer from standard in    case $ans in                       #Check choices      y*) echo $name;;                 #selected      q*) break;;                      #skip rest of arguments      *)  continue;;                   #skip item    esac  done}
#  Function acal   Display a nicer calendar#  but will accept Alphabetic monthfunction acal {  m=""  case $# in  0) cal; return;;                #no arguments  1) m=$1; y=`date +%Y`;;         #1 argument  2) m=$1; y=$2;;                 #2 arguments  esac  case $m in  Jan*|jan* ) m=1;;  Feb*|feb* ) m=2;;  Mar*|mar* ) m=3;;  Apr*|apr* ) m=4;;  May|may   } m=5;;  Jun*|jun* ) m=6;;  Jul*|jul* ) m=7;;  Aug*|aug* ) m=8;;  Sep*|sep* } m=9;;  Oct*|oct* ) m=10;;  Nov*|nov* ) m=11;;  Dec*|dec* ) m=12;;  [1-9]|1[0-2] ) ;; #numeric month  *)        ) y=$m; m="";;  esac  cal $m $y}
## Function selectex - Example select#function selectex () {  choices="/bin /usr /home"  select selection in $choices; do if [ $selection ]; then   ls $selection   break  else    echo 'Invalid selection'  fidone}
#  Function fwhich  Which command in $PATH is executed#function fwhich {  if [[ $# -eq 0 ]] ;    then cat << EndOfHelp  1>&2; return 2               Usage fwhich command   #Example of parsing the $PATH                  Return 0 - found                  Return 1 - not found                  Return 2 - No argumentsEndOfHelp  fi  for path in `echo $PATH | sed 's/^:/.:/                              s/::/.:/g                              s/:$/:./                              s/:/ /g'`  do    if [[ -x $path/$1 ]] ;  # does executable file exists here?      then echo $path/$1    # found it      return 0    fi  done  return 1                  # not found}
# Name:         overwrite   Copy standard input to output after EOFfunction overwrite {  if [[ $# -lt 2 ]] ;    then echo "Usage:  overwrite file command [args]" 1>&2; return 2  fi  file=$1; shift  new=/tmp/overwrite1$$; old=/tmp/overwrite2$$  trap 'rm -f $new $old; return 1' 1 2 15  # clean up files  "$@" > $new  if [[ $? -eq 0 ]] ;                      # collect output    then                                   # command completed successfully      cp $file $old                        # save original file      trap '' 1 2 15                       # we are committed; ignore signals      cp $new $file                        # copy new file into file      rm -f $new $old                      # remove temp files    else      echo "overwrite: $1 failed, $file unchanged" 1>$2      return 1  fi}
# Name:         zgrep# Purpose:      caseless grep of gzip files# Usage:        zgrep text files.gz#function zgrep { if [ $# -eq 0 ] ; then   echo "Usage: zgrep grep_text files.gz"    return 2 fi  text=$1  shift  while  [ $# -gt 0 ]    do       echo $1       gzip -cd $1 | grep -i $text       shift    done}
# Name:         hgrep# Purpose:      highlighting grep# Usage:        hgrep pattern files#function hgrep { if [ $# -lt 2 ] ; then   echo "Usage: hgrep pattern files"    return 2 fi  pattern=$1;shift  sep=$'\001'    #note use of $' ' to create control characters  bold=$'\e[1m'; off=$'\e[0m'  underline=$'\e[4m'; reverse=$'\e[7m'  #other choices of highlighting  sed -n "s${sep}${pattern}${sep}${reverse}&${off}${sep}gp" $*}

Introduction To Bash Shell Scripting

References

Review

Getting Help

Special Characters

Redirection, Pipes and Filters

Wildcards

Process Control

Regular Expressions

Shell Variables

Built-in Shell Variables

Environment Variables

User Variables

Positional Variables

Special Variables

Some Command Useful With Scripts

Making Scripts Colorful

Shell Functions and Scripts

Functions

Scripts

Script Debugging options

Operation With Variables

Variable Usage

Passing an variable to a program or script

Arrays

Setting Variable From Execution of Command

Arithmetic Operation

Arithmetic operator

Test Operation

String Operation

Pattern Operations

The handy 'expr'

Flow Control

if - general information

if/then/else or if/then/elif..

return from Function

for

while/until

break/continue

case

User Interfaces

select

Command Line Options

I/O in Scripts

Process Handling

Signals

Traps

Examples Useful Scripts or Function