UNIX Shells
CIS 218
shell is a command line interpreter that is the interface between the user and the OS.
It is also a programming language.
• Shells may be used interactively or non-interactively. In interactive mode, they accept
input typed from the keyboard. When executing non-interactively, shells execute
commands read from a file.
• The shell waits for synchronous commands to complete before accepting more input;
asynchronous commands continue to execute in parallel with the shell while it reads
and executes additional commands.
• The shell:
– analyzes each command
– performs substitutions
– determines what actions are to be performed
– performs the actions
• Shells also provide a small set of built-in commands (builtins) implementing
functionality impossible or inconvenient to obtain via separate utilities. For example,
cd, break, continue, and exec) cannot be implemented outside of the shell because
they directly manipulate the shell itself. Builtin commands do not generate separate
Which Shell?
sh – Bourne shell
– Most common, other shells are a superset
csh or tcsh – “C” shell
– C-like syntax
– Used by UNIX programmers
bash –Bourne again shell’ default on Linux
– Based on sh, newer versiond has some csh features.
korn – written by David Korn
– Based on sh – combines features of bash and csh.
PERL – Program Extract and Reporting Language
– sysadmin scripting language pre-dates Internet but used for web programmng.
Common shell facilities
Input-output redirection (STDIN, STDOUT):
prog < infile > outfile
Also << (here document) and >> (append)
Pipeline “|” commands
send the output (STDOUT) from one command to STDIN on a subsequent command.
Semicolon ; allows multiple commands on one line
Job control
- A job is a program whose execution has been initiated by the user.
- Foreground job: a program which has control of the terminal
- Background job: runs concurrently with the parent shell and does not take control of
the keyboard.
- Start a job in the background by appending &
- Commands: ^Z, jobs, fg, bg, kill
File Name Expansion
* stands in for 0 or more (any) characters
? stands in for exactly one (any) character
[..] stands for one character contained in the set of ..
sh[1-6] stands in for one of 1, 2, 3, 4, 5, 6
[^..] stands for one character NOT contained in the set of ..
[^oa] stands in for any char except o or a
~/ stands in for your home directory
- ls *.c
- rm file[1-6].?
- cd ~/bin
- ls ~krueger
- ls *.[^oa]
Shell Commands
You can run any program in a shell by calling it as you would on the command line.
When you run a program like grep or ls in a shell program, a new process is created.
There are also some built-in UNIX commands where no new process is created (in
addition to shell script commands).
Variables are placeholders, named locations in memory, where values are stored.
Whitespace or IFS delineate values (see Quoting)
– [set] name=value – assignment (no spaces around =)
– [set] name= nulls variable, variable exists, no value
– $name – replaced by value of name,
– ${name} – replaced by value of name, {} delineates varaiable name from adjacent
– unset name - to remove variable
Single value:
List of values (separated by spaces):
searchdirs="~/tests $HOME/test2 ."
String Quoting
Scripting languages are all about replacing text or strings. Usually with some form or
Escape “\” quotes the following single character.
Double quotes “” inhibit wildcard replacement only.
Single quotes ‘’ inhibit wildcard replacement, variable substitution and command
Back quotes ``cause command substitution: command output assigned to a variable.
Quoting is used to remove the special meaning of certain characters or words to the
Quoting can be used to disable special treatment for special characters, to prevent
reserved words from being recognized as such, and to prevent parameter expansion.
There are four quoting mechanisms: the escape character, single quotes, double
quotes nad back quotes.
String Quoting
Escape Character: A non-quoted backslash ‘\’ is the Bash escape character. It preserves the
literal value of the next character that follows, with the exception of newline. If a \newline pair
appears, and the backslash itself is not quoted, the \newline is treated as a line continuation (that
is, it is removed from the input stream and effectively ignored).
Single Quotes: Enclosing characters in single quotes (‘’’) preserves the literal value of each
character within the quotes. A single quote may not occur between single quotes, even when
preceded by a backslash.
Double Quotes: Enclosing characters in double quotes (‘"’) preserves the literal value of all
characters within the quotes, with the exception of ‘$’, ‘‘’, ‘\’, and, when history expansion is
enabled, ‘!’.
The characters ‘$’ and ‘‘’ retain their special meaning within double quotes. The backslash retains
its special meaning only when followed by one of the following characters: ‘$’, ‘‘’, ‘"’, ‘\’, or newline.
Within double quotes, backslashes that are followed by one of these characters are removed.
Backslashes preceding characters without a special meaning are left unmodified. A double quote
may be quoted within double quotes by preceding it with a backslash. If enabled, history
expansion will be performed unless an ‘!’ appearing in double quotes is escaped using a
backslash. The backslash preceding the ‘!’ is not removed.
The special parameters ‘*’ and ‘@’ have special meaning when in double quotes,
Command substitution: `` encloses a command and allows re-assignment of command output
to a variable. Example: DATE=`date`; echo $DATE
ANSI-C Quoting
A double-quoted string preceded by a dollar sign (‘$’) will cause the string to be
translated according to the current locale. If the current locale is C or POSIX, the
dollar sign is ignored. If the string is translated and replaced, the replacement is
Words of the form $'string' are treated specially. The word expands to string, with
backslash-escaped characters replaced as specified by the ANSI C standard.
Backslash escape sequences, if present, are decoded as follows:
ANSI-C Characters (man echo)
\a - alert (bell)
\b - backspace
\e - an escape character (not ANSI C)
\f - form feed
\n - newline
\r - carriage return
\t - horizontal tab
\v -vertical tab
\\ - backslash
\' -single quote
\nnn - the eight-bit character whose value is the octal value nnn (one to three digits)
\xHH -the eight-bit character whose value is the hexadecimal value HH (one or two hex digits)
\cx - a control-x character
SHELL Operation
Reads its input from a file from a string supplied as an argument to the ‘-c’ invocation option or
from the user’s terminal.
Breaks the input into words and operators, obeying the quoting rules separated by whitespace or
command separators (; | <LF>). Alias expansion is performed by this step
Parses the tokens into simple and compound commands. The first word or token in each string is
the command.
Performs the various shell expansions breaking the expanded tokens into lists of filenames and
commands and arguments.
Performs any necessary redirections and removes the redirection operators and their operands
from the argument list.
Executes the command.
Optionally waits for the command to complete and collects its exit status,
When you give the shell a command, it forks (spawns) a child process to execute the command.
While the child process is executing the command, the parent process sleeps. While a process is
sleeping, it does not use any computer time but remains inactive, waiting to wake up. When the
child process finishes executing the command, it tells its parent of its success or failure via its exit
status and then dies. The parent process (which is running the shell) wakes up and prompts for
another command.
Shell built-in commands do not spawn child processes. Neither do UNIX built-in commands
(previous slide).
Commands from the SHELL
A simple command is the kind of command encountered most often. It's just a
sequence of words separated by blanks, terminated by one of the shell's control
operators – usually RETURN or EOF <LF> character.
A pipeline is a sequence of simple commands separated by one of the control
operators ‘|’ or ‘|&’.
The format for a pipeline is
command1 [ [| or |&] command2 ...]
The output of each command in the pipeline is connected via a pipe to the input of
the next command. That is, each command reads the previous command's output.
This connection is performed before any redirections specified by the command.
With ‘|&’ is used, the standard error of command1 is connected to command2's
standard input through the pipe; it is shorthand for 2>&1 |. This implicit redirection of
the standard error is performed after any redirections specified by the command.
Commands from the SHELL
A command list is a sequence of one or more pipelines separated by one of the
operators ‘;’, ‘&’, ‘&&’, or ‘||’, and terminated one of ‘;’, ‘&’, or a newline <LF>
Of the list operators, ‘&&’ and ‘||’ have equal precedence, followed by ‘;’ and ‘&’, which
have equal precedence. Parentheses () can be used to group commands as a
single process or change precedence order within a string of commands.
A sequence of one or more newlines may appear in a list to delimit commands,
equivalent to a semicolon.
If a command is terminated by the control operator ‘&’, the shell executes the
command asynchronously in a subshell. This is known as executing the command in
the background. The shell does not wait for the command to finish, and the return
status is 0 (true).
The semicolon (;) is a command separator executes a series of commands
sequentially by entering them on a single command line and separating each from the
next with a semicolon (;). You initiate execution of the sequence of commands by
pressing RETURN: $ x ; y ; z<LF>
Commands from the SHELL
Commands separated by a ‘;’ are executed sequentially; the shell waits for each
command to terminate in turn. The return status is the exit status of the last
command executed.
AND and OR lists are sequences of one or more pipelines separated by the control
operators ‘&&’ and ‘||’, respectively. AND and OR lists are executed with left
Parentheses () can be used to group commands as a single process or
change precedence order within a string of commands.
An AND list has the form: command1 && command2 command2 is executed if,
and only if, command1 returns an exit status of zero.
An OR list has the form| command1 || command2 command2 is executed if, and
only if, command1 returns a non-zero exit status.
The return status of AND and OR lists is the exit status of the last command executed
in the list.
Commands from the SHELL
When you enter a long command line and the cursor reaches the right side of the
screen, you can use a backslash (\) character to continue the command on the next
line. The backslash quotes, or escapes, the NEWLINE character that follows it so that
the shell does not treat the NEWLINE as a command terminator. The > is a
secondary prompt (PS2) indicating that the shell is waiting for you to continue the
unfinished command.
You can use parentheses to group commands. The shell creates a copy of itself,
called a subshell, for each group. It treats each group of commands as a job and
creates a new process to execute each command .
Each subshell (job) has its own environment, meaning that it has its own set of
variables with values that can differ from those of other subshells. The following
command line executes commands a and b sequentially in the background while
executing c in the background. The shell prompt returns immediately. $ (a ; b) & c &
Shell (meta)characters
initiates execution of a command
Separates commands
Groups commands for execution by a subshell or identifies a function
Executes a command in the background
Sends output through a pipe
Redirects standard output
Appends standard output
Redirects standard input
Redirect the standard output and standard error to replace current contents.
Redirect the standard output an standard error to append to current contents.
Here document
Any string of zero or more characters in an ambiguous file reference
Any single character in an ambiguous file reference
Quotes the following character
Single quotes a string, preventing all substitution
Double quotes a string, allowing variable and command substitution
Performs command substitution
Character class in an ambiguous file reference
References a variable
Executes a command (only at the beginning of a line)
Begins a comment
Used to surround the contents of a function
null builtin Returns true
Boolean AND Executes command on right only if command on left succeeds (returns a zero exit status)
Boolean OR Executes command on right only if command on left fails (returns a nonzero exit status)
Boolean NOT Reverses exit status of a command
Performs command substitution (preferred form)
Evaluates an arithmetic expression

Slide 1