Statistics Function
Implementation
Traditional Programming Approach
Rolando V. Raqueño
Sunday, October 04, 2015
minimum.pro
function minimum, x
n = n_elements(x)
answer = x(0)
for i=1L, n-1 do begin
if( answer gt x(i) ) then $
answer = x(i)
endfor
return, answer
end
Rolando V. Raqueño
Sunday, October 04, 2015
sum.pro
function sum, x
n = n_elements( x )
answer = 0
for i = 0, n-1 do begin
answer = answer + x(i)
endfor
return, answer
end
Rolando V. Raqueño
Sunday, October 04, 2015
Overflow Problems
• What if the array is an integer array?
• Sums will have a maximum limit based on
the number of bits specified by the integer
type.
Rolando V. Raqueño
Sunday, October 04, 2015
Corrected sum.pro
function sum, x
n = n_elements( x )
answer = 0.0D
for i = 0L, n-1 do begin
answer = answer + x(i)
endfor
return, answer
end
Rolando V. Raqueño
Sunday, October 04, 2015
mean.pro
function mean, x
n = n_elements(x)
answer = sum(x) / n
return, answer
end
Rolando V. Raqueño
Sunday, October 04, 2015
Integer Problems
IDL> b=9/5
IDL> print,b
1
Rolando V. Raqueño
Sunday, October 04, 2015
Integer Problems
IDL> b=9/double(5)
IDL> print,b
1.8000000
Rolando V. Raqueño
Sunday, October 04, 2015
Corrected mean.pro
function mean, x
n = n_elements(x)
answer = sum(x) / double(n)
return, answer
end
Rolando V. Raqueño
Sunday, October 04, 2015
sum_squares.pro
function sum_squares, x
squares = x^2.0
answer= sum( squares )
return, answer
end
Rolando V. Raqueño
Sunday, October 04, 2015
variance.pro
function variance, x
n = double(n_elements(x))
answer=(sum_squares(x)-(sum(x)^2.0/n))/
(n-1)
return, answer
end
Rolando V. Raqueño
Sunday, October 04, 2015
sd.pro
function sd, x
answer = sqrt(variance(x))
return, answer
end
Rolando V. Raqueño
Sunday, October 04, 2015
Shell Programming
Rolando V. Raqueño
Sunday, October 04, 2015
UNIX Tip of the Day
• Directory maneuvering commands pushd,
popd, and, dirs
% cd /usr/tmp
% pwd
/usr/tmp
% pushd ~rvrpci
% pwd
/cis/staff/rvrpci
Rolando V. Raqueño
Sunday, October 04, 2015
UNIX Tip of the Day
% dirs
/cis/staff/rvrpci /usr/tmp
% pushd
% pwd
/usr/tmp
% dirs
/usr/tmp /cis/staff/rvrpci
% pushd /usr/local/bin
Rolando V. Raqueño
Sunday, October 04, 2015
UNIX Tip of the Day
% dirs
/usr/local/bin /usr/tmp
/cis/staff/rvrpci
% pwd
/usr/local/bin
% pushd
% dirs
/usr/tmp /usr/local/bin
/cis/staff/rvrpci
Rolando V. Raqueño
Sunday, October 04, 2015
UNIX Tip of the Day
% dirs
/usr/tmp /usr/local/bin
/cis/staff/rvrpci
% pwd
/usr/tmp
% pushd +2
% pwd
/cis/staff/rvrpci
Rolando V. Raqueño
Sunday, October 04, 2015
UNIX Tip of the Day
% dirs
/cis/staff/rvrpci /usr/tmp
/usr/local/bin
% popd
% dirs
/usr/tmp /usr/local/bin
Rolando V. Raqueño
Sunday, October 04, 2015
IMPORTANT UNIX Concepts
• Environment and Shell Variables
– These allow you to customize your UNIX
environment
– They are different in terms of their SCOPE
• SCOPE determines the visibility of a variable
Rolando V. Raqueño
Sunday, October 04, 2015
Other IMPORTANT UNIX Concepts
• Environment Variable
– Examples are TERM and DISPLAY
– Set a particular variable to a value by using the
setenv command
– You can print the value of a particular variable
or all the environment variable using the
printenv command
Rolando V. Raqueño
Sunday, October 04, 2015
% Environment Variables %
• Examples
– To set environment variables
% setenv TERM vt100
% setenv DOG Goofy
– print out the terminal type
% printenv TERM
vt100
– print out all environment variables
% printenv
Rolando V. Raqueño
Sunday, October 04, 2015
Shell Variables
• Shell variables are similar to Environment
variables except they have a limited scope,
i.e., they exist only in the shell which they
are defined.
• Environment variables on the other hand,
exist in all its children shells
• To illustrate this concept, let us look at the
following example
Rolando V. Raqueño
Sunday, October 04, 2015
Environment vs. Shell Variables
% set prompt
Parent Shell
Parent Shell
Parent Shell
Goofy
Parent Shell
Mickey
Parent Shell
=
>
>
>
"Parent Shell > "
setenv DOG Goofy
set mouse=Mickey
printenv DOG
> echo $mouse
> xterm &
(YOU SHOULD NOW HAVE A NEW xterm WINDOW)
THIS IS KNOWN AS
“SPAWNING A NEW (OR CHILD) PROCESS”
Rolando V. Raqueño
Sunday, October 04, 2015
Environment vs. Shell Variables
(IN THE NEW xterm WINDOW, DO THE FOLLOWING)
% set prompt = "Child Shell > "
Child Shell > printenv DOG
Goofy
Child Shell > echo $mouse
mouse: Undefined variable.
Rolando V. Raqueño
Sunday, October 04, 2015
Environment vs. Shell Variables
Child Shell
Child Shell
Child Shell
Pluto
Child Shell
Minnie
Child Shell
> setenv DOG Pluto
> set mouse=Minnie
> printenv DOG
> echo $mouse
> exit
(THE xterm WINDOW SHOULD NOW GO AWAY - THIS
PROCESS HAS NOW BEEN KILLED)
Rolando V. Raqueño
Sunday, October 04, 2015
Environment vs. Shell Variables
Parent
Parent
Goofy
Parent
Mickey
Parent
Shell >
Shell > printenv DOG
Shell > echo $mouse
Shell >
Rolando V. Raqueño
Sunday, October 04, 2015
Environment & Shell Variables
• Why is this important?
– UNIX uses Environment and Shell Variables
control a number of processes
– Customizes your working environment
– Variables used for UNIX Scripts
• They are typically defined and initialized in
your .login and .cshrc files
Rolando V. Raqueño
Sunday, October 04, 2015
Useful Shell Variables
filec
path
#Allows file completion
#List of command
directories
cdpath #List of candidate
directories to cd into
history #Number of commands to
remember
Rolando V. Raqueño
Sunday, October 04, 2015
What is shell programming?
• Shell programming
– automate a set of UNIX commands.
– Just like any programming language
– “wrappers”
• black box a customized collection of UNIX
commands.
– Example of shell programs
.login
.cshrc
Rolando V. Raqueño
Sunday, October 04, 2015
.login file
set path=($HOME/bin /usr/local/bin \
/usr/ucb /usr/sbin /bin /usr/bin \
/usr/bin/X11 .)
stty dec new
tset -I -Q
set mail=/usr/spool/mail/$USER
set editmode = emacs
umask 077
biff n
date
Rolando V. Raqueño
Sunday, October 04, 2015
.cshrc file
if ($?prompt) then
set notify
set history = 100
set savehist = 100
alias pd pushd
alias pop popd
alias vt100 "set term = vt100"
endif
Rolando V. Raqueño
Sunday, October 04, 2015
When these files are
executed?
.cshrc
– is automatically executed when you start a new
shell
.login
– only gets executed once when you first login
Can be re-executed by giving the source
command
% source .cshrc
Rolando V. Raqueño
Sunday, October 04, 2015
Other useful .login and
.cshrc entries
set filec
set cdpath=(~ ~rvrpci/pub ~/mythesis)
Other common entries
set path=( $path /usr/local/bin)
set path=(/usr/local/bin $path)
Rolando V. Raqueño
Sunday, October 04, 2015
User defined shell program
• Determine name of command
• Determine input, output, and option
arguments
• Determine UNIX commands to execute
• Establish error trapping
• Make shell program executable
Rolando V. Raqueño
Sunday, October 04, 2015
A simple shell program
• dd command to swap bytes
% dd if=input.dat of=output.dat
bs=2 conv=swab
• Very difficult to remember
• Very little utility to non-UNIX geeks
(normal people)
Rolando V. Raqueño
Sunday, October 04, 2015
We would rather see...
% swap_bytes input.dat output.dat
Rolando V. Raqueño
Sunday, October 04, 2015
Special Shell Variables Set
% swap_bytes input.dat output.dat
$0
command
Rolando V. Raqueño
$1
$argv[1]
$2
$argv[2]
Sunday, October 04, 2015
Another Special
Shell Variables
% swap_bytes input.dat output.dat
$#argv
Indicates how many arguments are present
In this case, 2
Rolando V. Raqueño
Sunday, October 04, 2015
shell program
swap_bytes
#!/bin/csh -f
dd if=$1 of=$2 bs=2 conv=swab
Rolando V. Raqueño
Sunday, October 04, 2015
Making swap_bytes
shell script executable
% ls -l swap_bytes
-rw------- ... swap_bytes
% chmod u+x swap_bytes
% ls -l swap_bytes
-rwx------ ... swap_bytes
Rolando V. Raqueño
Sunday, October 04, 2015
To run swap_bytes
• swap_bytes becomes just another
unix command!
% swap_bytes input.dat output.dat
Rolando V. Raqueño
Sunday, October 04, 2015
Limitation of swap_bytes
• No error trapping
• Should give usage when typing command
% swap_bytes
usage: swap_bytes input_file output_file
Rolando V. Raqueño
Sunday, October 04, 2015
Improvement to swap_bytes
#!/bin/csh -f
if ( $#argv != 2 ) then
echo "usage: $0 input_file output_file"
exit 1
endif
dd if=$1 of=$2 bs=2 conv=swab
Rolando V. Raqueño
Sunday, October 04, 2015
Commad exit status
• By convention
exit 0
Indicates successful command completion
exit 1 (or non-zero)
Indicates some error condition
Rolando V. Raqueño
Sunday, October 04, 2015
Informational message from
swap_bytes
• UNIX style informational message
% swap_bytes
usage: swap_bytes input_file output_file
Rolando V. Raqueño
Sunday, October 04, 2015
Interactive swap_bytes
• If you want a “friendlier” shell program
– Have it query the user for the inputs
• Another special shell variable can be
used
$<
Rolando V. Raqueño
Sunday, October 04, 2015
Interactive swap_bytes
#!/bin/csh -f
if ( $#argv != 2 ) then
echo -n "Please enter the input file> "
set input=$<
echo -n "Please enter the output file> "
set output=$<
endif
dd if=$input of=$output bs=2 conv=swab
Rolando V. Raqueño
Sunday, October 04, 2015
Interactive swap_bytes example
• User simply types the command
% swap_bytes
Please enter the input file> input.dat
Please enter the output file> output.dat
Rolando V. Raqueño
Sunday, October 04, 2015
UNIX Quotes
Rolando V. Raqueño
Sunday, October 04, 2015
A note about quotes in UNIX
%
%
%
%
%
%
%
%
%
set a=ls
echo a
echo $a
set b=“$a”
echo $b
set b=‘$a’
echo $b
set b=`$a`
echo $b
Rolando V. Raqueño
Sunday, October 04, 2015
A note about shell variables
• Shell variables can also double up as
arrays
• Using the previous example,
% echo $b
% echo $b[1]
% echo $#b
% echo $b[$#b]
Rolando V. Raqueño
Sunday, October 04, 2015
A more complex shell program
• In pbmplus utilities,
rawtopgm conversion exists
pgmtoraw conversion does not
• A version of pgmtoraw in a programming
language like C
– Time consuming
– Program will likely be used infrequently
• Solution: shell program
Rolando V. Raqueño
Sunday, October 04, 2015
pgmtoraw shell script design
•
•
•
•
•
•
Define input and output files
Figure out dimensions of input image
Determine number of bytes for input image
Determine number of bytes for header
Need to strip out the header bytes
Write out headerless image data
Rolando V. Raqueño
Sunday, October 04, 2015
Define input and output files
pgmtoraw
#!/bin/csh -f
set command_name=$0
set number_args=$#argv
if( $number_args != 1 ) then
echo “usage:$command_name input_file_name”
exit 1
endif
.
.
.
Rolando V. Raqueño
Sunday, October 04, 2015
Dimensions of input image
( pnmfile)
% more test_data.pgm
P2
3 3
255
1 2 3
4 5 6
7 8 9
% pnmfile test_data.pgm
test_data.pgm: PGM plain, 3 by 3
Rolando V. Raqueño
maxval 255
Sunday, October 04, 2015
pgmtoraw (continued)
set input_file=$1
set pnm_info = `pnmfile $input_file`
set image_type = $pnm_info[2]
set data_type = $pnm_info[3]
set width = $pnm_info[4]
set height = $pnm_info[6]
set maxval = $pnm_info[8]
set pixel_bytes = 1
@ image_bytes = $width * $height
Rolando V. Raqueño
Sunday, October 04, 2015
pgmtoraw (continued)
set file_info=`wc -c $input_file`
set bytes_in_file = $file_info[1]
@ header = $bytes_in_file - $image_bytes
dd if=$input_file bs=$pixel_bytes skip=$header
Rolando V. Raqueño
Sunday, October 04, 2015
Resulting pgmtoraw utility
• Uses existing routines to obtain information
pnmfile
wc
dd
• Functional tool written in 20 command lines
Rolando V. Raqueño
Sunday, October 04, 2015
Current Limitations of
Resulting pgmtoraw utility
• No check between “ASCII” vs. “RAW” pgm
if( data_type == ‘plain,’) ...
• No provisions for multibyte per pixel case
– Use pnmfile results to check for above cases
– endian case needs to be addressed for multibyte
case
• Above conditions can be addressed by suite
of UNIX commands
Rolando V. Raqueño
Sunday, October 04, 2015
Shell Scripts Wrappers
and IDL
• Another utility formerly missing in
pbmplus
jpegtopnm or pnmtojpeg
• IDL has jpeg read/write capability
– Create a “wrapper” that makes an idl
program pbmplus-like
Rolando V. Raqueño
Sunday, October 04, 2015
pnmtojpeg.pro
pro pnmtojpeg, input_file, output_file
read_ppm, input_file, image
write_jpeg, output_file, image
end
Rolando V. Raqueño
Sunday, October 04, 2015
Usage of pnmtojpeg.pro in IDL
IDL> pnmtojpeg,‘image.pnm’,’image.jpg’
Rolando V. Raqueño
Sunday, October 04, 2015
Usage of pnmtojpeg.pro in IDL
IDL> pnmtojpeg,‘image.pnm’,’image.jpg’
• For IDL to automatically find pnmtojpeg.pro
– It must be in the current working directory
– Directory containing pnmtojpeg.pro must be defined
in the ENVIRONMENT VARIABLE
• IDL_PATH
Rolando V. Raqueño
Sunday, October 04, 2015
IDL_PATH
environment variable
setenv IDL_DIR /cis/common/rsi/idl_5
setenv IDL_PATH
\+$IDL_DIR/lib:\+$IDL_DIR/examples:
\+/dirs/common/rsi/idl_5:\+/dirs/common/lib/id
l:\+~/lib/idl
Rolando V. Raqueño
Sunday, October 04, 2015
pnmtojpeg.csh
#!/bin/csh -f
echo pnmtojpeg “,” “’”$1”’” “,” “’”$2”’” | idl
Rolando V. Raqueño
Sunday, October 04, 2015
Usage of pnmtojpeg.csh
% pnmtojpeg.csh image.pnm image.jpg
Rolando V. Raqueño
Sunday, October 04, 2015
Limitation of pnmtojpeg.csh
• Does not conform to pbmplus piping,
i.e.,
% tifftopnm file.tif | pnmscale
2.0 > new_file.pnm
• No error trapping
Rolando V. Raqueño
Sunday, October 04, 2015
Usage cases of pnmtojpeg
(no command line arguments)
% tifftopnm file.tif | pnmscale
2.0 | pnmtojpeg > new_file.jpg
Rolando V. Raqueño
Sunday, October 04, 2015
Usage cases of pnmtojpeg
(1 command line argument)
% pnmtojpeg image.pnm > image.jpg
Rolando V. Raqueño
Sunday, October 04, 2015
Usage cases of pnmtojpeg
(2 command line arguments)
% pnmtojpeg image.pnm image.jpg
Rolando V. Raqueño
Sunday, October 04, 2015
Yet another wrapper
pnmtojpeg
#!/bin/csh -f
# If user interrupts process, jump to
stop
onintr stop
# $0 is the command name
# $#argv is the number of arguments
# $$ is the process id
Rolando V. Raqueño
Sunday, October 04, 2015
Code for no argument case
if($#argv == 0) then
set input_file = /usr/tmp/$0_input_$$
set output_file = /usr/tmp/$0_output_$$
cat > $input_file
pnmtojpeg.csh $input_file $output_file
cat $output_file
.
.
.
Rolando V. Raqueño
Sunday, October 04, 2015
Code for 1 argument case
else if($#argv ==1) then
set input_file = $1
set output_file = /usr/tmp/$0_output_$$
pnmtojpeg.csh $input_file $output_file
cat $output_file
.
.
.
Rolando V. Raqueño
Sunday, October 04, 2015
Code for 2 argument case
else
set input_file = $1
set output_file = $2
pnmtojpeg.csh $input_file $output_file
endif
#clean up when finished
stop:
rm -f /usr/tmp/$0_input_$$
rm -f /usr/tmp/$0_output_$$
Rolando V. Raqueño
Sunday, October 04, 2015
pnmtojpeg summary
• Produced a “new” pbmplus utility
• Used UNIX shell scripting
– Argument handling
– Scratch space /usr/tmp
– Process id handling
– Clean up
• Integrated IDL program and commands
• 21 lines of UNIX commands
Rolando V. Raqueño
Sunday, October 04, 2015
Summary
•
•
•
•
The “dot” files
Basics of Shell Scripting
Special Shell Variables
Seamless integration of UNIX to other
utilities (IDL)
Rolando V. Raqueño
Sunday, October 04, 2015
Other Shell Constructs
to keep in mind
• foreach
• while
• case
Rolando V. Raqueño
Sunday, October 04, 2015
Descargar

Shell Programming