CSC231 Bash Tutorial 6

--D. Thiebaut (talk) 22:18, 21 March 2017 (EDT)
Updated --D. Thiebaut (talk) 16:11, 19 October 2017 (EDT)

---

This lab deals with bash script files. Script files are program files that contain bash commands, and these commands can be interpreted by bash, one after the other, as if the user had typed them at the keyboard. Scripts can be quite complex, and contain functions, if-statements, loops, variables and other programming concepts found in programming languages. You will need to submit the solution to the last challenge to Moodle.

Reference

• This Bash tutorial heavily uses material presented in the excellent set of pages "Ryan's Tutorials" at http://ryanstutorials.net/bash-scripting-tutorial/. Do not hesitate to study these tutorials if you need additional information.

Backing up your files

Just in case you mess up and erase files in your account by mistake, you will make an archive of all your files and save it in your instructor's account:

cd
tar -czvf backup2.tgz *
rsubmit backup backup2.tgz

That's it! An archive of all your file should now be saved and available in case of accident!

Script Files

A Bash file, or Bash script, or script file, is a text file you can create and edit with emacs that contains a list of Linux commands, similar to the commands you type at the Linux prompt: ls, mkdir, cd, rm, nasm, ld, for, etc. In this section you will create such a file and have bash go through the whole list of commands and execute them, automatically.

Let's create a simple batch file that will allow you to automatically assemble, link, and run an assembly program. Furthermore, we'll link it with the 231Lib library, just to be safe. If the assembly program doesn't need the library, no harm will be done.

The commands you normally use to assemble, link and run are the following:

nasm -f elf progName.asm
ld -melf_i386 progName.o 231Lib.o -o progName
./progName

Let's create a script file called nald (for nasm ld) that will run these 3 commands automatically. For now, just copy the code without worrying too much about what happens. We'll explain what's going on later.

• emacs a new file called nald
• Store the following lines in it:

 
#! /bin/bash
 
nasm -f elf $1.asm
nasm -f elf 231Lib.asm
ld -melf_i386 $1.o 231Lib.o -o $1
./$1

• Make sure that the #-sign is the first character on the first line of the script.
• Make the script executable:

chmod +x nald

chmod is a command that changes the permissions of a file. "+x" means that you, everybody in the 231 class, and everybody who has an account on Aurora can execute your script.

• now run the script and pass it the name of one of your assembly files. Let's assume that you have a file called helloWorld.asm in your directory:

./nald  helloWorld

You should see the output of your program printed on the screen.

• Using emacs, make a slight modification to your helloWorld.asm program, just to verify that your script will re-assemble and link it correctly. Maybe, make it print something different.
• reassemble and run it in one command:

./nald  helloWorld

• You now have a nice script that will save you several keystrokes when you create your next assembly programs!

Explanations

Shebang

#! /bin/bash

the first line of the script starts with the pair "#!" which is referred to as "shebang" in the Linux world. It indicates that the commands in the script should be executed by the bash shell, which is located in the /bin directory of Aurora. If we had Python program that we would want to run automatically, we would use a shebang with Python's path. Let's try that for fun:

• Create a new file with emacs called hello.py

 #! /usr/bin/python
 
 from  __future__  import print_function
 
 def main():
      print( "Hello world of Python!" )
 
 main()

• Save the file, and make it executable

chmod +x hello.py

• run the program from the command line:

./hello.py
Hello world of Python!

• Notice that you didn't write "python hello.py" to run your program. Just "./hello.py." That's a nice trick.
• In summary, the shebang defines the interpreter to use to read the file and execute its contents.

$1

Command line parameters

inside the script, we can access the parameters that were typed on the command line using $1, $2, $3, and so on. To best understand how this works, create the following simple shell script with emacs called script1.sh:

#! /bin/bash

echo "You typed $# words on the command line"
echo "The whole collection of words is $@"
echo "Word 0 is $0"
echo "Word 1 is $1"
echo "Word 2 is $2"
echo "Word 3 is $3"
echo "Word 4 is $4"

• Make the script executable:

 chmod +x  script1.sh

• And try it with several command line parameters:

 ./script1.sh  hello world 3 20 2017

 ./script1.sh  CSC231

• so this is how the script can access the parameters passed on the command line. When you typed nald fileName on the command line earlier, the name of your file becomes $1 inside the script.

Challenge #1:

In a recent lab you created a command with a for loop that would run the N-Queens program for several chessboard sizes. A typical command to run the program for several board sizes would be:

 
 for i in `seq 8 14` ; do  
     echo -n "$i " 
     java NQueens $i -q 
 done

(if you have lost your NQueens program, you can get another copy with getcopy NQueens.java, and compile it with javac NQueens.java.)

• Create a bash script calls runNQueens.sh with the loop above in it. It will run all 7 cases automatically when called:

./runNQueens.sh 
8 0.158674 ms
9 0.093859 ms
10 0.160577 ms
11 0.113491 ms
12 0.355657 ms
13 0.185683 ms
14 12.889715 ms

Challenge #2:

Modify your script so that you can pass the boundaries of the board sizes you want to check, on the command line. For example, if you wanted to run the NQueens for board sizes ranging from 8 to 20, you'd run your script as follows:

./runNQueens.sh 8 20

Comments

• The #-sign is used as a comment. Bash scripts should be documented the same way regular programs are.
• Here's a documented version of the nald script. Please emulate this example when you write your bash scripts.

#! /bin/bash
# nald
# D. Thiebaut
# General script to assemble, link, and run assembly
# language programs.  Note that the 231Lib library will
# be linked automatically to the program, whether it needs
# it or not.
#
# Syntax:  nald  programName (without the .asm extension)
# --------

# assemble
nasm -f elf $1.asm
nasm -f elf 231Lib.asm

# link
ld -melf_i386 $1.o 231Lib.o -o $1

# execute
./$1

A Bash Scripts Behaves as a Regular Bash Command

• One nice feature of a bash script is that you can use it the same way you would a bash command. For example, if you wanted to capture the output of script1.sh in a file, you could simply type:

./script1.sh  hello world 3 20 2017  > dummy.txt

• Check the contents of the file dummy.txt and verify that the output of the script was captured correctly.
• Try this and capture the output of your runNQueens.sh script into a text file called queens.output. Verify that the queens.output file contains the output you want.

Bash Variables

You have already used a bash variable when you wrote your first bash loop:

for i in `seq 1 10` ; do
     echo $i
done

Rule for bash variables:

• When you create the variable, you do not use the $-sign.
• When you use the variable in an expression, you use the $-sign.

Setting Variables

• The syntax for setting a variable is simple:

variable=value

• Note that there are no spaces around the equal sign!
• Variables do not necessarily have to be used in scripts, you can also use them on the command line. Try these different commands on the Linux command line:

cs231a@aurora ~ $ semester=spring
cs231a@aurora ~ $ course=CSC231
cs231a@aurora ~ $ year=2017
cs231a@aurora ~ $ echo "$course, $semester $year" 

• If you want to set variables to string that contain several words, then enclose the string in double quotes:

cs231a@aurora ~ $ name="Smith College"
cs231a@aurora ~ $ echo $name

• You can also set a variable to be the output of a command.

cs231a@aurora ~ $ asmFiles=$( ls *.asm )
cs231a@aurora ~ $ objectFiles=$(  ls *.o )
cs231a@aurora ~ $ echo $asmFiles
cs231a@aurora ~ $ echo $objectFiles

• or the output of a pipe:

cs231a@aurora ~ $ noFiles=$( ls /etc | wc -l )
cs231a@aurora ~ $ echo "There are $noFiles files in the /etc directory"

Concatenating Variable Names to Strings

• Sometimes it is useful to concatenate a variable to a string. For example, if you wanted to create several files that have the same name, except for a number that identifies them, e.g. data_1March.txt, data_2March.txt, data_3March.txt, data_4March.txt, etc.

 for i in `seq 1 4`; do 
     fileName=data_$iOctober.txt
     touch $fileName
     echo "Created file $fileName"
 done

• I you run this loop, you will discover that instead of creating 4 files, it only create one. This is because Bash thinks your variable is $iOctober, and not $i concatenated to "October.txt". In order to help Bash figure out what is what, we can write the variable in a different form: ${i}. This is illustrated in the new bash loop below:

 for i in `seq 1 4`; do 
     fileName=data_${i}October.txt
     touch $fileName
     echo "Created file $fileName"
 done

• List the files in your directory with ls -ltr to list the newest files last, and verify that you now have the 4 new wanted files.

Challenge #3:

• Most Linux systems maintain a list of words in the file /usr/share/dict/words. This list is often used to pick words at random for various forms of testing. Check this file out, and verify that it indeed contains a list of words.
• How many words are in the file?
• Bash also has a system variable called $RANDOM, which contains a different random number between 0 and 32767 every time you use it. Verify that this is indeed the case on Aurora:

cs231a@aurora  $ echo $RANDOM
cs231a@aurora  $ echo $RANDOM
cs231a@aurora  $ echo $RANDOM

• Your challenge: write a script called randomWord.sh that outputs a random word from the '/usr/share/dict/words file every time it is executed, as illustrated below.

cs231a@aurora  $ ./randomWord.sh 
Iowans
cs231a@aurora  $ ./randomWord.sh 
angora
cs231a@aurora  $ ./randomWord.sh 
contractor
cs231a@aurora  $ ./randomWord.sh 
Ford's

Challenge #4:

• The previous script works because $RANDOM generates a number that is less than to total number of words in the file of words. When you want to pick a random line in a file that has fewer than 32,767 lines, you must use another approach. This is done often enough on Linux system that there is a special Linux command for this: shuf.
• Try it:

cs231a@aurora ~/handout $ shuf -n 1 /usr/share/dict/words
cs231a@aurora ~/handout $ shuf -n 1 /usr/share/dict/words
cs231a@aurora ~/handout $ shuf -n 1 /usr/share/dict/words

• You may want to read the man page for shuf to get a sense of what it really does:

 cs231a@aurora ~/handout $ man shuf

• Now that you have a new tool in your Linux "toolbox", copy the script you created for Challenge #3, and create a new one called randomWord2.sh. It will use a string (parameter) passed on the command line, and output a random word from /usr/share/dic/words that contains this string.

• Here are examples of how it should work:

cs231a@aurora ~/handout $ ./randomWord2.sh A
flack
cs231a@aurora ~/handout $ ./randomWord2.sh A 
Oregonian's
cs231a@aurora ~/handout $ ./randomWord2.sh A
enumerated
cs231a@aurora ~/handout $ ./randomWord2.sh aba
unabated
cs231a@aurora ~/handout $ ./randomWord2.sh aba
unabated
cs231a@aurora ~/handout $ ./randomWord2.sh aba
unabated
cs231a@aurora ~/handout $ ./randomWord2.sh al
zoological
cs231a@aurora ~/handout $ ./randomWord2.sh al
fallen
cs231a@aurora ~/handout $ ./randomWord2.sh al
heterosexuals

• note that sometimes the word won't really be random, but we don't care about this bug for this challenge.

Challenge #5:

• Linux always knows the current time and date. The command that returns it is date. Try this command at the prompt.
• You can force date to output the information in a different format by providing a special string indicating what you want to see. For example:

cs231a@aurora $  date +%Y-%m-%d
2017-10-20

• Create a new bash script called dateTag.sh to which you pass the name of a file on the command line, and that will make a copy of this file and prefix the name of the file with the current date. For example, if today's date is 2017-03-24, and you have a file in your directory called log.txt, then

cs231a@aurora $  dateTag.sh log.txt

will create a copy of log.txt with the name 2017-03-24_log.txt in your directory.

• Here is an example.

cs231a@aurora $ touch dummy.txt
cs231a@aurora $ dateTag.sh dummy.txt 
cs231a@aurora $ ls -ltr | tail -2
-rw------- 1 cs231a cs231a      166 Mar 24 11:24 dummy.txt
-rw------- 1 cs231a cs231a      166 Mar 24 11:25 2017-03-24_dummy.txt

Arithmetic With Bash Variables

• The Linux expr command can be used to do simple arithmetic. Try these commands at the prompt:

cs231a@aurora $ expr 3 + 4

cs231a@aurora  $ count=10
cs231a@aurora  $ echo $count
cs231a@aurora  $ expr $count + 2

cs231a@aurora  $ expr $count - 7

cs231a@aurora  $ expr $count \* 2

cs231a@aurora  $ expr $count / 3

cs231a@aurora  $ expr $count % 4

• Note: the reason we use "\*" when we want to multiply is because otherwise Bash would see a '*' character on the command line and automatically assume it to mean "All files in the current directory."

• We can do arithmetic in a bash script using the expr command, and a bit of additional syntax.
• Create a new script file called script3.sh:

#! /bin/bash
# script3.sh
# D. Thiebaut
# Computes and displays 10 powers of 2

count=1

for i in `seq 1 10` ; do
    echo "i = $i   count = $count"
    count=$( expr $count  \* 2 )
done

• This script is equivalent to the following Python program:

count = 1
for i in range( 1, 11 ):
    print( "i = ", i, "count = ", count )
    count = count * 2

• So arithmetic with Bash variable is fairly simple, although a bit heavy on the syntax.

Challenge #6 and Moodle Submission:

• Write a Bash script called teller.sh that accepts a number on the command line, and treats it as an amount of money, which it breaks down in a number of $20-bills, $10-bills, $5-bills, and $1-bills.
• Here is how your script should work. The first number output is the number of 20s, the second the number of 10s, the third the number of 5s, and the fourth, the number of 1s.

cs231a@aurora ~/handout $ ./teller.sh 10
0
1
0
0
cs231a@aurora ~/handout $ ./teller.sh 100 
5
0
0
0
cs231a@aurora ~/handout $ ./teller.sh 37 
1
1
1
2
cs231a@aurora ~/handout $ ./teller.sh 0
0
0
0
0

• When you are done and your script works similarly to the one above, submit it to Moodle.

Something Fun...

This exercise has to be done on a Mac, and not on aurora. This will work only if you have a Mac laptop... If you don't, maybe work on this with somebody next to you who has one...

• Open Terminal on your mac, and go to your home directory, then type the following commands:

cd 
touch fun.sh
open fun.sh -a TextEdit

• In TextEdit, type in the script below. Do not save it yet!

#! /bin/bash

for i in 1 2 3 4 5 ; do
   say "${i}"
   sleep 1
done

say "Time to go home"

• Before saving the file, still in TextEdit, select Format from the Menu and then click Make Plain Text. Save the file as
• make the file executable:

cd
chmod +x fun.sh

• Make sure the sound is ON. Run your script!

./fun.sh

Solutions to Challenges

Challenge 1

#! /bin/bash
# runNQueens.sh
# D. Thiebaut
# runs and times the execution of the NQueens
# program for board sizes ranging from 8 to 14.

for i in `seq 8 14` ; do  
     echo -n "$i " 
     java NQueens $i -q 
done

Challenge 2

#! /bin/bash
# runNQueens.sh
# D. Thiebaut
# runs and times the execution of the NQueens
# program for board sizes ranging from the first to second
# parameters passed on the command line.
# Example:
#     ./runNQueens.sh 10 21
#

for i in `seq $1 $2` ; do  
     echo -n "$i " 
     java NQueens $i -q 
done

Challenge 3

#! /bin/bash
# randomWord.sh
# D. Thiebaut
# Outputs a random word every time it is executed.

head -n $RANDOM /usr/share/dict/words | tail -n 1

Challenge 4

#! /bin/bash
# randomWord2.sh
# D. Thiebaut
#
# outputs a random word containing the string passed on the command line.

key=$1
words=/usr/share/dict/words

grep -i $key $words | shuf -n 1

Challege 5

#! /bin/bash
# dateTag.sh
# D. Thiebaut
 
tag=`date +%Y-%m-%d`

fileName=$1

cp $fileName ${tag}_$fileName