Tutorial: Moodle VPL -- Looping Through Tests in vpl evaluate.sh

--D. Thiebaut (talk) 15:14, 24 September 2014 (EDT)

---

---

Moodle VPL Tutorials

---

This VPL module tests how a program uses exceptions to parse a simple text file containing numbers. The vpl_evaluate.sh script generates 3 tests, and runs through all 3 of them. Vpl_evaluate.sh skips non digits and non minus characters when comparing the output of the submitted program to the expected output.

Setup

• Lab 5, Lab5ReadFileThrow.java*
• Due date: Thursday, September 25, 2014, 11:55 PM
• Requested files: Lab5ReadFileThrow.java (Download)
• Type of work: Individual work
• Grade settings: Maximum grade: 100
• Run: Yes Evaluate: Yes
• Automatic grade: Yes
• See http://cs.smith.edu/dftwiki/index.php/CSC212_Lab_5_2014 for additional details

Execution files

vpl_run.sh

#! /bin/bash

cat > vpl_execution <<EEOOFF
#! /bin/bash
 
prog1=Lab5ReadFileThrow

 
javac \${prog1}.java 
 
if ((\$? > 0)); then
     echo "Error compiling your program"
     exit
fi
 
# --- create test input files ---
cat > data1.txt <<EOF
2 1
10 2
30 5.5
6 0
Smith College 3
100 10
EOF

echo "Testing your program with a text file containing:"
cat data1.txt
echo ""
echo "Output of your program:"
java \${prog1} data1.txt
 
EEOOFF
 
chmod +x vpl_execution

vpl_evaluate.sh

#! /bin/bash
# D. Thiebaut
# Smith College
# vpl_evaluate.sh script looping through several tests, each test with
# its own input file and its own expected output file.  The output of the
# student program is tested against the expected output file.
#
cat > vpl_execution <<EEOOFF
#! /bin/bash
 
# --- program tested (no extension) ---
prog1=Lab5ReadFileThrow

# --- compile student program ---
javac \${prog1}.java 
 
if ((\$? > 0)); then
     echo "Comment :=>>  Error compiling your program"
     echo "Grade :=>> 20"
     exit
fi

# --- create test input files ---
cat > data1.txt <<EOF
2 1
10 2
30 5
EOF

cat > data2.txt <<EOF
1 1
2 1
10 5
10.3 5
30 6
EOF
    
cat > data3.txt <<EOF
10 2
Smith College
-1 -1
100 10
EOF

#--- create expected outputs, one for each input file above ---
cat > data1.out <<EOF
2 1 2
10 2 5
30 5 6
EOF

cat > data2.out <<EOF
1 1 1      
2 1 2
10 5 2
30 6 5
EOF

cat > data3.out <<EOF
10 2 5
-1 -1 1
100 10 10
EOF




for i in 1 2 3  ; do
   echo "Comment :=>>-TEST \$i"

   # ==============================================
   # TEST i
   # ==============================================
   #--- run program, capture output, display to student ---
   java \${prog1} data\${i}.txt  &> user.out
   cp user.out user.out.org

   #--- remove non numbers and non minus---
   cat user.out | sed 's/[^0-9\ -]*//g' > dummy.out
   mv dummy.out user.out
 
   #--- remove multiple spaces --- 
   cat user.out | sed 's/  */ /g' > dummy.out
   mv dummy.out user.out

   #--- remove blank lines ---
   cat user.out | sed '/^\s*$/d' > dummy.out
   mv dummy.out user.out

   #--- compute difference --- 
   diff -y -w --ignore-all-space user.out data\${i}.out > diff.out
   #echo "----- diff.out ------"
   #cat diff.out
   #echo "---------------------"
   diff -y -w --ignore-all-space user.out data\${i}.out > diff.out

   #--- reject if different ---
   if ((\$? > 0)); then
      echo "Comment :=>>- Your output is incorrect."

      #--- display test file ---
      echo "Comment :=>>- Your program tested with:"
      echo "<|--" 
      cat data\${i}.txt
      echo "--|>"

      echo "Comment :=>> ---------------"
      echo "Comment :=>>- Your output:"
      echo "Comment :=>> ---------------"
      echo "<|--"
      cat user.out.org
      echo "--|>"
      echo ""
      echo "Comment :=>> ---------------"
      echo "Comment :=>>- Expected output (only the numbers): "
      echo "Comment :=>> ---------------"
      echo "<|--"
      cat data\${i}.out
      echo "--|>"
   
      #--- consolation grade ---   
      grade=\$((grade+10))

      # --------------------- REWARD IF CORRECT OUTPUT -----------------
   else
      #--- good output ---
      echo "Comment :=>>- Congrats, your output is correct."
      echo "Comment :=>> --------------------------------."
      echo "<|--"
      cat user.out.org
      echo "--|>"
      grade=\$((grade+100/i))
   fi
done
if (( grade > 100 )); then
   grade=100
fi
echo "Grade :=>> \$grade"

EEOOFF
 
chmod +x vpl_execution

Test Program

...