Tutorial Moodle VPL Tips & Tricks

--D. Thiebaut (talk) 20:31, 11 June 2014 (EDT)

---

---

Moodle VPL Tutorials

---

This is a collection of tips, tricks, observations, recommendations, and modifications related to using VPL with Moodle.

Tips & Tricks

Bash Quick Solutions

To compare the student's output to the expected output, but not worry about order of lines output, or whitespace

   # code included in vpl_evaluate.sh.  All $-signs prefixed with back-slash.
   # $user contains output from student program
   # $expected contains expected output
   cat \$user | sort | xargs  > \${user}_sorted
   cat \$expected | sort | xargs > \${expected}_sorted
   diff -y -w -B --ignore-all-space \${user}_sorted \${expected}_sorted > diff.out

   if ((\$? > 0)); then
      echo "Comment :=>> Your output is incorrect."
      echo "Comment :=>> ---------------"
      echo "Comment :=>>- Your output:"
      echo "Comment :=>> ---------------"
      echo "<|--"
      cat \$user
      echo "--|>"
      echo ""
      echo "Comment :=>> ---------------"
      echo "Comment :=>>- Expected output "
      echo "Comment :=>> ---------------"
      echo "<|--"
      cat \$expected
      echo "--|>"
   else
      echo "Comment :=>> Correct output."
      noTestsPassed=\$((noTestsPassed+1))
   fi

To remove comments from a Java program, including /* ... */, /** ... */, and //

  
  prog1=yourprogramname         # the name of the java code, '''without''' the ".java" extension
  cat $prog1.java | sed 's://.*$::g' | sed '/\/\*\*/,/\*\// {s/.*\*\/.*//p; d}' > _$prog1.java

To remove the comments from an assembly program (.asm)

 prog1=yourprogramname         # then name of the asm code, '''without''' the ".asm" extension
 cat ${prog1}.asm | sed 's:;.*$::g'  > _${prog1}.asm

To stop programs with endless loops, use the timeout command.

(Note: the \$ are required inside vpl_evaluate.sh when creating vpl_execute.sh)

  timeout 30 java \$prog1 < dataIn.txt  &> user.out
  retcode=\$?

  #--- if we killed the waiter, everything is good ---
  if [ \$retcode -eq  124 ]; then
      echo "Comment :=>>-Your program has stopped working."
  else
      echo "Comment :=>>-Your program ran to completion."
  fi

To add @suppresswarnings() to Java programs that don't contain it

addSuppressWarnings() {
    if [ "\$1" ]; then   # if there's a parameter
        prog=\$1
        if grep SuppressWarnings $prog ; then
           :
        else
           #echo "Not found"
           sed -i 's/^class/@SuppressWarnings("unchecked")\nclass/'  \${prog}
           sed -i 's/^public class/@SuppressWarnings("unchecked")\npublic class/'  \${prog}
        fi
    fi
}

###### (and in the code of ''vpl_evaluate.sh'', call function this way: ######
# --- add a @SuppressWarnings( "unchecked" ) label
addSuppressWarnings( \${prog1}.java )

Output of Evaluate Step

• The output of the Evaluate action is a bit illogical:

 +------------------------------+
 |  3 tests run/ 0 tests failed   | 
 +------------------------------+

• To change it into something more intuitive, such as this:

 +------------------------------+
 |  3 tests run/ 3 tests passed  |  
 +------------------------------+

we simply need to modify the code of vpl_evaluate.cpp on the moodle server, in /var/www/html/moodle/mod/vpl/jail/default_scripts (or whatever directory the vpl files reside).

• The diff of the modification is shown below:

1058,1059c1058,1060
< 				printf(">| %2d %s run/%2d %s failed |\n",
< 						nruns, nruns==1?stest[0]:stest[1], nerrors, nerrors==1?stest[0]:stest[1]);
---
> 				printf(">| %2d %s run/%2d %s passed |\n",
> 						nruns, nruns==1?stest[0]:stest[1], nruns-nerrors, 
> 				                (nruns-nerrors)==1?stest[0]:stest[1]);

Generating a vpl_evaluate.cases File Quickly

Recommendations

1. write a solution program first.
2. run it on several cases you want to test your students' programs with.
3. capture the outputs of the solution program on different cases. Copy paste into the vpl_evaluate.cases file. Don't worry yet about making it exactly correct.
4. evaluate the solution program. You will get mismatches between the generated output and the expected output. Just copy/paste the generated output back into the vpl_evaluate.cases file. You will also be able to include the extra characters generated by the input statements.
5. You should be ready to evaluate the solution program and get 100/100 as a grade.

Creating a vpl_execution script that measures the size of a program

Imagine that the assignment for the student is to create a program that solves a particular problem (which can be tested separately for the correctness of its output), but the requirement is to make the program executable as small as possible. The grade given to the submitted program will be 100 if the size is equal to the size of the solution program, and some lower grade depending on how large the file is.
Here's a bash script that will do the trick

#! /bin/bash
# $file contains the name of the executable.  The executable
# is created by assembling ${file}.asm, generating ${file}.o,
# and linking ${file}.o.
# The script outputs 0 if the executable crashes, or just
# an integer that is the size of the executable, in bytes, if
# it runs correctly.

file="printStars"  # whatever executable is to be tested

# assemble and link the executable
nasm -f elf -F stabs ${file}.asm
ld -melf_i386 -o $file ${file}.o

# run the executable in a subshell to catch any segmentation fault error
# that may result.
{ ./$file &> /dev/null ; } &> log

# if an error occured, automatically return 0
if [ $? -eq 139 ]; then
    echo "0";
    exit 0;
fi

# print out the size of the file
size=$(stat -c%s "$file")
echo $size

The part of the script that reads { ./$file &> /dev/null ; } &> log is a bit complex. The reason for this is that we want to catch a potential crash by the program being tested, prevent it from outputting some segmentation fault message that would have come up, and automatically output 0 for it, and not its size.

If the program runs correctly, its output is discarded and the script simply outputs the size in bytes of the executable.

Creating a vpl_execution Script that Outputs the Grade a Program Based on its Size

This time we take the previous version of vpl_execution and make it output the grade (0 to 100) based on how small the program executable is. The smallest possible size is assumed to be known.

#! /bin/bash
# $file contains the name of the executable.  The executable
# is created by assembling ${file}.asm, generating ${file}.o,
# and linking ${file}.o.
# The script outputs 0 if the executable crashes, or just
# an integer that is the size of the executable, in bytes, if
# it runs correctly.

file="printStars"  # whatever executable is to be tested
bestSize=1065    # the size of the executable version of the solution program
                         # in bytes.

# assemble and link the executable
nasm -f elf -F stabs ${file}.asm
ld -melf_i386 -o $file ${file}.o

# run the executable in a subshell to catch any segmentation fault error
# that may result.
{ ./$file &> /dev/null ; } &> log

# if an error occured, automatically return 0
if [ $? -eq 139 ]; then
    echo "0";
    exit 0;
fi

# print out the size of the file
size=$(stat -c%s "$file")
#echo $size

# compare size to best size recorded and output
# grade based on size.

increment=20   # size increment that takes away 10 more points
               # from the grade

# possible grades
grades=( 100 90 80 70 60 50 40 30 20 10 0 )

# find bin corresponding to size of executable, based on
# $bestSize, and bins of width $increment
for i in {0..10} ; do
    bestSize=$(($bestSize+$increment))
    if [ $size -le $bestSize ] ; then
	echo ${grades[$i]}
	exit 0
    fi  
done
echo "0"