XGrid Tutorial Part 1: Monte Carlo

This tutorial is intended for running distributed programs on an Apple Mac X-Grid at Smith College. Most of the steps presented here should work on other Apple grids, except for the specific details of login and host addresses. This document supersedes and extends the original tutorial CSC334 Introduction to the XGrid at Smith College.

Setup

This section assumes that you are using your own computer, or a computer in a lab, and that you have a secure shell client available on your computer. You will use the secure shell to connect to a host in the XGrid and run your programs from there.

XGrid General Setup

(taken from http://www.macresearch.org/the_xgrid_tutorials_part_i_xgrid_basics)

Important Concepts

• job: typically a program or collection of programs, along with their data files submitted to the XGrid.
• tasks: a division of a job into smaller pieces containing a program or programs with its/their data file(s).

• controller: the main computer in the XGrid in charge of distributing work to the agents
• agent: the other computers in the XGrid

• client: the user or computer where the user sits, where jobs are issued.

Connection, Bash Profile, Test

The XGrid system is available for the Mac platform only. In order to use it, you will have to connect to a Mac Pro that is a client and also the server of a grid (named XGrid).

• Use one of the Windows, Linux, or Mac computers available to you and open an SSH window.

• Connect to DT's Mac Pro using your Smith CSC352 account.

This section is only visible to computers located at Smith College

• When prompted for a password, use the one given out in class.

• Create a .bash_profile file in your account on the MacPro for simplifying the connection to the Grid (you will need to do this only once, the first time you connect to the Mac Pro):

emacs -nw .bash_profile

• Copy/paste the text below in it.

export EDITOR=/usr/bin/emacs
export TERM=xterm-color
export PATH=$PATH:.

#export PS1="\w> "
PS1='[\h]\n[\t] \w\$: '
PS2='> '

# setup xgrid access
This section is only visible to computers located at Smith College

• Make sure you replace the string xxx_xxx_xxx_xxx with the one that will be given to you.

• Use Control-X Control-C to save the file.

• Make the operating system read this file as if you had just logged in:

source .bash_profile

(Another way would have been to logout and login again.)

• Check that the XGrid is up and that you can connect to it:

xgrid -grid attributes -gid 0

the output should be similar to what is shown below:

{
   gridAttributes =     {
       gridMegahertz = 0;
       isDefault = YES;
       name = Xgrid;
   };
}

You are discovering the format used by Apple to code information. It is similar to XML, but uses braces. It is called the PLIST format. We'll see it again later, when we deal with batch jobs.

Your First Distributed Program: Monte Carlo

In this section you will learn how to run a Python program on the XGrid in synchronous and asynchronous modes.

First, the serial version of the program

We'll pick something simple to start with: a monte-carlo Python program that generates an approximation of Pi. You can find some good information on the monte-carlo method here. A good flash demonstration is availablehere as well.

The Python code is shown below.

#!/usr/bin/env python
# montecarlo.py
# taken from http://www.eveandersson.com/pi/monte-carlo-circle   
# Usage: 
#      python montecarlo.py N
# or
#      ./montecarlo N
#
# where N is the number of samples wanted.
# the approximation of pi is printed after the samples have been 
# generated.
#
import random
import math
import sys

# get the number of samples from the command line
# or assign 1E6 as the default number of samples.
N = 1000000
if len( sys.argv ) > 1:
   N = int( sys.argv[1] )

# generate the samples and keep track of counts
count_inside = 0
for count in range( N ):
    d = math.hypot(random.random(), random.random())
    if d < 1: count_inside += 1
count += 1

# print approximation of pi
print 30 * '-'
print n, ":", 4.0 * count_inside / count

Running the serial program

• Store the program in a file called montecarlo.py, and make it executable as follows:

 chmod +x montecarlo.py

• Run the program and measure its execution time for 1000 samples:

 time ./montecarlo.py 1000

 ------------------------------
 1000 : 3.136 

 real	0m0.026s
 user	0m0.016s
 sys	0m0.010s

• To test several number of samples, we can use the Bash for-loop, as follows (This command can be entered directly at the command prompt):

 for N in 1000 10000 100000 1000000 10000000; do time ./montecarlo.py $N; done

• Here's the output:

------------------------------
1000 : 3.136

real	0m0.026s
user	0m0.016s
sys	0m0.010s
------------------------------
10000 : 3.1592

real	0m0.035s
user	0m0.026s
sys	0m0.009s
------------------------------
100000 : 3.14472

real	0m0.141s
user	0m0.130s
sys	0m0.011s
------------------------------
1000000 : 3.14366

real	0m1.208s
user	0m1.183s
sys	0m0.023s
------------------------------
10000000 : 3.1414768

real	0m11.901s
user	0m11.732s
sys	0m0.162s

If you have a Mac and Want to Monitor your Distributed Programs

This section can be skipped. It allows one to monitor the grid from a GUI, and observe the parallel execution of several copies of a distributed program.

Install Apple Admin Tool

• If you do not have a Server folder in the Applications folder, and you don't have an XGrid Admin application in the Server folder, then download the admin, otherwise skip to the next step.
• Download and install package for Mac OS X 10.6 from Apple

• Install
• Launch

Connecting to the XGridMac server

• Locate the Server folder in your applications, and in it the XGrid icon
• Open it
• Click + to add a new a new controller.
• Enter xgridmac.xxxxxx.xxx and the password given to you in class.
• Observe that you should have access to a collection of Macs around the building