CSC334 Introduction to the XGrid at Smith College

--D. Thiebaut 21:34, 3 November 2008 (UTC)

---

This is an introduction/laboratory on using an XGrid system in CSC334, Fall 08, at Smith College.

Introduction

This first tutorial introduces the basic concepts of an XGrid system, and how to program it from the command line. The document assumes that the access is performed from a Windows XP PC which connects to an XGrid system through a Mac Pro intermediary.

Setup for CSC334

   

XGrid General Setup

(taken from http://images.apple.com/server/macosx/docs/Xgrid_Admin_and_HPC_v10.5.pdf)

(taken from http://www.macresearch.org/the_xgrid_tutorials_part_i_xgrid_basics)

Important Concepts

• job: typically a program or collection of programs, along with their data files submitted to the XGrid.
• tasks: a division of a job into smaller pieces containing a program or programs with its/their data file(s).

• controller: the main computer in the XGrid in charge of distributing work to the agents
• agent: the other computers in the XGrid

• client: the user or computer where the user sits, where jobs are issued.

Getting Started!

The XGrid system is available from the Mac platform only. In order to use it, you will have to connect to a Mac Pro that is a client of the XGrid.

Its address in xgridmac.dyndns.org.

• Use one of the Windows XP PC (or Linux PC), and open an SSH window.

• Connect to DT's Mac Pro using your Smith or MtHolyoke email name.

ssh -Y username@xgridmac.dyndns.org

• When prompted for a password, use the one given out in class.

• Create a .bash_profile file for simplifying the connection to the Grid (you will need to do this only once, the first time you connect to the Mac Pro):

emacs -nw .bash_profile

and copy/paste the text below in it.

export EDITOR=/usr/bin/emacs
export TERM=xterm-color

#export PS1="\w> "
PS1='[\h]\n[\t] \w\$: '
PS2='> '

# setup xgrid access
export XGRID_CONTROLLER_HOSTNAME=mathgrid0.smith.edu
export XGRID_CONTROLLER_PASSWORD=xxx_xxx_xxx_xxx

Make sure you replace the string xxx_xxx_xxx_xxx with the one that will be given to you.

Use Control-X Control-C to save the file.

• Make this file take effect:

source .bash_profile

• Check that the XGrid is up and that you can connect to it:

xgrid -grid attributes -gid 0

the output should be similar to what is shown below:

{
   gridAttributes =     {
       gridMegahertz = 0;
       isDefault = YES;
       name = Xgrid;
   };
}

You are discovering the format used by Apple to code information. It is similar to XML, but uses braces. It is called the PLIST format. We'll see it again later, when we deal with batch jobs.

A First Example: Histogram

We are going to use a very simplistic program as our target for exploring XGrid programming. The program will simply generate a given number of random amino acids, and will compute their histogram.

Here is an example of how it will work:

./genAminoHisto.pl 1000    

ASP: 46
PRO: 50
ILE: 51
LYS: 61
GLY: 46
TRP: 47
CYS: 48
PHE: 49 
GLN: 54
SER: 55
ASN: 38
VAL: 49
LEU: 47
TYR: 58
GLU: 53
ARG: 51
THR: 53 
MET: 52
ALA: 46 
HIS: 46

In this case we ask the Perl program to generate 1000 random amino acids, and to compute a histogram (frequency of occurrence) for them. We see that we roughly get 50 samples of each of the 20 amino acids.

EXERCISE #1 Generate the pseudo code for the program.

The code for the program is available here.

References

• Tutorial Part I by Charles Parnot, maintainer of one of the largest XGrid systems at Stanford.
• Command-Line Tutorial Part II
• XGrid Programming at Smith, maintained by D. Thiebaut
• A movie of a presentation of the XGrid (click on the movie icon).
• Tutorials on setting up and using an XGrid system.
• A movie on the use of Perl on an XGrid system.