Tutorial: Running MPI Programs on Hadoop Cluster

--D. Thiebaut (talk) 13:57, 15 October 2013 (EDT)
Revised: --D. Thiebaut (talk) 12:02, 15 March 2017 (EDT)

---

Setup Password-Less ssh to the Hadoop cluster

This section is only visible to computers located at Smith College

Test MPI

MPI should already be installed, and your account ready to access it. To verify this, create a simple MPI "Hello World!" program, compile it, and run it.

/* C Example */
#include <stdio.h>
#include <mpi.h>


int main (argc, argv)
     int argc;
     char *argv[];
{
  int rank, size;

  MPI_Init (&argc, &argv);      /* starts MPI */
  MPI_Comm_rank (MPI_COMM_WORLD, &rank);        /* get current process id */
  MPI_Comm_size (MPI_COMM_WORLD, &size);        /* get number of processes */
  printf( "Hello world from process %d of %d\n", rank, size );
  MPI_Finalize();
  return 0;
}

• Compile and Run

mpicc -o hello helloWorld.c
mpirun -np 2 ./hello
Hello world from process 0 of 2
Hello world from process 1 of 2

• If you see the two lines starting with "Hello world" on your screen, MPI was successfully installed on your system!

Configuration

• create a file called hosts in the directory where the mpi programs are located.
• Store the following IP addresses in it:

This section is only visible to computers located at Smith College