Difference between revisions of "Tutorial: Running MPI Programs on Hadoop Cluster"

From dftwiki3
Jump to: navigation, search
(Setup Password-Less ssh to the Hadoop cluster)
Line 26: Line 26:
 
  mv id_rsa.pub id_rsa.mpi.pub
 
  mv id_rsa.pub id_rsa.mpi.pub
 
  ssh yourusername@hadoop02.dyndns.org mkdir -p .ssh
 
  ssh yourusername@hadoop02.dyndns.org mkdir -p .ssh
  cat id_rsa.mpi.pub | ssh yourusername@hadoop02.dyndns.org 'cat >> .ssh/authorized_keys'
+
  cat ~/.ssh/id_rsa.mpi.pub | ssh yourusername@hadoop02.dyndns.org 'cat >> .ssh/authorized_keys'
 
</source>
 
</source>
 
<br />
 
<br />
Line 37: Line 37:
 
:::<source lang="text">
 
:::<source lang="text">
 
  ssh yourusername@hadoop03.dyndns.org mkdir -p .ssh
 
  ssh yourusername@hadoop03.dyndns.org mkdir -p .ssh
  cat id_rsa.mpi.pub | ssh yourusername@hadoop03.dyndns.org 'cat >> .ssh/authorized_keys'
+
  cat ~/.ssh/id_rsa.mpi.pub | ssh yourusername@hadoop03.dyndns.org 'cat >> .ssh/authorized_keys'
  
 
  ssh yourusername@hadoop04.dyndns.org mkdir -p .ssh
 
  ssh yourusername@hadoop04.dyndns.org mkdir -p .ssh
  cat id_rsa.mpi.pub | ssh yourusername@hadoop04.dyndns.org 'cat >> .ssh/authorized_keys'
+
  cat ~/.ssh/id_rsa.mpi.pub | ssh yourusername@hadoop04.dyndns.org 'cat >> .ssh/authorized_keys'
 
</source>
 
</source>
 
<br />
 
<br />
Line 48: Line 48:
 
</onlysmith>
 
</onlysmith>
 
<br />
 
<br />
 +
 
=Test MPI=
 
=Test MPI=
 
<br />
 
<br />

Revision as of 11:42, 15 March 2017

--D. Thiebaut (talk) 13:57, 15 October 2013 (EDT)
Revised: --D. Thiebaut (talk) 12:02, 15 March 2017 (EDT)




Setup Password-Less ssh to the Hadoop cluster



This section is only visible to computers located at Smith College


Test MPI


MPI should already be installed, and your account ready to access it. To verify this, create a simple MPI "Hello World!" program, compile it, and run it.

/* C Example */
#include <stdio.h>
#include <mpi.h>


int main (argc, argv)
     int argc;
     char *argv[];
{
  int rank, size;

  MPI_Init (&argc, &argv);      /* starts MPI */
  MPI_Comm_rank (MPI_COMM_WORLD, &rank);        /* get current process id */
  MPI_Comm_size (MPI_COMM_WORLD, &size);        /* get number of processes */
  printf( "Hello world from process %d of %d\n", rank, size );
  MPI_Finalize();
  return 0;
}


  • Compile and Run


mpicc -o hello helloWorld.c
mpirun -np 2 ./hello
Hello world from process 0 of 2
Hello world from process 1 of 2
  • If you see the two lines starting with "Hello world" on your screen, MPI was successfully installed on your system!


Configuration for Running On Multiple Servers


This section is only visible to computers located at Smith College