Difference between revisions of "Tutorial: Running MPI Programs on Hadoop Cluster"

From dftwiki3
Jump to: navigation, search
(Setup Password-Less ssh to the Hadoop cluster)
Line 30: Line 30:
 
   
 
   
  
:* Now ssh to hadoop02 and verify that you can ssh without password, as the authentication is now done through rsa keys.
+
:* Now ssh to '''hadoop02''' and verify that you can ssh without password, as the authentication is now done through rsa keys.
:* exit from hadoop02 and find yourself again on hadoop01
+
:* exit from hadoop02 and find yourself again on '''hadoop01'''
:* repeat the last 2 commands above (ssh and cat) for hadoop03, and hadoop04.
+
:* repeat the last 2 commands above (ssh and cat) for '''hadoop03''', and '''hadoop04''':
:* verify that you can ssh to hadoop03 and hadoop04 without password.
+
<br />
 +
:::<source lang="text">
 +
ssh yourusername@hadoop03.dyndns.org mkdir -p .ssh
 +
cat id_rsa.mpi.pub | ssh dominique@hadoop03.dyndns.org 'cat >> .ssh/authorized_keys'
 +
ssh yourusername@hadoop04.dyndns.org mkdir -p .ssh
 +
cat id_rsa.mpi.pub | ssh dominique@hadoop04.dyndns.org 'cat >> .ssh/authorized_keys'
 +
</source>
 +
<br />
 +
 
 +
:* verify that you can ssh to '''hadoop03''' and '''hadoop04''' without password.
 
* You should now be all set with passwordless ssh to the hadoop01, 02, 03, and 04 cluster.
 
* You should now be all set with passwordless ssh to the hadoop01, 02, 03, and 04 cluster.
 
</onlysmith>
 
</onlysmith>

Revision as of 11:08, 15 March 2017

--D. Thiebaut (talk) 13:57, 15 October 2013 (EDT)
Revised: --D. Thiebaut (talk) 12:02, 15 March 2017 (EDT)



Setup Password-Less ssh to the Hadoop cluster



This section is only visible to computers located at Smith College

Configuration

  • create a file called hosts in the directory where the mpi programs are located.
  • Store the following IP addresses in it:

This section is only visible to computers located at Smith College