Difference between revisions of "Tutorial: Running MPI Programs on Hadoop Cluster"

From dftwiki3
Jump to: navigation, search
(→‎Configuration)
(→‎Configuration)
Line 38: Line 38:
 
<onlysmith>
 
<onlysmith>
 
   
 
   
−
  131.229.103.46
+
  +----+-------------+-----------------+-----------------+
−
  131.229.103.62
+
| id | system_name | ip              | system_location |
−
  131.229.103.63
+
+----+-------------+-----------------+-----------------+
 +
|  1 | ford345-09  | 131.229.103.62  | Ford 345        |
 +
|  2 | ford345-03  | 131.229.103.46 | Ford 345        |
 +
|  3 | ford345-04  | 131.229.103.63  | Ford 345        |
 +
|  4 | ford345-05  | 131.229.103.64  | Ford 345        |
 +
|  5 | ford345-06  | 131.229.103.57  | Ford 345        |
 +
|  6 | ford345-L  | 131.229.103.52  | Ford 345        |
 +
|  7 | ford345-10  | 131.229.103.186 | Ford 342        |
 +
|  8 | ford345-02  | 131.229.103.188 | Ford 342        |
 +
|  9 | ford342-07  | 131.229.103.122 | Ford 342        |
 +
  | 10 | ford342-08  | 131.229.103.109 | Ford 342        |
 +
| 11 | ford342-05  | 131.229.103.214 | Ford 342        |
 +
| 12 | ford342-02  | 131.229.103.158 | Ford 342        |
 +
| 13 | ford342-04  | 131.229.103.174 | Ford 342        |
 +
| 14 | ford342-09  | 131.229.103.173 | Ford 342        |
 +
| 15 | ford342-10  | 131.229.103.172 | Ford 342        |
 +
  | 16 | ford342-L  | 131.229.103.143 | Ford 342        |
 +
| 17 | ford345-08  | 131.229.100.178 | Ford 342        |
 +
| 18 | ford345-07  | 131.229.101.176 | Ford 342        |
 +
| 19 | ford342-06  | 131.229.97.201  | Ford 342        |
 +
| 20 | ford342-03  | 131.229.101.192 | Ford 342        |
 +
+----+-------------+-----------------+-----------------+
 
    
 
    
 
</onlysmith>
 
</onlysmith>

Revision as of 15:57, 15 October 2013

--D. Thiebaut (talk) 13:57, 15 October 2013 (EDT)

Setup to run MPI on multiple Linux-Mint computers

  • Follow general directions from this page, or follow the steps summarized below.
  • First set up password-less login setup

Setup password-less ssh

  • Follow directions in this page, or follow the steps summarized below.
  • We'll use 3 of the Linux-Mint machines for this demo.

This section is only visible to computers located at Smith College

  • login to one of the Linux-Mint machines with your 352a-xx account.
  • enter the following commands:
ssh-keygen -t rsa
ls .ssh
cd .ssh
mv id_rsa id_rsa.mpi
mv id_rsa.pub id_rsa.mpi.pub
ls -l
total 12
-rw------- 1 352a 352a 1675 Oct 15 13:33 id_rsa.mpi
-rw------- 1 352a 352a  396 Oct 15 13:33 id_rsa.mpi.pub
-rw-r--r-- 1 352a 352a  444 Oct 14 14:26 known_hosts
cat id_rsa.mpi.pub > authorized_keys
emacs -nw config     (and add the next line to the file)
IdentityFile ~/.ssh/id_rsa.mpi
  • Now login to another Linux-Mint machine. You should be able to login without having to enter a password.

Configuration

  • create a file called hosts in the directory where the mpi programs are located.
  • Store the following IP addresses in it:

This section is only visible to computers located at Smith College

Retrieved from "http://dominiquethiebaut.com/dftwiki3/index.php?title=Tutorial:_Running_MPI_Programs_on_Hadoop_Cluster&oldid=18249"