Difference between revisions of "CSC352 Notes 2013"
(→Setting up Eclipse for Hadoop) |
(→Setting up Eclipse for Hadoop) |
||
Line 15: | Line 15: | ||
* setup eclipse | * setup eclipse | ||
: http://v-lad.org/Tutorials/Hadoop/17%20-%20set%20up%20hadoop%20location%20in%20the%20eclipse.html | : http://v-lad.org/Tutorials/Hadoop/17%20-%20set%20up%20hadoop%20location%20in%20the%20eclipse.html | ||
+ | ** localhost | ||
+ | ** Map/Reduce Master: localhost, 9101 | ||
+ | ** DFS Master: user M/R Master ht, localhost, 9000 | ||
+ | ** user name: hadoop-user | ||
+ | ** SOCKS proxy: host, 1080 | ||
+ | * Open DFS Locations | ||
+ | ** localhost | ||
+ | *** (2) | ||
+ | **** tmp(1) | ||
+ | ***** hadoop-thiebaut (1) | ||
+ | ****** mapred (1) | ||
+ | ******* system (0) | ||
+ | **** user(1) | ||
+ | ***** thiebaut (2) | ||
+ | ****** hello.txt | ||
+ | ****** readme.txt | ||
=Notes on doing example in Yahoo Tutorial, Module 2= | =Notes on doing example in Yahoo Tutorial, Module 2= |
Revision as of 18:48, 15 December 2009
Setting up Eclipse for Hadoop
- Java 1.6
- http://v-lad.org/Tutorials/Hadoop/03%20-%20Prerequistes.html
- download Eclipse 3.3.2 (Europa) from http://www.eclipse.org/downloads/packages/release/europa/winter
- Use Hadoop 0.19.1
- open eclipse and deploy (Mac)
- uncompress hadoop 19.1
- copy the eclipse-plugin from hadoop to the plugin directory of eclipse
- start hadoop on the Mac and follow directions from http://v-lad.org/Tutorials/Hadoop page:
start-all.sh
- setup eclipse
- localhost
- Map/Reduce Master: localhost, 9101
- DFS Master: user M/R Master ht, localhost, 9000
- user name: hadoop-user
- SOCKS proxy: host, 1080
- Open DFS Locations
- localhost
- (2)
- tmp(1)
- hadoop-thiebaut (1)
- mapred (1)
- system (0)
- mapred (1)
- hadoop-thiebaut (1)
- user(1)
- thiebaut (2)
- hello.txt
- readme.txt
- thiebaut (2)
- tmp(1)
- (2)
- localhost
Notes on doing example in Yahoo Tutorial, Module 2
http://developer.yahoo.com/hadoop/tutorial/module2.html
cd ../hadoop/examples/ cat > HDFSHelloWorld.java mkdir hello_classes javac -classpath /Users/thiebaut/hadoop/hadoop-0.19.2-core.jar -d hello_classes HDFSHelloWorld.java emacs HDFSHelloWorld.java -nw javac -classpath /Users/thiebaut/hadoop/hadoop-0.19.2-core.jar -d hello_classes HDFSHelloWorld.java ls ls hello_classes/ jar -cvf helloworld.jar -C hello_classes/ . ls ls -ltr find . -name "*" -print hadoop jar helloworld.jar HDFSHelloWorld Hello, world!