[hadoop] 30G Access log 수행 결과

namenode 1대

slave 3대(dataNode)
(30분)
13/12/06 01:16:28 INFO mapreduce.Job: Running job: job_1386259770316_0001
13/12/06 01:16:36 INFO mapreduce.Job: Job job_1386259770316_0001 running in uber mode : false
13/12/06 01:16:36 INFO mapreduce.Job: map 0% reduce 0%
13/12/06 01:17:20 INFO mapreduce.Job: map 1% reduce 0%
13/12/06 01:17:44 INFO mapreduce.Job: map 2% reduce 0%
13/12/06 01:18:24 INFO mapreduce.Job: map 3% reduce 0%
13/12/06 01:47:08 INFO mapreduce.Job: map 100% reduce 100%
13/12/06 01:47:09 INFO mapreduce.Job: Job job_13862597 70316_0001 completed successfully
slave 6대(dataNode)
(기존 30분 -> 17분으로 단축)
13/12/06 02:32:39 INFO mapreduce.Job: Running job: job_1386264218442_0002
13/12/06 02:32:46 INFO mapreduce.Job: Job job_1386264218442_0002 running in uber mode : false
13/12/06 02:32:46 INFO mapreduce.Job: map 0% reduce 0%
13/12/06 02:49:50 INFO mapreduce.Job: map 100% reduce 100%
13/12/06 02:49:51 INFO mapreduce.Job: Job job_1386264218442_0002 completed successfully

Advertisements

[hadoop] WARN unable to load native hadoop library

64bit에서 hadoop을 실행시키면 발생하는 경고창이라고 한다….

완전히 문제를 해결하기 위해서는 64bit환경에서 모듈을 컴파일 해야 한다고 한다.

http://stackoverflow.com/questions/19943766/hadoop-unable-to-load-native-hadoop-library-for-your-platform-error-on-centos