[hadoop] 30G Access log 수행 결과

namenode 1대

slave 3대(dataNode)
(30분)
13/12/06 01:16:28 INFO mapreduce.Job: Running job: job_1386259770316_0001
13/12/06 01:16:36 INFO mapreduce.Job: Job job_1386259770316_0001 running in uber mode : false
13/12/06 01:16:36 INFO mapreduce.Job: map 0% reduce 0%
13/12/06 01:17:20 INFO mapreduce.Job: map 1% reduce 0%
13/12/06 01:17:44 INFO mapreduce.Job: map 2% reduce 0%
13/12/06 01:18:24 INFO mapreduce.Job: map 3% reduce 0%
13/12/06 01:47:08 INFO mapreduce.Job: map 100% reduce 100%
13/12/06 01:47:09 INFO mapreduce.Job: Job job_13862597 70316_0001 completed successfully
slave 6대(dataNode)
(기존 30분 -> 17분으로 단축)
13/12/06 02:32:39 INFO mapreduce.Job: Running job: job_1386264218442_0002
13/12/06 02:32:46 INFO mapreduce.Job: Job job_1386264218442_0002 running in uber mode : false
13/12/06 02:32:46 INFO mapreduce.Job: map 0% reduce 0%
13/12/06 02:49:50 INFO mapreduce.Job: map 100% reduce 100%
13/12/06 02:49:51 INFO mapreduce.Job: Job job_1386264218442_0002 completed successfully

Advertisements

Javascript에서 Map 구현하기

/**
* The FreeBSD Copyright
*
* Copyright 1992-2012 The FreeBSD Project. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions are met:
*
* Redistributions of source code must retain the above copyright notice, this
* list of conditions and the following disclaimer. Redistributions in binary
* form must reproduce the above copyright notice, this list of conditions and
* the following disclaimer in the documentation and/or other materials provided
* with the distribution. THIS SOFTWARE IS PROVIDED BY THE FREEBSD PROJECT ``AS
* IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
* THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
* PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE FREEBSD PROJECT OR CONTRIBUTORS
* BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
* POSSIBILITY OF SUCH DAMAGE.
*
* The views and conclusions contained in the software and documentation are
* those of the authors and should not be interpreted as representing official
* policies, either expressed or implied, of the FreeBSD Project.
*
* @author kipyung.sung
*/

function newMap() {
  var map = {};
  map.value = {};
  map.getKey = function(id) {
    return "k_"+id;
  };
  map.put = function(id, value) {
    var key = map.getKey(id);
    map.value[key] = value;
  };
  map.contains = function(id) {
    var key = map.getKey(id);
    if(map.value[key]) {
      return true;
    } else {
      return false;
    }
  };
  map.get = function(id) {
    var key = map.getKey(id);
    if(map.value[key]) {
      return map.value[key];
    }
    return null;
  };
  map.remove = function(id) {
    var key = map.getKey(id);
    if(map.contains(id)){
      map.value[key] = undefined;
    }
  };

  return map;
}

javascript에서 JSON 객체를 사용하여 정적 map을 구현할 수 있으나

동적 Map이 필요한 경우 다음과 같이 활용할 수 있다.

위 코드의 Key Point는 getKey

map.value가 데이터를 저장하는 JSON의 역할을 맡게 된다.

var map = newMap();

map.put(1, “hello world”);
map.put(2, “I Love NY”);

map.value의 상태는 다음과 같게 된다.

{“k_1″:”hello world”
, “k_2″:”I Love NY”}

map.value.k_1과 map.value[“k_1”] 표현식이 같은 원리를 이용하여 구현함.

위 코드를 카피 하거나 수정하거나 재사용하는 것은 허용하며
소스코드 사용 결과에 대해 책임을 지지 않음을 알려 드립니다.