Show page

Differences

This shows you the differences between two versions of the page.

--- utile:instalare_hadoop_hbase [2014/10/13 08:37]
Andrei Roșu-Cojocaru created
+++ utile:instalare_hadoop_hbase [2014/12/18 12:54] (current)
Andrei Roșu-Cojocaru
@@ Line 1: / Line 1: @@
-====== Instalare Hadoop 2.5.1 ======
+====== Instalare Hadoop 2.6.0 ======
 Pentru ca framework-ul Hadoop să poată funcționa, este necesar ca în prealabil să se instaleze:
@@ Line 31: / Line 31: @@
 Are you sure you want to continue connecting (yes/no)? yes
 Warning: Permanently added 'localhost' (ECDSA) to the list of known hosts.
-aipi2014@localhost's password:
+Welcome to Ubuntu 14.10 (GNU/Linux 3.16.0-28-generic i686)
-Welcome to Ubuntu 14.04.1 LTS (GNU/Linux 3.13.0-35-generic i686)
  * Documentation:  https://help.ubuntu.com/
@@ Line 46: / Line 45: @@
 Alte configurări cu privire la serverul SSH pot fi realizate în fișierul ''/etc/ssh/sshd_config'':
-  * opțiunea ''PubkeyAuthentication'' trebuie să aibă valoarea ''true'';
+  * opțiunea ''PubkeyAuthentication'' trebuie să aibă valoarea ''yes'';
   * dacă opțiunea ''AllowUsers'' este activă, utilizatorul curent trebuie să se regăsească în această listă.
@@ Line 63: / Line 62: @@
 <code>
 aipi2014@ubuntu:~$ sudo nano /etc/sysctl.conf
+</code>
+<file bash sysctl.conf>
 ...
 net.ipv6.conf.all.disable_ipv6 = 1
 net.ipv6.conf.default.disable_ipv6 = 1
 net.ipv6.conf.lo.disable_ipv6 = 1
-</code>
+</file>
 Actualizarea acestor valori se realizează în momentul în care se repornește sistemul de operare.
@@ Line 91: / Line 92: @@
   * se descarcă cea mai recentă versiune stabilă (lista poate fi consultată pe [[http://hadoop.apache.org/releases.html#Download|Hadoop Releases]])\\ <code>
 aipi2014@ubuntu:~$ cd /tmp
-aipi2014@ubuntu:/tmp$ wget http://mirrors.advancedhosters.com/apache/hadoop/common/stable/hadoop-2.5.1.tar.gz
+aipi2014@ubuntu:/tmp$ wget http://mirrors.hostingromania.ro/apache.org/hadoop/common/hadoop-2.6.0/hadoop-2.6.0.tar.gz
 </code>
   * se copiază în /usr/local, creându-se și o legătură simbolică spre aceasta\\ <code>
-aipi2014@ubuntu:/tmp$ tar xzf hadoop-2.5.1.tar.gz
+aipi2014@ubuntu:/tmp$ tar xzf hadoop-2.6.0.tar.gz
-aipi2014@ubuntu:/tmp$ sudo mv hadoop-2.5.1 /usr/local/
+aipi2014@ubuntu:/tmp$ sudo mv hadoop-2.6.0 /usr/local/
-aipi2014@ubuntu:/tmp$ sudo ln -s /usr/local/hadoop-2.5.1/ /usr/local/hadoop
+aipi2014@ubuntu:/tmp$ sudo ln -s /usr/local/hadoop-2.6.0/ /usr/local/hadoop
-aipi2014@ubuntu:/tmp$ rm -f hadoop-2.5.1.tar.gz
+aipi2014@ubuntu:/tmp$ rm -f hadoop-2.6.0.tar.gz
 </code>
   * se specifică utilizatorul curent drept proprietarul acestui director\\ <code>
@@ Line 121: / Line 122: @@
 <code>
 aipi2014@ubuntu:~$ sudo nano ~/.bashrc
+</code>
+<file bash .bashrc>
 ...
-export JAVA_HOME=/usr/local/java/jdk1.8.0_20
+export JAVA_HOME=/usr/local/java/jdk1.8.0_25
 export HADOOP_INSTALL=/usr/local/hadoop
 export PATH=$PATH:$HADOOP_INSTALL/bin:$HADOOP_INSTALL/sbin
@@ Line 132: / Line 135: @@
 export HBASE_INSTALL=/usr/local/hbase
 export PATH=$PATH:$HBASE_INSTALL/bin
-</code>
+</file>
 <note tip>Încărcarea acestor valori specificate pentru variabilele de sistem se face prin comanda:\\ <code>
@@ Line 144: / Line 147: @@
 <code>
 aipi2014@ubuntu:/usr/local/hadoop/etc/hadoop$ sudo nano hadoop-env.sh
-...
-export JAVA_HOME=/usr/local/java/jdk1.8.0_20
 </code>
+<file bash hadoop-env.sh>
+...
+export JAVA_HOME=/usr/local/java/jdk1.8.0_25
+</file>
 ==== core-site.xml ====
@@ Line 157: / Line 162: @@
 aipi2014@ubuntu:/usr/local/hadoop/etc/hadoop$ sudo nano core-site.xml
 </code>
 <file xml core-site.xml>
 <configuration>
@@ Line 169: / Line 173: @@
   </property>
 </configuration>
 </file>
@@ Line 179: / Line 182: @@
 aipi2014@ubuntu:/usr/local/hadoop/etc/hadoop$ sudo nano yarn-site.xml
 </code>
 <file xml yarn-site.xml>
 <configuration>
@@ Line 201: / Line 203: @@
 aipi2014@ubuntu:/usr/local/hadoop/etc/hadoop$ nano mapred-site.xml
 </code>
 <file xml mapred-site.xml>
 <configuration>
@@ Line 221: / Line 222: @@
 aipi2014@ubuntu:/usr/local/hadoop/etc/hadoop$ sudo nano hdfs-site.xml
 </code>
 <file xml hdfs-site.xml>
 <configuration>
@@ Line 245: / Line 245: @@
 <code>
 aipi2014@ubuntu:~$ hdfs namenode -format
-/10/10 22:20:25 INFO namenode.NameNode: STARTUP_MSG:
+/12/17 14:15:34 INFO namenode.NameNode: STARTUP_MSG:
 /************************************************************
 STARTUP_MSG: Starting NameNode
 STARTUP_MSG:   host = ubuntu/127.0.1.1
 STARTUP_MSG:   args = [-format]
-STARTUP_MSG:   version = 2.5.1
+STARTUP_MSG:   version = 2.6.0
-STARTUP_MSG:   classpath = /usr/local/hadoop-2.5.1/etc/hadoop:/usr/local/hadoop/share/hadoop/common/lib/jackson-xc-1.9.13.jar:/usr/local/hadoop/share/hadoop/common/lib/jetty-6.1.26.jar:/usr/local/hadoop/share/hadoop/common/lib/commons-beanutils-1.7.0.jar:/usr/local/hadoop/share/hadoop/common/lib/commons-codec-1.4.jar:/usr/local/hadoop/share/hadoop/common/lib/jersey-core-1.9.jar:/usr/local/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar:/usr/local/hadoop/share/hadoop/common/lib/jaxb-impl-2.2.3-1.jar:/usr/local/hadoop/share/hadoop/common/lib/log4j-1.2.17.jar:/usr/local/hadoop/share/hadoop/common/lib/jackson-jaxrs-1.9.13.jar:/usr/local/hadoop/share/hadoop/common/lib/xz-1.0.jar:/usr/local/hadoop/share/hadoop/common/lib/commons-configuration-1.6.jar:/usr/local/hadoop/share/hadoop/common/lib/commons-logging-1.1.3.jar:/usr/local/hadoop/share/hadoop/common/lib/guava-11.0.2.jar:/usr/local/hadoop/share/hadoop/common/lib/paranamer-2.3.jar:/usr/local/hadoop/share/hadoop/common/lib/activation-1.1.jar:/usr/local/hadoop/share/hadoop/common/lib/servlet-api-2.5.jar:/usr/local/hadoop/share/hadoop/common/lib/jsr305-1.3.9.jar:/usr/local/hadoop/share/hadoop/common/lib/hamcrest-core-1.3.jar:/usr/local/hadoop/share/hadoop/common/lib/commons-net-3.1.jar:/usr/local/hadoop/share/hadoop/common/lib/jettison-1.1.jar:/usr/local/hadoop/share/hadoop/common/lib/apacheds-i18n-2.0.0-M15.jar:/usr/local/hadoop/share/hadoop/common/lib/jsp-api-2.1.jar:/usr/local/hadoop/share/hadoop/common/lib/snappy-java-1.0.4.1.jar:/usr/local/hadoop/share/hadoop/common/lib/commons-compress-1.4.1.jar:/usr/local/hadoop/share/hadoop/common/lib/netty-3.6.2.Final.jar:/usr/local/hadoop/share/hadoop/common/lib/commons-lang-2.6.jar:/usr/local/hadoop/share/hadoop/common/lib/jaxb-api-2.2.2.jar:/usr/local/hadoop/share/hadoop/common/lib/mockito-all-1.8.5.jar:/usr/local/hadoop/share/hadoop/common/lib/java-xmlbuilder-0.4.jar:/usr/local/hadoop/share/hadoop/common/lib/jasper-runtime-5.5.23.jar:/usr/local/hadoop/share/hadoop/common/lib/xmlenc-0.52.jar:/usr/local/hadoop/share/hadoop/common/lib/junit-4.11.jar:/usr/local/hadoop/share/hadoop/common/lib/jsch-0.1.42.jar:/usr/local/hadoop/share/hadoop/common/lib/jetty-util-6.1.26.jar:/usr/local/hadoop/share/hadoop/common/lib/zookeeper-3.4.6.jar:/usr/local/hadoop/share/hadoop/common/lib/jackson-core-asl-1.9.13.jar:/usr/local/hadoop/share/hadoop/common/lib/commons-collections-3.2.1.jar:/usr/local/hadoop/share/hadoop/common/lib/commons-math3-3.1.1.jar:/usr/local/hadoop/share/hadoop/common/lib/protobuf-java-2.5.0.jar:/usr/local/hadoop/share/hadoop/common/lib/jets3t-0.9.0.jar:/usr/local/hadoop/share/hadoop/common/lib/httpclient-4.2.5.jar:/usr/local/hadoop/share/hadoop/common/lib/avro-1.7.4.jar:/usr/local/hadoop/share/hadoop/common/lib/commons-cli-1.2.jar:/usr/local/hadoop/share/hadoop/common/lib/jersey-server-1.9.jar:/usr/local/hadoop/share/hadoop/common/lib/hadoop-annotations-2.5.1.jar:/usr/local/hadoop/share/hadoop/common/lib/jasper-compiler-5.5.23.jar:/usr/local/hadoop/share/hadoop/common/lib/slf4j-api-1.7.5.jar:/usr/local/hadoop/share/hadoop/common/lib/api-util-1.0.0-M20.jar:/usr/local/hadoop/share/hadoop/common/lib/asm-3.2.jar:/usr/local/hadoop/share/hadoop/common/lib/commons-beanutils-core-1.8.0.jar:/usr/local/hadoop/share/hadoop/common/lib/commons-httpclient-3.1.jar:/usr/local/hadoop/share/hadoop/common/lib/apacheds-kerberos-codec-2.0.0-M15.jar:/usr/local/hadoop/share/hadoop/common/lib/api-asn1-api-1.0.0-M20.jar:/usr/local/hadoop/share/hadoop/common/lib/commons-digester-1.8.jar:/usr/local/hadoop/share/hadoop/common/lib/hadoop-auth-2.5.1.jar:/usr/local/hadoop/share/hadoop/common/lib/httpcore-4.2.5.jar:/usr/local/hadoop/share/hadoop/common/lib/commons-io-2.4.jar:/usr/local/hadoop/share/hadoop/common/lib/jackson-mapper-asl-1.9.13.jar:/usr/local/hadoop/share/hadoop/common/lib/commons-el-1.0.jar:/usr/local/hadoop/share/hadoop/common/lib/stax-api-1.0-2.jar:/usr/local/hadoop/share/hadoop/common/lib/jersey-json-1.9.jar:/usr/local/hadoop/share/hadoop/common/hadoop-common-2.5.1.jar:/usr/local/hadoop/share/hadoop/common/hadoop-common-2.5.1-tests.jar:/usr/local/hadoop/share/hadoop/common/hadoop-nfs-2.5.1.jar:/usr/local/hadoop/share/hadoop/hdfs:/usr/local/hadoop/share/hadoop/hdfs/lib/jetty-6.1.26.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/commons-codec-1.4.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/jersey-core-1.9.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/log4j-1.2.17.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/commons-daemon-1.0.13.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/commons-logging-1.1.3.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/guava-11.0.2.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/servlet-api-2.5.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/jsr305-1.3.9.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/jsp-api-2.1.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/netty-3.6.2.Final.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/commons-lang-2.6.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/jasper-runtime-5.5.23.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/xmlenc-0.52.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/jetty-util-6.1.26.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/jackson-core-asl-1.9.13.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/protobuf-java-2.5.0.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/commons-cli-1.2.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/jersey-server-1.9.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/asm-3.2.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/commons-io-2.4.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/jackson-mapper-asl-1.9.13.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/commons-el-1.0.jar:/usr/local/hadoop/share/hadoop/hdfs/hadoop-hdfs-2.5.1.jar:/usr/local/hadoop/share/hadoop/hdfs/hadoop-hdfs-2.5.1-tests.jar:/usr/local/hadoop/share/hadoop/hdfs/hadoop-hdfs-nfs-2.5.1.jar:/usr/local/hadoop-2.5.1/share/hadoop/yarn/lib/jackson-xc-1.9.13.jar:/usr/local/hadoop-2.5.1/share/hadoop/yarn/lib/jetty-6.1.26.jar:/usr/local/hadoop-2.5.1/share/hadoop/yarn/lib/commons-codec-1.4.jar:/usr/local/hadoop-2.5.1/share/hadoop/yarn/lib/jersey-core-1.9.jar:/usr/local/hadoop-2.5.1/share/hadoop/yarn/lib/jaxb-impl-2.2.3-1.jar:/usr/local/hadoop-2.5.1/share/hadoop/yarn/lib/log4j-1.2.17.jar:/usr/local/hadoop-2.5.1/share/hadoop/yarn/lib/jersey-client-1.9.jar:/usr/local/hadoop-2.5.1/share/hadoop/yarn/lib/jersey-guice-1.9.jar:/usr/local/hadoop-2.5.1/share/hadoop/yarn/lib/jline-0.9.94.jar:/usr/local/hadoop-2.5.1/share/hadoop/yarn/lib/jackson-jaxrs-1.9.13.jar:/usr/local/hadoop-2.5.1/share/hadoop/yarn/lib/xz-1.0.jar:/usr/local/hadoop-2.5.1/share/hadoop/yarn/lib/commons-logging-1.1.3.jar:/usr/local/hadoop-2.5.1/share/hadoop/yarn/lib/guava-11.0.2.jar:/usr/local/hadoop-2.5.1/share/hadoop/yarn/lib/guice-servlet-3.0.jar:/usr/local/hadoop-2.5.1/share/hadoop/yarn/lib/activation-1.1.jar:/usr/local/hadoop-2.5.1/share/hadoop/yarn/lib/servlet-api-2.5.jar:/usr/local/hadoop-2.5.1/share/hadoop/yarn/lib/jsr305-1.3.9.jar:/usr/local/hadoop-2.5.1/share/hadoop/yarn/lib/jettison-1.1.jar:/usr/local/hadoop-2.5.1/share/hadoop/yarn/lib/commons-compress-1.4.1.jar:/usr/local/hadoop-2.5.1/share/hadoop/yarn/lib/netty-3.6.2.Final.jar:/usr/local/hadoop-2.5.1/share/hadoop/yarn/lib/commons-lang-2.6.jar:/usr/local/hadoop-2.5.1/share/hadoop/yarn/lib/jaxb-api-2.2.2.jar:/usr/local/hadoop-2.5.1/share/hadoop/yarn/lib/guice-3.0.jar:/usr/local/hadoop-2.5.1/share/hadoop/yarn/lib/jetty-util-6.1.26.jar:/usr/local/hadoop-2.5.1/share/hadoop/yarn/lib/zookeeper-3.4.6.jar:/usr/local/hadoop-2.5.1/share/hadoop/yarn/lib/jackson-core-asl-1.9.13.jar:/usr/local/hadoop-2.5.1/share/hadoop/yarn/lib/commons-collections-3.2.1.jar:/usr/local/hadoop-2.5.1/share/hadoop/yarn/lib/protobuf-java-2.5.0.jar:/usr/local/hadoop-2.5.1/share/hadoop/yarn/lib/javax.inject-1.jar:/usr/local/hadoop-2.5.1/share/hadoop/yarn/lib/aopalliance-1.0.jar:/usr/local/hadoop-2.5.1/share/hadoop/yarn/lib/commons-cli-1.2.jar:/usr/local/hadoop-2.5.1/share/hadoop/yarn/lib/jersey-server-1.9.jar:/usr/local/hadoop-2.5.1/share/hadoop/yarn/lib/leveldbjni-all-1.8.jar:/usr/local/hadoop-2.5.1/share/hadoop/yarn/lib/asm-3.2.jar:/usr/local/hadoop-2.5.1/share/hadoop/yarn/lib/commons-httpclient-3.1.jar:/usr/local/hadoop-2.5.1/share/hadoop/yarn/lib/commons-io-2.4.jar:/usr/local/hadoop-2.5.1/share/hadoop/yarn/lib/jackson-mapper-asl-1.9.13.jar:/usr/local/hadoop-2.5.1/share/hadoop/yarn/lib/stax-api-1.0-2.jar:/usr/local/hadoop-2.5.1/share/hadoop/yarn/lib/jersey-json-1.9.jar:/usr/local/hadoop-2.5.1/share/hadoop/yarn/hadoop-yarn-server-resourcemanager-2.5.1.jar:/usr/local/hadoop-2.5.1/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-2.5.1.jar:/usr/local/hadoop-2.5.1/share/hadoop/yarn/hadoop-yarn-server-common-2.5.1.jar:/usr/local/hadoop-2.5.1/share/hadoop/yarn/hadoop-yarn-applications-unmanaged-am-launcher-2.5.1.jar:/usr/local/hadoop-2.5.1/share/hadoop/yarn/hadoop-yarn-server-applicationhistoryservice-2.5.1.jar:/usr/local/hadoop-2.5.1/share/hadoop/yarn/hadoop-yarn-server-tests-2.5.1.jar:/usr/local/hadoop-2.5.1/share/hadoop/yarn/hadoop-yarn-server-web-proxy-2.5.1.jar:/usr/local/hadoop-2.5.1/share/hadoop/yarn/hadoop-yarn-common-2.5.1.jar:/usr/local/hadoop-2.5.1/share/hadoop/yarn/hadoop-yarn-api-2.5.1.jar:/usr/local/hadoop-2.5.1/share/hadoop/yarn/hadoop-yarn-server-nodemanager-2.5.1.jar:/usr/local/hadoop-2.5.1/share/hadoop/yarn/hadoop-yarn-client-2.5.1.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/jersey-core-1.9.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/log4j-1.2.17.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/jersey-guice-1.9.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/xz-1.0.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/guice-servlet-3.0.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/paranamer-2.3.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/hamcrest-core-1.3.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/snappy-java-1.0.4.1.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/commons-compress-1.4.1.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/netty-3.6.2.Final.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/guice-3.0.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/junit-4.11.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/jackson-core-asl-1.9.13.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/protobuf-java-2.5.0.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/avro-1.7.4.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/javax.inject-1.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/aopalliance-1.0.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/jersey-server-1.9.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/hadoop-annotations-2.5.1.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/leveldbjni-all-1.8.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/asm-3.2.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/commons-io-2.4.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/jackson-mapper-asl-1.9.13.jar:/usr/local/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.5.1-tests.jar:/usr/local/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.5.1.jar:/usr/local/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-2.5.1.jar:/usr/local/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.5.1.jar:/usr/local/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-core-2.5.1.jar:/usr/local/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-app-2.5.1.jar:/usr/local/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-plugins-2.5.1.jar:/usr/local/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-common-2.5.1.jar:/usr/local/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-shuffle-2.5.1.jar:/contrib/capacity-scheduler/*.jar
+STARTUP_MSG:   classpath = /usr/local/hadoop-2.6.0/etc/hadoop:/usr/local/hadoop/share/hadoop/common/lib/httpcore-4.2.5.jar:/usr/local/hadoop/share/hadoop/common/lib/httpclient-4.2.5.jar:/usr/local/hadoop/share/hadoop/common/lib/zookeeper-3.4.6.jar:/usr/local/hadoop/share/hadoop/common/lib/slf4j-api-1.7.5.jar:/usr/local/hadoop/share/hadoop/common/lib/curator-client-2.6.0.jar:/usr/local/hadoop/share/hadoop/common/lib/commons-digester-1.8.jar:/usr/local/hadoop/share/hadoop/common/lib/jettison-1.1.jar:/usr/local/hadoop/share/hadoop/common/lib/jackson-jaxrs-1.9.13.jar:/usr/local/hadoop/share/hadoop/common/lib/commons-codec-1.4.jar:/usr/local/hadoop/share/hadoop/common/lib/commons-el-1.0.jar:/usr/local/hadoop/share/hadoop/common/lib/jaxb-api-2.2.2.jar:/usr/local/hadoop/share/hadoop/common/lib/commons-collections-3.2.1.jar:/usr/local/hadoop/share/hadoop/common/lib/commons-cli-1.2.jar:/usr/local/hadoop/share/hadoop/common/lib/commons-math3-3.1.1.jar:/usr/local/hadoop/share/hadoop/common/lib/jasper-compiler-5.5.23.jar:/usr/local/hadoop/share/hadoop/common/lib/commons-compress-1.4.1.jar:/usr/local/hadoop/share/hadoop/common/lib/commons-io-2.4.jar:/usr/local/hadoop/share/hadoop/common/lib/jackson-mapper-asl-1.9.13.jar:/usr/local/hadoop/share/hadoop/common/lib/mockito-all-1.8.5.jar:/usr/local/hadoop/share/hadoop/common/lib/gson-2.2.4.jar:/usr/local/hadoop/share/hadoop/common/lib/api-asn1-api-1.0.0-M20.jar:/usr/local/hadoop/share/hadoop/common/lib/avro-1.7.4.jar:/usr/local/hadoop/share/hadoop/common/lib/api-util-1.0.0-M20.jar:/usr/local/hadoop/share/hadoop/common/lib/guava-11.0.2.jar:/usr/local/hadoop/share/hadoop/common/lib/jersey-core-1.9.jar:/usr/local/hadoop/share/hadoop/common/lib/snappy-java-1.0.4.1.jar:/usr/local/hadoop/share/hadoop/common/lib/jasper-runtime-5.5.23.jar:/usr/local/hadoop/share/hadoop/common/lib/htrace-core-3.0.4.jar:/usr/local/hadoop/share/hadoop/common/lib/commons-beanutils-core-1.8.0.jar:/usr/local/hadoop/share/hadoop/common/lib/hamcrest-core-1.3.jar:/usr/local/hadoop/share/hadoop/common/lib/commons-net-3.1.jar:/usr/local/hadoop/share/hadoop/common/lib/paranamer-2.3.jar:/usr/local/hadoop/share/hadoop/common/lib/jetty-6.1.26.jar:/usr/local/hadoop/share/hadoop/common/lib/jsp-api-2.1.jar:/usr/local/hadoop/share/hadoop/common/lib/java-xmlbuilder-0.4.jar:/usr/local/hadoop/share/hadoop/common/lib/curator-recipes-2.6.0.jar:/usr/local/hadoop/share/hadoop/common/lib/apacheds-kerberos-codec-2.0.0-M15.jar:/usr/local/hadoop/share/hadoop/common/lib/commons-logging-1.1.3.jar:/usr/local/hadoop/share/hadoop/common/lib/curator-framework-2.6.0.jar:/usr/local/hadoop/share/hadoop/common/lib/asm-3.2.jar:/usr/local/hadoop/share/hadoop/common/lib/jersey-server-1.9.jar:/usr/local/hadoop/share/hadoop/common/lib/log4j-1.2.17.jar:/usr/local/hadoop/share/hadoop/common/lib/jets3t-0.9.0.jar:/usr/local/hadoop/share/hadoop/common/lib/servlet-api-2.5.jar:/usr/local/hadoop/share/hadoop/common/lib/jetty-util-6.1.26.jar:/usr/local/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar:/usr/local/hadoop/share/hadoop/common/lib/jsr305-1.3.9.jar:/usr/local/hadoop/share/hadoop/common/lib/jsch-0.1.42.jar:/usr/local/hadoop/share/hadoop/common/lib/commons-httpclient-3.1.jar:/usr/local/hadoop/share/hadoop/common/lib/jackson-core-asl-1.9.13.jar:/usr/local/hadoop/share/hadoop/common/lib/stax-api-1.0-2.jar:/usr/local/hadoop/share/hadoop/common/lib/xz-1.0.jar:/usr/local/hadoop/share/hadoop/common/lib/netty-3.6.2.Final.jar:/usr/local/hadoop/share/hadoop/common/lib/junit-4.11.jar:/usr/local/hadoop/share/hadoop/common/lib/commons-configuration-1.6.jar:/usr/local/hadoop/share/hadoop/common/lib/commons-lang-2.6.jar:/usr/local/hadoop/share/hadoop/common/lib/xmlenc-0.52.jar:/usr/local/hadoop/share/hadoop/common/lib/jersey-json-1.9.jar:/usr/local/hadoop/share/hadoop/common/lib/hadoop-annotations-2.6.0.jar:/usr/local/hadoop/share/hadoop/common/lib/jaxb-impl-2.2.3-1.jar:/usr/local/hadoop/share/hadoop/common/lib/apacheds-i18n-2.0.0-M15.jar:/usr/local/hadoop/share/hadoop/common/lib/protobuf-java-2.5.0.jar:/usr/local/hadoop/share/hadoop/common/lib/commons-beanutils-1.7.0.jar:/usr/local/hadoop/share/hadoop/common/lib/jackson-xc-1.9.13.jar:/usr/local/hadoop/share/hadoop/common/lib/activation-1.1.jar:/usr/local/hadoop/share/hadoop/common/lib/hadoop-auth-2.6.0.jar:/usr/local/hadoop/share/hadoop/common/hadoop-common-2.6.0-tests.jar:/usr/local/hadoop/share/hadoop/common/hadoop-nfs-2.6.0.jar:/usr/local/hadoop/share/hadoop/common/hadoop-common-2.6.0.jar:/usr/local/hadoop/share/hadoop/hdfs:/usr/local/hadoop/share/hadoop/hdfs/lib/commons-codec-1.4.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/commons-el-1.0.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/commons-cli-1.2.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/commons-io-2.4.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/jackson-mapper-asl-1.9.13.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/xercesImpl-2.9.1.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/guava-11.0.2.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/jersey-core-1.9.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/jasper-runtime-5.5.23.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/htrace-core-3.0.4.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/jetty-6.1.26.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/xml-apis-1.3.04.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/jsp-api-2.1.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/commons-logging-1.1.3.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/asm-3.2.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/jersey-server-1.9.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/log4j-1.2.17.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/servlet-api-2.5.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/jetty-util-6.1.26.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/jsr305-1.3.9.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/jackson-core-asl-1.9.13.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/netty-3.6.2.Final.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/commons-daemon-1.0.13.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/commons-lang-2.6.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/xmlenc-0.52.jar:/usr/local/hadoop/share/hadoop/hdfs/lib/protobuf-java-2.5.0.jar:/usr/local/hadoop/share/hadoop/hdfs/hadoop-hdfs-2.6.0-tests.jar:/usr/local/hadoop/share/hadoop/hdfs/hadoop-hdfs-2.6.0.jar:/usr/local/hadoop/share/hadoop/hdfs/hadoop-hdfs-nfs-2.6.0.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/lib/zookeeper-3.4.6.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/lib/jettison-1.1.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/lib/jersey-client-1.9.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/lib/jline-0.9.94.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/lib/jackson-jaxrs-1.9.13.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/lib/commons-codec-1.4.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/lib/jaxb-api-2.2.2.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/lib/commons-collections-3.2.1.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/lib/commons-cli-1.2.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/lib/commons-compress-1.4.1.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/lib/commons-io-2.4.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/lib/jackson-mapper-asl-1.9.13.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/lib/jersey-guice-1.9.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/lib/leveldbjni-all-1.8.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/lib/guava-11.0.2.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/lib/jersey-core-1.9.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/lib/javax.inject-1.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/lib/jetty-6.1.26.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/lib/commons-logging-1.1.3.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/lib/asm-3.2.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/lib/jersey-server-1.9.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/lib/log4j-1.2.17.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/lib/servlet-api-2.5.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/lib/jetty-util-6.1.26.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/lib/jsr305-1.3.9.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/lib/commons-httpclient-3.1.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/lib/jackson-core-asl-1.9.13.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/lib/stax-api-1.0-2.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/lib/xz-1.0.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/lib/guice-servlet-3.0.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/lib/netty-3.6.2.Final.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/lib/aopalliance-1.0.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/lib/commons-lang-2.6.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/lib/jersey-json-1.9.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/lib/jaxb-impl-2.2.3-1.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/lib/protobuf-java-2.5.0.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/lib/jackson-xc-1.9.13.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/lib/activation-1.1.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/lib/guice-3.0.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-server-common-2.6.0.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-api-2.6.0.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-applications-unmanaged-am-launcher-2.6.0.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-server-tests-2.6.0.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-server-nodemanager-2.6.0.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-server-web-proxy-2.6.0.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-registry-2.6.0.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-2.6.0.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-server-resourcemanager-2.6.0.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-server-applicationhistoryservice-2.6.0.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-common-2.6.0.jar:/usr/local/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-client-2.6.0.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/commons-compress-1.4.1.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/commons-io-2.4.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/jackson-mapper-asl-1.9.13.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/jersey-guice-1.9.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/avro-1.7.4.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/leveldbjni-all-1.8.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/jersey-core-1.9.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/snappy-java-1.0.4.1.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/javax.inject-1.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/hamcrest-core-1.3.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/paranamer-2.3.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/asm-3.2.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/jersey-server-1.9.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/log4j-1.2.17.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/jackson-core-asl-1.9.13.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/xz-1.0.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/guice-servlet-3.0.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/netty-3.6.2.Final.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/junit-4.11.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/aopalliance-1.0.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/hadoop-annotations-2.6.0.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/protobuf-java-2.5.0.jar:/usr/local/hadoop/share/hadoop/mapreduce/lib/guice-3.0.jar:/usr/local/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-core-2.6.0.jar:/usr/local/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-common-2.6.0.jar:/usr/local/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-app-2.6.0.jar:/usr/local/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.6.0-tests.jar:/usr/local/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-shuffle-2.6.0.jar:/usr/local/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-plugins-2.6.0.jar:/usr/local/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.6.0.jar:/usr/local/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.6.0.jar:/usr/local/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-2.6.0.jar:/contrib/capacity-scheduler/*.jar
-STARTUP_MSG:   build = https://git-wip-us.apache.org/repos/asf/hadoop.git -r 2e18d179e4a8065b6a9f29cf2de9451891265cce; compiled by 'jenkins' on 2014-09-05T23:11Z
+STARTUP_MSG:   build = https://git-wip-us.apache.org/repos/asf/hadoop.git -r e3496499ecb8d220fba99dc5ed4c99c8f9e33bb1; compiled by 'jenkins' on 2014-11-13T21:10Z
-STARTUP_MSG:   java = 1.8.0_20
+STARTUP_MSG:   java = 1.8.0_25
 ************************************************************/
-/10/10 22:20:25 INFO namenode.NameNode: registered UNIX signal handlers for [TERM, HUP, INT]
+/12/17 14:15:35 INFO namenode.NameNode: registered UNIX signal handlers for [TERM, HUP, INT]
-/10/10 22:20:25 INFO namenode.NameNode: createNameNode [-format]
+/12/17 14:15:35 INFO namenode.NameNode: createNameNode [-format]
-Java HotSpot(TM) Client VM warning: You have loaded library /usr/local/hadoop-2.5.1/lib/native/libhadoop.so.1.0.0 which might have disabled stack guard. The VM will try to fix the stack guard now.
+/12/17 14:15:43 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
-It's highly recommended that you fix the library with 'execstack -c <libfile>', or link it with '-z noexecstack'.
+/12/17 14:15:45 WARN common.Util: Path /home/aipi2014/hdfs/name should be specified as a URI in configuration files. Please update hdfs configuration.
-/10/10 22:00:00 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
+/12/17 14:15:45 WARN common.Util: Path /home/aipi2014/hdfs/name should be specified as a URI in configuration files. Please update hdfs configuration.
-Formatting using clusterid: CID-ea5581cc-e9ae-4ad4-8b99-a44ccb3f4c02
+Formatting using clusterid: CID-43be76c0-81e9-42b7-bae9-b210dbd132f6
-/10/10 22:00:00 INFO namenode.FSNamesystem: fsLock is fair:true
+/12/17 14:15:45 INFO namenode.FSNamesystem: No KeyProvider found.
-/10/10 22:00:00 INFO blockmanagement.DatanodeManager: dfs.block.invalidate.limit=1000
+/12/17 14:15:46 INFO namenode.FSNamesystem: fsLock is fair:true
-/10/10 22:00:00 INFO blockmanagement.DatanodeManager: dfs.namenode.datanode.registration.ip-hostname-check=true
+/12/17 14:15:47 INFO blockmanagement.DatanodeManager: dfs.block.invalidate.limit=1000
-/10/10 22:00:00 INFO blockmanagement.BlockManager: dfs.namenode.startup.delay.block.deletion.sec is set to 000:00:00:00.000
+/12/17 14:15:47 INFO blockmanagement.DatanodeManager: dfs.namenode.datanode.registration.ip-hostname-check=true
-/10/10 22:00:00 INFO blockmanagement.BlockManager: The block deletion will start around 2014 Oct 10 22:20:27
+/12/17 14:15:47 INFO blockmanagement.BlockManager: dfs.namenode.startup.delay.block.deletion.sec is set to 000:00:00:00.000
-/10/10 22:00:00 INFO util.GSet: Computing capacity for map BlocksMap
+/12/17 14:15:47 INFO blockmanagement.BlockManager: The block deletion will start around 2014 Dec 17 14:15:47
-/10/10 22:00:00 INFO util.GSet: VM type       = 32-bit
+/12/17 14:15:47 INFO util.GSet: Computing capacity for map BlocksMap
-/10/10 22:00:00 INFO util.GSet: 2.0% max memory 966.7 MB = 19.3 MB
+/12/17 14:15:47 INFO util.GSet: VM type       = 32-bit
-/10/10 22:00:00 INFO util.GSet: capacity      = 2^22 = 4194304 entries
+/12/17 14:15:47 INFO util.GSet: 2.0% max memory 966.7 MB = 19.3 MB
-/10/10 22:00:00 INFO blockmanagement.BlockManager: dfs.block.access.token.enable=false
+/12/17 14:15:47 INFO util.GSet: capacity      = 2^22 = 4194304 entries
-/10/10 22:00:00 INFO blockmanagement.BlockManager: defaultReplication         = 1
+/12/17 14:15:50 INFO blockmanagement.BlockManager: dfs.block.access.token.enable=false
-/10/10 22:00:00 INFO blockmanagement.BlockManager: maxReplication             = 512
+/12/17 14:15:50 INFO blockmanagement.BlockManager: defaultReplication         = 1
-/10/10 22:00:00 INFO blockmanagement.BlockManager: minReplication             = 1
+/12/17 14:15:50 INFO blockmanagement.BlockManager: maxReplication             = 512
-/10/10 22:00:00 INFO blockmanagement.BlockManager: maxReplicationStreams      = 2
+/12/17 14:15:50 INFO blockmanagement.BlockManager: minReplication             = 1
-/10/10 22:00:00 INFO blockmanagement.BlockManager: shouldCheckForEnoughRacks  = false
+/12/17 14:15:50 INFO blockmanagement.BlockManager: maxReplicationStreams      = 2
-/10/10 22:00:00 INFO blockmanagement.BlockManager: replicationRecheckInterval = 3000
+/12/17 14:15:50 INFO blockmanagement.BlockManager: shouldCheckForEnoughRacks  = false
-/10/10 22:00:00 INFO blockmanagement.BlockManager: encryptDataTransfer        = false
+/12/17 14:15:50 INFO blockmanagement.BlockManager: replicationRecheckInterval = 3000
-/10/10 22:00:00 INFO blockmanagement.BlockManager: maxNumBlocksToLog          = 1000
+/12/17 14:15:50 INFO blockmanagement.BlockManager: encryptDataTransfer        = false
-/10/10 22:00:00 INFO namenode.FSNamesystem: fsOwner             = aipi2014 (auth:SIMPLE)
+/12/17 14:15:50 INFO blockmanagement.BlockManager: maxNumBlocksToLog          = 1000
-/10/10 22:00:00 INFO namenode.FSNamesystem: supergroup          = supergroup
+/12/17 14:15:51 INFO namenode.FSNamesystem: fsOwner             = aipi2014 (auth:SIMPLE)
-/10/10 22:00:00 INFO namenode.FSNamesystem: isPermissionEnabled = true
+/12/17 14:15:51 INFO namenode.FSNamesystem: supergroup          = supergroup
-/10/10 22:00:00 INFO namenode.FSNamesystem: HA Enabled: false
+/12/17 14:15:51 INFO namenode.FSNamesystem: isPermissionEnabled = true
-/10/10 22:00:00 INFO namenode.FSNamesystem: Append Enabled: true
+/12/17 14:15:51 INFO namenode.FSNamesystem: HA Enabled: false
-/10/10 22:00:00 INFO util.GSet: Computing capacity for map INodeMap
+/12/17 14:15:51 INFO namenode.FSNamesystem: Append Enabled: true
-/10/10 22:00:00 INFO util.GSet: VM type       = 32-bit
+/12/17 14:15:56 INFO util.GSet: Computing capacity for map INodeMap
-/10/10 22:00:00 INFO util.GSet: 1.0% max memory 966.7 MB = 9.7 MB
+/12/17 14:15:56 INFO util.GSet: VM type       = 32-bit
-/10/10 22:00:00 INFO util.GSet: capacity      = 2^21 = 2097152 entries
+/12/17 14:15:56 INFO util.GSet: 1.0% max memory 966.7 MB = 9.7 MB
-/10/10 22:00:00 INFO namenode.NameNode: Caching file names occuring more than 10 times
+/12/17 14:15:56 INFO util.GSet: capacity      = 2^21 = 2097152 entries
-/10/10 22:00:00 INFO util.GSet: Computing capacity for map cachedBlocks
+/12/17 14:16:00 INFO namenode.NameNode: Caching file names occuring more than 10 times
-/10/10 22:00:00 INFO util.GSet: VM type       = 32-bit
+/12/17 14:16:00 INFO util.GSet: Computing capacity for map cachedBlocks
-/10/10 22:00:00 INFO util.GSet: 0.25% max memory 966.7 MB = 2.4 MB
+/12/17 14:16:00 INFO util.GSet: VM type       = 32-bit
-/10/10 22:00:00 INFO util.GSet: capacity      = 2^19 = 524288 entries
+/12/17 14:16:00 INFO util.GSet: 0.25% max memory 966.7 MB = 2.4 MB
-/10/10 22:00:00 INFO namenode.FSNamesystem: dfs.namenode.safemode.threshold-pct = 0.9990000128746033
+/12/17 14:16:00 INFO util.GSet: capacity      = 2^19 = 524288 entries
-/10/10 22:00:00 INFO namenode.FSNamesystem: dfs.namenode.safemode.min.datanodes = 0
+/12/17 14:16:01 INFO namenode.FSNamesystem: dfs.namenode.safemode.threshold-pct = 0.9990000128746033
-/10/10 22:00:00 INFO namenode.FSNamesystem: dfs.namenode.safemode.extension     = 30000
+/12/17 14:16:01 INFO namenode.FSNamesystem: dfs.namenode.safemode.min.datanodes = 0
-/10/10 22:00:00 INFO namenode.FSNamesystem: Retry cache on namenode is enabled
+/12/17 14:16:01 INFO namenode.FSNamesystem: dfs.namenode.safemode.extension     = 30000
-/10/10 22:00:00 INFO namenode.FSNamesystem: Retry cache will use 0.03 of total heap and retry cache entry expiry time is 600000 millis
+/12/17 14:16:01 INFO namenode.FSNamesystem: Retry cache on namenode is enabled
-/10/10 22:00:00 INFO util.GSet: Computing capacity for map NameNodeRetryCache
+/12/17 14:16:01 INFO namenode.FSNamesystem: Retry cache will use 0.03 of total heap and retry cache entry expiry time is 600000 millis
-/10/10 22:00:00 INFO util.GSet: VM type       = 32-bit
+/12/17 14:16:01 INFO util.GSet: Computing capacity for map NameNodeRetryCache
-/10/10 22:00:00 INFO util.GSet: 0.029999999329447746% max memory 966.7 MB = 297.0 KB
+/12/17 14:16:01 INFO util.GSet: VM type       = 32-bit
-/10/10 22:00:00 INFO util.GSet: capacity      = 2^16 = 65536 entries
+/12/17 14:16:01 INFO util.GSet: 0.029999999329447746% max memory 966.7 MB = 297.0 KB
-/10/10 22:00:00 INFO namenode.NNConf: ACLs enabled? false
+/12/17 14:16:01 INFO util.GSet: capacity      = 2^16 = 65536 entries
-/10/10 22:00:00 INFO namenode.NNConf: XAttrs enabled? true
+/12/17 14:16:02 INFO namenode.NNConf: ACLs enabled? false
-/10/10 22:00:00 INFO namenode.NNConf: Maximum size of an xattr: 16384
+/12/17 14:16:02 INFO namenode.NNConf: XAttrs enabled? true
-/10/10 22:00:00 INFO namenode.FSImage: Allocated new BlockPoolId: BP-455075811-127.0.1.1-1412968829123
+/12/17 14:16:02 INFO namenode.NNConf: Maximum size of an xattr: 16384
-/10/10 22:00:00 INFO common.Storage: Storage directory /home/aipi2014/hdfs/namenode has been successfully formatted.
+/12/17 14:16:07 INFO namenode.FSImage: Allocated new BlockPoolId: BP-756723125-127.0.1.1-1418818563112
-/10/10 22:00:00 INFO namenode.NNStorageRetentionManager: Going to retain 1 images with txid >= 0
+/12/17 14:16:07 INFO common.Storage: Storage directory /home/aipi2014/hdfs/name has been successfully formatted.
-/10/10 22:00:00 INFO util.ExitUtil: Exiting with status 0
+/12/17 14:16:08 INFO namenode.NNStorageRetentionManager: Going to retain 1 images with txid >= 0
-/10/10 22:00:00 INFO namenode.NameNode: SHUTDOWN_MSG:
+/12/17 14:16:08 INFO util.ExitUtil: Exiting with status 0
+/12/17 14:16:08 INFO namenode.NameNode: SHUTDOWN_MSG:
 /************************************************************
 SHUTDOWN_MSG: Shutting down NameNode at ubuntu/127.0.1.1
@@ Line 321: / Line 322: @@
 aipi2014@ubuntu:~$ start-dfs.sh
 Starting namenodes on [localhost]
-localhost: starting namenode, logging to /usr/local/hadoop-2.5.1/logs/hadoop-aipi2014-namenode-ubuntu.out
+localhost: starting namenode, logging to /usr/local/hadoop-2.6.0/logs/hadoop-aipi2014-namenode-ubuntu.out
-localhost: starting datanode, logging to /usr/local/hadoop-2.5.1/logs/hadoop-aipi2014-datanode-ubuntu.out
+localhost: starting datanode, logging to /usr/local/hadoop-2.6.0/logs/hadoop-aipi2014-datanode-ubuntu.out
 Starting secondary namenodes [0.0.0.0]
 The authenticity of host '0.0.0.0 (0.0.0.0)' can't be established.
-ECDSA key fingerprint is bb:40:19:2c:64:82:15:d6:0f:a3:0e:a2:b2:a7:ae:5a.
+ECDSA key fingerprint is 36:dc:33:57:17:ce:fc:3d:d6:11:52:5e:b5:7f:24:7c.
 Are you sure you want to continue connecting (yes/no)? yes
 .0.0.0: Warning: Permanently added '0.0.0.0' (ECDSA) to the list of known hosts.
-.0.0.0: starting secondarynamenode, logging to /usr/local/hadoop-2.5.1/logs/hadoop-aipi2014-secondarynamenode-ubuntu.out
+.0.0.0: starting secondarynamenode, logging to /usr/local/hadoop-2.6.0/logs/hadoop-aipi2014-secondarynamenode-ubuntu.out
 </code>
     * pornire: ''start-dfs.sh''
@@ Line 335: / Line 336: @@
 aipi2014@ubuntu:~$ start-yarn.sh
 starting yarn daemons
-starting resourcemanager, logging to /usr/local/hadoop-2.5.1/logs/yarn-aipi2014-resourcemanager-ubuntu.out
+starting resourcemanager, logging to /usr/local/hadoop-2.6.0/logs/yarn-aipi2014-resourcemanager-ubuntu.out
-localhost: starting nodemanager, logging to /usr/local/hadoop-2.5.1/logs/yarn-aipi2014-nodemanager-ubuntu.out
+localhost: starting nodemanager, logging to /usr/local/hadoop-2.6.0/logs/yarn-aipi2014-nodemanager-ubuntu.out
 </code>
     * pornire: ''start-yarn.sh''
@@ Line 356: / Line 357: @@
 Framework-ul Hadoop pune la dispoziția utilizatorilor interfețe web ce permit monitorizarea cluster-ului:
-  * procesul NameNode ([[http://localhost:50070|]]) oferă informații cu privire la sistemul de fișiere distribuit HDFS, putând fi consultată inclusiv structura acestuia și descărcate resurse din cadrul său;\\ {{ :utile:hadoop_namenode.jpg?nolink&700 }}
+  * procesul NameNode ([[http://localhost:50070|]]) oferă informații cu privire la sistemul de fișiere distribuit HDFS, putând fi consultată inclusiv structura acestuia și descărcate resurse din cadrul său;\\ {{ :utile:hadoop_namenode.png?nolink&700 }}
-  * procesul ResourceManager ([[http://localhost:8088|]]) conține date referitoare la sarcinile care au fost rulate, starea acestora, putând fi consultate și statistici cu privire la aplicațiile executate.\\ {{ :utile:hadoop_resourcemanager.jpg?nolink&700 }}
+  * procesul ResourceManager ([[http://localhost:8088|]]) conține date referitoare la sarcinile care au fost rulate, starea acestora, putând fi consultate și statistici cu privire la aplicațiile executate.\\ {{ :utile:hadoop_resourcemanager.png?nolink&700 }}
-====== Instalare HBase 0.98.6.1 ======
+====== Instalare HBase 0.98.8 ======
 ===== Obținerea celei mai recente versiuni =====
@@ Line 365: / Line 366: @@
   * se descarcă cea mai recentă versiune stabilă (lista poate fi consultată pe [[http://www.apache.org/dyn/closer.cgi/hbase/|HBase Downloads]])\\ <code>
 aipi2014@ubuntu:~$ cd /tmp
-aipi2014@ubuntu:/tmp$ wget http://mirrors.koehn.com/apache/hbase/stable/hbase-0.98.6.1-hadoop2-bin.tar.gz
+aipi2014@ubuntu:/tmp$ wget http://mirrors.hostingromania.ro/apache.org/hbase/stable/hbase-0.98.8-hadoop2-bin.tar.gz
 </code>
   * se copiază în /usr/local, creându-se și o legătură simbolică spre aceasta\\ <code>
-aipi2014@ubuntu:/tmp$ tar xzf hbase-0.98.6.1-hadoop2-bin.tar.gz
+aipi2014@ubuntu:/tmp$ tar xzf hbase-0.98.8-hadoop2-bin.tar.gz
-aipi2014@ubuntu:/tmp$ sudo mv hbase-0.98.6.1-hadoop2 /usr/local/
+aipi2014@ubuntu:/tmp$ sudo mv hbase-0.98.8-hadoop2 /usr/local/
-aipi2014@ubuntu:/tmp$ sudo ln -s /usr/local/hbase-0.98.6.1-hadoop2/ /usr/local/hbase
+aipi2014@ubuntu:/tmp$ sudo ln -s /usr/local/hbase-0.98.8-hadoop2/ /usr/local/hbase
-aipi2014@ubuntu:/tmp$ rm -f hbase-0.98.6.1-hadoop2-bin.tar.gz
+aipi2014@ubuntu:/tmp$ rm -f hbase-0.98.8-hadoop2-bin.tar.gz
 </code>
   * se specifică utilizatorul curent drept proprietarul acestui director\\ <code>
@@ Line 379: / Line 380: @@
 ===== Suprascrierea bibliotecilor Hadoop =====
-În directorul $HBASE_INSTALL/lib, HBase stochează biblioteci Hadoop pe care le-ar folosi doar în situația în care nu ar fi utilizat în corelație cu acest framework (modul standalone). Întrucât această colecție de biblioteci nu este actualizată, acest proces trebuie realizat manual (bibliotecile respective din distribuția curentă Hadoop sunt copiate în contextul HBase).
+În directorul ''$HBASE_INSTALL/lib'', HBase stochează biblioteci Hadoop pe care le-ar folosi doar în situația în care nu ar fi utilizat în corelație cu acest framework (modul standalone). Întrucât această colecție de biblioteci nu este actualizată, acest proces trebuie realizat manual (bibliotecile respective din distribuția curentă Hadoop sunt copiate în contextul HBase).
 <code>
@@ Line 403: / Line 404: @@
 <file sh hbase-env.sh>
 ...
-export JAVA_HOME=/usr/local/java/jdk1.8.0_20
+export JAVA_HOME=/usr/local/java/jdk1.8.0_25
 export HBASE_REGIONSERVERS=/usr/local/hbase/conf/regionservers
 export HBASE_MANAGES_ZK=true
@@ Line 424: / Line 425: @@
 aipi2014@ubuntu:/usr/local/hbase/conf$ nano hbase-site.xml
 </code>
 <file xml hbase-site.xml>
 <configuration>
@@ Line 493: / Line 493: @@
 HBase Shell; enter 'help<RETURN>' for list of supported commands.
 Type "exit<RETURN>" to leave the HBase Shell
-Version 0.98.6.1-hadoop2, r96a1af660b33879f19a47e9113bf802ad59c7146, Sun Sep 14 21:27:25 PDT 2014
+.98.8-hadoop2, r6cfc8d064754251365e070a10a82eb169956d5fe, Fri Nov 14 18:26:29 PST 2014
 hbase(main):001:0> list
@@ Line 507: / Line 507: @@
 Prin intermediul interfeței web, disponibilă la [[http://localhost:60010|]], utilizatorul are la dispoziție doar operații de consultare a stării bazei de date sau a vizualizării tabelelor.
-{{ :utile:hadoop_hbase.jpg?nolink&700 }}
+{{ :utile:hadoop_hbase.png?nolink&700 }}
+====== Plugin Hadoop pentru Eclipse =====
+Dezvoltarea aplicațiilor Hadoop / HDFS în Eclipse poate fi facilitată de folosirea unui plugin:
+  * [[http://aipi2014.andreirosucojocaru.ro/hadoop-eclipse-kepler-plugin-2.2.0.jar|Hadoop 2.2.0]]
+  * [[http://aipi2014.andreirosucojocaru.ro/hadoop-eclipse-kepler-plugin-2.6.0.jar|Hadoop 2.6.0]]
+Arhiva ''.jar'' corespunzătoare acestuia trebuie copiată în subdirectorul ''plugins'' al mediului de dezvoltare Eclipse pentru a putea fi folosit corespunzător.
+<note important>Există posibilitatea de a exista unele probleme de compatibilitate cu anumite versiuni Hadoop / Eclipse.
+Pentru remedierea acestuia, plugin-ul trebuie compilat din surse.
+**1.** Se indică versiunile curente ale bibliotecilor Hadoop existente în versiunea instalată pe mașina locală, în fișierele
+  * ''~/hadoop2x-eclipse-plugin/ivy/libraries.properties''
+  * ''~/hadoop2x-eclipse-plugin/src/ivy/libraries.properties''
+Specificarea unor valori incorecte pentru aceste proprietăți va conduce la eșuarea procesului de obținere a unui plugin pe baza versiunilor curente de Eclipse și Hadoop instalate pe mașina locală.
+**2.** Se precizează căile către versiunile de Hadoop / Eclipse existente pe mașina pe care se dorește folosirea acestuia.
+<code>
+aipi2014@ubuntu:~/hadoop2x-eclipse-plugin$ cd src/contrib/eclipse-plugin
+aipi2014@ubuntu:~/hadoop2x-eclipse-plugin/src/contrib/eclipse-plugin$ ant jar -Dversion=2.6.0 -Declipse.home=/opt/eclipse -Dhadoop.home=/usr/local/hadoop
+</code>
+Arhiva ''.jar'' corespunzătoare plugin-ului obținut va fi plasată în ''~/hadoop2x-eclipse-plugin/build/contrib/eclipse-plugin''.
+</note>
+Se selectează ''Window'' → ''Open Perspective'' → ''Other...'' și din cadrul listei de perspective disponibile se selectează opțiunea Map/Reduce.
+{{ :utile:hadoop_plugin01.jpg?nolink }}
+În bara de perspective va apărea secțiunea ''Map/Reduce Locations'' în care vor trebui configurați anumiți parametrii legați de localizarea framework-ului Hadoop, prin:
+  - click dreapta în zona de lucru a perspectivei; sau
+  - accesarea pictogramei reprezentând un elefant și semnul + ({{:utile:hadoop_plugin_pictograma.jpg?nolink}}).
+{{ :utile:hadoop_plugin03a.jpg?nolink&750 }}
+{{ :utile:hadoop_plugin03b.jpg?nolink&750 }}
+Definirea unei locații Hadoop presupune:
+  * asocierea unei denumiri (nu există restricții în acest sens);
+  * specificarea adresei mașinii și a portului pe care rulează sistemul de fișiere HDFS (secțiunea ''DFS Master''): ''localhost:9000'';
+  * specificarea adresei mașinii și a portului pe care rulează procesul care gestionează aplicațiile de tip Map/Reduce (secțiunea ''Map/Reduce(V2) Master''): ''localhost:50020'';
+  * indicarea utilizatorului care va dezvolta aplicații de tip Map/Reduce.
+{{ :utile:hadoop_plugin02.jpg?nolink }}
+Astfel, în secțiunea Project Explorer va fi disponibil un navigator pentru sistemul de fișiere HDFS (DFS Locations) care oferă posibilitatea interacțiunii cu sistemul de fișiere local prin operații de tip încărcare / descărcare de diferite resurse.
+{{ :utile:hadoop_plugin04.jpg?nolink }}
+Configurarea unui proiect Map/Reduce se face prin ''New'' → ''Project'' → ''Map/Reduce'' → ''Map/Reduce Project'', specificându-se și locația unde a fost instalat framework-ul Hadoop pe discul local (''Use Default Hadoop'' → ''Configure Hadoop install directory...'')
+{{ :utile:hadoop_plugin05.jpg?nolink&600 }}
+{{ :utile:hadoop_plugin06.jpg?nolink&600 }}
+{{ :utile:hadoop_plugin07.jpg?nolink&600 }}
+Ulterior, clasele specifice acestui tip de aplicaţii (clasa principală, respectiv clasele Mapper şi Reducer) se vor genera automat (''New'' → ''Other'' → ''Map/Reduce'' de unde se selectează tipurile de clasă respective: ''MapReduce Driver'', ''Mapper'' şi ''Reducer''), utilizatorul completând numai conţinutul metodelor responsabile cu sarcinile executate pe fiecare cluster în parte şi cu prelucrarea rezultatelor finale pe baza informaţiilor provenite de la proces care a rulat o anumită parte a sarcinii.
+{{ :utile:hadoop_plugin08.jpg?nolink&600 }}
+{{ :utile:hadoop_plugin09.jpg?nolink&600 }}
+De asemenea, pentru clasele de tip ''Mapper'' / ''Reducer'', vor trebui specificate tipul de intrare şi tipul de ieşire prin indicarea clasei de bază parametrizate pe care o vor moșteni.
+{{ :utile:hadoop_plugin10.jpg?nolink&600 }}
+{{ :utile:hadoop_plugin11.jpg?nolink&600 }}
+{{ :utile:hadoop_plugin12.jpg?nolink&600 }}
+{{ :utile:hadoop_plugin13.jpg?nolink&600 }}
+Întrucât se doreşte ca rezultatele procesărilor distribuite să fie afişate în cadrul consolei Eclipse, se va crea un fişier ''log4j.properties'' care va fi plasat în subdirectorul ''bin'' al proiectului respectiv:
+<file java log4j.properties>
+log4j.rootLogger=INFO,console
+log4j.appender.console=org.apache.log4j.ConsoleAppender
+log4j.appender.console.target=System.err
+log4j.appender.console.layout=org.apache.log4j.PatternLayout
+log4j.appender.console.layout.ConversionPattern=%d{yy/MM/dd HH:mm:ss} %p %c{2}: %m%n
+</file>
+Rularea proiectului va trebui realizată în contexul Hadoop, astfel încât să fie invocate procesele care sunt responsabile cu distribuirea sarcinii, respectiv cu preluarea rezultatelor obţinute în urma prelucrărilor pe diferite sarcini. Astfel, din meniul contextual al proiectului va trebui să se specifice ''Run As'' → ''Run On Hadoop''.
+{{ :utile:hadoop_plugin14.jpg?nolink&500 }}
+Pe măsură ce este rulată aplicaţia, în consolă vor fi afişate actualizările privind sarcinile de tip map şi sarcinile de tip reduce, exprimate procentual.
+După ce aplicaţia a fost executată, vor fi afişate statistici referitoare la numărul de octeţi citiţi şi scrişi (din sistemul de fişiere local respectiv din sistemul de fişiere distribuit HDFS), numărul de operaţii de citire şi scriere (cu evidenţierea celor care au durat mai mult timp) ca şi numărul de înregistrări procesate în cadrul sarcinilor de tip map şi de tip reduce.
 ====== Resurse ======

Informaţii Administrative

Resurse

Laboratoare

Laborator 00 (partea I) - Introducere în aplicații de tip ERP
Laborator 00 (partea a II-a) - Sistemul de control al versiunilor GIT
Laborator 01 - Utilizarea sistemului de gestiune pentru baze de date MySQL
Laborator 02 - Gestiunea informațiilor dintr-o bază de date MySQL prin Java Database Connectivity
Laborator 03 - Dezvoltarea interfețelor grafice cu utilizatorul folosind JavaFX
Laborator 04 - Proiectarea de aplicații distribuite folosind mecanismul RMI
Laborator 05 - Dezvoltarea de aplicații distribuite folosind tehnologia CORBA
Laborator 06 - Construirea unui serviciu web folosind tehnologia JAX-WS
Laborator 07 - Realizarea de aplicații web folosind Java Servlets
Laborator 08 - Realizarea de aplicații web folosind JavaServer Pages
Laborator 09 - Comunicații Asincrone folosind Java Message Service
Laborator 10 - Proiectarea de aplicații distribuite folosind framework-ul Hadoop
- Sistemul Distribuit de Fișiere HDFS
- Sistemul de Gestiune pentru Baze de Date HBase
Laborator 11 - Proiectarea unui depozit de date cu Oracle Warehouse Builder 11gR2

Teme de Casă

Tutoriale

utile/instalare_hadoop_hbase.1413178620.txt.gz · Last modified: 2014/10/13 08:37 by Andrei Roșu-Cojocaru

Show page Old revisions

Media Manager Back to top