2017年6月18日 星期日

Hadoop 學習系列(二)

Hadoop with Java
Hbase



1. https://hadoop.apache.org/docs/r2.8.0/hadoop-mapreduce-client/hadoop-mapreduce-client-core/MapReduceTutorial.html

2.https://www.javatpoint.com/hadoop-tutorial

3.http://www.yiibai.com/hbase/hbase_installation.html

Fsck 說明

Unix/Linux 啟動後先會檢視檔案 /etc/fstab 中的 pass 欄 (第六欄) 檢查檔案系統的次序。pass 值為 0 時表示永不檢查,非零亦由小至大逐個檢查。以下面例子,GNU/Linux 會先檢查 /dev/sda6 ,然後檢查 /dev/sda1。
proc       /proc          proc     defaults        0  0
/dev/sda6  /              ext3     defaults        0  1
/dev/sda1  /boot          ext3     defaults        0  2
/dev/sda5  none           swap     sw              0  0


fsck mount Vs umount

Q. Can I run fsck or e2fsck command when Linux file system is mounted? Do you advice to run fsck on a live file system? I am using Cent OS.
A. No. Do not run fsck on a live or mounted file system. fsck is used to check and optionally repair one or more Linux file systems. Running fsck on a mounted filesystem can usually result in disk / data corruption. So please do not do it. You have two choices
(a) Take down system to single user mode and unmout system
(b) Boot from the installation CD into rescue mode

Ref :
https://blog.gtwang.org/linux/linux-fsck-examples/
https://www.cyberciti.biz/faq/can-i-run-fsck-or-e2fsck-when-linux-file-system-is-mounted/
https://serverfault.com/questions/29889/why-cant-you-fsck-a-mounted-partition

Linux Timezone settings

由於業務需求,服務器會內在不同的地區,先在這裡簡單介紹更改時區的問題。
日後會補充美國夏令及冬令時區轉換的問題。



CentOS 7

$ date

$ ls -l /etc/localtime

# timedatectl list-timezones

##*** Grep possible Asian timezones ***##
# timedatectl list-timezones | grep Asia


The syntax is as follows to set timezone and need to run as root user:
# timedatectl set-timezone time_zone
In this example, set timezone to America/Chicago
# timedatectl set-timezone America/Chicago
Change the current timezone in CentOS 6 and older
Type the following commands as root:
cp /etc/localtime /root/old.timezone
rm /etc/localtime
ln -s /usr/share/zoneinfo/America/Chicago /etc/localtime
Verify new settings by typing the following two commands:
date
ls -l /etc/localtime


Ref :
https://www.cyberciti.biz/faq/centos-linux-6-7-changing-timezone-command-line/

Hadoop 學習系列(一)

HDFS - Hadoop Distributed File System (HDFS)
1. http://www.mail-archive.com/common-user@hadoop.apache.org/msg15171.html


Hadoop on Centos 6
http://javadev.org/docs/hadoop/centos/6/installation/single-node-installation-on-centos-6/

YARN
https://www.ibm.com/developerworks/library/bd-yarn-intro/
https://www.ibm.com/developerworks/cn/opensource/os-cn-hadoop-yarn/


https://hadoop.apache.org/docs/current/hadoop-mapreduce-client/hadoop-mapreduce-client-core/MapReduceTutorial.html



嘗試運行wordCount, 出現stuck的情況。

Wordcount program is stuck in hadoop-2.3.0

https://stackoverflow.com/questions/23397763/wordcount-program-is-stuck-in-hadoop-2-3-0



Hadoop :
1. http://www.cc.ntu.edu.tw/chinese/epaper/0036/20160321_3609.html
2. https://blog.gtwang.org/linux/linux-hadoop-single-node-cluster-tutorial/
3. http://limitedcode.blogspot.hk/2014/10/hdfs-hadoop-distributed-file-system-hdfs.html

Tutorials :
1. https://www.digitalocean.com/community/tutorials