With the heat of Hadoop, the deployment and monitor becoming our system engineers’ focus.
More solution coming out now. such as Serengeti, but it aims for the Vsphere.
Bigtop, is a project under Apache. not bad, will give it a try.
HDP, Hortonworks Data Platform has been release recently.
Will try mention as much details doing the installation as I can.
- OS, Although HDP offically support CentOS 5.x/ RHEL 5.x. we still found some issue when doing the installation.
such as missing nodes when doing the deployment(can be fixed by update php5.1 to php5.3 but will bring more issue).
so we skip CentOS 5.x and use CentOS 6.x. it’s cool and have less issue ;), recommend install the OS with minimal packages.
# tips, please update the release file.
sed -i.bak ‘s/6.2/5.8/g’ /etc/redhat-release
- Disable Selinux and Iptables.
chkconfig iptables off
- password less auth
ssh-copy-id target_host (don’t forget the deployment host itself.)
- DNS, add the new nodes to DNS with PTR record. or you can add them to /etc/hosts.
- Enable Repo
# ruby repo
rpm -Uvh http://passenger.stealthymonkeys.com/rhel/6/passenger-release.noarch.rpm
rpm -Uvh http://public-repo-1.hortonworks.com/HDP-220.127.116.11/repos/centos5/hdp-release-18.104.22.168-1.el5.noarch.rpm
rpm -Uvh http://download.fedoraproject.org/pub/epel/6/i386/epel-release-6-7.noarch.rpm
- Install HMC (HDP core package)
Note: when you start the HMC. it ask you download JDK. recommend download by yourself from Oracle site.
[[email protected] ~]# yum install -y hmc php-process
[[email protected] ~]# service hmc start
Do you agree to Oracle’s Java License at
Would you like us to download the JDK binaries for you?(y/n)n
Please download jdk-6u31-linux-x64.bin and jdk-6u31-linux-i586.bin from Oracle to /var/run/hmc/downloads/
Starting HMC Installer [ OK ]
- Deploy HDP.
# update puppet for the nagios. remove php-pecl-json. because php5.3 already include it by default.
line 11+ /etc/puppet/master/modules/hdp-nagios/manifests/server/packages.pp
Then next follow the guide. Good luck.