Archive for August, 2015

(Fixed) I/O error trying to sync with MASTER: connection lost

Error:

[2162] 26 Aug 15:26:26.795 # I/O error trying to sync with MASTER: connection lost

Above error happened when there is poor network connection, and you do the slave sync first time.

Possible Solution: increase slave buffer from 256m 64m 60 to 512M 128m 120

x.x.x.x:6379> config get client-output-buffer-limit
1) “client-output-buffer-limit”
2) “normal 0 0 0 slave 268435456 67108864 60 pubsub 33554432 8388608 60”
Read the rest of this entry »

No Comments

Install ZeroMQ plugin for MySQL

Following steps in CentOS 6.x

Download cmake , use the binary one. such as: cmake-3.2.3-Linux-x86_64.sh
http://www.cmake.org/download/

Install Cmake
chmod +x cmake-3.2.3-Linux-x86_64.sh
./cmake-3.2.3-Linux-x86_64.sh

Get mysql zeromq plugin:
git clone https://github.com/netkiller/mysql-zmq-plugin.git
Read the rest of this entry »

No Comments

php access hive2 server by thrift

You can get some guys did on https://github.com/search?utf8=%E2%9C%93&q=php+thrift+hive

When you try to access Hive2 without SASL. you may met following alerts:
TTransportException’ with message ‘TSocket: timed out reading 67108844 bytes from

Try add following lines to /etc/hive/conf/hive-site.xml

hive.server2.authentication
NOSASL
Client authentication types.
NONE: no authentication check
LDAP: LDAP/AD based authentication
KERBEROS: Kerberos/GSSAPI authentication
CUSTOM: Custom authentication provider
(Use with property hive.server2.custom.authentication.class)

Notes: NOSASL not equal NONE. NONE is default and means Plain SASL.

No Comments

(fixed) sqoop import from Oracle to Hive throw Heap size error

If you use CDH and Sqoop. you probably met following issue:

15/08/07 15:21:55 INFO manager.SqlManager: Using default fetchSize of 1000
15/08/07 15:21:55 INFO tool.CodeGenTool: Beginning code generation
15/08/07 15:21:56 INFO manager.OracleManager: Time zone has been set to GMT
15/08/07 15:21:56 INFO manager.SqlManager: Executing SQL statement: select xxx where (1 = 0)
Exception in thread “main” java.lang.OutOfMemoryError: Java heap space
at java.lang.reflect.Array.newArray(Native Method)
at java.lang.reflect.Array.newInstance(Array.java:70)
at oracle.jdbc.driver.BufferCache.get(BufferCache.java:226)
at oracle.jdbc.driver.PhysicalConnection.getCharBuffer(PhysicalConnection.java:7698)
at oracle.jdbc.driver.OracleStatement.prepareAccessors(OracleStatement.java:1013)
at oracle.jdbc.driver.T4CTTIdcb.receiveCommon(T4CTTIdcb.java:277)

This related with HDFS client HEAP settings. you can fix it by increase HEAP size for HDFS client.

from CDH:
Client Java Heap Size in Bytes.
Read the rest of this entry »

No Comments