when openssh will kick out idle users ?

when reading CIS security baseline. it mentions following lines:

Having no timeout value associated with a connection could allow an unauthorized user access to another user’s ssh session (e.g. user walks away from their computer and doesn’t lock the screen). Setting a timeout value at least reduces the risk of this happening..
While the recommended setting is 300 seconds (5 minutes), set this timeout value based on site policy. The recommended setting for ClientAliveCountMax is 0. In this case, the client session will be terminated after 5 minutes of idle time and no keepalive messages will be sent.

Review our settings:
ClientAliveInterval 300
ClientAliveCountMax 1

According to man sshd_config
If ClientAliveInterval (see below) is set to 15, and ClientAliveCountMax is left at the default, unresponsive SSH clients will be disconnected after approximately 45 seconds. This option applies to protocol version 2 only.

Interesting thing is: you won’t be kicked out after 45s if you set as above with Protocol 2.

From my test: the timeout will ONLY work when you set ClientAliveCountMax to 0. and idle time set to what you want kick out the user.

1 Comment

(Fixed) I/O error trying to sync with MASTER: connection lost


[2162] 26 Aug 15:26:26.795 # I/O error trying to sync with MASTER: connection lost

Above error happened when there is poor network connection, and you do the slave sync first time.

Possible Solution: increase slave buffer from 256m 64m 60 to 512M 128m 120

x.x.x.x:6379> config get client-output-buffer-limit
1) “client-output-buffer-limit”
2) “normal 0 0 0 slave 268435456 67108864 60 pubsub 33554432 8388608 60”
Read the rest of this entry »

No Comments

Install ZeroMQ plugin for MySQL

Following steps in CentOS 6.x

Download cmake , use the binary one. such as: cmake-3.2.3-Linux-x86_64.sh

Install Cmake
chmod +x cmake-3.2.3-Linux-x86_64.sh

Get mysql zeromq plugin:
git clone https://github.com/netkiller/mysql-zmq-plugin.git
Read the rest of this entry »

No Comments

php access hive2 server by thrift

You can get some guys did on https://github.com/search?utf8=%E2%9C%93&q=php+thrift+hive

When you try to access Hive2 without SASL. you may met following alerts:
TTransportException’ with message ‘TSocket: timed out reading 67108844 bytes from

Try add following lines to /etc/hive/conf/hive-site.xml

Client authentication types.
NONE: no authentication check
LDAP: LDAP/AD based authentication
KERBEROS: Kerberos/GSSAPI authentication
CUSTOM: Custom authentication provider
(Use with property hive.server2.custom.authentication.class)

Notes: NOSASL not equal NONE. NONE is default and means Plain SASL.

No Comments

(fixed) sqoop import from Oracle to Hive throw Heap size error

If you use CDH and Sqoop. you probably met following issue:

15/08/07 15:21:55 INFO manager.SqlManager: Using default fetchSize of 1000
15/08/07 15:21:55 INFO tool.CodeGenTool: Beginning code generation
15/08/07 15:21:56 INFO manager.OracleManager: Time zone has been set to GMT
15/08/07 15:21:56 INFO manager.SqlManager: Executing SQL statement: select xxx where (1 = 0)
Exception in thread “main” java.lang.OutOfMemoryError: Java heap space
at java.lang.reflect.Array.newArray(Native Method)
at java.lang.reflect.Array.newInstance(Array.java:70)
at oracle.jdbc.driver.BufferCache.get(BufferCache.java:226)
at oracle.jdbc.driver.PhysicalConnection.getCharBuffer(PhysicalConnection.java:7698)
at oracle.jdbc.driver.OracleStatement.prepareAccessors(OracleStatement.java:1013)
at oracle.jdbc.driver.T4CTTIdcb.receiveCommon(T4CTTIdcb.java:277)

This related with HDFS client HEAP settings. you can fix it by increase HEAP size for HDFS client.

from CDH:
Client Java Heap Size in Bytes.
Read the rest of this entry »

No Comments

OS Tuning tips for Hadoop Cluster

1.Decrease swappiness.
A value from 0 to 100 which controls the degree to which the system swaps. A high value prioritizes system performance, aggressively swapping processes out of physical memory when they are not active. A low value prioritizes interactivity and avoids swapping processes out of physical memory for as long as possible, which decreases response latency. The default value is 60.

Default value: 60
Recommend value: 5
Online Change: Y
# update online
echo 5 > /proc/sys/vm/swappiness

# update permanently , edit /etc/sysctl.conf and add following line:
vm.swappiness = 5
Read the rest of this entry »

No Comments

Migrate existing hadoop to CDH

Don’t need to sell CDH’s benefits. you should know it before want to migrate 🙂

Very Important, The following has been tested in my lab, all goes fine. can’t grantee if also works for you.
I migrate from Apache Hadoop 2.2 to CDH 5.3 or 5.4 all works.

## Backup namenode
# cd /mnt/hadoop/hdfs/name
# tar -cvf /root/nn_backup_data.tar .


## Install CDH WITHOUT create any service.
Read the rest of this entry »

No Comments

Hive Issue: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask

2015-06-25 11:16:52,554 INFO [main]: metastore.HiveMetaStore (HiveMetaStore.java:logInfo(623)) – 0: get_table : db=lntest tbl=tmp_download
2015-06-25 11:16:52,554 INFO [main]: HiveMetaStore.audit (HiveMetaStore.java:logAuditEvent(305)) – ugi=abcuser ip=unknown-ip-addr cmd=get_table : db=lntest tbl=
2015-06-25 11:16:52,555 ERROR [main]: bonecp.ConnectionHandle (ConnectionHandle.java:markPossiblyBroken(388)) – Database access problem. Killing off this connection and all r
emaining connections in the connection pool. SQL State = 08S01
2015-06-25 11:16:52,557 ERROR [main]: bonecp.BoneCP (BoneCP.java:destroyConnection(221)) – Error in attempting to close connection
com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: Communications link failure during rollback(). Transaction resolution unknown.
at sun.reflect.GeneratedConstructorAccessor114.newInstance(Unknown Source)

Problem related with MySQL for Hive settings:
mysql> show global variables like ‘%time%’ ;
Read the rest of this entry »

No Comments

How to upgrade WAP4400N firmware from 1.2.14 to 1.2.19

Seems it’s a old wireless Router. no new firmware released now.

Why I want to upgrade firmeware ? just for fun. didn’t feel much improvement. 🙂
Just got a official link

I had to use an GCC 3.x box. so I choose CentOS 4.8. so old.

Download firmware from above link, following the cross compile guide. will be straight forward.
Packages installed: gcc bison flex gcc-c++ zlib-devel
After a looong compile. got the new firemware 1.2.19.

Upgrade by your own risk 🙂

Get the firmware: http://www.linuxdict.com/soft/wap4400n_v1219.img

No Comments

Howto: Restore VG with LVM backup.

Like cbr in VxVM, LVM has also backup in /etc/lvm/archive/, so if you accidentally remove VG, or VG got destroyed. you can try following procedure.

Note: better do disk2disk copy before doing following. No grantee for your data.

[root@mytesthost:~]# cd /etc/lvm/archive/

[root@mytesthost:/etc/lvm/archive]# ls -lart .
total 36
-rw——- 1 root root 919 Oct 22 13:34 VG-data_00000-1897227548.vg
-rw——- 1 root root 923 Oct 22 13:51 VG-data_00001-659586034.vg
Read the rest of this entry »


No Comments