doubaokun/metaq-php

MetaQ PHP client library

#MetaQ PHP Client library

• Support producer short connection and long connection, this require the config file
• Producer random select a broker-partition for load balance, no proxy required
• Support master async salve group cluster, circle backups
• Producer auto failover
• Consumer will read from both master and slaves
• Slave broker can be read by consumers
• Store consumer offset in zookeeper
• Consumer do not require the config file, they will fetch meta data from zookeeper
• Consumer will auto-rebalance if consumer group changes
• Suggest to use superviord auto restarting when consumer process dies. see http://supervisord.org/

##MetaQ server installation and configurations:

wget http://fnil.net/downloads/metaq-server-1.4.6.2.tar.gz

Web UI for MetaQ: http://ip:8120/
Web UI for zookeeper: https://github.com/qiuxiafei/zk-web.git

##Notes

Remember to set

updateConsumerOffsets=false

Master node configuration:

[system]
brokerId=0
numPartitions=1
serverPort=8123
dashboardHttpPort=8120
unflushThreshold=1000
unflushInterval=10000
maxSegmentSize=1073741824
maxTransferSize=1048576
deletePolicy=delete,168
deleteWhen=0 0 6,18 * * ?
flushTxLogAtCommit=1
stat=true
getProcessThreadCount=40
putProcessThreadCount=40
dataPath=/home/bruce/meta2

;; Update consumers offsets to current max offsets when consumers offsets are out of range of current broker's messages.
;; It must be false in production.But recommend to be true in development or test.
updateConsumerOffsets=false

[zookeeper]
zk.zkConnect=localhost:2181
zk.zkSessionTimeoutMs=30000
zk.zkConnectionTimeoutMs=30000
zk.zkSyncTimeMs=5000

;; Topics section
[topic=t1]
numPartitions=5

Slave configuration:

slaveId=1
slaveGroup=meta-slave-group
slaveMaxDelayInMills=50
autoSyncMasterConfig=true

Install PHP zookeeper extension:

1. Install libzookeeper
2. pecl install zookeeper-0.2.2

metaq-php's Issues

php消费者接受java消息生产者发送的消息有乱码

重现场景:
java 客户端使用Metamorphosis源码metamorphosis-example工程中的TradeSender发送消息(原来代码发送的是java对象,已经改为发送字符串,形如: abcde), 参照reademe安装php端zk依赖以后, 执行php SimpleConsumer.php, SimpleConsumer.php中部分主要代码如下:
17 while (1) {
18 $msgs = $metaq->getNext();
19 foreach ($msgs as $msg) {
20 $strMsg = $msg['msg'];
21 var_dump($strMsg);
22 # $obj = json_decode($strMsg);
23 # var_dump($obj);
24 }
输出结果:
??tabcde
结果中 ??t 为乱码?

生产者发送到topic，总是报找不到no broker accept topic test

php 客户端是否能够支持启动以后指定客户端的offset ?

php 客户端是否能够支持启动以后指定客户端的offset ? java 客户端中,可以通过consumerConfig 设置启动时候自动拉去某一个offset的值, 消费者经常会有需求就是启动的时候不要拉去之前的消息, 在某些场景下面, 以前的消息没啥用呢..... bruce 大神是否有计划支持一把 ？ 谢谢拉～