codislabs/redis-port

redis-port's Issues

Target key is busy

迁移数据从codis到redis时，报错 panic Ｔarget key is busy，请问这是什么原因，怎么解决？谢谢

redis4.0.1迁移到codis3.2.2报错

2018/07/19 14:34:08 sync.go:123: [INFO] +
2018/07/19 14:34:09 sync.go:73: [INFO] rdb file = 4920652048
2018/07/19 14:34:09 utils.go:361: [PANIC] parse rdb header error
[error]: verify version, invalid RDB version number 8
1 /home/travis/gopath/src/github.com/CodisLabs/redis-port/pkg/rdb/loader.go:44
github.com/CodisLabs/redis-port/pkg/rdb.(*Loader).Header
0 /home/travis/gopath/src/github.com/CodisLabs/redis-port/cmd/utils.go:360
main.newRDBLoader.func1
... ...
[stack]:
0 /home/travis/gopath/src/github.com/CodisLabs/redis-port/cmd/utils.go:361
main.newRDBLoader.func1
... ...
目前不支持4.x迁移？只是单个实例的redis 啊，没用cluster

port data eror

I use redis-port port data from redis to codis and some error like below:
2017/03/16 11:02:25 sync.go:235: [INFO] total=2908349195 - 2864928461 [ 98%] entry=5111118
2017/03/16 11:02:26 sync.go:235: [INFO] total=2908349195 - 2868982634 [ 98%] entry=5120844
2017/03/16 11:02:27 sync.go:235: [INFO] total=2908349195 - 2873281677 [ 98%] entry=5130978
2017/03/16 11:02:28 sync.go:235: [INFO] total=2908349195 - 2878549522 [ 98%] entry=5141108
2017/03/16 11:02:29 sync.go:235: [INFO] total=2908349195 - 2884614954 [ 99%] entry=5157670
2017/03/16 11:02:30 sync.go:235: [INFO] total=2908349195 - 2891919471 [ 99%] entry=5172628
2017/03/16 11:02:31 sync.go:235: [INFO] total=2908349195 - 2898386154 [ 99%] entry=5185882
2017/03/16 11:02:32 sync.go:235: [INFO] total=2908349195 - 2903861685 [ 99%] entry=5196923
2017/03/16 11:02:33 sync.go:235: [INFO] total=2908349195 - 2908300354 [ 99%] entry=5207640
2017/03/16 11:02:33 sync.go:235: [INFO] total=2908349195 - 2908349187 [ 99%] entry=5211041
2017/03/16 11:02:33 sync.go:237: [INFO] sync rdb done
2017/03/16 11:02:33 sync.go:259: [PANIC] parse command arguments failed
[error]: expect Array, but got <*redis.BulkBytes>
2 /usr/local/share/go/src/github.com/CodisLabs/redis-port/pkg/redis/resp.go:163
github.com/CodisLabs/redis-port/pkg/redis.AsArray
1 /usr/local/share/go/src/github.com/CodisLabs/redis-port/pkg/redis/handler.go:88
github.com/CodisLabs/redis-port/pkg/redis.ParseArgs
0 /usr/local/share/go/src/github.com/CodisLabs/redis-port/cmd/sync.go:258
main.(*cmdSync).SyncCommand.func2
this redis version is : redis_version:3.0.6
I has read this issue:#33
and hope some one can give me some help ,thanks

can not sync data from redis to codis

I use redis 2.8.21 of codis2.x, when i sync like this: ./redis-port sync --from=10.135.29.168:6381 --target=10.180.161.49:16379 --sockfile=redisport.tmp --filesize=2GB --psync , an error occurred :

2016/07/12 17:49:06 [PANIC] invalid sync response = '$EOF:915130babe4028703009b80a46956b6cac9adb54
', n = 0
[error]: strconv.ParseInt: parsing "EOF:915130babe4028703009b80a46956b6cac9adb54": invalid syntax

Then i capture tcp package, redis server return like below:
`*3
$5
## psync
$1
?
$2
-1
+FULLRESYNC 3c5def1ebe88c5fbc14c1b3a32dfed886dd5c81d 700308201
$EOF:8a4c50b7076680e1c6654b8901e22369946f24f9
REDIS0006....key-93794.AbCi.value-93794
[root@localhost conf.]# 2015/12/01 17:51:42 read tcp .10.58.65.240:52433->....64:2181: i/o timeout

I check the code,redis-port process by:

        if rsp[0] != '$' {
            log.Panicf("invalid sync response, rsp = '%s'", rsp)
        }
        n, err := strconv.Atoi(rsp[1 : len(rsp)-2])
        if err != nil || n <= 0 {
            log.PanicErrorf(err, "invalid sync response = '%s', n = %d", rsp, n)
        }

But redis process by:

retval = rdbSaveRioWithEOFMark(&slave_sockets,NULL);

int rdbSaveRioWithEOFMark(rio *rdb, int *error) {
    char eofmark[REDIS_EOF_MARK_SIZE];

    getRandomHexChars(eofmark,REDIS_EOF_MARK_SIZE);
    if (error) *error = 0;
    if (rioWrite(rdb,"$EOF:",5) == 0) goto werr;
    if (rioWrite(rdb,eofmark,REDIS_EOF_MARK_SIZE) == 0) goto werr;
    if (rioWrite(rdb,"\r\n",2) == 0) goto werr;
    if (rdbSaveRio(rdb,error) == REDIS_ERR) goto werr;
    if (rioWrite(rdb,eofmark,REDIS_EOF_MARK_SIZE) == 0) goto werr;
    return REDIS_OK;

how can solve this problem???

多 DB 的 port

目前 redis-port 似乎只可以支持将原 redis db 0 的数据同步到 codis。
是否可以支持将 Redis 其他 db 的数据同步到 codis, 如将 原 db1 的数据同步到 codis db 0

Can not solve migrating multiple db to single target db

I want to migrate Redis to Codis(good job dude) and I have multiple db on my redis server. But Codis only use db0. So I just want to migrate multiple db to single target db. Maybe should add a command to deal with this demand.

WRONGTYPE Operation against a key holding the wrong kind of value

2018/04/27 10:59:50 sync.go:103: [INFO] +
2018/04/27 10:59:51 sync.go:109: [INFO] sync: runid = "969c3a1e19ca54b66b2ad780d60edd43472a3a09", offset = 519121821
2018/04/27 10:59:51 sync.go:111: [INFO] sync: rdb file = 1663327129 (1.55gb)
2018/04/27 10:59:51 sync.go:208: [INFO] sync: (r/f,s/f,s) = (read,rdb.forward,rdb.skip/rdb.forward,rdb.skip)
2018/04/27 10:59:52 sync.go:250: [INFO] sync: rdb = 1663327129 - [ 24.74%] (r/f,s/f,s)=(786432/7628,0/0,0) ~ (768.00kb/-,-/-,-) ~ speed=(768.00kb/7628,0/0,0)
2018/04/27 10:59:53 sync.go:250: [INFO] sync: rdb = 1663327129 - [ 48.86%] (r/f,s/f,s)=(1441792/14493,0/0,0) ~ (1.38mb/-,-/-,-) ~ speed=(640.00kb/6865,0/0,0)
2018/04/27 10:59:54 sync.go:250: [INFO] sync: rdb = 1663327129 - [ 67.32%] (r/f,s/f,s)=(2752512/18423,0/0,0) ~ (2.62mb/-,-/-,-) ~ speed=(1.25mb/3930,0/0,0)
2018/04/27 10:59:54 libs.go:462: [PANIC] fetch redigo reply failed
[error]: WRONGTYPE Operation against a key holding the wrong kind of value
[stack]:
2 /home/travis/gopath/src/github.com/CodisLabs/redis-port/cmd/libs.go:462
main.redigoGetResponse
1 /home/travis/gopath/src/github.com/CodisLabs/redis-port/cmd/libs.go:588
main.doRestoreDBEntry.func3
0 /home/travis/gopath/src/github.com/CodisLabs/redis-port/cmd/libs.go:314
main.(*Job).Run.func1
... ...

做同步的时候报错WRONGTYPE Operation against a key holding the wrong kind of value

关于数据的sync

[root@iZ23y2cpqz2Z redis-port]# ./bin/redis-port sync --from 10.168.22.140:6379 --filterdb=3 --target 10.251.241.246:19000
2016/07/28 12:39:17 [INFO] set ncpu = 2, parallel = 2
2016/07/28 12:39:17 [INFO] sync from '10.168.22.140:6379' to '10.251.241.246:19000'
2016/07/28 12:39:18 [INFO] +
2016/07/28 12:39:19 [INFO] -
2016/07/28 12:39:19 [INFO] rdb file = 37806381
2016/07/28 12:39:20 [INFO] total=37806381 - 9613493 [ 25%] entry=0 ignore=1472
2016/07/28 12:39:21 [INFO] total=37806381 - 20463960 [ 54%] entry=0 ignore=3010
2016/07/28 12:39:22 [INFO] total=37806381 - 34201728 [ 90%] entry=0 ignore=5363
2016/07/28 12:39:23 [INFO] total=37806381 - 37806381 [100%] entry=0 ignore=7258
2016/07/28 12:39:23 [INFO] sync rdb done
2016/07/28 12:39:24 [INFO] sync: +forward=1 +nbypass=0 +nbytes=14
2016/07/28 12:39:25 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:39:26 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:39:27 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:39:28 [INFO] sync: +forward=1 +nbypass=0 +nbytes=14
2016/07/28 12:39:29 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:39:30 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:39:31 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:39:32 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:39:33 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:39:34 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:39:35 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:39:36 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:39:37 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:39:38 [INFO] sync: +forward=1 +nbypass=0 +nbytes=14
2016/07/28 12:39:39 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:39:40 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:39:41 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:39:42 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:39:43 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:39:44 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:39:45 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:39:46 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:39:47 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:39:48 [INFO] sync: +forward=1 +nbypass=0 +nbytes=14
2016/07/28 12:39:49 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:39:50 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:39:51 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:39:52 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:39:53 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:39:54 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:39:55 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:39:56 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:39:57 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:39:58 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:39:59 [INFO] sync: +forward=1 +nbypass=0 +nbytes=14
2016/07/28 12:40:00 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:40:01 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:40:02 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:40:03 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:40:04 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:40:05 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:40:06 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:40:07 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:40:08 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:40:09 [INFO] sync: +forward=1 +nbypass=0 +nbytes=14
2016/07/28 12:40:10 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:40:11 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:40:12 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:40:13 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:40:14 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:40:15 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:40:16 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:40:17 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:40:18 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:40:19 [INFO] sync: +forward=1 +nbypass=0 +nbytes=14
2016/07/28 12:40:20 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:40:21 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:40:22 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:40:23 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:40:24 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:40:25 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:40:26 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:40:27 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:40:28 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:40:29 [INFO] sync: +forward=1 +nbypass=0 +nbytes=14
2016/07/28 12:40:30 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:40:31 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:40:32 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:40:33 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:40:34 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:40:35 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:40:36 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:40:37 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:40:38 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:40:39 [INFO] sync: +forward=1 +nbypass=0 +nbytes=14
2016/07/28 12:40:40 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:40:41 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:40:42 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:40:43 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:40:44 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:40:45 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:40:46 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:40:47 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:40:48 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:40:49 [INFO] sync: +forward=1 +nbypass=0 +nbytes=14
2016/07/28 12:40:50 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:40:51 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:40:52 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:40:53 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:40:54 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:40:55 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:40:56 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:40:57 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:40:58 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:40:59 [INFO] sync: +forward=1 +nbypass=0 +nbytes=14
2016/07/28 12:41:00 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:41:01 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:41:02 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:41:03 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:41:04 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:41:05 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:41:06 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:41:07 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:41:08 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:41:09 [INFO] sync: +forward=1 +nbypass=0 +nbytes=14
2016/07/28 12:41:10 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:41:11 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:41:12 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:41:13 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:41:14 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:41:15 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:41:16 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:41:17 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:41:18 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:41:19 [INFO] sync: +forward=1 +nbypass=0 +nbytes=14
2016/07/28 12:41:20 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:41:21 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:41:22 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:41:23 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:41:24 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:41:25 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:41:26 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:41:27 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:41:28 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:41:29 [INFO] sync: +forward=1 +nbypass=0 +nbytes=14
2016/07/28 12:41:30 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:41:31 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:41:32 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:41:33 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:41:34 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:41:35 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:41:36 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:41:37 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:41:38 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:41:39 [INFO] sync: +forward=1 +nbypass=0 +nbytes=14
2016/07/28 12:41:40 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:41:41 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:41:42 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:41:43 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:41:44 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:41:45 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:41:46 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:41:47 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:41:48 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:41:49 [INFO] sync: +forward=1 +nbypass=0 +nbytes=14
2016/07/28 12:41:50 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:41:51 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:41:52 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:41:53 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:41:54 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:41:55 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:41:56 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:41:57 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:41:58 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:41:59 [INFO] sync: +forward=1 +nbypass=0 +nbytes=14
2016/07/28 12:42:00 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:42:01 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:42:02 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:42:03 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:42:04 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:42:05 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
2016/07/28 12:42:06 [INFO] sync: +forward=0 +nbypass=0 +nbytes=0
......

以上的过程只要有sync rdb done 就算成功了吗,还是的一直等待那, 因为一直在forward; 刷屏;

我只能ctrl + c了.

restore 出错unknown command 'slotsrestore'

./bin/redis-port restore -i dump.rdb -t 127.0.0.1:6379 -n 8
2015/10/20 16:29:06 [INFO] set ncpu = 8, parallel = 8
2015/10/20 16:29:06 [INFO] restore from 'dump.rdb' to '127.0.0.1:6379'
2015/10/20 16:29:06 [PANIC] restore command error
[error]: ERR unknown command 'slotsrestore'
[stack]:
1 /usr/local/codis/src/github.com/wandoulabs/redis-port/cmd/utils.go:207
main.restoreRdbEntry
0 /usr/local/codis/src/github.com/wandoulabs/redis-port/cmd/restore.go:102
main.func·013
... ...

redis-port在同步两个redis时，如何找到两个redis已经完全同步的时间点？

问题的场景是这样的：
利用redis-port同步redis1(master)和redis2(slave)。在完成了rdb的dump、传输、加载过程以及同步这段时间内的命令之后，锁住redis1（要求不能锁太久，在1秒之内），然后去做其他事情。做完之后，结束主从关系，并解锁redis1。

问题是：
1: 在锁redis1之前，如何判断主从已经达到同步状态？

2: 结束主从关系之前，如何确定主从已经达到同步状态？

为什么redis-port出现连不上redis的时候也要退出

我做了一个测试，在执行sync的时候，把redis弄挂掉，结果redis-port退出了。如果codis出现连不上zk退出还有一点点原因的话，redis-port执行数据同步的时候出现redis连接异常也要退出进程就实在无法理解了。为什么不可以在出现异常的时候仅仅打错误日志，每隔一段时间不停地尝试连接redis，不要杀掉进程了呢？

Error when sync or dump

2016/08/02 14:04:33 [INFO] set ncpu = 4, parallel = 4
2016/08/02 14:04:33 [INFO] sync from '10.1.101.168:6379' to '10.1.100.219:7018'
2016/08/02 14:04:34 [INFO] -
2016/08/02 14:04:35 [INFO] -
2016/08/02 14:04:36 [INFO] -
2016/08/02 14:04:37 [INFO] -
2016/08/02 14:04:38 [INFO] -
2016/08/02 14:04:39 [INFO] -
2016/08/02 14:04:39 [PANIC] invalid sync response = '$EOF:d7a06afab211ab2508092e1c019e4cdbac1908bb
', n = 0
[error]: strconv.ParseInt: parsing "EOF:d7a06afab211ab2508092e1c019e4cdbac1908bb": invalid syntax
[stack]:
0 /usr/lib/golang/src/github.com/CodisLabs/redis-port/cmd/utils.go:122
main.waitRdbDump.func1

decode 出现这个，是什么原因

[root@92780d52277f 6379]# ../../cmd decode --n=2 --i=./dump.rdb --o=test
2017/08/21 09:12:56 main.go:189: [INFO] set ncpu = 2, parallel = 2
2017/08/21 09:12:56 decode.go:45: [INFO] decode from './dump.rdb' to 'test'
2017/08/21 09:12:56 utils.go:361: [PANIC] parse rdb header error
[error]: verify version, invalid RDB version number 8
1 /home/gopath/src/github.com/CodisLabs/redis-port/pkg/rdb/loader.go:44
github.com/CodisLabs/redis-port/pkg/rdb.(*Loader).Header
0 /home/gopath/src/github.com/CodisLabs/redis-port/cmd/utils.go:360
main.newRDBLoader.func1
... ...
[stack]:
0 /home/gopath/src/github.com/CodisLabs/redis-port/cmd/utils.go:361
main.newRDBLoader.func1

sync.go里247～252行的作用是什么？

下面的代码为了做什么？清空buffer ?
go func() {
p := make([]byte, ReaderBufferSize)
for {
iocopy(c, ioutil.Discard, p, len(p))
}
}()

redis-port执行抛错utils.go:106

go get 下载redis-port，make编译完成后，执行dump抛错，如下：
[root@ redis-port]# ./bin/redis-port dump -f 192.168.0.1:19000 -o data/IP-0-1.rdb
2015/12/11 17:40:08 [INFO] set ncpu = 1, parallel = 1
2015/12/11 17:40:08 [INFO] dump from '192.168.0.1:19000' to 'data/IP-0-1.rdb'
2015/12/11 17:40:08 [PANIC] read sync response = ''

[stack]:
0 /usr/local/go/gocode/src/github.com/wandoulabs/redis-port/cmd/utils.go:106
main.func路025
... ...
[root@ redis-port]#

redis-port 命令行解析错误

redis-port 解析命令行，按照正确的usage输入命令，但是正常功能无法执行，总是输出
Usage:
redis-port decode [--ncpu=N] [--parallel=M] [--input=INPUT] [--output=OUTPUT]
redis-port restore [--ncpu=N] [--parallel=M] [--input=INPUT] --target=TARGET [--auth=AUTH] [--extra] [--faketime=FAKETIME] [--filterdb=DB] ...
查阅最新的代码，
源码文件中 redis-port/blob/master/cmd/main.go 85行 d, err := docopt.Parse(usage, nil, true, "", false)
docopt.Parse第二个参数为nil 是不是有问题，觉得应该是 os.Args ???

使用redis-port迁移集群版redis数据到主从版redis报错

请问下这个工具支持实时迁移redis集群版数据到主从版吗，redis版本2.8，谢谢！

从redis cluster迁移

@heroliy

有支持 redis cluster 迁移到codis的方法吗？貌似这个不支持

同步时，在rdb文件传输时，如果有很大的key，超过了ReadBufferSize 会出什么问题？

如题～

迁移过程中失败

redis源版本
redis_version:2.8.4
迁移到codis。
文件差不多有5个G

2015/09/10 12:39:00 [PANIC] parse rdb entry error

7   /usr/local/codis/src/github.com/wandoulabs/redis-port/pkg/rdb/reader.go:75
        github.com/wandoulabs/redis-port/pkg/rdb.(*rdbReader).Read
6   /usr/local/go/src/io/io.go:298
        io.ReadAtLeast
5   /usr/local/go/src/io/io.go:316
        io.ReadFull
4   /usr/local/codis/src/github.com/wandoulabs/redis-port/pkg/rdb/reader.go:234
        github.com/wandoulabs/redis-port/pkg/rdb.(*rdbReader).readFull
3   /usr/local/codis/src/github.com/wandoulabs/redis-port/pkg/rdb/reader.go:240
        github.com/wandoulabs/redis-port/pkg/rdb.(*rdbReader).readBytes
2   /usr/local/codis/src/github.com/wandoulabs/redis-port/pkg/rdb/reader.go:170
        github.com/wandoulabs/redis-port/pkg/rdb.(*rdbReader).readString
1   /usr/local/codis/src/github.com/wandoulabs/redis-port/pkg/rdb/loader.go:125
        github.com/wandoulabs/redis-port/pkg/rdb.(*Loader).NextBinEntry
0   /usr/local/codis/src/github.com/wandoulabs/redis-port/cmd/utils.go:247
        main.newRDBLoader.func1
    ... ...

[stack]:
0 /usr/local/codis/src/github.com/wandoulabs/redis-port/cmd/utils.go:248
main.newRDBLoader.func1

数据迁移目标codis集群的database选择

你好，请问redis-port支持target 的database选择吗？
例如，我要把某个redis集群中的数据迁移到codis3.2中的database 3

迁移数据时需要把原集群里key的一部分和节点相关的信息移除掉

你好！我们原先的Redis集群方案，存储在Redis的key里额外包含了和具体redis节点相关的信息。现在想切换到Codis，想把这部分信息移除掉。请问有办法在迁移的时候修改key去实现这样的效果么？

codis集群 3.2往Redis单实例 2.8版本迁移出错

redis-port版本：
version: 2017-05-02 12:36:19 +0800 @79e0f6ca38fbaf6f5c36ea704cf07a96ce9e9d79 @v1.2.1-2-g79e0f6c
compile: 2017-08-24 20:25:19 +0800 by go version go1.8.3 linux/amd64

命令：
redis-port sync --ncpu=2 --parallel=12 --from=127.0.0.1:7384 --password=****** --psync --target=127.0.0.1:7397 --auth=****** --redis --sockfile=buffer_7384.tmp --filesize=8gb

结果：
2018/09/19 11:40:03 utils.go:216: [PANIC] RESTORE command error
[error]: ERR wrong number of arguments for 'restore' command
[stack]:
1 /data/gopath/src/github.com/CodisLabs/redis-port/cmd/utils.go:216
main.restoreRdbEntry
0 /data/gopath/src/github.com/CodisLabs/redis-port/cmd/sync.go:212
main.(*cmdSync).SyncRDBFile.func1.1
... ...

DO NOT sync data from redis to codis-proxy

Because codis-proxy dose not guarantee to execute redis commands orderly

redis使用redis-port迁移至codis时报错

[root@go_cache bin]# ./redis-port sync -f 127.0.0.1:6379 -t 172.31.12.92:6499 -n 8
2015/05/24 08:30:07 [INFO] set ncpu = 8, parallel = 8
2015/05/24 08:30:07 [INFO] sync from '127.0.0.1:6379' to '172.31.12.92:6499'
2015/05/24 08:30:08 [INFO] rdb file = 1653523
2015/05/24 08:30:08 [PANIC] parse rdb checksum error
[error]: checksum validation failed
    1   /data/rpmbuild/rpmbuild/project/redis-port/BUILD/src/github.com/wandoulabs/redis-port/pkg/rdb/loader.go:51
            github.com/wandoulabs/redis-port/pkg/rdb.(*Loader).Footer
    0   /data/rpmbuild/rpmbuild/project/redis-port/BUILD/src/github.com/wandoulabs/redis-port/cmd/utils.go:184
            main.func·024
        ... ...
[stack]:
    0   /data/rpmbuild/rpmbuild/project/redis-port/BUILD/src/github.com/wandoulabs/redis-port/cmd/utils.go:185
            main.func·024
        ... ...
[root@go_cache bin]#

出现以上错误就中断了

redis-port:panic: runtime error: invalid memory address or nil pointer dereference

使用redis-port同步数据从redis到codis，数据量有7G，400多万个key，在sync rdb阶段crash，感觉像是在建立到target连接读取返回值时hang住了？实验了多次，每次停在了不同的key上，应该和源数据没关系？下面是crash log。多谢
panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xb code=0x1 addr=0x0 pc=0x41caa9]

goroutine 7 [running]:
main.newRDBLoader.func1(0xc820075ec0, 0xc82001e8a0, 0xc820079e00)
/home/easemob/go/src/github.com/left2right/redis-port/cmd/utils.go:331 +0x639
created by main.newRDBLoader
/home/easemob/go/src/github.com/left2right/redis-port/cmd/utils.go:344 +0x68

goroutine 1 [select]:
main.(_cmdSync).SyncRDBFile(0xc820079e00, 0xc82001e8a0, 0x7fffd855a4e3, 0x26, 0x0, 0x0, 0x3b8a4c5f)
/home/easemob/go/src/github.com/left2right/redis-port/cmd/sync.go:227 +0xb2b
main.(_cmdSync).Main(0xc820079e00)
/home/easemob/go/src/github.com/left2right/redis-port/cmd/sync.go:91 +0x863
main.main()
/home/easemob/go/src/github.com/left2right/redis-port/cmd/main.go:377 +0x24c3

goroutine 17 [syscall, 6 minutes, locked to thread]:
runtime.goexit()
/usr/local/go/src/runtime/asm_amd64.s:1721 +0x1

goroutine 8 [chan receive, 6 minutes]:
main.(_cmdSync).SyncRDBFile.func1(0xc820075f20, 0x7fffd855a4e3, 0x26, 0x0, 0x0, 0xc820075ec0, 0xc820079e00)
/home/easemob/go/src/github.com/left2right/redis-port/cmd/sync.go:222 +0x136
created by main.(_cmdSync).SyncRDBFile
/home/easemob/go/src/github.com/left2right/redis-port/cmd/sync.go:224 +0x110

goroutine 9 [IO wait]:
net.runtime_pollWait(0x2b88bf6e7388, 0x72, 0xc820076140)
/usr/local/go/src/runtime/netpoll.go:157 +0x60
net.(_pollDesc).Wait(0xc82127eae0, 0x72, 0x0, 0x0)
/usr/local/go/src/net/fd_poll_runtime.go:73 +0x3a
net.(_pollDesc).WaitRead(0xc82127eae0, 0x0, 0x0)
/usr/local/go/src/net/fd_poll_runtime.go:78 +0x36
net.(_netFD).Read(0xc82127ea80, 0xc82001d000, 0x1000, 0x1000, 0x0, 0x2b88bfd21028, 0xc820076140)
/usr/local/go/src/net/fd_unix.go:232 +0x23a
net.(_conn).Read(0xc820030018, 0xc82001d000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
/usr/local/go/src/net/net.go:172 +0xe4
bufio.(_Reader).fill(0xc8200740c0)
/usr/local/go/src/bufio/bufio.go:97 +0x1e9
bufio.(_Reader).ReadSlice(0xc8200740c0, 0xa, 0x0, 0x0, 0x0, 0x0, 0x0)
/usr/local/go/src/bufio/bufio.go:328 +0x21a
github.com/garyburd/redigo/redis.(_conn).readLine(0xc8200ac000, 0x0, 0x0, 0x0, 0x0, 0x0)
/home/easemob/go/src/github.com/left2right/redis-port/Godeps/_workspace/src/github.com/garyburd/redigo/redis/conn.go:338 +0x5a
github.com/garyburd/redigo/redis.(_conn).readReply(0xc8200ac000, 0x0, 0x0, 0x0, 0x0)
/home/easemob/go/src/github.com/left2right/redis-port/Godeps/_workspace/src/github.com/garyburd/redigo/redis/conn.go:411 +0x57
github.com/garyburd/redigo/redis.(_conn).Do(0xc8200ac000, 0x69ff40, 0xc, 0xc824c0a7e0, 0x3, 0x3, 0x0, 0x0, 0x0, 0x0)
/home/easemob/go/src/github.com/left2right/redis-port/Godeps/_workspace/src/github.com/garyburd/redigo/redis/conn.go:559 +0x6b2
main.restoreRdbEntry(0x2b88bfd270e8, 0xc8200ac000, 0xc824bf0680)
/home/easemob/go/src/github.com/left2right/redis-port/cmd/utils.go:286 +0x1c6a
main.(_cmdSync).SyncRDBFile.func1.1(0xc8212056c0, 0x7fffd855a4e3, 0x26, 0x0, 0x0, 0xc820075ec0, 0xc820079e00)
/home/easemob/go/src/github.com/left2right/redis-port/cmd/sync.go:216 +0x377
created by main.(*cmdSync).SyncRDBFile.func1
/home/easemob/go/src/github.com/left2right/redis-port/cmd/sync.go:219 +0xe7

/bin/sh: line 1: ./autogen.sh: No such file or directory

redis-4.x-cgo分支make的时候提示找不到这个文件...

使用redis-port sync 之后一直不停的刷新下面内容

您好：
我使用redis-port sync...命令来将redis的数据同步到codis中，但是一直出现下面内容在刷新，
请问：
1、sync是不是永远会执行下去？
2、“sync: +forward=12547 +nbypass=0 +nbytes=1588130”具体每个值是什么意思呢？比如+forward、+nbypass？
下面是刷屏内容：
2016/09/21 18:34:59 [INFO] sync: +forward=12547 +nbypass=0 +nbytes=1588130
2016/09/21 18:35:00 [INFO] sync: +forward=12121 +nbypass=0 +nbytes=1524604
2016/09/21 18:35:01 [INFO] sync: +forward=12721 +nbypass=0 +nbytes=1609632
2016/09/21 18:35:02 [INFO] sync: +forward=11180 +nbypass=0 +nbytes=1405242
2016/09/21 18:35:03 [INFO] sync: +forward=11215 +nbypass=0 +nbytes=1414128
2016/09/21 18:35:04 [INFO] sync: +forward=11677 +nbypass=0 +nbytes=1474217
2016/09/21 18:35:05 [INFO] sync: +forward=12818 +nbypass=0 +nbytes=1617235
2016/09/21 18:35:06 [INFO] sync: +forward=12489 +nbypass=0 +nbytes=1578302
2016/09/21 18:35:07 [INFO] sync: +forward=12898 +nbypass=0 +nbytes=1624360
2016/09/21 18:35:08 [INFO] sync: +forward=12558 +nbypass=0 +nbytes=1584386
2016/09/21 18:35:09 [INFO] sync: +forward=10635 +nbypass=0 +nbytes=1342169
2016/09/21 18:35:10 [INFO] sync: +forward=10354 +nbypass=0 +nbytes=1309103
2016/09/21 18:35:11 [INFO] sync: +forward=12291 +nbypass=0 +nbytes=1548088
2016/09/21 18:35:12 [INFO] sync: +forward=4331 +nbypass=0 +nbytes=549110
2016/09/21 18:35:13 [INFO] sync: +forward=11431 +nbypass=0 +nbytes=1441636
2016/09/21 18:35:14 [INFO] sync: +forward=10927 +nbypass=0 +nbytes=1376418
......

redis-port 迁移数据报错

2017/11/01 17:27:40 main.go:189: [INFO] set ncpu = 4, parallel = 16
2017/11/01 17:27:40 sync.go:56: [INFO] sync from '10.10.10.52:6679' to '10.10.10.55:8899'
2017/11/01 17:27:40 utils.go:139: [PANIC] invalid psync response, fullsync
[error]: bad resp CRLF end
6 /home/zhouyi/go/src/github.com/CodisLabs/redis-port/pkg/redis/decoder.go:166
github.com/CodisLabs/redis-port/pkg/redis.(*decoder).decodeSingleLineBulkBytesArray
5 /home/zhouyi/go/src/github.com/CodisLabs/redis-port/pkg/redis/decoder.go:85
github.com/CodisLabs/redis-port/pkg/redis.(*decoder).decodeResp
4 /home/zhouyi/go/src/github.com/CodisLabs/redis-port/pkg/redis/decoder.go:28
github.com/CodisLabs/redis-port/pkg/redis.Decode
3 /home/zhouyi/go/src/github.com/CodisLabs/redis-port/cmd/utils.go:137
main.sendPSyncFullsync
2 /home/zhouyi/go/src/github.com/CodisLabs/redis-port/cmd/sync.go:115
main.(*cmdSync).SendPSyncCmd
1 /home/zhouyi/go/src/github.com/CodisLabs/redis-port/cmd/sync.go:67
main.(*cmdSync).Main
0 /home/zhouyi/go/src/github.com/CodisLabs/redis-port/cmd/main.go:199
main.main
... ...
[stack]:
3 /home/zhouyi/go/src/github.com/CodisLabs/redis-port/cmd/utils.go:139
main.sendPSyncFullsync
2 /home/zhouyi/go/src/github.com/CodisLabs/redis-port/cmd/sync.go:115
main.(*cmdSync).SendPSyncCmd
1 /home/zhouyi/go/src/github.com/CodisLabs/redis-port/cmd/sync.go:67
main.(*cmdSync).Main
0 /home/zhouyi/go/src/github.com/CodisLabs/redis-port/cmd/main.go:199
main.main

迁移使用的命令是
./redis-port sync --psync --ncpu=4 -p 16 --from=10.10.10.52:6679 --filterdb=1 --target=10.10.10.55:8899 --sockfile=test.tmp --filesize=32gb
redis客户端是3.2版本的

无法恢复数据

/data/package/redis-port/redis-port restore -i save.rdb -t 192.168.30.200:6380
2018/04/18 17:53:20 main.go:189: [INFO] set ncpu = 4, parallel = 4
2018/04/18 17:53:20 restore.go:53: [INFO] restore from 'save.rdb' to '192.168.30.200:6380'
2018/04/18 17:53:20 utils.go:211: [PANIC] SLOTSRESTORE command error
[error]: ERR unknown command 'SLOTSRESTORE'
[stack]:
1 /home/travis/gopath/src/github.com/CodisLabs/redis-port/cmd/utils.go:211
main.restoreRdbEntry
0 /home/travis/gopath/src/github.com/CodisLabs/redis-port/cmd/restore.go:102
main.(*cmdRestore).RestoreRDBFile.func1.1
... ...

Twemproxy 迁移数据到codis报错

当我使用./redis-port sync --parallel=1 --from=192.168.81.170:12121 --password=XXX --target=192.168.81.220:19000 --auth=XXXX 命令将数据从Twemproxy 迁移到codis报如下错误：
2018/08/01 12:30:17 main.go:189: [INFO] set ncpu = 4, parallel = 4
2018/08/01 12:30:17 sync.go:56: [INFO] sync from '192.168.81.170:12121' to '192.168.81.220:19000'
2018/08/01 12:30:18 utils.go:109: [PANIC] read sync response = ''
[error]: EOF
[stack]:
0 /home/travis/gopath/src/github.com/CodisLabs/redis-port/cmd/utils.go:109
main.waitRdbDump.func1
... ...
请问是什么原因，怎么解决？用的版本是redis-port-v1.2.1-go1.7.5-linux。

编译命令

能给下编译命令吗？

-ERR Unable to perform background save

2015/07/15 15:21:33 [panic]: invalid sync response, rsp = '-ERR Unable to perform background save
'
/home/go/src/github.com/wandoulabs/codis/ext/redis-port/cmd/utils.go:103
github.com/wandoulabs/codis/ext/redis-port/cmd.func·016

编译提示：import cycle not allowed

编译提示：import cycle not allowed

求解

use of closed network connection

错误信息如下

[error]: read tcp 172.16.1.42:42002: use of closed network connection
[stack]:
    1   /root/disaster-recover/cmd/utils.go:223
            main.iocopy
    0   /root/disaster-recover/cmd/main.go:484
            main.func·007

不是每次都出现这个错误 ，偶尔出现。
484行就是如下的一段代码：

481 go func() {
482     p := make([]byte, ReaderBufferSize)
483     for {
484         iocopy(c, ioutil.Discard, p, len(p))
485     }
486 }()

迁移的时候异常

大神们好！ 首先感谢这个工具让我有机会能把数据轻松迁移到codis。

最近做迁移的时候遇到一个异常 详细日志如下

2017/10/20 17:06:12 sync.go:235: [INFO] total=3797108680 -    550792228 [ 14%]  entry=5794370
2017/10/20 17:06:13 sync.go:235: [INFO] total=3797108680 -    551975102 [ 14%]  entry=5810504
2017/10/20 17:06:14 sync.go:235: [INFO] total=3797108680 -    552631996 [ 14%]  entry=5819927
2017/10/20 17:06:15 sync.go:235: [INFO] total=3797108680 -    552631996 [ 14%]  entry=5819927
2017/10/20 17:06:16 sync.go:235: [INFO] total=3797108680 -    552631996 [ 14%]  entry=5819927
2017/10/20 17:06:17 sync.go:235: [INFO] total=3797108680 -    552631996 [ 14%]  entry=5819927
2017/10/20 17:06:18 sync.go:235: [INFO] total=3797108680 -    552631996 [ 14%]  entry=5819927
2017/10/20 17:06:19 sync.go:235: [INFO] total=3797108680 -    552631996 [ 14%]  entry=5819927
2017/10/20 17:06:20 sync.go:235: [INFO] total=3797108680 -    552631996 [ 14%]  entry=5819927
2017/10/20 17:06:21 sync.go:235: [INFO] total=3797108680 -    552631996 [ 14%]  entry=5819927
2017/10/20 17:06:22 sync.go:235: [INFO] total=3797108680 -    552631996 [ 14%]  entry=5819927
2017/10/20 17:06:23 sync.go:235: [INFO] total=3797108680 -    552631996 [ 14%]  entry=5819927
2017/10/20 17:06:24 sync.go:235: [INFO] total=3797108680 -    552631996 [ 14%]  entry=5819927
2017/10/20 17:06:25 sync.go:235: [INFO] total=3797108680 -    552631996 [ 14%]  entry=5819927
2017/10/20 17:06:26 sync.go:235: [INFO] total=3797108680 -    552631996 [ 14%]  entry=5819927
2017/10/20 17:06:27 sync.go:235: [INFO] total=3797108680 -    552631996 [ 14%]  entry=5819927
2017/10/20 17:06:28 sync.go:235: [INFO] total=3797108680 -    552631996 [ 14%]  entry=5819927
2017/10/20 17:06:29 sync.go:235: [INFO] total=3797108680 -    552631996 [ 14%]  entry=5819927
2017/10/20 17:06:30 sync.go:235: [INFO] total=3797108680 -    552631996 [ 14%]  entry=5819927
2017/10/20 17:06:31 sync.go:235: [INFO] total=3797108680 -    552631996 [ 14%]  entry=5819927
2017/10/20 17:06:32 sync.go:235: [INFO] total=3797108680 -    552631996 [ 14%]  entry=5819927
2017/10/20 17:06:33 sync.go:235: [INFO] total=3797108680 -    552631996 [ 14%]  entry=5819927
2017/10/20 17:06:34 sync.go:235: [INFO] total=3797108680 -    552631996 [ 14%]  entry=5819927
2017/10/20 17:06:35 sync.go:235: [INFO] total=3797108680 -    552631996 [ 14%]  entry=5819927
2017/10/20 17:06:36 sync.go:235: [INFO] total=3797108680 -    552631996 [ 14%]  entry=5819927
2017/10/20 17:06:37 sync.go:235: [INFO] total=3797108680 -    552631996 [ 14%]  entry=5819927
2017/10/20 17:06:38 sync.go:235: [INFO] total=3797108680 -    552631996 [ 14%]  entry=5819927
2017/10/20 17:06:39 sync.go:235: [INFO] total=3797108680 -    552631996 [ 14%]  entry=5819927
2017/10/20 17:06:40 sync.go:235: [INFO] total=3797108680 -    552631996 [ 14%]  entry=5819927
2017/10/20 17:06:41 sync.go:235: [INFO] total=3797108680 -    552631996 [ 14%]  entry=5819927
2017/10/20 17:06:42 utils.go:365: [PANIC] parse rdb entry error
[error]: EOF
    7   /home/travis/gopath/src/github.com/CodisLabs/redis-port/pkg/rdb/reader.go:74
            github.com/CodisLabs/redis-port/pkg/rdb.(*rdbReader).Read
    6   /home/travis/.gimme/versions/go1.7.5.linux.amd64/src/io/io.go:307
            io.ReadAtLeast
    5   /home/travis/.gimme/versions/go1.7.5.linux.amd64/src/io/io.go:325
            io.ReadFull
    4   /home/travis/gopath/src/github.com/CodisLabs/redis-port/pkg/rdb/reader.go:243
            github.com/CodisLabs/redis-port/pkg/rdb.(*rdbReader).readFull
    3   /home/travis/gopath/src/github.com/CodisLabs/redis-port/pkg/rdb/reader.go:249
            github.com/CodisLabs/redis-port/pkg/rdb.(*rdbReader).readBytes
    2   /home/travis/gopath/src/github.com/CodisLabs/redis-port/pkg/rdb/reader.go:157
            github.com/CodisLabs/redis-port/pkg/rdb.(*rdbReader).readString
    1   /home/travis/gopath/src/github.com/CodisLabs/redis-port/pkg/rdb/loader.go:142
            github.com/CodisLabs/redis-port/pkg/rdb.(*Loader).NextBinEntry
    0   /home/travis/gopath/src/github.com/CodisLabs/redis-port/cmd/utils.go:364
            main.newRDBLoader.func1
        ... ...
[stack]:
    0   /home/travis/gopath/src/github.com/CodisLabs/redis-port/cmd/utils.go:365
            main.newRDBLoader.func1

重试过好几次了,依旧这样

redis-port跑不满cpu

32核的机器，用-n 24 -p 1024 但是cpu、网卡、硬盘都没瓶颈，所以导入速度还不是很理想，是这些参数还可以加大么？

rdb version 7

redis 3.2开始使用quickList类型来存储了，而且不再兼容前面的版本，会支持rdb version 7吗？

这是什么问题

2017/09/13 22:05:02 utils.go:365: [PANIC] parse rdb entry error
[error]: EOF
10 /home/travis/gopath/src/github.com/CodisLabs/redis-port/pkg/rdb/reader.go:74
github.com/CodisLabs/redis-port/pkg/rdb.(*rdbReader).Read
9 /home/travis/.gimme/versions/go1.7.5.linux.amd64/src/io/io.go:523
io.(*teeReader).Read
8 /home/travis/gopath/src/github.com/CodisLabs/redis-port/pkg/rdb/reader.go:72
github.com/CodisLabs/redis-port/pkg/rdb.(*rdbReader).Read
7 /home/travis/.gimme/versions/go1.7.5.linux.amd64/src/io/io.go:307
io.ReadAtLeast
6 /home/travis/.gimme/versions/go1.7.5.linux.amd64/src/io/io.go:325
io.ReadFull
5 /home/travis/gopath/src/github.com/CodisLabs/redis-port/pkg/rdb/reader.go:243
github.com/CodisLabs/redis-port/pkg/rdb.(*rdbReader).readFull
4 /home/travis/gopath/src/github.com/CodisLabs/redis-port/pkg/rdb/reader.go:249
github.com/CodisLabs/redis-port/pkg/rdb.(*rdbReader).readBytes
3 /home/travis/gopath/src/github.com/CodisLabs/redis-port/pkg/rdb/reader.go:179
github.com/CodisLabs/redis-port/pkg/rdb.(*rdbReader).readString
2 /home/travis/gopath/src/github.com/CodisLabs/redis-port/pkg/rdb/reader.go:132
github.com/CodisLabs/redis-port/pkg/rdb.(*rdbReader).readObjectValue
1 /home/travis/gopath/src/github.com/CodisLabs/redis-port/pkg/rdb/loader.go:146
github.com/CodisLabs/redis-port/pkg/rdb.(*Loader).NextBinEntry
0 /home/travis/gopath/src/github.com/CodisLabs/redis-port/cmd/utils.go:364
main.newRDBLoader.func1
... ...
[stack]:
0 /home/travis/gopath/src/github.com/CodisLabs/redis-port/cmd/utils.go:365
main.newRDBLoader.func1

Redis cluster支持问题

请问redis-port支不支持redis cluster呢？比如：cluster to cluster的迁移支不支持？README没看到这方面的说明呢！

redis-port 导入数据时出错

rdb文件有800m，使用下面参数导入， 前面是的redi ，后面的codis-proxy

帮我看下是什么原因导致的。

2015/07/15 16:32:13 [INFO] total=758189300 - 168525824 [ 22%] entry=481532
2015/07/15 16:32:14 [INFO] total=758189300 - 168525824 [ 22%] entry=481532
2015/07/15 16:32:14 [PANIC] parse rdb entry error
[error]: EOF
7 /home/test/op_tools/src/github.com/wandoulabs/redis-port/pkg/rdb/reader.go:75
github.com/wandoulabs/redis-port/pkg/rdb.(_rdbReader).Read
6 /usr/local/go/src/io/io.go:298
io.ReadAtLeast
5 /usr/local/go/src/io/io.go:316
io.ReadFull
4 /home/test/op_tools/src/github.com/wandoulabs/redis-port/pkg/rdb/reader.go:234
github.com/wandoulabs/redis-port/pkg/rdb.(_rdbReader).readFull
3 /home/test/op_tools/src/github.com/wandoulabs/redis-port/pkg/rdb/reader.go:240
github.com/wandoulabs/redis-port/pkg/rdb.(_rdbReader).readBytes
2 /home/test/op_tools/src/github.com/wandoulabs/redis-port/pkg/rdb/reader.go:148
github.com/wandoulabs/redis-port/pkg/rdb.(_rdbReader).readString
1 /home/test/op_tools/src/github.com/wandoulabs/redis-port/pkg/rdb/loader.go:125
github.com/wandoulabs/redis-port/pkg/rdb.(*Loader).NextBinEntry
0 /home/test/op_tools/src/github.com/wandoulabs/redis-port/cmd/utils.go:247
main.func·026
... ...
[stack]:
0 /home/test/op_tools/src/github.com/wandoulabs/redis-port/cmd/utils.go:248
main.func·026
... ...

同步后，内存和key数量不一致

master的内存使用是9G，440W的key，用这个工具同步到其他redis上，显示使用内存只有6G，400W的key。
少了几个内存使用和40W的key。
也简单的验证了一些key，都是存在的。 但是还是不明白差的地方在哪，是否能保证数据一致？

ERR wrong number of arguments for 'restore' command

./redis-port restore --input=/root/dump1.rdb --target=10.159.3.16:6379 --auth=abcabc --redis
2017/09/14 15:13:09 main.go:189: [INFO] set ncpu = 2, parallel = 2
2017/09/14 15:13:09 restore.go:53: [INFO] restore from '/root/dump1.rdb' to '10.159.3.16:6379'
2017/09/14 15:13:09 utils.go:216: [PANIC] RESTORE command error
[error]: ERR wrong number of arguments for 'restore' command

i followed the help information ,but failed . why?
redis-port restore [--ncpu=N] [--parallel=M] [--input=INPUT] [--faketime=FAKETIME] [--extra] [--filterdb=DB] --target=TARGET [--auth=AUTH] [--redis|--codis]

redis2.x迁移至4.x报错

源版本：Redis 2.8
目标版本：Redis 4.0

2018/08/19 19:53:50 [PANIC] parse rdb entry error
[error]: EOF
    10  /home/pan.liangp/go/src/github.com/CodisLabs/redis-port/pkg/rdb/reader.go:85
            github.com/CodisLabs/redis-port/pkg/rdb.(*rdbReader).Read
    9   /usr/local/go/src/io/io.go:525
            io.(*teeReader).Read
    8   /home/pan.liangp/go/src/github.com/CodisLabs/redis-port/pkg/rdb/reader.go:83
            github.com/CodisLabs/redis-port/pkg/rdb.(*rdbReader).Read
    7   /usr/local/go/src/io/io.go:309
            io.ReadAtLeast
    6   /usr/local/go/src/io/io.go:327
            io.ReadFull
    5   /home/pan.liangp/go/src/github.com/CodisLabs/redis-port/pkg/rdb/reader.go:290
            github.com/CodisLabs/redis-port/pkg/rdb.(*rdbReader).readFull
    4   /home/pan.liangp/go/src/github.com/CodisLabs/redis-port/pkg/rdb/reader.go:296
            github.com/CodisLabs/redis-port/pkg/rdb.(*rdbReader).ReadBytes
    3   /home/pan.liangp/go/src/github.com/CodisLabs/redis-port/pkg/rdb/reader.go:216
            github.com/CodisLabs/redis-port/pkg/rdb.(*rdbReader).ReadString
    2   /home/pan.liangp/go/src/github.com/CodisLabs/redis-port/pkg/rdb/reader.go:169
            github.com/CodisLabs/redis-port/pkg/rdb.(*rdbReader).readObjectValue
    1   /home/pan.liangp/go/src/github.com/CodisLabs/redis-port/pkg/rdb/loader.go:169
            github.com/CodisLabs/redis-port/pkg/rdb.(*Loader).NextBinEntry
    0   /home/pan.liangp/go/src/github.com/CodisLabs/redis-port/cmd/utils.go:733
            main.newRDBLoader.func1
        ... ...
[stack]: 
    0   /home/pan.liangp/go/src/github.com/CodisLabs/redis-port/cmd/utils.go:734
            main.newRDBLoader.func1
        ... ...

psync instead of sync

use sync command is not resistent to network error, can you improve it?

请问可以在readme里写下安装过程吗

网上很少写如何安装redis-port的。希望可一附加安装过程。

迁移大list问题

大神们好！ ,在迁移过程中出现了一个错误 如下:

2017/02/15 18:25:55 [PANIC] parse command arguments failed
[error]: expect Array, but got <*redis.BulkBytes>
    2   /root/.gvm/pkgsets/go1.5/global/src/github.com/CodisLabs/redis-port/pkg/redis/resp.go:163
            github.com/CodisLabs/redis-port/pkg/redis.AsArray
    1   /root/.gvm/pkgsets/go1.5/global/src/github.com/CodisLabs/redis-port/pkg/redis/handler.go:88
            github.com/CodisLabs/redis-port/pkg/redis.ParseArgs
    0   /root/.gvm/pkgsets/go1.5/global/src/github.com/CodisLabs/redis-port/cmd/sync.go:258
            main.(*cmdSync).SyncCommand.func2
        ... ...
[stack]:
    0   /root/.gvm/pkgsets/go1.5/global/src/github.com/CodisLabs/redis-port/cmd/sync.go:259
            main.(*cmdSync).SyncCommand.func2
        ... ...

猜测可能是某个key的的value太大导致的

目前有个list 类型的key value 大约有几千万条 - - ...

我该如何解决这个问题呢？

make

/bin/sh: 2: ./autogen.sh: not found
Makefile:59: recipe for target 'jemalloc' failed
make[1]: *** [jemalloc] Error 127
Makefile:63: recipe for target 'build-jemalloc' failed
make: *** [build-jemalloc] Error 2

运行redis-port，一直报错panic: runtime error

redis版本2.8.19
go版本1.6.3

./redis-port sync -f 127.0.0.1:6379 -t 127.0.0.1:6381 -n 1
panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xb code=0x1 addr=0x0 pc=0x8181ffb]

goroutine 1 [running]:
panic(0x8214460, 0x18528030)
/usr/local/go/src/runtime/panic.go:481 +0x326
sync/atomic.LoadUint64(0x1855c104, 0x805c9d2, 0x8249610)
/usr/local/go/src/sync/atomic/asm_386.s:159 +0xb
github.com/CodisLabs/redis-port/pkg/libs/log.(_LogLevel).Test(0x1855c104, 0x4, 0x0, 0x18d414d0)
/home/leechanx/go/src/github.com/CodisLabs/redis-port/pkg/libs/log/log.go:72 +0x21
github.com/CodisLabs/redis-port/pkg/libs/log.(_Logger).isDisabled(0x1855c0f0, 0x4, 0x0, 0x0)
/home/leechanx/go/src/github.com/CodisLabs/redis-port/pkg/libs/log/log.go:167 +0x45
github.com/CodisLabs/redis-port/pkg/libs/log.Infof(0x8258ff0, 0x13, 0x0, 0x0, 0x0)
/home/leechanx/go/src/github.com/CodisLabs/redis-port/pkg/libs/log/log.go:529 +0x3e
main.main()
/home/leechanx/redis-port/cmd/main.go:176 +0x1136

可以给一份安装文档吗？

可以给一份安装文档吗？go语言不会