why pipeliningFlushPolicy default is not flushOnClose?

[Blusez]

why pipeliningFlushPolicy default is not flushOnClose?

[mp911de]

Pipelining is a mechanism to send commands without awaiting completion of the previous command. That's exactly the default configuration setting to have a balance between command processing and latency. Since a framework cannot make any assumptions on how long you will wait between sending commands you can configure LettuceConnectionFactory.pipeliningFlushPolicy to your needs.

[Blusez]

Pipelining is a mechanism to send commands without awaiting completion of the previous command. That's exactly the default configuration setting to have a balance between command processing and latency. Since a framework cannot make any assumptions on how long you will wait between sending commands you can configure LettuceConnectionFactory.pipeliningFlushPolicy to your needs.

If the default config is to submit a command every time, then what is the difference with loop to direct the command?
In jedis and lettuce, the default implementation of pipeline is the last to submit.
Why does spring-data want to change this?

[mp911de]

then what is the difference with loop to direct the command?

Care to elaborate? I do not understand the question.

In jedis and lettuce, the default implementation of pipeline is the last to submit.

That's not true. Lettuce's async API submits commands without awaiting previous command completion. Also, there's no pipelining API. Also, Jedis sends commands to the pipeline directly, however, Jedis uses a BufferedOutputStream that buffers a few commands (up to 8kb of data IIRC).

[Blusez]

then what is the difference with loop to direct the command?

I mean if default policy is flushEachCommand ,it same as :

for (int i = 0; i < 100; i++) {
 RedisTemplate.opsForValue().get("key:"+i);
}

in lettuce:,it pipeline is write all commands to the transport layer:

In my code I wanted to be able to submit all commands together, less network io, but after using spring-data -lettuce, all requests were submitted one by one,it causing the machine's out-of-heap memory to continue to rise, which made me feel very confused.

I notice Lettuce pipelining behaviour is different than Jedis pipelining.

and I think using pipeline, the default is to want to submit commands in batches, not one by one.

[mp911de]

flushCommands is in Lettuce only an optimization to use netty's buffering mechanism for additional command buffering. Using flushCommands can be dangerous especially in multi-threaded arrangements.

If you're want to optimize for performance then using Lettuce directly is a good alternative.

[Blusez]

What I mean is that pipeline should be more focused on batch submission, like the jedis implementation whitch buffer all commands and flush them to the transport at once, then it doesn't bother users.

But now the default policy in spring-data-redis is flushEachCommand,it can cause a lot of trouble for upgrades, and upgrades and switch implementations should not be such a big difference.so I think the default policy should be flushOnClose.

By the way ,in a production environment, direct use of redis pipeline policy of spring-data-redis , under certain conditions(it could be the Internet.), will cause off-heap memory to continue to rise, what is the reason for this? spring-data-redis version is 2.7.1.

[mp911de]

There's a difference between pipelining and batching. We do not want to force all users into a particular scheme that appears convenient in a specific application.

Your other question relates to Lettuce's buffering feature upon disconnect. Please refer to ClientOptions.disconnectedBehavior and requestQueueSize for configuration options.

[Blusez]

But when we use pipeline, we want to be able to execute it in batches. I think this is more general. Each command is submitted as a special scenario.

Inotice that the client disconnected buffer queue is an unbounded queue, but why use off-heap memory here？