swift-server / redistack Goto Github PK
View Code? Open in Web Editor NEWNon-blocking, event-driven Swift client for Redis.
Home Page: http://swiftpackageindex.com/swift-server/RediStack/documentation
License: Apache License 2.0
Non-blocking, event-driven Swift client for Redis.
Home Page: http://swiftpackageindex.com/swift-server/RediStack/documentation
License: Apache License 2.0
I have configured RedisQueue Driver for my vapor project. While running the app the console becomes flooded with continuous debug log from the driver connection. I can barely see the other logs. Is there any way to disable the RedisQueue driver connection log? Screenshot of my console is attached. (Notes: I am running the service with docker compose)
From the proposal discussion thread: https://forums.swift.org/t/discussion-nioredis-nio-based-redis-driver/22455/11
Change
public enum RESPValue {
case .simpleString(String)
case .bulkString([UInt8])
}
to
public enum RESPValue {
case .simpleString(ByteBuffer)
case .bulkString(ByteBuffer?)
}```
I tried to connect to the redis server using RediStack. And the client and the server are both on the same server. When I tried to connect to the redis-server using unix socket (unix:///tmp/mm.sock) , I got the folllowing error :
RediStack.RedisConnection.Configuration.ValidationError.(unknown context at $100654b1c).Kind.invalidURLScheme)
Right now the RESPValue.bulkString
stores a reference to Data
, but it might be possible / easy to replace it with a ByteBuffer
The unit tests for all the scan
command iterations are failing non-deterministically.
They should be re-written so that they do fail reliably.
Now that swift-log
is publicly available - NIORedis should pull it in as a dependency and provide logs.
Today's afternoon, I tried to integrate the Redis to vapor 4 (my swift version is 5.4)
according to this doc , I copied the url ( .package(url: "https://gitlab.com/mordil/RediStack.git", .branch("master"))) to my Package.swift
but the complier reported an unwanted problem, it indicated :" https://gitlab.com/Mordil/RediStack: An unknown error occurred. unexpected return value from ssl handshake -9806 (-1)"
Meanwhile, I found another reference on vapor's doc (https://docs.vapor.codes/4.0/redis/overview/). Unfortunately, it was also unable to be fixed.
Finally, I attempted to substitute the "gitlab" by "github", then the dependency package can be install on my app
To be honesty, I had spent more to an hour to address this issue. so , would you mind to change the url into the right way ?
Look into adding convenience APIs for Redis 5's Streams
Right now - the RedisPipeline.execute()
method resolves [RESPValue]
- but this creates an unwieldy API compared to how RedisClient
in general behaves.
In enqueue(operation:)
the eventual resolves value is just ignored and discarded - but it would be nice to be able to capture the type information to somehow instead return [RESPValueEncodable]
or have a way to grab in a reliable, non-optional, type-safe way the actual return values from the RedisClient
interface so that it's consistent.
Right now, commands like zpopmax(from:max:)
have a relatively complex response structure that users of RedisPipeline
are left to figure out themselves, lest we just expose our helper method _mapSortedSetResponse(_:scoreIsFirst:)
Hi there,
When connected to our Redis Cluster, we're getting a lot of errors of the form:
MOVED 12858 <internal-ip>:6379
From looking at the post here: https://forums.swift.org/t/redistack-future-plans/65703
and the fact that there isn't a RedisClusterClient
yet, my understanding is that this client doesn't yet support cluster mode, and my best option is to reduce it to a single shard/replica for now. Is that correct?
ERR Protocol error: expected '$', got '+'
I'm setting door_uuid
to {"locked":true,"open":false}
(where uuid is a uuid)
We should add an index to the RESP3ParsingError
, so that we can track where in the buffer parsing failed. This can be immensely helpful when trying to debug RESP3 values with a more complex structure.
Could we add an index here? That way we can correlate the error back to a specific location in the received ByteBuffer. Definitely helps debugging should this ever trigger
Originally posted by @Joannis in #71 (comment)
I was ready to use TLS to connect, but I kept getting a clue. I've noticed that version 1.5 of release even adds nio-ssl as the default dependency, but so far I've been trying for hours and don't understand how to use TLS connections.
Maybe this is a very basic question, but I can't find a relevant post and can't find a relevant answer, can you prompt me, thank you!
The Redis retry configuration is potentially confusing, and downright problematic for Vapor.
RedisConnectionPool.Configuration.init()
is implemented like this. Note that it specifies a default timeout in the signature of 60 seconds, but if nil
is passed in, it uses an internal timeout of 10 ms.
public init(
initialServerConnectionAddresses: [SocketAddress],
maximumConnectionCount: RedisConnectionPoolSize,
connectionFactoryConfiguration: ConnectionFactoryConfiguration,
minimumConnectionCount: Int = 1,
connectionBackoffFactor: Float32 = 2,
initialConnectionBackoffDelay: TimeAmount = .milliseconds(100),
connectionRetryTimeout: TimeAmount? = .seconds(60),
onUnexpectedConnectionClose: ((RedisConnection) -> Void)? = nil,
poolDefaultLogger: Logger? = nil
) {
self.initialConnectionAddresses = initialServerConnectionAddresses
self.maximumConnectionCount = maximumConnectionCount
self.factoryConfiguration = connectionFactoryConfiguration
self.minimumConnectionCount = minimumConnectionCount
self.connectionRetryConfiguration = (
(initialConnectionBackoffDelay, connectionBackoffFactor),
connectionRetryTimeout ?? .milliseconds(10) // always default to a baseline 10ms
)
self.onUnexpectedConnectionClose = onUnexpectedConnectionClose
self.poolDefaultLogger = poolDefaultLogger ?? .redisBaseConnectionPoolLogger
}
The Vapor wrapper around this passes nil
down through its call stack.
It's not clear what the intent is in RediStack. Is a default timeout 60 seconds or 10 milliseconds? Certainly the public documentation implies it's 60 seconds, and only by a careful reading of the actual code can you determine that it ends up as 10 ms if you pass nil
.
I am using the RediStack library in my Swift project with Vapor, and I need to send multiple commands to Redis together as a batch to improve performance. However, I couldn't find a direct way to send multiple commands together using the existing API.
The send(command: String...)
function appears to set connection.sendCommandsImmediately = true
under the hood, which sends each command immediately and cannot be configured otherwise. Even if I construct RedisCommand
myself, I don't see any way to send it manually, only through send(command: String...)
. Please correct me if I'm wrong.
I found several old discussions about this topic, but all of them seem irrelevant.
enum ExecutionModel {
case managed(threadCount: Int)
case unmanaged(EventLoopGroup)
}```
compilation fails with:
/.build/checkouts/RediStack/Sources/RediStack/Cluster/SwiftPolyfill.swift:43:55: error: 'AsyncStream' is only available in macOS 10.15 or newer
) -> (stream: AsyncStream<Element>, continuation: AsyncStream<Element>.Continuation) {
^
/.build/checkouts/RediStack/Sources/RediStack/Cluster/SwiftPolyfill.swift:40:17: note: add @available attribute to enclosing static method
static func makeStream(
^
/.build/checkouts/RediStack/Sources/RediStack/Cluster/SwiftPolyfill.swift:39:1: note: add @available attribute to enclosing extension
extension AsyncStream {
^
We should not depend on the NIO umbrella module.
Instead we should explicitly
import NIOCore
import NIOPosix
import NIOConcurrencyHelpers
import NIOEmbedded
where needed.
We should also add explicit dependencies in the Package.swift
This code in scripts/soundness.sh
can be commented in again and CI passes:
# This checks for the umbrella NIO module.
printf "=> Checking for imports of umbrella NIO module... "
if git grep --color=never -i "^[ \t]*import \+NIO[ \t]*$" > /dev/null; then
printf "\033[0;31mUmbrella imports found.\033[0m\n"
git grep -i "^[ \t]*import \+NIO[ \t]*$"
exit 1
fi
printf "\033[0;32mokay.\033[0m\n"
RediStack 1.4.0 triggers multiple Thread Sanitizer failures due to lack of thread-safety in metrics when multiple connections are in use simultaneously, as in the unit tests for Vapor's RediStack integration provider. swift-demangle
d transcript of the TSan output is attached. Full test run output is available here.
Two of the graduation paths for the SSWG Incubation process out of sandbox are:
- Document that it is being used successfully in production by at least three independent end users which, in the SSWG judgement, are of adequate quality and scope
- Have a healthy number of committers
Eventually, it would be great to be promoted by the SSWG to a higher maturity level, but I need the communities help.
โ ๏ธ I have no intention to abandon or transition ownership of the project, this is simply a call to action raise this project's maturity and stability._
There are a few ways you can help, check out the full RFC on the source GitLab repo.
From the proposal discussion thread: https://forums.swift.org/t/discussion-nioredis-nio-based-redis-driver/22455/15
Is there are reason we shouldn't also have a declarative way for the command themselves? Something like
public enum RESPCommand {
case get(RESPValueConvertible)
case increment(RESPValueConvertible)
case set(RESPValueConvertible, RESPValueConvertible)
// ...
}
That way, batching of requests can become more natural and we don't need
public func enqueue<T>(operation: (RedisClient) -> EventLoopFuture<T>) -> RedisPipeline
where we have this weird
operation: (RedisClient) -> EventLoopFuture<T>
which I commented about before. Correct me if I'm wrong but I believe theT
is just there because we need support all of the singular operations like$0.get(...)
which already return anEventLoopFuture
but of different types... If we created an enumRESPCommand
we could have
connection.sendBatch([.get("foo"), .set("bar", 1), .increment("buz")])
I'm opening this up for discussion of what people may want to do with Lua scripting integration through the Swift client against Redis.
The list of supported commands: https://redis.io/commands#scripting
When using a RedisPipeline
created from a RedisConnection
, both have access to the same Channel
instance.
The former will write
then flush
in two separate steps, while the latter writeAndFlush
in a single step.
When using both concurrently - say on separate threads - or even if you for some reason need to break up the pipeline with other non-pipeline calls, the results are unexpected as the connection's command is written in between the pipeline's.
func test_concurrentChannelUsage() throws {
let pipeline = connection.makePipeline()
.enqueue { $0.get("key") }
.enqueue { $0.increment("key") }
.enqueue { $0.increment("key", by: 30) }
_ = try connection.increment("key").wait()
pipeline.enqueue { $0.decrement("key") }
let results = try pipeline.execute().wait()
XCTAssertEqual(results.count, 4)
XCTAssertTrue(results[0].isNull)
XCTAssertEqual(results[1].int, 1)
XCTAssertEqual(results[2].int, 31)
XCTAssertEqual(results[3].int, 30)
}
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.