basho / riak-dotnet-client Goto Github PK
View Code? Open in Web Editor NEWThe Riak client for .NET
License: Apache License 2.0
The Riak client for .NET
License: Apache License 2.0
I have read this (and your announcement about intending to keep Xml), so am I familiar with your stance ;0
http://news.ycombinator.com/item?id=2799823
We are using your client internally against our test Riak cluster... and the config was one thing that immediately jumped out at me as something I didn't like. Having done a lot of ASP.NET / MVC / WCF development, I have a very strong bias against XML config at all costs -- it tends to grow out of hand and become a real beast to read, understand and maintain over time.
I would advocate for supporting both .config file based and code-based config, and based on your code, you can do this w/out a lot of extra effort. You're already most of the way there given your config is expressed through interfaces already. We're working around this by implementing the config interfaces ourselves with POCOs that have no connection to the .NET config classes, but I'd like to see you support that out of the box. All you need to provide really are hand-rolled validation routines that match up with the config validation attributes. (I've done this sort of thing internally on other projects)
I would also like to make a couple of comments about your remarks on the ycombinator thread.
There's a good chance that we'll be building out a few pieces on node.js for instance. I think if you're a team looking at Riak, chances are you'll probably have at least a couple of client stacks working off it, which would again emphasize the need for a centralized load balancer.
So in short, I think it's awesome that your client does round robin load balancing -- and there are some scenarios where that might be useful. However, I think in any real production use, I can't see a lot of people using that. If I were to guess, I would expect that the majority of .NET teams would be deploying on Mono, where installing HAProxy is something that's more familiar anyhow. In our world, we'll be building out a service layer with ServiceStack, and almost certainly be deploying on Mono.
Inputs can be in one of 4 formats:
Refactor the input mechanism to support one of those four mechanisms.
Implement a WriteJson method for each so that this code will be much purdier.
Placeholder for discussion about how to best create cross-platform builds for CorrugatedIron.
Hi,
I may have missed something, but I'm wondering if there's support for object lookup using secondary indexes, as documented at http://docs.basho.com/riak/latest/tutorials/querying/Secondary-Indexes/, e.g.
curl localhost:8098/buckets/users/index/twitter_bin/jsmith123
Thanks,
Felix
The greater than key filter has an incorrect class name of GreatherThan.
Add vnode vclock flag in configuration to use the Riak 1.0 behavior of generating the vclock at the coordinating vnode
Background
But I'm starting to think that there's a lot of ceremony that goes along with building these in-memory repos, and that it would be nice to push this responsibility off to CorrugatedIron.
Since the Riak datamodel is so simple, it can be pretty closely simulated with a ConcurrentDictionary<string, object>
.
I haven't looked too closely at your code in a while, but I think that this would involve
I think the RavenDB client has a feature like that if I recall correctly.
Thoughts?
Implement Luwak API.
This should be implemented.
Remove Either<uint?, string> for R, RW, W, DW, and PR options. These options only take a uint32. Instead developers should make use of RiakConstants.QuorumOptions to define One, Quorum, All, or Default.
The current Riak Search API requires that the developer understand more about MR jobs than is really necessary. The API should be improved to make it easier for developers to query Search indexes.
RiakConstants sets the default value for R to 2. This should be 4294967295-4 or 'default'.
In the effort of lowering latency and whatnot, we should change DeleteBucket to use a streaming ListKeys operation and batch up deletes to re-use the same connection.
This would be getting the results of riak_admin status
through code.
Create scripts to automagically package up code for NuGet, binaries for github.
Valid content types will have a slash and be of a specific format. RiakObject should check for valid content types.
The Apache License isn't present in all source code files (e.g. IRiakClient.cs). Add it.
MR can support search via
{
"inputs": {
"bucket":"mybucket",
"query":"foo OR bar"
},
"query":...
}```
or
{
"inputs": {
"bucket":"mybucket",
"query":"foo OR bar",
"filter":"field2:baz"
},
"query":...
}
We need both an agreed upon conflict resolution mechanism and API.
Riak 1.0 provides many new and magical bucket properties (these can be found in the WebMachine source). CorrugatedIron should expose these.
The FluentInterface takes care of this. We should reduce the public footprint of the library and move contents of CorrugatedIron.Models.MapReduce.KeyFilters to be internal.
Back in the day it was important to provide the facility to handle client IDs when talking to Riak. These days they're ignored by the server.
As of v1.0 of CI, supporting Riak 1.2, we should remove the API calls that deal with Client IDs as they're no longer relevant.
The key filter API doesn't fit with the rest of the API. Modify it and make it more fluent-y.
Remove calls to MarkClean(). This belongs in higher level code than the current version of CorrugatedIron.
The performance profile will be used to generate a list of known performance pains and to create work items for a 1.1 release.
Hello
First of all, thanks for making CorrugatedIron available to everyone.
I was trying to run a long running M/R and realized I needed to set a higher timeout for the query.
AFAIK CorrugatedIron does not have any option to set the timeout for M/R queries am I right? If so, may I contribute to this project by implementing the option to set the query timeout?
Thanks in advance.
Verify that PUTs are being sent to the server with vector clocks properly attached.
If the vector clock is not set, we should examine the object being sent and pull the vclock from that. Should that not be an option, we throw an exception since lord only knows what's going to happen if we're not sending vclocks to the server. Some kind of sibling clown car, no doubt.
Nagle's Algorithm:
In most cases โ especially when using the PBC API, which tends to use small messages โ clients should set the TCP_NODELAY flag on opened socket connections to Riak, which disables Nagle's Algorithm. Latency profiles having a minimum of 40 milliseconds often indicate the presence of Nagle on either end of the connection. If the client application has significant round-trip latency to the Riak cluster, disabling Nagle will have little effect, but for well-connected clients it can significantly reduce latency.
The SetObject method will bomb if users are attempting to save objects with a content type that is not Application/JSON, protocol buffers, or XML. text based content types should be allowed.
Using xbuild on OS X generates a bucket of errors related to resolving DLL paths. Similar problems arise when you attempt to use MonoDevelop, too. Not sure if this is related to NuGet based shenanigans, however it makes an automated build outside of Windows a touch more painful.
Gist of output available at https://gist.github.com/4105672
Version info:
XBuild Engine Version 2.10.9.0
Mono, Version 2.10.9.0
Copyright (C) Marek Sieradzki 2005-2008, Novell 2008-2011.
Implement Riak Search API.
Listing keys from a bucket in a Riak is a bad idea. This has been documented by Basho and discussed a number of times. Despite this, developers do at some point during development require the ability to list the keys that are in a bucket.
CI should have some documentation which tell the developer that the code that they're using is not code that's intended for production. We can do this via:
The entire lib needs to be profiled and benchmarked then compared against other clients to see how it performs. Bottlenecks and major issues with performance need to be highlighted and subsequently addressed.
The ServiceStack json serializer is significantly faster than Newtonsoft's: http://www.servicestack.net/benchmarks/
We're looking to use Riak for logging and we need to get past some perf bottlenecks to do this. Faster serialization will be crucial.
Hello guys
Just been testing out CorrugatedIron, and got some message framing errors - so I had a look into it and had a small refactoring in RiakPbcSocket so that you can ensure you don't get partial network messages:
private byte[] receiveAll(byte[] resultBuffer)
{
int totalBytesReceived = 0;
int lengthToReceive = resultBuffer.Length;
while (lengthToReceive > 0)
{
int bytesReceived = PbcSocket.Receive(resultBuffer, totalBytesReceived, lengthToReceive, 0);
if (bytesReceived == 0)
{
throw new RiakException("Unable to read data from the source stream - Timed Out.");
}
totalBytesReceived += bytesReceived;
lengthToReceive -= bytesReceived;
}
return resultBuffer;
}
Then, the two receive calls become:
var header = receiveAll(new byte[5]);
and
var resultBuffer = receiveAll(new byte[size - 1]);
Hope that helps.
Matt
Remove UpdateLastModified calls from RiakObject. This should be handled by Riak. There is always a chance of clock skew and conditional requests over HTTP may not work.
Reduce the lines of code to configure and load an instance of a cluster for use in tools like a REPL.
The DeleteBucket call currently only takes a RW value, and we should expose another overload which takes in the RiakDeleteOptions instance instead so that other values can be set, such as DW.
Hi again,
As I'm working with the sync & async clients, I'm wondering if you guys have any examples of the async/callback pattern. I've worked with this type of pattern in languages like node.js before, but am still trying to understand best practices in C#. Using .NET 4.5 here FYI, so most of the goodies that are needed are already available.
Here's the example that I'm thinking of.
public int ReturnsAnInt() {
RiakResult<RiakObject> result = Client.Get(someBucket, someKey);
return ComputeIntFromRiakResult(result);
}
This is all well and good ... I end up getting my int after the Get completes, and I can go about my business. But it would be great if I could leverage the Async client in Riak to do this. I have a feeling that's quite possible, and that I'm probably just being a bit thickheaded about it....
public int WouldLoveToReturnAnInt() {
// returns void
Client.Async.Get(someBucket, someKey, delegate(RiakResult<RiakObject> result) {
int i = ComputeIntFromRiakResult(result);
});
// can't compile this --
// is there some way that I can have my computed int back before I return in the top-level method?
}
Apologies for such a basic question here, and no rush to respond. But I think that once this is documented in CI it should help out other folks as well.
Cheers,
Felix
After removing OJ's ass from his shoulders and putting his head back on, he alleges to have removed the mess that was the CommandRunner. Deep down OJ always knew that the command approach for this problem was the wrong one, but for some reason his desire to have connection idling in there was more important.
Now that the CommandRunner is gone, and the extra thread and context switching along with it, the live unit tests take, on average, less than 1/3 of the time they used to take. That isn't a coincidence, and I don't think that connection idling is worth that kind of overhead. (OJ loves threading, someone let me write threading code.)
So it's been ripped out, and replaced with nothing, which means that the connection idling functionality no longer exists.
Adding idling isn't as simple as it sounds, given the pooling features that we have in place. Contention on the connection is a problem when you want to move it into an idle state (and hence kill off the connection) right at the same time as a client wants to use the connection. The simple option is to lock the connection each time it's used. Locking adds more overhead, and I'm really not keen on the idea of littering the connection code with locks.
There are a few ideas on how to manage this but OJ isn't sure how solid they are. In short, there's a big smelly dream, but that's it.
If I pull CorrugatedIron from nuget, my code bombs when I do Put or Get like so:
Could not load file or assembly 'Newtonsoft.Json, Version=4.0.2.0, Culture=neutral, PublicKeyToken=30ad4fe6b2a6aeed' or one of its dependencies. The located assembly's manifest definition does not match the assembly reference. (Exception from HRESULT: 0x80131040)
If I build CorrugatedIron from source and then reference the freshly-built lib from my code everything works fine. It appears CorrugatedIron is tied to Newtonsoft 4.0.2.0 but nuget (being the eager beaver) pulls a later version 4.0.5.0.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.