shubhamranjan/dotnet-etcd

A C# .NET (dotnet) GRPC client for etcd v3 +

Hi 👋, I'm Shubham Ranjan

A Software Engineer 👨‍💻 from India building stuff for scale

• 🔭 I’m currently working on : dotnet-etcd

• 🌱 I’m currently learning : rust, flutter

• 👨‍💻 All of my projects are available at : https://github.com/shubhamranjan?tab=repositories

• 📝 I regulary write articles on : https://medium.com/@shubham.ranjan

• 💬 Ask me about : dotnet, docker, kubernetes

• 📫 How to reach me : [email protected]

Blogs posts

• Monolith to microservices using dotnet core

Technologies I have explored

Github Stats

dotnet-etcd's Issues

How to read past version of keys

Is your feature request related to a problem? Please describe.
I cannot find any doc about read past version of keys.

Describe the solution you'd like
In etcdctl:

etcdctl get --prefix --rev=4 foo # access the versions of keys at revision 4

So how can do the same with dotnet-etcd?

Additional context
I'm using latest version of dotnet etcd for now (5.2.1), .NET Core 6.
Thanks

Invalid port specified

Describe the bug
Providing connection string with only host and port, adding second port to url.

To Reproduce
Initialize with connection string like "localhost:2379" will try to create url from following string "localhost:2379:2379".

Expected behavior
Create internal url with only http prefix (or https when ssl enabled).

Is there any reason of looking first for 3 colons? Swapping ifs order could resolve problem.

dotnet-etcd/dotnet-etcd/etcdClient.cs

Lines 115 to 123 in c82573f

	if (host.Split(':').Length < 3)
	{
	host += $":{Convert.ToString(port, CultureInfo.InvariantCulture)}";
	}
	if (!(host.StartsWith(InsecurePrefix, StringComparison.InvariantCultureIgnoreCase) || host.StartsWith(SecurePrefix, StringComparison.InvariantCultureIgnoreCase)))
	{
	host = options.Credentials == ChannelCredentials.Insecure ? $"{InsecurePrefix}{host}" : $"{SecurePrefix}{host}";
	}

etcd-client support for k8s headless service

i want to use k8s headless service for connection to my etcd cluster with 3 nodes.
does dotnet-etcd support this feature? or i need pass to etcdClient one url for each etcd nodes?

Initialization error

EtcdClient client = new EtcdClient("host1:port1:,...., hostN:portN");
Error reporting using constructor

Using this client with IConfiguration

Hello,

Is there a way to use this client with IConfiguration?

Support IConfiguration and DI

• Users should be able to inject the client
• If not config is passed, lib should fall back to IConfiguration to check for values. The config format will be predefined
• Users should be able to pass the config

Lock permission denied

Describe the bug
I have 2 instances of the same service with the following simply logic
Instance A:
create lease => leaseA
lock("someName", leaseA)

Instance B:
create lease => leaseB
lock("someName", leaseB)

Expected behavior:
Instance B wait, until unlock or lease expired

Real behavior:
Instance B sometimes receive error permission denied on client side, on server side: "auth: user name is empty"

We use tls auth and in wireshark i see that all request use tls without auth error. We use long live certificates

My experimental project
https://github.com/careless6666/etcd-cluster-issue/blob/main/Service/Controllers/DebugController.cs

etcd configuration:
#[member]
ETCD_NAME="stgetcd1z1.h.companyName"
ETCD_DATA_DIR="/var/lib/etcd/default"
ETCD_SNAPSHOT_COUNT=5000
ETCD_HEARTBEAT_INTERVAL=150
ETCD_ELECTION_TIMEOUT=3000
ETCD_QUOTA_BACKEND_BYTES=0
ETCD_LISTEN_CLIENT_URLS="https://0.0.0.0:2379/"
ETCD_ADVERTISE_CLIENT_URLS="https://stgetcd1z1.h.companyname:2379/"
ETCD_MAX_SNAPSHOTS=5
ETCD_MAX_WALS=5
ETCD_ENABLE_V2=false

#[proxy]
ETCD_PROXY="off"
ETCD_PROXY_FAILURE_WAIT=5000
ETCD_PROXY_REFRESH_INTERVAL=30000
ETCD_PROXY_DIAL_TIMEOUT=1000
ETCD_PROXY_WRITE_TIMEOUT=5000
ETCD_PROXY_READ_TIMEOUT=0

#[cluster]
ETCD_LISTEN_PEER_URLS="https://0.0.0.0:2380/"
ETCD_INITIAL_ADVERTISE_PEER_URLS="https://stgetcd1z1.h.companyname:2380/"
ETCD_INITIAL_CLUSTER_STATE="new"
ETCD_INITIAL_CLUSTER="stgetcd1z1.h.companyName=https://stgetcd1z1.h.companyName:2380,stgetcd2z1.h.companyName=https://stgetcd2z1.h.companyName:2380,stgetcd3z1.h.companyName=https://stgetcd3z1.h.companyName:2380,stgetcd4z1.h.companyName=https://stgetcd4z1.h.companyName:2380,stgetcd5z1.h.companyName=https://stgetcd5z1.h.companyName:2380,"
ETCD_INITIAL_CLUSTER_TOKEN="eiphohCipoowe9OnaiPhiefu"
ETCD_DISCOVERY_FALLBACK="proxy"
ETCD_STRICT_RECONFIG_CHECK=true
ETCD_FORCE_NEW_CLUSTER=false

#[security]
ETCD_CERT_FILE="/etc/etcd/ssl/stgetcd1z1_server_cert.pem"
ETCD_KEY_FILE="/etc/etcd/ssl/stgetcd1z1_server_key.pem"
ETCD_CLIENT_CERT_AUTH=true
ETCD_TRUSTED_CA_FILE="/etc/etcd/ssl/ca.pem"
ETCD_PEER_CERT_FILE="/etc/etcd/ssl/stgetcd1z1_peer_cert.pem"
ETCD_PEER_KEY_FILE="/etc/etcd/ssl/stgetcd1z1_peer_key.pem"
ETCD_PEER_CLIENT_CERT_AUTH=true
ETCD_PEER_TRUSTED_CA_FILE="/etc/etcd/ssl/peer_ca.pem"
ETCD_PEER_CERT_ALLOWED_CN="stg-etcd-common"

#[logging]
ETCD_DEBUG=False
ETCD_LOG_LEVEL=info

#ExecStart=/bin/sh -c "GOMAXPROCS=$(nproc) /usr/local/bin/etcd $DAEMON_ARGS"

DAEMON_ARGS:
--listen-metrics-urls http://0.0.0.0:2385/ --auto-compaction-mode=periodic --auto-compaction-retention=1h

version 3.5.5

We can't reproduce with error on latest commit on main branch with grpc connection dns:///etcd-stg-3.companyName:2379, but version 6.0.1 not available new constructor with grpc options, can you publish new version?

cannot cancel watch

Describe the bug
i try to cancel a watch with below code, but failed.

To Reproduce

using Etcdserverpb;
using Google.Protobuf;
using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.Text;

namespace dotnet_etcd
{
    class TestCancelWatch
    {
        dotnet_etcd.EtcdClient myConsulClient;

        const long WID = 4445;

        public void Start()
        {
            myConsulClient = new dotnet_etcd.EtcdClient("http://127.0.0.1:2379");
            WatchKey("/zzz");
        }

        public void Stop()
        {
            UnWatch();
        }

        void WatchKey(string keyName)
        {
            WatchRequest req = new WatchRequest {
                CreateRequest = new WatchCreateRequest
                {
                    Key = ByteString.CopyFromUtf8(keyName),
                    WatchId = WID,
                }
            };

            myConsulClient.Watch(req, (Etcdserverpb.WatchResponse respon) => {
                Debug.Assert(respon.WatchId == WID);
                Console.WriteLine("watch event!!!");
            });
        }

        // unwatch
        public void UnWatch()
        {
            WatchRequest req = new WatchRequest
            {
                CancelRequest = new WatchCancelRequest
                {
                    WatchId = WID
                }
            };

            myConsulClient.Watch(req, (Etcdserverpb.WatchResponse respon) => {

                // never be executed!!!! bug?
                Console.WriteLine($"cancel:{respon.Canceled}: {respon.CancelReason}");
            });
        }
    }


    class Program
    {
        static void Main(string[] args)
        {
            var watch = new TestCancelWatch();

            while (true)
            {
                var str = Console.ReadLine();
                if (str == "s")
                {
                    watch.Start();
                }
                else if (str == "u")
                {
                    watch.Stop();
                }
            }
        }
    }
}

Expected behavior
input 's'
input 'u'
console print "cancel: ..."

Screenshots

Additional context
dotnet-etcd: 3.0.0
etcd:3.4.3

GrpcChannelOptions parameter configuration

Is your feature request related to a problem? Please describe.
I would like to set MaxReceiveMessageSize to a larger value, but since GrpcChannelOptions is created in the EtcdClient constructor, this is not possible.

Describe the solution you'd like
a Action<GrpcChannelOptions> configureChannelOptions = null parameter added to the constructor.
and configureChannelOptions?.Invoke(options) at

dotnet-etcd/dotnet-etcd/etcdClient.cs

Line 104 in 4db4737

Describe alternatives you've considered
The whole GrpcChannelOptions object can't be passed into the constructor because the MethodConfigs, throttlingPolicy, handler, .. properties are set in the constructor already. And when passing in the entire object, you can't get those from the constructor.

And creating a custom configuration class with only the relevant properties, and copying them over sounds like wasted effort.

Additional context
Reason for changing the GrpcChannelOptions property:

Grpc.Core.RpcException: Status(StatusCode="ResourceExhausted", Detail="Received message exceeds the maximum configured message size.")
   at Grpc.Net.Client.StreamExtensions.ReadMessageAsync[TResponse](Stream responseStream, GrpcCall call, Func`2 deserializer, String grpcEncoding, Boolean singleMessage, CancellationToken cancellationToken)
   at Grpc.Net.Client.Internal.GrpcCall`2.RunCall(HttpRequestMessage request, Nullable`1 timeout)
   at dotnet_etcd.EtcdClient.<>c__DisplayClass49_0.<<GetAsync>b__0>d.MoveNext()

MemberAdd - accept url only, no name

Usually when adding a member we also set a name:
etcdctl member add NAME http://a.b.c.d:2380

Is it optional or useless?

Run Lease on linux, System.MissingMethodException: Method not found: 'Google.Protobuf.FieldCodec...

My code like this:

            var leaseId = etcdClient.LeaseGrant(new LeaseGrantRequest
            {
                TTL = Info.alive_tick
            }).ID;

            etcdClient.Put(new PutRequest
            {
                Key = ByteString.CopyFromUtf8(key_alive),
                Value = ByteString.CopyFromUtf8(Info.ValueYes),
                Lease = leaseId
            });

When I run on mac, it is ok.
But when I run on linux(centos 7), It throw Exception:

Unhandled Exception: System.TypeInitializationException: The type initializer for 'Etcdserverpb.Lease' threw an exception. ---> System.TypeInitializationException: The type initializer for 'Etcdserverpb.LeaseTimeToLiveResponse' threw an exception. ---> System.MissingMethodException: Method not found: 'Google.Protobuf.FieldCodec`1<Google.Protobuf.ByteString> Google.Protobuf.FieldCodec.ForBytes(UInt32)'.
   --- End of inner exception stack trace ---
   at Etcdserverpb.LeaseTimeToLiveResponse.get_Parser()
   at Etcdserverpb.Lease..cctor()
   --- End of inner exception stack trace ---
   at Etcdserverpb.Lease.LeaseClient.LeaseGrant(LeaseGrantRequest request, CallOptions options)
   at Etcdserverpb.Lease.LeaseClient.LeaseGrant(LeaseGrantRequest request, Metadata headers, Nullable`1 deadline, CancellationToken cancellationToken)
   at dotnet_etcd.EtcdClient.LeaseGrant(LeaseGrantRequest request)
   at battlecheck.core.etcd.BeDiscovery.Start() in /Users/zhangzhen/serverthreekingdom/src/vcs.taiyouxi.net/jws2/battlecheck/battlecheck/core/etcd/BeDiscovery.cs:line 48
   at battlecheck.Program.Main(String[] args) in /Users/zhangzhen/serverthreekingdom/src/vcs.taiyouxi.net/jws2/battlecheck/battlecheck/Program.cs:line 42

Cluster health?

It would be nice to have this client command:

etcdctl cluster-health

Should the host of EtcdClient be array?

I found in Golang, the host of EtcdClient is slice.
But dotnet-etcd the host of EtcdClient is string.
So I only can connect single-point of etcd. Is it right?

Few async methods are not awaitable

Hi,

Methods like 'PutAsync()' are marked as 'void'. Can this return type changed to Task and return whatever the underlying async response comes from KVClient's corresponding async methods? This will make the caller to apply proper 'await' patterns.

Example:

Existing:

        public async void PutAsync(string key, string val)
        {
            try
            {
                await _kvClient.PutAsync(new PutRequest
                {
                    Key = ByteString.CopyFromUtf8(key),
                    Value = ByteString.CopyFromUtf8(val)
                }, _headers);
            }
            catch
            {
                ResetConnection();
                throw;
            }
       }

Suggested Change:

    public async Task<PutResponse> PutAsync(string key, string val)
    {
        try
        {
            return await _kvClient.PutAsync(new PutRequest
            {
                Key = ByteString.CopyFromUtf8(key),
                Value = ByteString.CopyFromUtf8(val)
            }, _headers);

        }
        catch
        {
            ResetConnection();
            throw;
        }
    }

Thank you

Interface for EtcdClient

Can you create an interface for EtcdClient so when used/injected in integrations it can be mocked and therefore the integrations can be unit tested easily?

Legacy Grpc.Core fails on Linux Alpine container

Describe the bug
The EtcdClient fails to connect on a Linux Alpine container running .net 5.

The underlying grpc library fails with the following error:

System.IO.IOException: Error loading native library "/app/runtimes/linux-x64/native/libgrpc_csharp_ext.x64.so". Error loading shared library ld-linux-x86-64.so.2: No such file or directory (needed by /app/runtimes/linux-x64/native/libgrpc_csharp_ext.x64.so)
   at Grpc.Core.Internal.UnmanagedLibrary..ctor(String[] libraryPathAlternatives)
   at Grpc.Core.Internal.NativeExtension.LoadNativeMethodsUsingExplicitLoad()
   at Grpc.Core.Internal.NativeExtension.LoadNativeMethods()
   at Grpc.Core.Internal.NativeExtension..ctor()
   at Grpc.Core.Internal.NativeExtension.Get()
   at Grpc.Core.Internal.NativeMethods.Get()
   at Grpc.Core.GrpcEnvironment.GrpcNativeInit()
   at Grpc.Core.GrpcEnvironment..ctor()
   at Grpc.Core.GrpcEnvironment.AddRef()
   at Grpc.Core.Channel..ctor(String target, ChannelCredentials credentials, IEnumerable`1 options)
   at Grpc.Core.Channel..ctor(String host, Int32 port, ChannelCredentials credentials, IEnumerable`1 options)
   at dotnet_etcd.multiplexer.Balancer..ctor(List`1 nodes, String caCert, String clientCert, String clientKey, Boolean publicRootCa)
   at dotnet_etcd.EtcdClient..ctor(String connectionString, Int32 port, String caCert, String clientCert, String clientKey, Boolean publicRootCa)

Please note that Grpc.Net.Client is the recommended library for the future and that Grpc.Core will go into maintenance. This might be worth looking into as a solution for this bug.

https://grpc.io/blog/grpc-csharp-future/

Avoiding using 'async void' as function signature

Like calling LeaseKeepAlive, I have no way to detect whether it throws exception or not.
Please consider using async Task to return a Task object to user.

Implement the gRPC Naming like go-clientv3

Can implement the Naming like go client?

https://github.com/etcd-io/etcd/tree/master/clientv3/naming

Use methods with generics to implement retry

Is there any reason why there is so much retry logic copied throughout the codebase Using a method with generics and a Func this could be refactored to a more maintainable codebase.

For example this method:

public AuthenticateResponse Authenticate(AuthenticateRequest request, Grpc.Core.Metadata headers = null,
    DateTime? deadline = null,
    CancellationToken cancellationToken = default(CancellationToken))
{
    AuthenticateResponse response = new AuthenticateResponse();
    bool success = false;
    int retryCount = 0;
    while (!success)
    {
        try
        {
            response = _balancer.GetConnection().authClient
                .Authenticate(request, headers, deadline, cancellationToken);
            success = true;
        }
        catch (RpcException ex) when (ex.StatusCode == StatusCode.Unavailable)
        {
            retryCount++;
            if (retryCount >= _balancer._numNodes)
            {
                throw ex;
            }
        }
    }

    return response;
}

Can become:

public AuthenticateResponse Authenticate(AuthenticateRequest request, Grpc.Core.Metadata headers = null,
    DateTime? deadline = null,
    CancellationToken cancellationToken = default(CancellationToken))
{
    return CallEtcd((con) => con.authClient.Authenticate(request, headers, deadline, cancellationToken));
}

With the use of a simple helper function:

private TResponse CallEtcd<TResponse>(Func<Connection, TResponse> etcdCallFunc)
{
    TResponse response = default(TResponse);
    bool success = false;
    int retryCount = 0;
    while (!success)
    {
        try
        {
            response = etcdCallFunc.Invoke(_balancer.GetConnection());
            success = true;
        }
        catch (RpcException ex) when (ex.StatusCode == StatusCode.Unavailable)
        {
            retryCount++;
            if (retryCount >= _balancer._numNodes)
            {
                throw;
            }
        }
    }

    return response;
}

I have also changed 'throw ex' to simple 'throw' because 'throw ex' removes the original stack trace and creates a new one on the place of the throw while a simple 'throw' keeps the original stack trace that should provide more details. Especially with the helper function: without this change all stack traces would point to the helper function as the source of the exception.

When using "Watch" or "WatchRange" function, how to catch "Grpc.Core.RpcException" unhandled exception when network disconnected?

Is your feature request related to a problem? Please describe.
When using "Watch" or "WatchRange" function, we got "Grpc.Core.RpcException" unhandled exception when network disconnected.

Describe the solution you'd like
Some EtcdClient event or static event that can help us to catch expcetion in this situation.
Or do you know any Grpc.Core method or event that can help us to handle exception?

Screenshots
Exception:

Our code:

Additional context
.net core 3.0 console app
dotnet-etcd v3.0.0
etcd docker image: quay.io/coreos/etcd:v3.3.15

user name is empty error is shown if authentication is enabled

Describe the bug
When Authentication is enabled. I am not able to get the values.

user name is empty

is returned.

To Reproduce
_etcdClient = new dotnet_etcd.EtcdClient(_client.Settings.ConnectionString, _client.Settings.Port, _client.Settings.UserName,_client.Settings.Password);

a
Expected behavior
I should get the value ( ETCD cli does return the value)

Etcd value getting trimmed out

I'm using etcd to store my app settings including the database connection string and I ran into this issue where the value is getting trimmed out.

I'm saving my connection string using the following command and when I read execute the get command I'm able to see the saved connection string.

etcdctl.exe --user root:password put ConnectionString "Data Source=(DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)(HOST=xxx.domain.com)(PORT=1521)))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=XXX)));User Id=testuser;Password=password1;Statement Cache Size=10; Validate Connection=true;Connection Lifetime=120;Connection Timeout=60;Min Pool Size=10;Incr Pool Size=5; Decr Pool Size=2;Max Pool Size=100;Self Tuning=false;"

Issue

However while using the etcd client when I read the keys i see the value for the key ConnectionString is getting trimmed out.

Following is the value returned by the etcd client

Data

i.e the after the space all the values got trimmed out.

Thank you for this library and the support !!

Cluster distribution

I have a application where I would create an etcd client when needed and not keep a single etcd client around for all requests.

If I read through the code it looks like the round-robin technique of handing out the etcd node connections always starts with the first node of the provided node list. This results in an uneaven load distribution where the first node would get far more requests if the clients are short-lived.

Maybe it is better to initialize the _lastNodeIndex of the EtcdClient with a random selected node from the node list? Or change the GetNextNodeIndex method so it chooses a random node when the _lastNodeIndex is -1.
Setting it on the _lastNodeIndex directly on construction would be easier but the number stored in _lastNodeIndex would not actualy be the last node used.

Add optional ExceptionHandler in LeaseKeepAlive

Is your feature request related to a problem? Please describe.
I am not able to log the exceptions thrown in the LeaseKeepAlive function.

Describe the solution you'd like
I would like an optional exceptionHandler parameter just like in the Watch method.

Store data in hierarchically organized directories

Hello,

Can we Store data in hierarchically organized directories, as in a standard filesystem?
Something like -
/config
/database
/feature-flags
/verbose-logging
/redesign

etcdclient

Describe the bug
A clear and concise description of what the bug is.
EtcdClient client = new EtcdClient("127.0.0.1",2379);

error:

System.UriFormatException : "invalid URI: invalid URI scheme.

To Reproduce
Steps to reproduce the behavior:

1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots
If applicable, add screenshots to help explain your problem.

Additional context
Add any other context about the problem here.

connection close

What is the way to judge that the connection is no longer available for the client that has been connected (the server is closed, the network is broken)

Put overload with PutRequest argument

Current function does not provide ability to specify lease

Exception type when cancelling async operations

Some background on the problem I have with how cancellations are thrown in the grpc library
Currently, the new Grpc.Net.Client throws a RpcException with a 'Canceled' StatusCode when async operations are canceled. This means that a Task's state IsFaulted and not IsCanceled.

Also, handling exceptions when awaiting these tasks needs to be treated differently. Instead of catching all cancellations with the OperationCanceledException base type, they need to deal with a RpcException instead:

var cancellationTokenSource = new CancellationTokenSource();
cancellationTokenSource.CancelAfter(0);

try { await etcdClient.StatusASync(new StatusRequest(), cancellationToken:cancellationTokenSource.Token); }
catch (RpcException rpc) when (rpc.StatusCode == StatusCode.Cancelled) { /* handle cancellation */ }
catch (Exception e) { /* handle errors */ };

The solution I prefer
Cancellations must throw OperationCanceledException. The fix for this is as follows:

When creating the GrpcChannel in the Balancer, update the GrpcChannelOptions to include this property:
ThrowOperationCanceledOnCancellation = true

Alternatives I've considered
Currently, I'm wrapping each call to the etcd library to catch the 'Canceled' RpcException, and rethrow it as OperationCanceledException.

Add name option when adding new member.

MemberAddRequest() method
I't would be really nice to be able to set name parameter when registering/adding new nodes (clusters).
I'm implementing discovery service for consul and etcd, but it is impossible to do it for etcd now, because I need to find it by it's name. Go library for etcd already has this option.

Whether watch child node changes is supported

etcd watch function support asynchronous method

when i use dotnet-etcd some watch method,it was blocked my code and i can not run code after watch method invoked

code:

etcdClient.Watch("services", WatchDelegate, cancellationToken: cancellationTokenSource.Token); //blocked

//Set
await etcdClient.PutAsync(new PutRequest { Key = ByteString.CopyFromUtf8("services"), Value = ByteString.CopyFromUtf8("test") });
await etcdClient.PutAsync("services/demo", "test");

future:

add asynchronous method

Task watchTask = etcdClient.WatchAsync("services", WatchDelegate, cancellationToken: cancellationTokenSource.Token); //unblocked
//Set
await etcdClient.PutAsync(new PutRequest { Key = ByteString.CopyFromUtf8("services"), Value = ByteString.CopyFromUtf8("test") });
await etcdClient.PutAsync("services/demo", "test");
await watchTask;   //blocked

Possible NullReference

Hello, thanks for implementing #171, however it looks like:

dotnet-etcd/dotnet-etcd/etcdClient.cs

Line 99 in 035b2df

configureChannelOptions.Invoke(options);

is missing handling for case when user does not supply any options Action (default is null), then you get a null-reference on construction of etcdClient.

Regards,
Jiri

Add node to cluster or create a new cluster.

Hello, is it possible to use the client to add node to existing cluster, or create a cluster?
I mean if it is possible to use commands like:
etcdctl member add nodex http://a.b.c.d:2380

Thank you

GetRange() / GetRangeVal() get too many results

Describe the bug
The GetRange and GetRangeVal methods return a too large result.
This error has been introduced with the changes to GetEndRange.

It just sets a zero byte. Thus the end entry point of the range is the prefix provided. The end however is not within it's structure but at the end of a kv pairs by their default sorting from the first match of the prefix provided.

To Reproduce
Create sample data
Put("/xIssueTest", "some value")
Put("/yIssueTest", "some other value")

GetRangeVal("/yIssueTest") <-- returns with count 1
GetRangeVal("/xIssueTest") <-- returns with count 2

Expected behavior
As it was before. The change to GetRangeEnd()
GetRange("/some/config/key/itemList/") <-- should return any kv pair starting with that prefix only

Exception handling issue in Watch API

Describe the bug
In the 3.1.0 release, a new exception handler parameter was added to the watch API, however this only works properly when StatusCode is not Unavailable

dotnet-etcd/dotnet-etcd/watchClient.cs

Lines 71 to 78 in cd0afcb

	if (exceptionHandler != null)
	{
	exceptionHandler(ex);
	}
	else
	{
	throw ex;
	}

This is because the original exception is still thrown when StatusCode is Unavailable and thus this is causing side effects since the etcd watch logic is inside a Task.

dotnet-etcd/dotnet-etcd/watchClient.cs

Line 93 in cd0afcb

throw ex;

To Reproduce
Steps to reproduce the behavior:

1. Create a connection when etcd is down
2. Use the Watch API (any overload) with exception handling
3. Exception is thrown from Task

Expected behavior
Exception details should be handled in the exceptionHandler action

Prefix request with Limit

Etcd supports the following:

$ etcdctl get --prefix --limit=2 foo
foo
bar
foo1
bar1

Which allows for a prefix query with a limit. Unfortunately the RangeRequest class doesn't have a Prefix property and the GetAsync overload with prefix argument doesn't take limit as argument.

GetRange with RangeResponse result

It would be useful to have ability to receive actual RangeResponse instead of key/value dictionary

Error when install dotnet-etcd v3.0.0-alpha

I used nuget install dotnet-etcd v3.0.0-alpha, error like below,

Restoring packages for /Users/zhangzhen/serverthreekingdom/src/vcs.taiyouxi.net/jws2/battlecheck/battlecheck/battlecheck.csproj...
NU1605: Detected package downgrade: Google.Protobuf from 3.9.1 to 3.9.0-rc1. Reference the package directly from the project to select a different version. 
 battlecheck -> dotnet-etcd 3.0.0-alpha -> Google.Protobuf (>= 3.9.1) 
 battlecheck -> Google.Protobuf (>= 3.9.0-rc1)
Package restore failed. Rolling back package changes for 'battlecheck'.
NuGet.PackageManagement.PackageReferenceRollbackException: Package restore failed. Rolling back package changes for 'battlecheck'.
  at NuGet.PackageManagement.NuGetPackageManager.ExecuteBuildIntegratedProjectActionsAsync (NuGet.ProjectManagement.Projects.BuildIntegratedNuGetProject buildIntegratedProject, System.Collections.Generic.IEnumerable`1[T] nuGetProjectActions, NuGet.ProjectManagement.INuGetProjectContext nuGetProjectContext, System.Threading.CancellationToken token) [0x00c45] in <7a2d2afbbb674cd48052281699c39e01>:0 
  at NuGet.PackageManagement.NuGetPackageManager.ExecuteNuGetProjectActionsAsync (NuGet.ProjectManagement.NuGetProject nuGetProject, System.Collections.Generic.IEnumerable`1[T] nuGetProjectActions, NuGet.ProjectManagement.INuGetProjectContext nuGetProjectContext, NuGet.Protocol.Core.Types.PackageDownloadContext downloadContext, System.Threading.CancellationToken token) [0x00121] in <7a2d2afbbb674cd48052281699c39e01>:0 
  at NuGet.PackageManagement.NuGetPackageManager.InstallPackageAsync (NuGet.ProjectManagement.NuGetProject nuGetProject, NuGet.Packaging.Core.PackageIdentity packageIdentity, NuGet.PackageManagement.ResolutionContext resolutionContext, NuGet.ProjectManagement.INuGetProjectContext nuGetProjectContext, NuGet.Protocol.Core.Types.PackageDownloadContext downloadContext, System.Collections.Generic.IEnumerable`1[T] primarySources, System.Collections.Generic.IEnumerable`1[T] secondarySources, System.Threading.CancellationToken token) [0x00121] in <7a2d2afbbb674cd48052281699c39e01>:0 
  at NuGet.PackageManagement.NuGetPackageManager.InstallPackageAsync (NuGet.ProjectManagement.NuGetProject nuGetProject, NuGet.Packaging.Core.PackageIdentity packageIdentity, NuGet.PackageManagement.ResolutionContext resolutionContext, NuGet.ProjectManagement.INuGetProjectContext nuGetProjectContext, System.Collections.Generic.IEnumerable`1[T] primarySources, System.Collections.Generic.IEnumerable`1[T] secondarySources, System.Threading.CancellationToken token) [0x000d3] in <7a2d2afbbb674cd48052281699c39e01>:0 
  at JetBrains.ProjectModel.NuGet.Operations.NuGetInstallOperation+<>c__DisplayClass2_0.<InstallAsync>b__0 (NuGet.ProjectManagement.NuGetProject nuGetProject, System.Threading.CancellationToken token) [0x000a5] in <4f50b4f4bae54c32bce2af009baac90a>:0 
  at JetBrains.ProjectModel.NuGet.Operations.NuGetInstallOperation.InstallInternal (JetBrains.ProjectModel.IProject project, System.String packageId, System.String packgeUserStr, JetBrains.ProjectModel.NuGet.Configs.NuGetFeedContext feedContext, System.Int32 nestedLevel, JetBrains.ProjectModel.NuGet.Logging.NuGetNotificationMode mode, JetBrains.ProjectModel.NuGet.Operations.NuGetOperationReporter reporter, System.Func`3[T1,T2,TResult] nativeInstall) [0x003e6] in <4f50b4f4bae54c32bce2af009baac90a>:0 

@ Installing dotnet-etcd in battlecheck finished (4.852 sec)
[Notification][Install] Install failed (project: battlecheck, package: dotnet-etcd v3.0.0-alpha)
Package restore failed. Rolling back package changes for 'battlecheck'.

Etcd client connection issue when etcd restarts

I'm using etcd client in my web api application and have implemented watch which works fine.
However i'm facing the following issue.

Etcd connection gets disconnected

To Reproduce
Steps to reproduce the behavior:

1. During web api bootstrap initialize the etcd client and read the keys , registered a watch and loaded the IConfiguration with the keys.
2. Change any value of the key in etcd
3. We are able to get the call back with the updated value.
4. Stop etcd and start it again.
5. Change the value of any key
6. Unable to receive the watch callback. It seems like with the etcd restart the connection was lost and has not reestablished the connection.

Expected behavior
Application should be able to receive the callback notification with the updated key / value

Blocking on call in GUI-Thread

If I call per example GetAsync in the GUI thread and Wait for the result, then there is a deadlock.
It seams to be like this in this article: https://medium.com/bynder-tech/c-why-you-should-use-configureawait-false-in-your-library-code-d7837dce3d7f
So there is a work around, but it will be great to fix it direcly in the lib.

GetRange includes line return to response

Describe the bug
RangeResponse GetRange(
string prefixKey,
Metadata headers = null,
DateTime? deadline = null,
CancellationToken cancellationToken = default (CancellationToken))
includes line return to key values.

To Reproduce
Steps to reproduce the behavior:

1. Load some keys to etcd
2. Fetch them with GetRange method
3. Look to each key Value.Span property
4. You'll see \r at the end of value.

Expected behavior
Do not include line return as GetVal method does.

Additional context
We are using prefixes like /API_NAME/Key_Name "key_Value"

Lease keep alive missing

Need ability to keep lease alive

Add support for etcd Auth

Add support for the following auth operations.

• AuthEnable // AuthEnable enables authentication.
• AuthDisable // AuthDisable disables authentication.
• Authenticate // Authenticate processes an authenticate request.
• UserAdd // UserAdd adds a new user.
• UserGet // UserGet gets detailed user information.
• UserList // UserList gets a list of all users.
• UserDelete // UserDelete deletes a specified user.
• UserChangePassword // UserChangePassword changes the password of a specified user.
• UserGrantRole // UserGrant grants a role to a specified user.
• UserRevokeRole // UserRevokeRole revokes a role of specified user.
• RoleAdd // RoleAdd adds a new role.
• RoleGet // RoleGet gets detailed role information.
• RoleList // RoleList gets lists of all roles.
• RoleDelete // RoleDelete deletes a specified role.
• RoleGrantPermission // RoleGrantPermission grants a permission of a specified key or range to a specified role.
• RoleRevokePermission // RoleRevokePermission revokes a key or range permission of a specified role.

Is there a way to return all keys

How can I get all the keys. Similar to the way we run the following command which returns all the keys.

etcdctl.exe --user root:password get --from-key '

Basic auth doesn't work

Describe the bug
Basic auth doesn't work

To Reproduce
Steps to reproduce the behavior:

1. Enable basic auth on etcd cluster
2. Try using username and pass word via client

Expected behavior
Method calls should work

Additional context
Basic auth variable has not been handled inside code. That needs to be handled via headers.

Add examples about how to use all etcd operations

Is your feature request related to a problem? Please describe.
I want to use ETCD's ttl, and I noticed that there are some lease methods (LeaseGrant and so on) exposed from client, but I can't figure out how to use them.

Describe the solution you'd like
Add some usage examples in README.md.

Thank you for your creating the great client again.

add separated community contrib project or repo

Is your feature request related to a problem? Please describe.
we want to contribute several methods that allow to get the maximum reliability from interaction with ETCD.
For example, to keep lease alive by sending requests to several nodes.
However, these solutions may not be suitable for everyone and use external dependencies.

Describe the solution you'd like
it would be nice to make the source code more open for extension (get balancer nodes list for example)(we will make pull requests).
And create a separate .csproj and .nuget for code that not everyone needs.

Describe alternatives you've considered
alternatively, create a separate repository.
examples:
https://github.com/Polly-Contrib
https://github.com/serilog-contrib

Add timeout and cancellation token support

Hi, first of all thank you for this library!

I'm building a simple proof of concept project that can do distributed locking using etcd using the 'lockClient' part of the EtcdClient. In this PoC would like to pass a timeout while aquiring a lock, but the deadline and/or cancellationtoken of the gRPC client are not part of the public API of the client. Therefor there is no nice way to implement the timeout. I would like to add both fields to the public API so I can implement this timeout.

cluster

Is your feature request related to a problem? Please describe.
How does the cluster work? You demonstrated an etcd.

Describe the solution you'd like
A clear and concise description of what you want to happen.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.