couchbase/go-couchbase

Couchbase client in Go

A smart client for couchbase in go

This is a unoffical version of a Couchbase Golang client. If you are looking for the Offical Couchbase Golang client please see [CB-go])[https://github.com/couchbaselabs/gocb].

This is an evolving package, but does provide a useful interface to a couchbase server including all of the pool/bucket discovery features, compatible key distribution with other clients, and vbucket motion awareness so application can continue to operate during rebalances.

It also supports view querying with source node randomization so you don't bang on all one node to do all the work.

Install

go get github.com/couchbase/go-couchbase

Example

c, err := couchbase.Connect("http://dev-couchbase.example.com:8091/")
if err != nil {
	log.Fatalf("Error connecting:  %v", err)
}

pool, err := c.GetPool("default")
if err != nil {
	log.Fatalf("Error getting pool:  %v", err)
}

bucket, err := pool.GetBucket("default")
if err != nil {
	log.Fatalf("Error getting bucket:  %v", err)
}

bucket.Set("someKey", 0, []string{"an", "example", "list"})

go-couchbase's Issues

perf/perf.go:101:10: assignment mismatch: 2 variables but 3 values

perf does not build with Go 1.10 x86_64

cd /builddir/build/BUILD/go-couchbase-a064ca416844fb169255a76fbdac7e80101b6049/_build/src/github.com/couchbase/go-couchbase/perf
/usr/lib/golang/pkg/tool/linux_amd64/compile -o $WORK/b001/_pkg_.a -trimpath $WORK/b001 -shared -p main -complete -installsuffix shared -bui
ldid xg8dqXZtFmAoDE_d3Yjw/xg8dqXZtFmAoDE_d3Yjw -goversion go1.10 -D "" -importcfg $WORK/b001/importcfg -pack ./generate-json.go ./perf.go
# github.com/couchbase/go-couchbase/perf
perf/perf.go:101:10: assignment mismatch: 2 variables but 3 values
perf/perf.go:101:22: not enough arguments in call to b.GetBulk
        have ([]string)
        want ([]string, time.Time, []string)
error: Bad exit status from /var/tmp/rpm-tmp.tCh9vC (%build)

TAP close / forwardTapEvents race?

For the TAP client part of go-couchbase, there might be a race condition between tap.Close() and forwardTapEvents. See...

couchbaselabs/cbft#13

Enable Sourcegraph

I want to use Sourcegraph code search and code review with go-couchbase. A project maintainer needs to enable it to set up a webhook so the code is up-to-date there.

Could you please enable go-couchbase on @sourcegraph by going to https://sourcegraph.com/github.com/couchbaselabs/go-couchbase and clicking on Settings? (It should only take 15 seconds.)

Thank you!

Remove bucketlist binary from Github repo

It looks like the binary for the bucketlist example was committed to the repository inflating the size of the repo from about 300KB to 6MB. Can this be be removed please?

https://github.com/couchbaselabs/go-couchbase/tree/master/examples/bucketlist

Type issues

this is my struct that I'm inserting to the db:
type GamePlayer struct {
ID uint64
Name string
}

when I get it, the ID field returns as a float64 type.

This happens for all INT types

Access to new CAS value after Add or SetCas

I'd like to be able to obtain the newest CAS value after a successful Add or SetCas call. Currently a user of this library is required to do a followup GetCas after one of those calls if they intend to modify the value again later, leading to wasted time and network traffic.

Schema injection

Is it possible to inject schema using this client library? If so, how? if not, how is the prevention mechanism...

Great work,
Thanks
ma

Questions on bulk operations

I noticed there was a BulkGet but no corresponding BulkSet. Any reason for that or is it just not commonly needed?

Also for the BulkGet, is the main motivation to allow more efficient throughput by reducing round trips, or just an easier api to use?

/cc @steveyen @adamcfraser

examples/basic/basic.go broken?

Hi

I'm not sure if I could run it before, but I got this today.

2015/04/07 21:38:02 set: MCResponse status=0x20, opcode=SASL_AUTH, opaque=0, msg: Auth failure

Can anyone please have a look?

Thanks

Is using unsafe required?

See https://issues.couchbase.com/browse/GOCBC-38 -- users cannot run this on Google App Engine since packages that use "unsafe" is forbidden.

This repo has the same issue. Is using unsafe absolutely required?

Logging doesn't always use the same unit

From @vmx in couchbase/sync_gateway#850

In the Sync Gateway logging the timings are sometimes seconds and sometimes milliseconds. That makes processing the log files hard. Here's an example

2015/05/04 18:00:52 go-couchbase: call to ViewCustom("sync_gateway", "access") in github.com/couchbase/sync_gateway/db.(*DatabaseContext).ComputeChannelsForPrincipal took 660.192061ms
2015/05/04 18:00:52 go-couchbase: call to ViewCustom("sync_gateway", "access") in github.com/couchbase/sync_gateway/db.(*DatabaseContext).ComputeChannelsForPrincipal took 1.013949095s

Auth issue when connecting to default bucket via cbdatasource

I'm connecting to a default bucket via CBGT, which uses the go-couchbase cbdatasource connection library.

The problem appears to be in this method:

func maybeAddAuth(req *http.Request, ah AuthHandler) error {
    if hah, ok := ah.(HTTPAuthHandler); ok {
        return hah.SetCredsForRequest(req)
    }
    if ah != nil {
        user, pass, _ := ah.GetCredentials()
        req.Header.Set("Authorization", "Basic "+
            base64.StdEncoding.EncodeToString([]byte(user+":"+pass)))
    }
    return nil
}

Even though user and pass are both empty, it tries to add the Basic Authorization header, which ends up failing to connect with 401 errors.

A workaround that appears to work is:

func maybeAddAuth(req *http.Request, ah AuthHandler) error {
    if hah, ok := ah.(HTTPAuthHandler); ok {
        return hah.SetCredsForRequest(req)
    }
    if ah != nil {
        user, pass, _ := ah.GetCredentials()
        if user != "" {
            req.Header.Set("Authorization", "Basic "+
                base64.StdEncoding.EncodeToString([]byte(user+":"+pass)))
        }
    }
    return nil
}

This works around the issue by only adding the Basic Auth header if the user is non-empty.

I'm not sure why it has a non-nil AuthHandler if the username and password are both empty. That might be a bug somewhere up in the callstack of the code that calls this method. Even so, checking for a non-empty user seems like good defensive coding, since it doesn't seem like a valid use case to set a Basic Auth header with an empty user.

Is "import C" actually needed?

https://github.com/couchbase/go-couchbase/blob/master/platform/platform.go#L22

It's not apparent where it's being used. Just wondering if it's a leftover.

Xattr support in cbdatasource for DCP events

For mobile convergence work, we need:

• Ability to start a DCP stream and request Xattrs in DCP feed events
• DCP events should include Xattrs

References

• XATTR Client Access
• Add XATTR support to KV

Need MapUpser API

Need MapUpser API

Current TCPKeepalive defaults result in 5 hours before dead peer connections are reaped

See couchbase/sync_gateway#859 (comment) and couchbase/sync_gateway#859 (comment)

Add API to allow tuning of connection pool

As reported in couchbase/sync_gateway#842, we're seeing situations where we have lots of goroutines in Sync Gateway that appear to be blocked waiting for connections from the pool. We're also seeing huge discrepancies in reported round trip times by go-couchbase vs Couchbase Server, probably because that time includes waiting to get a connection (and as mentioned, goroutines are blocked waiting for connections from the pool).

We're still in the root cause analysis phase, but I have some questions on the connection pooling in go-couchbase:

• What is the current connection pool size defaulting to?
• Are any known "best practices" regarding what the optimal pool size is?

/cc @steveyen @adamcfraser

Get the mapper for the bucket's stats

Instead of getting the basic stats, we need to get the details stats for each bucket, the corresponding restful api for the stats is:
http://${hostsname}:${port}/pools/${pool_name}/buckets/${bucket_name}/stats

cbdatasource with options.SeqEnd, same mutations over and over

When I set the options.SeqEnd to a set of vbucket high-water marks, I end up getting the same mutation delivered to me over and over again.

Also, is there any callback/notification that the configure end has been reached and things can be shut down?

cc @steveyen

Syntax error on client.go

I got errors on Travis CI :
../../couchbase/go-couchbase/client.go:180: syntax error: unexpected range, expecting {
../../couchbase/go-couchbase/client.go:184: non-declaration statement outside function body
../../couchbase/go-couchbase/client.go:185: syntax error: unexpected }

for range vsm.ServerList {
    gs := <-ch
    rv[gs.Server] = gs
}

@steveyen : what syntax "for range" you used here ?

When Couchbase cluster is rebalancing, the sdk's CRUD operation is possbile to return error

When Couchbase cluster is rebalancing, the sdk's CRUD operation is possbile to return error:

Getting data from Couchbase cluster when balancing might not be successful,

in client.go's Do function, even after bucket.refresh, CRUD operation might not be successful

# github.com/couchbaselabs/go-couchbase ../../src/github.com/couchbaselabs/go-couchbase/upr.go:150: too many arguments in call to singleFeed.uprFeed.UprRequestStream

After commit cf12df6. I am unable to build.

github.com/couchbaselabs/go-couchbase ../../src/github.com/couchbaselabs/go-couchbase/upr.go:150: too many arguments in call to singleFeed.uprFeed.UprRequestStream

TestTraceConsolidateByTitle

With:
go 1.9.4 x86_64

Hello,
I try to build and test your project and it fails with :

--- FAIL: TestTraceConsolidateByTitle (0.00s)
trace_test.go:173: expected string "bye (2 times)\nbuh (2 times)", got "bye (2x)\nbuh (2x)"
trace_test.go:179: expected string "bye (2 times)\nprefixbuh (2 times)", got "bye (2x)\nprefixbuh (2x)"
FAIL

Could you help me please ?

Thanks

There is no function get_from_replica like java's api

Does this work with CouchDB?

Googling for CouchDB libraries for Go results in links to couch-go, which in turn all direct here.

I'm guessing that this project has long since abandoned CouchDb support, am I correct?

A small blurb in the Readme explaining why we're redirected here from couch-go, and whether or not this project is still relevant to CouchDB might be nice.

Tap vs Upr

I don't understand how to use Tap or Upr. I don't understand what they're even doing. I'm looking for a small pub/sub feed system. I checked the Couchbase docs and there's no reference to "tap" or "upr".

all ops failed with EOF error after failing over a node

I have the following setup

• a two-node cluster (node 1 in group 1 and node 2 in group 2)
• 3 buckets with 1 replicas
• a minimal load (several ops per second) is running.
• commit 9dc9fdd

As soon as I fail over a node from GUI, all ops starts failing with EOF error.

And it never recovers from that even after "add back" and "rebalance" is done.

Incr operation failing sometimes

In my testcase I increment a counter three times in a row using:

value, err := bucket.Incr(key, amount, amount, 0)

When the bucket has been recently destroyed one of the 3 Incr operations (not necessary the first operation) quite often fails with :

MCResponse status=NOT_STORED, opcode=INCREMENT, opaque=0, msg: Not stored

Any idea what could cause this?

WriteCas doesn't return new CAS value

WriteCas only returns an error, but it should be returning an (err, newCas) pair.

In comparison, gocb's Insert method returns a Cas value.

In order to work around this, Sync Gateway currently needs to the following kludgy workaround:

bucket.Bucket.WriteCas(k, flags, exp, cas, v, couchbase.WriteOptions(opt))
observeResult, err := bucket.Bucket.Observe(k)
LogTo("DIndex+", "WriteCas: post-write cas value is %d", observeResult.Cas)
return observeResult.Cas, err

From an "Api neatness" standpoint, it would be preferable to modify the existing method rather than add a whole new method, even if it breaks some existing users when they try to point to the latest commit on the master branch. (they'd get a compile time error)

Data race detected in go-couchbase/tap.go

Spun out of couchbase/sync_gateway#1400

Using go-couchbase commit ac7dc18:

sg3/sync_gateway_error.log:==================
sg3/sync_gateway_error.log:WARNING: DATA RACE
sg3/sync_gateway_error.log:Read by goroutine 48:
sg3/sync_gateway_error.log:  github.com/couchbase/go-couchbase.(*TapFeed).closeNodeFeeds()
sg3/sync_gateway_error.log:      /home/centos/sync_gateway/src/github.com/couchbase/go-couchbase/tap.go:121 +0x37
sg3/sync_gateway_error.log:  github.com/couchbase/go-couchbase.(*TapFeed).Close()
sg3/sync_gateway_error.log:      /home/centos/sync_gateway/src/github.com/couchbase/go-couchbase/tap.go:135 +0x98
sg3/sync_gateway_error.log:  github.com/couchbase/sync_gateway/base.(*couchbaseFeedImpl).Close()
sg3/sync_gateway_error.log:      <autogenerated>:1 +0x61
sg3/sync_gateway_error.log:  github.com/couchbase/sync_gateway/db.(*changeListener).Stop()
sg3/sync_gateway_error.log:      /home/centos/sync_gateway/src/github.com/couchbase/sync_gateway/db/change_listener.go:82 +0x77
sg3/sync_gateway_error.log:  github.com/couchbase/sync_gateway/db.(*DatabaseContext).Close()
sg3/sync_gateway_error.log:      /home/centos/sync_gateway/src/github.com/couchbase/sync_gateway/db/database.go:262 +0xa6
sg3/sync_gateway_error.log:  github.com/couchbase/sync_gateway/rest.(*ServerContext)._removeDatabase()
sg3/sync_gateway_error.log:      /home/centos/sync_gateway/src/github.com/couchbase/sync_gateway/rest/server_context.go:980 +0x2a7
sg3/sync_gateway_error.log:  github.com/couchbase/sync_gateway/rest.(*ServerContext).ReloadDatabaseFromConfig()
sg3/sync_gateway_error.log:      /home/centos/sync_gateway/src/github.com/couchbase/sync_gateway/rest/server_context.go:528 +0xbf
sg3/sync_gateway_error.log:  github.com/couchbase/sync_gateway/rest.(*handler).handleDbOnline.func1()
sg3/sync_gateway_error.log:      /home/centos/sync_gateway/src/github.com/couchbase/sync_gateway/rest/admin_api.go:87 +0x24f

Full logs in context

How to change a value of beam.smp in couchbase

Hello,

I could find some info from couchbase page
Excessive memory consumption by the beam.smp process on Lnux
If the XDCR Max Replications per Bucket value is raised (such as to 128), then the beam.smp process uses excessive memory. The solution is to reset to 32 or lower.

I could see some beam.smp errors .. I just want to change or see the value of beam.smp for my couchbase server..

Can anyone let me know how to find the value of beam.smp

Thanks,
Dev

bad strings.Replace call arguments

#87

- strings.Replace(buf.String(), "`", "` + \"`\" + `", 0))
+ strings.Replace(buf.String(), "`", "` + \"`\" + `", -1))

Reporting an issue, but it's not convenient for me to send a gerrit CL.
You can see a fix in a referenced PR.

cbdatasource Receiver OnError() question

In Receiver:

	// Invoked in advisory fashion by the BucketDataSource when it
	// encounters an error.  The BucketDataSource will continue to try
	// to "heal" and restart connections, etc, as necessary.  The
	// Receiver has a recourse during these error notifications of
	// simply Close()'ing the BucketDataSource.
	OnError(error)

How do we distinguish between actionable errors and ignorable errors?

More concretely, in Sync Gateway we are seeing this error come through:

2017-04-26T19:51:11.697-04:00 DCP: OnError: pkt.Receive, err: EOF
2017-04-26T19:51:11.697-04:00 WARNING: Feed%!(EXTRA string=Error processing DCP stream: %v, *errors.errorString=pkt.Receive, err: EOF) -- base.(*DCPReceiver).OnError() at dcp_feed.go:123

and not sure of the cause and any action we should take, like restarting one more more vbucket streams or the entire DCP feed.

No Secured Connection Examples

Security is a major issue. Without having a secured connection to Couchbase, it is possible for attackers to initialize a man in the middle attack and steal sensitive information which could be used to compromise couchbase accounts to inject or even modify information. It would be good to see an example of how to secure a connection to Couchbase with this lib.

AuthHandler doc comment "default" bucket safety tips

Originally from... couchbaselabs/cbgt#32

Doc improvement request, related to AuthHandler / pools.go:

// AuthHandler is a callback that gets the auth username and password
// for the given bucket.
type AuthHandler interface {
GetCredentials() (string, string, string)
}

"Should get an upgrade in the comment that specs out the contract "if using the 'default' bucket, the username should be 'default'". Either that or the more generic "the username returned by this should never be empty"."

how to touch a document

i cannot find any way to touch a document (to just only update the expiration)

not enough arguments in call to newConnectionPool

With:
go 1.9.4 x86_64

Hello,
I try to build and test your project and it fails with

Testing: "/builddir/build/BUILD/go-couchbase-a064ca416844fb169255a76fbdac7e80101b6049/_build/src/github.com/couchbase/go-couchbase"

• GOPATH=/builddir/build/BUILD/go-couchbase-a064ca416844fb169255a76fbdac7e80101b6049/_build:/usr/share/gocode
• go test -buildmode pie -compiler gc -ldflags '-extldflags '''-Wl,-z,relro ''''

github.com/couchbase/go-couchbase

./conn_pool_test.go:40:25: not enough arguments in call to newConnectionPool
have (string, *basicAuth, number, number)
want (string, AuthHandler, bool, int, int)
./conn_pool_test.go:132:25: not enough arguments in call to newConnectionPool
have (string, *basicAuth, number, number)
want (string, AuthHandler, bool, int, int)
./conn_pool_test.go:173:25: not enough arguments in call to newConnectionPool
have (string, *basicAuth, number, number)
want (string, AuthHandler, bool, int, int)
./conn_pool_test.go:200:25: not enough arguments in call to newConnectionPool
have (string, *basicAuth, number, number)
want (string, AuthHandler, bool, int, int)
./conn_pool_test.go:229:25: not enough arguments in call to newConnectionPool
have (string, *basicAuth, number, number)
want (string, AuthHandler, bool, int, int)
./conn_pool_test.go:260:25: not enough arguments in call to newConnectionPool
have (string, *basicAuth, number, number)
want (string, AuthHandler, bool, int, int)
./conn_pool_test.go:306:25: not enough arguments in call to newConnectionPool
have (string, *basicAuth, number, number)
want (string, AuthHandler, bool, int, int)
./conn_pool_test.go:327:25: not enough arguments in call to newConnectionPool
have (string, *basicAuth, number, number)
want (string, AuthHandler, bool, int, int)
./conn_pool_test.go:336:24: not enough arguments in call to newConnectionPool
have (string, *basicAuth, number, number)
want (string, AuthHandler, bool, int, int)
./conn_pool_test.go:351:25: not enough arguments in call to newConnectionPool
have (string, *basicAuth, number, number)
want (string, AuthHandler, bool, int, int)
./conn_pool_test.go:351:25: too many errors
FAIL github.com/couchbase/go-couchbase [build failed]

Could you help me please ?

Thanks

Add ability to update expiry on get

Sync Gateway has a requirement to be able to update the expiry on a get. I think it's reasonable to add this as a separate API (GetAndTouch), as opposed to modifying the existing Get.

Concurrency - Warning: Finalizing a bucket with active connections.

What is the correct usage to prevent such a warning
https://github.com/couchbase/go-couchbase/blob/master/pools.go#L757

I would be great if there was documentation on correct usage within the context of an application server

• Should connections be created per go routine
• Should buckets be created per go routine
• When should a resources be released e.g
  • Connection Pool Close
  • Connection Pool Return

TestConnPool / TestConnPoolSoonAvailable unit test failure

Using origin/master (commit 35d65cd), I see errors while trying go test...

$ go test -v
=== RUN TestWriteOptionsString
--- PASS: TestWriteOptionsString (0.00s)
=== RUN TestConnPool
--- FAIL: TestConnPool (0.01s)
conn_pool_test.go:117: Expected error on second pool close
=== RUN TestConnPoolSoonAvailable
--- PASS: TestConnPoolSoonAvailable (0.01s)
conn_pool_test.go:169: Callback report: map[create:7 short-circuit:1 avail1:2], timings: [1.273003ms 1.155947ms 1.162186ms 1.150928ms 1.149961ms 1.157836ms 1.141134ms 2.313256ms 1.168166ms 1.229084ms]
=== RUN TestConnPoolClosedFull
--- PASS: TestConnPoolClosedFull (0.01s)
=== RUN TestConnPoolWaitFull
--- PASS: TestConnPoolWaitFull (0.01s)
=== RUN TestConnPoolWaitFailFull
--- PASS: TestConnPoolWaitFailFull (0.01s)
=== RUN TestConnPoolWaitDoubleFailFull
--- PASS: TestConnPoolWaitDoubleFailFull (0.01s)
=== RUN TestConnPoolNil
--- PASS: TestConnPoolNil (0.00s)
=== RUN TestConnPoolClosed
--- PASS: TestConnPoolClosed (0.00s)
=== RUN TestConnPoolCloseWrongPool
--- PASS: TestConnPoolCloseWrongPool (0.00s)
=== RUN TestConnPoolCloseNil
--- PASS: TestConnPoolCloseNil (0.00s)
=== RUN TestConnPoolStartTapFeed
--- PASS: TestConnPoolStartTapFeed (0.00s)
=== RUN TestPoolsResponse
--- PASS: TestPoolsResponse (0.00s)
=== RUN TestPool
--- PASS: TestPool (0.00s)
=== RUN TestCommonAddressSuffixEmpty
--- PASS: TestCommonAddressSuffixEmpty (0.00s)
=== RUN TestCommonAddressSuffixUncommon
--- PASS: TestCommonAddressSuffixUncommon (0.00s)
=== RUN TestCommonAddressSuffixCommon
--- PASS: TestCommonAddressSuffixCommon (0.00s)
=== RUN TestBucketConnPool
--- PASS: TestBucketConnPool (0.00s)
=== RUN TestBucketConnPoolConcurrent
--- PASS: TestBucketConnPoolConcurrent (0.00s)
=== RUN TestCleanupHost
--- PASS: TestCleanupHost (0.00s)
=== RUN TestFindCommonSuffix
--- PASS: TestFindCommonSuffix (0.00s)
=== RUN TestParseURL
--- PASS: TestParseURL (0.00s)
=== RUN TestVBHash
--- PASS: TestVBHash (0.00s)
=== RUN TestViewError
--- PASS: TestViewError (0.00s)
=== RUN TestViewURL
2015/08/25 08:20:25 Non-healthy node; node details:
2015/08/25 08:20:25 Hostname=, Status=, CouchAPIBase=, ThisNode=false
2015/08/25 08:20:25 Non-healthy node; node details:
2015/08/25 08:20:25 Hostname=, Status=, CouchAPIBase=::gopher:://localhost:80x92/, ThisNode=false
2015/08/25 08:20:25 Non-healthy node; node details:
2015/08/25 08:20:25 Hostname=, Status=, CouchAPIBase=http:://localhost:8092/, ThisNode=false
--- PASS: TestViewURL (0.00s)
=== RUN TestBadViewParam
--- PASS: TestBadViewParam (0.00s)
FAIL
exit status 1
FAIL github.com/couchbase/go-couchbase 0.140s
Steves-MacBook-Pro:go-couchbase steveyen$

Getting error 'EOF' when to read data from a bucket in CB 5.0.1

Hello,

I recently built a new Couchbase server (community 5.0.1) and started getting an error when I tried to read any item in my bucket. The code shown below is a simple test code to read data from CB, it works well with CB 4.5. However, EOF error when it calls 'bucket.Get' consists with CB 5.0.1, the only difference from the code is authentication part since CB 5.0 requires using authentication. Would this be my fault or something that go-couchbase does not support with 5.0.1?

Thanks!

func main() {
c, err := couchbase.Connect("http://test1:test123@mycouchbase:8091/") // CB 5.0.1
// c, err := couchbase.Connect("http://@mycouchbase:8091/") // CB 4.5
if err != nil {
log.Fatalf("Error connecting: %v", err)
}

pool, err := c.GetPool("default")
if err != nil {
	log.Fatalf("Error getting pool:  %v", err)
}

bucket, err := pool.GetBucket("Mybucket")
if err != nil {
	log.Fatalf("Error getting bucket:  %v", err)
}
val := make(map[string]interface{})
err = bucket.Get("mykey1", &val)
if err != nil {
	log.Fatalf("Error getting data from bucket:  %v", err)   // <- Error 'EOF'
}

}

Broken link on project description

Hello,

The link to documentation in project description is broken: https://godoc.org/github.com/couchbaselabs/go-couchbase

Maybe it should be replaced to https://godoc.org/github.com/couchbase/go-couchbase

upr_restart panics

This panic is on a mac (64 vbuckets), but I don't think thats related as it only works on the first few vbuckets anyway.

Also, it has no replicas, but again seems like it shouldn't be related, as I would expect it to only be talking to the masters anyway.

I did change the URL to point to :8091 instead of :9000.

$ ./upr_restart 
09:48:08.736652 Default None UPR_STREAMREQ for vbucket 0 successful
09:48:08.736855 Default None UPR_STREAMREQ for vbucket 1 successful
2014/09/19 09:48:09  Mutation count 0, Snapshot markers 0
2014/09/19 09:48:10 Restarting ....
2014/09/19 09:48:10 go-couchbase: Error connecting to upr feed of 127.0.0.1:11210: EOF
panic: Cannot connect to bucket EOF

goroutine 16 [running]:
runtime.panic(0x29bca0, 0xc2080c3dc0)
    /Users/mschoch/Documents/research/go/go/src/pkg/runtime/panic.c:279 +0xf5
main.main()
    /Users/mschoch/go/src/github.com/couchbaselabs/go-couchbase/examples/upr_restart/restart.go:60 +0x6b3

goroutine 19 [finalizer wait]:
runtime.park(0x15c80, 0x4ae470, 0x4acf69)
    /Users/mschoch/Documents/research/go/go/src/pkg/runtime/proc.c:1369 +0x89
runtime.parkunlock(0x4ae470, 0x4acf69)
    /Users/mschoch/Documents/research/go/go/src/pkg/runtime/proc.c:1385 +0x3b
runfinq()
    /Users/mschoch/Documents/research/go/go/src/pkg/runtime/mgc0.c:2644 +0xcf
runtime.goexit()
    /Users/mschoch/Documents/research/go/go/src/pkg/runtime/proc.c:1445

goroutine 20 [IO wait]:
net.runtime_pollWait(0x55f638, 0x72, 0x0)
    /Users/mschoch/Documents/research/go/go/src/pkg/runtime/netpoll.goc:146 +0x66
net.(*pollDesc).Wait(0xc20802a060, 0x72, 0x0, 0x0)
    /Users/mschoch/Documents/research/go/go/src/pkg/net/fd_poll_runtime.go:84 +0x46
net.(*pollDesc).WaitRead(0xc20802a060, 0x0, 0x0)
    /Users/mschoch/Documents/research/go/go/src/pkg/net/fd_poll_runtime.go:89 +0x42
net.(*netFD).accept(0xc20802a000, 0x3a00c8, 0x0, 0x55e418, 0x23)
    /Users/mschoch/Documents/research/go/go/src/pkg/net/fd_unix.go:409 +0x343
net.(*UnixListener).AcceptUnix(0xc208044220, 0x0, 0x0, 0x0)
    /Users/mschoch/Documents/research/go/go/src/pkg/net/unixsock_posix.go:293 +0x73
net.(*UnixListener).Accept(0xc208044220, 0x0, 0x0, 0x0, 0x0)
    /Users/mschoch/Documents/research/go/go/src/pkg/net/unixsock_posix.go:304 +0x4b
github.com/couchbaselabs/retriever/logger.doHandleConnections(0xc2080540c0, 0x327530, 0xa)
    /Users/mschoch/go/src/github.com/couchbaselabs/retriever/logger/accept.go:50 +0x2e9
github.com/couchbaselabs/retriever/logger.handleConnections(0xc2080540c0, 0x327530, 0xa)
    /Users/mschoch/go/src/github.com/couchbaselabs/retriever/logger/accept.go:24 +0x3b
created by github.com/couchbaselabs/retriever/logger.NewLogger
    /Users/mschoch/go/src/github.com/couchbaselabs/retriever/logger/logger.go:136 +0x2a0

goroutine 17 [syscall]:
runtime.goexit()
    /Users/mschoch/Documents/research/go/go/src/pkg/runtime/proc.c:1445

goroutine 22 [IO wait]:
net.runtime_pollWait(0x55f588, 0x72, 0x0)
    /Users/mschoch/Documents/research/go/go/src/pkg/runtime/netpoll.goc:146 +0x66
net.(*pollDesc).Wait(0xc20802a220, 0x72, 0x0, 0x0)
    /Users/mschoch/Documents/research/go/go/src/pkg/net/fd_poll_runtime.go:84 +0x46
net.(*pollDesc).WaitRead(0xc20802a220, 0x0, 0x0)
    /Users/mschoch/Documents/research/go/go/src/pkg/net/fd_poll_runtime.go:89 +0x42
net.(*netFD).Read(0xc20802a1c0, 0xc208063000, 0x1000, 0x1000, 0x0, 0x55e418, 0x23)
    /Users/mschoch/Documents/research/go/go/src/pkg/net/fd_unix.go:232 +0x34c
net.(*conn).Read(0xc208036040, 0xc208063000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
    /Users/mschoch/Documents/research/go/go/src/pkg/net/net.go:122 +0xe7
net/http.noteEOFReader.Read(0x55f840, 0xc208036040, 0xc208038108, 0xc208063000, 0x1000, 0x1000, 0x4c03a0, 0x0, 0x0)
    /Users/mschoch/Documents/research/go/go/src/pkg/net/http/transport.go:1203 +0x72
net/http.(*noteEOFReader).Read(0xc208044600, 0xc208063000, 0x1000, 0x1000, 0xc20804e178, 0x0, 0x0)
    <autogenerated>:124 +0xca
bufio.(*Reader).fill(0xc2080044e0)
    /Users/mschoch/Documents/research/go/go/src/pkg/bufio/bufio.go:97 +0x1b3
bufio.(*Reader).Peek(0xc2080044e0, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
    /Users/mschoch/Documents/research/go/go/src/pkg/bufio/bufio.go:132 +0x101
net/http.(*persistConn).readLoop(0xc2080380b0)
    /Users/mschoch/Documents/research/go/go/src/pkg/net/http/transport.go:782 +0x95
created by net/http.(*Transport).dialConn
    /Users/mschoch/Documents/research/go/go/src/pkg/net/http/transport.go:600 +0x93f

goroutine 23 [select]:
net/http.(*persistConn).writeLoop(0xc2080380b0)
    /Users/mschoch/Documents/research/go/go/src/pkg/net/http/transport.go:885 +0x38f
created by net/http.(*Transport).dialConn
    /Users/mschoch/Documents/research/go/go/src/pkg/net/http/transport.go:601 +0x957

goroutine 25 [select]:
github.com/couchbase/gomemcached/client.sendCommands(0xc2080aeb70, 0xc208005020, 0xc208005080)
    /Users/mschoch/go/src/github.com/couchbase/gomemcached/client/upr_feed.go:196 +0x204
created by github.com/couchbase/gomemcached/client.(*Client).NewUprFeed
    /Users/mschoch/go/src/github.com/couchbase/gomemcached/client/upr_feed.go:224 +0x185

goroutine 26 [select]:
github.com/couchbase/gomemcached/client.(*UprFeed).runFeed(0xc20804e630, 0xc208005200)
    /Users/mschoch/go/src/github.com/couchbase/gomemcached/client/upr_feed.go:537 +0x91d
created by github.com/couchbase/gomemcached/client.(*UprFeed).StartFeed
    /Users/mschoch/go/src/github.com/couchbase/gomemcached/client/upr_feed.go:367 +0x65

goroutine 27 [chan send]:
github.com/couchbaselabs/go-couchbase.(*UprFeed).forwardUprEvents(0xc208018b40, 0xc2080ba200, 0xc208004f60, 0xc2080ab7d0, 0xf)
    /Users/mschoch/go/src/github.com/couchbaselabs/go-couchbase/upr.go:240 +0x226
created by github.com/couchbaselabs/go-couchbase.(*UprFeed).connectToNodes
    /Users/mschoch/go/src/github.com/couchbaselabs/go-couchbase/upr.go:222 +0x3e5

goroutine 28 [select]:
github.com/couchbaselabs/go-couchbase.(*UprFeed).run(0xc208018b40)
    /Users/mschoch/go/src/github.com/couchbaselabs/go-couchbase/upr.go:161 +0x3f5
created by github.com/couchbaselabs/go-couchbase.(*Bucket).StartUprFeed
    /Users/mschoch/go/src/github.com/couchbaselabs/go-couchbase/upr.go:115 +0x269

goroutine 29 [select]:
github.com/couchbase/gomemcached/client.sendCommands(0xc2080aef00, 0xc2080e2f00, 0xc2080e2f60)
    /Users/mschoch/go/src/github.com/couchbase/gomemcached/client/upr_feed.go:196 +0x204
created by github.com/couchbase/gomemcached/client.(*Client).NewUprFeed
    /Users/mschoch/go/src/github.com/couchbase/gomemcached/client/upr_feed.go:224 +0x185

go-couchbase is transforming 404 errors from couchbase to 500 errors querying non-existent views

If you query a non-existent view from go-couchbase, it will get a 404 back from Couchbase Server, but then return a 500 error to the go-couchbase client.

fatal error: all goroutines are asleep - deadlock!

Hello,

I would like to report an error encountered recently after update with golang v1.9.1 . The following error frequently happens when it calls. Not always, but sometime happens. Can anybody please look into this and hopefully fix this soon?

Thank you!

fatal error: all goroutines are asleep - deadlock!

goroutine 1 [select]:
net/http.(persistConn).roundTrip(0xc042092fc0, 0xc04206aae0, 0x0, 0x0, 0x0)
C:/Go/src/net/http/transport.go:1970 +0x60f
net/http.(Transport).RoundTrip(0x8008a0, 0xc0420fa000, 0x8008a0, 0x0, 0x0)
C:/Go/src/net/http/transport.go:413 +0x999
net/http.send(0xc0420fa000, 0x7ceca0, 0x8008a0, 0x0, 0x0, 0x0, 0xc042076048, 0x4
1034d, 0xc042073390, 0x1)
C:/Go/src/net/http/client.go:249 +0x1b0
net/http.(Client).send(0x7fc020, 0xc0420fa000, 0x0, 0x0, 0x0, 0xc042076048, 0x0
, 0x1, 0x12200ffffffff)
C:/Go/src/net/http/client.go:173 +0x104
net/http.(Client).Do(0x7fc020, 0xc0420fa000, 0x0, 0xc0420735c8, 0x4437de)
C:/Go/src/net/http/client.go:602 +0x294
github.com/couchbase/go-couchbase.doHTTPRequest(0xc0420fa000, 0x0, 0x0, 0x0)
C:/goproject/src/github.com/couchbase/go-couchbase/pools.go:534 +0x9f
github.com/couchbase/go-couchbase.queryRestAPI(0xc042080100, 0x6a6c3a, 0x6, 0x0,
0x0, 0x638a40, 0xc0420ec1f8, 0x0, 0x0)
C:/goproject/src/github.com/couchbase/go-couchbase/pools.go:632 +0x26a
github.com/couchbase/go-couchbase.(Client).parseURLResponse(0xc0420ec1e0, 0x6a6
c3a, 0x6, 0x638a40, 0xc0420ec1f8, 0x0, 0x410801)
C:/goproject/src/github.com/couchbase/go-couchbase/pools.go:652 +0x6f
github.com/couchbase/go-couchbase.ConnectWithAuth(0xc042074030, 0x24, 0x0, 0x0,
0x346010, 0x0, 0x0, 0x805280, 0x3406a8, 0x0, ...)
C:/goproject/src/github.com/couchbase/go-couchbase/pools.go:773 +0xdf
github.com/couchbase/go-couchbase.Connect(0xc042074030, 0x24, 0x0, 0x0, 0x0, 0x0
, 0x0, 0x0, 0x0, 0x0, ...)
C:/goproject/src/github.com/couchbase/go-couchbase/pools.go:792 +0xcd
main.(repoCouchbase3).open(0xc042073cc8, 0xc042074030, 0x24, 0xc0420560c0, 0x7,
0xc0420d8660, 0x20)
C:/goproject/src/:298 +0xd9
main.bucketsImpl(0xc042074030, 0x24, 0xc0420560c0, 0x7, 0xc0420521f0, 0x1, 0x1,
0x0, 0x0)
C:/goproject/src/:245 +0xcf
laszlo/tree.Main(0x6bab10, 0x6bab20, 0x6baaf8, 0x6bab00, 0x6bab30, 0x6baae8, 0x6
87000)
C:/goproject/src/:194 +0x13c9
main.main()
C:/goproject/src/******:32 +0x222

goroutine 22 [IO wait]:
internal/poll.runtime_pollWait(0xb24f70, 0x72, 0x0)
C:/Go/src/runtime/netpoll.go:173 +0x5e
internal/poll.(*pollDesc).wait(0xc042094c98, 0x72, 0x7cd200, 0x0, 0x0)
C:/Go/src/internal/poll/fd_poll_runtime.go:85 +0xb5
internal/poll.(*ioSrv).ExecIO(0x802d78, 0xc042094b58, 0x6baa30, 0x0, 0x0, 0x0)
C:/Go/src/internal/poll/fd_windows.go:195 +0x13a
internal/poll.(*FD).Read(0xc042094b40, 0xc04210a000, 0x1000, 0x1000, 0x0, 0x0, 0
x0)
C:/Go/src/internal/poll/fd_windows.go:439 +0x266
net.(*netFD).Read(0xc042094b40, 0xc04210a000, 0x1000, 0x1000, 0x345430, 0x0, 0x0
)
C:/Go/src/net/fd_windows.go:151 +0x59
net.(*conn).Read(0xc042076058, 0xc04210a000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
C:/Go/src/net/net.go:176 +0x74
net/http.(*persistConn).Read(0xc042092fc0, 0xc04210a000, 0x1000, 0x1000, 0x20, 0
x677300, 0xc042105b68)
C:/Go/src/net/http/transport.go:1391 +0x147
bufio.(*Reader).fill(0xc0420ec600)
C:/Go/src/bufio/bufio.go:97 +0x121
bufio.(*Reader).Peek(0xc0420ec600, 0x1, 0xc042016060, 0xc042105c80, 0x805280, 0x
3406a8, 0x0)
C:/Go/src/bufio/bufio.go:129 +0x41
net/http.(*persistConn).readLoop(0xc042092fc0)
C:/Go/src/net/http/transport.go:1539 +0x18c
created by net/http.(*Transport).dialConn
C:/Go/src/net/http/transport.go:1186 +0xa35

goroutine 23 [select]:
net/http.(*persistConn).writeLoop(0xc042092fc0)
C:/Go/src/net/http/transport.go:1759 +0x16c
created by net/http.(*Transport).dialConn
C:/Go/src/net/http/transport.go:1187 +0xa5a

Questions

1. for what version of Couchbase is this?
2. is this uses native protocol or REST?

does this library really work

I changed the example a little bit
err1 := bucket.Set("someKey", 0, []string{"an", "example", "list"})
if err1 != nil {
log.Fatalf("Error setting bucket: %v", err1)
}
because I always get error.
on Couchbase 4, I got
Error setting bucket: MCResponse status=AUTH_ERROR, opcode=SASL_AUTH, opaque=0, msg: Auth failure
on Couchbase 5, I got
res=MCResponse status=SUCCESS, opcode=GET, opaque=0, msg: ,err=EOF

Why GetBulk() use 4 goroutines to fetch data?

I want to use GetBulk to fetch data, which fetches multiple keys concurrently. But the source code of this function use only 4 worker goroutines to fetch concurrently, I think the count is len(kdm) is better. And the channel of key, wch, I think it should be allocated with enough buffer, like this, wch := make(chan uint16, len(kdm).

func (b *Bucket) processBulkGet(kdm map[uint16][]string,
    ch chan map[string]*gomemcached.MCResponse, ech chan error) {
    wch := make(chan uint16)
    defer close(ch)
    defer close(ech)

    wg := &sync.WaitGroup{}
    worker := func() {
        defer wg.Done()
        for k := range wch {
            b.doBulkGet(k, kdm[k], ch, ech)
        }
    }

    for i := 0; i < 4; i++ {
        wg.Add(1)
        go worker()
    }

    for k := range kdm {
        wch <- k
    }
    close(wch)
    wg.Wait()
}

cannot get or set values to a 'Memcached' type bucket

Using the following stub code below against a cluster with a 'Memcached' type bucket called 'memcached_bucket' I get the following error everytime I try to read or write a value:

ahurt1:local ahurt$ go build cb-test.go; ./cb-test
2015/03/16 17:08:05 Error setting key: go-couchbase: vbmap smaller than vbucket list: 6536 vs. []

Thank you in advance!

package main

import (
    "bytes"
    "github.com/couchbase/go-couchbase"
    "log"
)

const (
    CB_HOST   = "http://some-host:8091/"
    CB_BUCKET = "memcached_bucket"
)

func main() {
    c, err := couchbase.Connect(CB_HOST)
    if err != nil {
        log.Fatalf("Error connecting:  %v", err)
    }

    pool, err := c.GetPool("default")
    if err != nil {
        log.Fatalf("Error getting pool:  %v", err)
    }

    bucket, err := pool.GetBucket(CB_BUCKET)
    if err != nil {
        log.Fatalf("Error getting bucket:  %v", err)
    }

    buff := bytes.NewBufferString("someValue")

    err = bucket.SetRaw("someKey", 0, buff.Bytes())
    if err != nil {
        log.Fatalf("Error setting key: %v", err)
    }
}

Does not work with Docker

I'm running my Couchbase server in Docker with port mapping to my local machine for development.
This works fine for the .NET SDK, however, when using this Go SDK, the SDK tries to talk to the Docker IP.

2016/09/26 09:03:27 dial tcp 172.17.0.2:11210: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.
exit status 1

The 172.17.0.2 is the internal IP for the container, and is not accessible from the outside.
Do note that this is not the IP I provided when trying to connect to the server:
I provided http://localhost:8091,

Does this mean that we cannot use Couchbase hosted in Docker for development environments?