Terrible Write Performance with DynamoDB Local

[taoensso]

Terrible Write Performance with DynamoDB Local

[ptaoussanis]

Hi Scott, have you confirmed that this is a problem with Faraday specifically and not DDB / your DDB configuration?

Could you possibly try check your write performance against DDB local directly? Faraday is a very thin layer over the DDB API; barring bugs - it shouldn't significantly impact the perf you'd see against the Java API.

Disclaimer: I've never used DDB local myself.

[ptaoussanis]

Also, could you please clarify what intro.md is / what the samples look like?

[tcoupland]

Another idea might be to checkout faraday and run the tests in local mode. Seeing how fast they run might help narrow down the problem!

[skbach]

Hi,

I should've been more clear. I'm pretty certain the problem is NOT with Faraday, and maybe I shouldn't have polluted your issue tracker with Dynamo issues :). However, Amazon has been unresponsive and several other people online are reporting similar write speeds. I figured you had probably tested the code using Local, and was wondering if there was some configuration (either in the Client API or Dynamo Service) that could improve it.

By intro.md I mean the intro on this github page. I tested using the sample (far/put-item client-opts
:my-table...etc.

I've since given up on dynamo and moved to mongo, and now my write performance is nearly 500 x faster (from 120 ms per item to 250 microseconds per item) on the same box. (even with the most conservative mongo writeconcern settings). That's why I figured there must be something trivially wrong with my dynamo setup.

Thanks.

[ptaoussanis]

I would guess that this is a config issue, but I'm not sure. In any case, I'd note that DDB local is (afaik) intended for development and testing purposes; not for use as a production database itself. It's conceivable to me that they'd intentionally even simulate some network latency by default, but you or I would need to look at the docs to check.

I'd add that even if it worked as a production local DB, you'd be nullifying most of the point of using DDB in the first place. If you're not looking for the scaling / services aspect of DDB, then better choices exist for a local DB. Not sure what your actual requirements are, but you may also want to investigate Riak and PostgreSQL. I'm not a fan of MongoDB, but that's certainly also an option if you're satisfied with it and it fits your particular requirements.

Best of luck, cheers! :-)