ktheory/dalli-elasticache

A wrapper for Dalli with support for AWS ElastiCache

dalli-elasticache's Issues

Cannot connect to elasticache

This error shows on run application on passenger

These are the possible causes:
There may be a syntax error in the application's code. Please check for such errors and fix them.
A required library may not installed. Please install all libraries that this application requires.
The application may not be properly configured. Please check whether all configuration files are written correctly, fix any incorrect configurations, and restart this application.
A service that the application relies on (such as the database server or the Ferret search engine server) may not have been started. Please start that service.
Further information about the error may have been written to the application's log file. Please check it in order to analyse the problem.
Error message:
getaddrinfo: Name or service not known
Exception class:
SocketError

Using an ubuntu flavor memcache instance breaks the regex

dalli-elasticache/lib/dalli/elasticache/auto_discovery/stats_response.rb

Line 19 in 565bf94

@text = response_text.to_s

is:
STAT version 1.4.14 (Ubuntu)

Results in the error:
undefined method `[]' for nil:NilClass

Which was traced back to:

dalli-elasticache/lib/dalli/elasticache/auto_discovery/stats_response.rb

Line 26 in 565bf94

Gem::Version.new(VERSION_REGEX.match(@text)[1])

The string 'STAT version 1.4.14 (Ubuntu)' is causing the regex to return nil because it failed to find matches

Set up Travis CI configuration

The .travis.yml file is currently configured to watch https://github.com/zmillman/dalli-elasticache for updates.

@ktheory needs to set up the configuration and update the badge on the README.

Having issues with Ruby/dalli-elasticache inside a Docker Container.

Hi, We are having problems with Passenger and RoR application running on Docker. getting getaddrinfo: Name or service not known (SocketError) in nginx/errors.log, but all works on Ruby Console via 'bundle exec rails c'. We are Running Ruby 2.3.1 on nginx/1.10.1 Phusion_Passenger/5.0.29. Also all networking and DNS resolution working fine on Docker container, tested with ping, nslookup and raw telnet to memcache server. Below are the additional errors below, don't see any issues with remote MySQL and Redis; any help would be greatly appreciated.
Thanks,

getaddrinfo: Name or service not known (SocketError)
/usr/local/rvm/gems/ruby-2.3.1/gems/socksify-1.7.0/lib/socksify.rb:178:in initialize' /usr/local/rvm/gems/ruby-2.3.1/gems/socksify-1.7.0/lib/socksify.rb:178:in initialize'
/usr/local/rvm/gems/ruby-2.3.1/gems/dalli-elasticache-0.2.0/lib/dalli/elasticache/auto_discovery/endpoint.rb:64:in new' /usr/local/rvm/gems/ruby-2.3.1/gems/dalli-elasticache-0.2.0/lib/dalli/elasticache/auto_discovery/endpoint.rb:64:in remote_command'
/usr/local/rvm/gems/ruby-2.3.1/gems/dalli-elasticache-0.2.0/lib/dalli/elasticache/auto_discovery/endpoint.rb:47:in get_stats_from_remote' /usr/local/rvm/gems/ruby-2.3.1/gems/dalli-elasticache-0.2.0/lib/dalli/elasticache/auto_discovery/endpoint.rb:27:in stats'
/usr/local/rvm/gems/ruby-2.3.1/gems/dalli-elasticache-0.2.0/lib/dalli/elasticache/auto_discovery/endpoint.rb:37:in engine_version' /usr/local/rvm/gems/ruby-2.3.1/gems/dalli-elasticache-0.2.0/lib/dalli/elasticache/auto_discovery/endpoint.rb:52:in get_config_from_remote'
/usr/local/rvm/gems/ruby-2.3.1/gems/dalli-elasticache-0.2.0/lib/dalli/elasticache/auto_discovery/endpoint.rb:32:in config' /usr/local/rvm/gems/ruby-2.3.1/gems/dalli-elasticache-0.2.0/lib/dalli/elasticache.rb:37:in servers'
/home/app/webapp/config/environments/production.rb:40:in `block in <top (required)>'

Documentation on how often to refresh nodes

The README states how you can refresh the client, but it does not state how often this should occur or by what means.

I couldn't find much documentation on how often or why Elasticache nodes might drop off or join. The only thing the AWS docs mention is that:

If a node fails, ElastiCache takes down that node and spins up a replacement. The replacement process takes a few minutes. During this time the metadata in all the nodes still shows the endpoint for the failed node, but any attempt to interact with the node will fail. Therefore, your logic should always include retry logic.
Reference

What are users of this gem expected to implement in terms of refreshing? I see in the Dalli-Elasticache docs that it refreshes upon app server restart. However, should we implement a refresh as a cron? Every 30minutes? Sample code would be most appreciated!

I've also posted a similar question on Stack overflow: https://stackoverflow.com/questions/47170376/how-to-refresh-clustered-redis-elasticache-nodes-in-a-rails-app
Thanks.

Simpler Rails cache store configuration

If feels kind of verbose to create a Dalli::ElastiCache instance, get a list of servers, then use a :dalli_store for the actual application store in production.rb.

I don't know how tricky wrapping ActiveSupport::Cache::DalliStore is, but my ideal configuration would look something more like this:

# in config/environments/production.rb

config.cache_store = :dalli_elasticache_store, "my-cluster.cfg.use1.cache.amazonaws.com:11211", {:expires_in => 1.day}

Other thoughts: Should the store default to reading the endpoint configuration from a config/dalli-elasticache.yml file or be configured in an initializer?

Missing release tag 0.1.2

Can you please tag release v0.1.2. Thx.

Support for TLS

Dalli already has support for connections with TLS, would it be possible to support the same for auto-discovery? Attempting to get the servers right now fails with: Errno::ECONNRESET: Connection reset by peer.

Perhaps the ssl_context we pass to Dalli through dalli_options could be used for the socket connection as well?

configuration example kaputt

config_endpoint = "my-cluster-name.abc123.cfg.use1.cache.amazonaws.com:11211"
elasticache     = Dalli::ElastiCache.new(endpoint)

shouldnt that be

elasticache     = Dalli::ElastiCache.new(config_endpoint)

Does not work with Google Cloud MemoryStore

GCP MemoryStore memcached also supports this discovery command: https://cloud.google.com/memorystore/docs/memcached/using-auto-discovery

However, when you run the STATS command you get back the following:

STAT version UNKNOWN

So this version check won't work:

dalli-elasticache/lib/dalli/elasticache/auto_discovery/endpoint.rb

Line 52 in 2224e4c

if engine_version < Gem::Version.new("1.4.14")

.

I hardcoded the check to default to the new version and after that it worked to discover the servers.

Instructions on how to use with Rails App?

Could you please provide some instructions on how to use this in a rails application?

installing this gem triggers large number of gem updates when doing bundle install

Would love to use this gem but it trigger a large number of gem updates when doing a bundle install. Briefly looked in repo to locate dependency that triggered so many updates but not much luck. Any way to lessen the (dependency) impact?

List of gems updated

Installing polyglot 0.3.5 (was 0.3.4)
Installing active_utils 2.2.3 (was 1.0.5)
Installing money 6.1.1 (was 5.1.1)
Installing activemerchant 1.43.3 (was 1.32.1)
Installing tzinfo 0.3.41 (was 0.3.39)
Installing addressable 2.3.6 (was 2.3.3)
Installing awesome_print 1.2.0 (was 1.1.0)
Installing safe_yaml 1.0.3
Installing crack 0.4.2 (was 0.3.2)
Installing netrc 0.7.7
Installing rest-client 1.7.2 (was 1.6.7)
Installing rmagick 2.13.3 (was 2.13.2)
Installing aws-sdk 1.51.0 (was 1.28.1)
Installing thread_safe 0.3.4
Installing descendants_tracker 0.0.4 (was 0.0.1)
Installing ice_nine 0.11.0
Installing axiom-types 0.1.1
Installing bcrypt 3.1.7
Installing coderay 1.1.0 (was 1.0.9)
Installing better_errors 1.1.0 (was 0.9.0)
Installing will_paginate 3.0.7 (was 3.0.4)
Installing bootstrap-will_paginate 0.0.10 (was 0.0.9)
Installing business_time 0.7.3 (was 0.6.1)
Installing net-ssh 2.9.1 (was 2.8.0)
Installing net-scp 1.2.1 (was 1.1.2)
Installing excon 0.39.5 (was 0.31.0)
Installing formatador 0.2.5 (was 0.2.4)
Installing fog-core 1.23.0
Installing fog-json 1.0.0
Installing inflecto 0.0.2
Installing fog-brightbox 0.3.0
Installing fog-softlayer 0.3.14
Installing ipaddress 0.8.0
Installing fog 1.23.0 (was 1.20.0)
Installing ffi 1.9.3 (was 1.9.0)
Installing childprocess 0.5.3 (was 0.3.9)
Installing chunky_png 1.3.1 (was 1.2.7)
Installing cocaine 0.5.4 (was 0.5.3)
Installing coercible 1.0.0
Installing coffee-script-source 1.7.1 (was 1.6.3)
Installing execjs 2.2.1 (was 1.4.0)
Installing coffee-script 2.3.0 (was 2.2.0)
Installing commonjs 0.2.7 (was 0.2.6)
Installing sass 3.2.19 (was 3.2.7)
Installing compass 0.12.7 (was 0.12.2)
Installing compass_twitter_bootstrap 2.3.1 (was 2.2.2.2)
Installing composite_primary_keys 5.0.14 (was 5.0.13)
Installing dalli 2.7.2 (was 2.6.2)
Installing dalli-elasticache 0.1.2
Installing debugger-ruby_core_source 1.3.5 (was 1.2.4)
Installing orm_adapter 0.5.0 (was 0.4.0)
Installing warden 1.2.3 (was 1.2.1)
Installing devise 3.3.0 (was 2.2.3)
Installing devise-encryptable 0.2.0 (was 0.1.1)
Installing docile 1.1.5
Installing email_preview 1.5.4 (was 1.5.3)
Installing equalizer 0.0.9
Installing geocoder 1.2.4 (was 1.1.6)
Installing multi_xml 0.5.5 (was 0.5.3)
Installing httparty 0.13.1 (was 0.10.2)
Installing hipchat 1.3.0 (was 0.8.0)
Installing holder_rails 2.3.2 (was 1.7.0)
Installing holidays 1.0.6 (was 1.0.5)
Installing htmlentities 4.3.2 (was 4.3.1)
Installing httpi 2.2.5 (was 2.1.0)
Installing rubyzip 1.1.6 (was 0.9.9)
Installing selenium-webdriver 2.42.0 (was 2.33.0)
Installing rainbow 2.0.0 (was 1.1.4)
Installing jquery-rails 3.1.1 (was 2.2.1)
Installing jquery-ui-rails 5.0.0 (was 4.0.2)
Installing kgio 2.9.2 (was 2.8.0)
Installing koala 1.10.0 (was 1.10.0rc2)
Installing less 2.5.1 (was 2.3.1)
Installing less-rails 2.5.0 (was 2.3.2)
Installing meta-tags 2.0.0 (was 1.5.0)
Installing method_source 0.8.2 (was 0.8.1)
Installing mixpanel 4.1.1 (was 3.5.2)
Installing mono_logger 1.1.0
Installing mysql2 0.3.16 (was 0.3.14)
Installing newrelic_rpm 3.9.1.236 (was 3.9.0.229)
Installing numbers_and_words 0.10.0 (was 0.9.0)
Installing slop 3.6.0 (was 3.4.4)
Installing pry 0.10.0 (was 0.9.12)
Installing rack-mini-profiler 0.9.2 (was 0.9.1)
Installing rack-protection 1.5.3 (was 1.4.0)
Installing raindrops 0.13.0 (was 0.10.0)
Installing redis 3.1.0 (was 3.0.3)
Installing redis-namespace 1.5.1 (was 1.2.1)
Installing sinatra 1.4.5 (was 1.3.6)
Installing resque 1.25.2 (was 1.23.1)
Installing resque-cleaner 0.3.0 (was 0.2.10)
Installing resque-meta 2.0.1 (was 2.0.0)
Installing rufus-scheduler 2.0.24 (was 2.0.18)
Installing resque-scheduler 3.0.0 (was 2.0.0)
Installing ruby-prof 0.15.1 (was 0.13.0)
Installing sassy-buttons 0.2.6 (was 0.1.4)
Installing savon 2.6.0 (was 2.5.1)
Installing simplecov-html 0.8.0 (was 0.7.1)
Installing simplecov 0.9.0 (was 0.7.1)
Installing spork 1.0.0rc4 (was 1.0.0rc3)
Installing spork-rails 4.0.0 (was 3.2.1)
Installing strong_parameters 0.2.3 (was 0.2.1)
Installing therubyracer 0.12.1 (was 0.12.0)
Installing timecop 0.7.1 (was 0.6.1)
Installing twitter-bootstrap-rails 2.2.8 (was 2.2.6)
Installing uglifier 2.5.3 (was 1.3.0)
Installing unf 0.1.4 (was 0.1.3)
Installing unicorn 4.8.3 (was 4.6.2)
Installing videojs_rails 4.6.1 (was 4.1.0)
Installing virtus 1.0.3 (was 0.5.5)

I think you broke my internet

I noticed that you just bumped the micro version of the gem. I also noticed that you have been checking in new code all the live-long day, whilst retaining the v0.1.0 file.

#servers is hanging, and it shouldn't be. Have you experienced this?

Fail silently

If the servers timeout then the application won't boot :(

Is there a way to fail silently?

Is it pure memcached compatible?

I'm testing this gem with memcached with this simple script

#!/usr/bin/env ruby                                                             

require 'dalli-elasticache'


endpoint    = "localhost:11211"
elasticache = Dalli::ElastiCache.new(endpoint)

#puts elasticache.public_methods

elasticache.set('test_key', 'test_value')

puts elasticache.get('test_key')                         

I got this...

vagrant@ror:~$ ./test.rb 
Traceback (most recent call last):
./test.rb:11:in `<main>': undefined method `set' for #<Dalli::ElastiCache:0x00005556b58a7990> (NoMethodError)
Did you mean?  send

I'm doing something wrong?

I'm using those versions:

dalli (3.2.0)
dalli-elasticache (0.2.0)

Thanks.