ember-cli-deploy / ember-cli-deploy-redis Goto Github PK
View Code? Open in Web Editor NEWAn ember-cli-deploy-plugin to upload index.html to a Redis store
License: MIT License
An ember-cli-deploy-plugin to upload index.html to a Redis store
License: MIT License
- Value already exists for key: app-name:index:default
Value already exists for key: app-name:index:default
undefinedPipeline aborted
It's my first deploy of the app
"ember-cli-deploy": "0.5.1",
"ember-cli-deploy-build": "0.1.0",
"ember-cli-deploy-manifest": "0.1.0",
"ember-cli-deploy-redis": "0.1.0",
"ember-cli-deploy-s3": "0.1.0",
TypeError: this._listRevisions(...).then is not a function
TypeError: this._listRevisions(...).then is not a function
at CoreObject.fetchRevisions (/Users/lmelia/p/yapp/dashboard/node_modules/ember-cli-deploy-redis/lib/redis.js:75:43)
at CoreObject.fetchInitialRevisions (/Users/lmelia/p/yapp/dashboard/node_modules/ember-cli-deploy-redis/index.js:149:47)
Backstory and investigation for this is in ember-cli-deploy/ember-cli-deploy/issues/327 . Specifically in this comment
It looks like IORedis
should cover all the cases that we need for this plugin
we need a champion willing to attempt a port to IORedis
(see comment from @blimmer below)
Hi Folks,
We're running into a scenario where our Redis store reaches its memory capacity. When that happens, ember deploy fails because there is no room to deploy the index.html to our Redis instance.
We're using RedisToGo, so one approach may be to set our maxmemory-policy
to allkeys-lru
since that policy doesn't require keys to have expire set.
CONFIG SET maxmemory-policy allkeys-lru
Unfortunately, RedisToGo seems to have obfuscated the CONFIG
command (for security reasons no doubt) and its default maxmemory-policy
is volatile-lru
which requires keys to have an expire set in order to make space for any new data added once max memory is met.
Without access to CONFIG
our only option is to set an expire on our ember cli deployment keys. It doesn't appear that ember-cli-deploy-redis sets an expire by default. Is there a configuration option or some other approach we could take to set an expire during ember cli deploy?
Thanks Everyone!
Heroku redis addon url's generally have a placeholder for the username, ie
redis://x:345932ywwe659weotwy@some-host:12345
That x
in that url is pointless and breaks the redis package we use. The required url is:
redis://:345932ywwe659weotwy@some-host:12345
We should make this plugin handle the url if it includes the username placeholder.
I have run into an issue with compatibility of using the lightning deploy strategy and serving a service worker. the sw.js file needs to be served up from the root domain and not from a CDN url so I'm wondering if it's possible to deploy both the index.html and sw.js files to redis where I can then adapt my server to also dish out the service worker from the main domain?
so far I've tried updating the filePattern but it feels like this is set to locate 1 file to upload and doesn't support many files... but I'm likely missing something so thought I'd check here. TIA for the help!
Redis is running on a server that is not publicly accessible (it's in a private subnet) so I'm using ssh tunnel to connect from a public facing server that is in the same VPC.
If I ssh into that public facing server, I can connect to the redis instance on the private server just fine using redis-cli. But I get I'm getting timeout errors when I try to deploy to production using the following config:
'redis': {
allowOverwrite: true,
keyPrefix: 'pigeon:index',
url: 'redis://:' + env('REDIS_PASSWORD') + '@' + env('REDIS_HOST') + ':6379',
},
'ssh-tunnel': {
username: env('SSH_USER'),
host: env('SSH_HOST'),
privateKeyPath: env('SSH_KEY'),
}
I'm guessing this is some sort of configuration issue on my end since I don't see any other issues about it. But I get this same error using the host, password, port configuration variation rather than specifying a connection url:
Error: Redis connection to internal-pigeon-db-production-1812613668.us-east-1.elb.amazonaws.com:6379 failed - connect ETIMEDOUT 192.168.1.148:6379
at RedisClient.on_error (/Applications/MAMP/htdocs/pigeon-web-app/node_modules/ember-cli-deploy-lightning-pack/node_modules/ember-cli-deploy-redis/node_modules/redis/index.js:196:24)
at Socket.<anonymous> (/Applications/MAMP/htdocs/pigeon-web-app/node_modules/ember-cli-deploy-lightning-pack/node_modules/ember-cli-deploy-redis/node_modules/redis/index.js:106:14)
at emitOne (events.js:77:13)
at Socket.emit (events.js:169:7)
at emitErrorNT (net.js:1269:8)
at nextTickCallbackWith2Args (node.js:442:9)
at process._tickCallback (node.js:356:17)
Any ideas about what might be causing this?
Edit: Explained how I'm using ssh tunnel.
Hi,
From the description of the plugin, This plugin uploads a file, presumably index.html, to a specified Redis store.
, it seems that uploading multiple files is not supported. In my use-case, besides an index.html
, I would also like to upload a assets/print.css
to Redis. Will such a use-case ever be supported by this plugin?
Thank you,
Yoran
Prember pre-renders routes into a static directory structure with index.html files at build time. I'd like to be able to use the lightning strategy for deployment and pre-render my static routes. This can be supported by grabbing the files from s3 as assets, but you lose the ability to effectively use revisions, uploading the html as assets using the s3 plugin means the index.html files will be overwritten as they are not fingerprinted. I would like to upload all the html into redis just as it stores the main ember index.html file to retain the ability to have revisions.
I've noticed in the past a PR was rejected, as a use case didn't really exist at the time and the suggestion of using the plugin multiple times as part of the pipeline was the correct advice. The past issues and pr were about uploading a single other file. An app may have tens or hundreds of static routes being prerended by prember so creating a plugin entry for each html file would be cumbersome, prone to error and not very scalable.
I'm creating this issue to spark at the least a discussion about if prembers rising popularity changes the conversation around supporting multiple file support and if it should be pursued as part of this plugin.
I've specified the 'url' key in my redis config object, but it does not seem to be taken in to account and deploy tries to use default host and port.
Using [email protected]
works find if I configure 'host', 'port' and 'password' instead
hi I would like to see something like pub/sub when deploying changes, so when a didDeploy
event happens this add-on can exec a PUBLISH ${keyPrefix}:deploy ${revisionKey}
and a fastboot-notifier can be listen on that channel events and trigger a notification to download the new app version. let me know if that sound good to you and if I can help implementing this. thanks
Hello guys and merry christmas ๐ ,
Without any version changes or any altered setup in my laptop I noticed $ ember deploy
now throws an exception:
+- didFail
TypeError: this.readConfig is not a function
TypeError: this.readConfig is not a function
at Object.redisDeployClient (/Users/mariogintili/dev/sunlight/web-client/node_modules/ember-cli-deploy-redis/index.js:55:36)
at CoreObject.readConfig (/Users/mariogintili/dev/sunlight/web-client/node_modules/ember-cli-deploy-plugin/index.js:63:30)
at CoreObject.fetchInitialRevisions (/Users/mariogintili/dev/sunlight/web-client/node_modules/ember-cli-deploy-redis/index.js:137:38)
at Object.fn (/Users/mariogintili/dev/sunlight/web-client/node_modules/ember-cli-deploy/lib/tasks/pipeline.js:93:21)
at Pipeline._notifyPipelinePluginHookExecution (/Users/mariogintili/dev/sunlight/web-client/node_modules/ember-cli-deploy/lib/models/pipeline.js:175:19)
at tryCatch (/Users/mariogintili/dev/sunlight/web-client/node_modules/rsvp/dist/rsvp.js:538:12)
at invokeCallback (/Users/mariogintili/dev/sunlight/web-client/node_modules/rsvp/dist/rsvp.js:553:13)
at /Users/mariogintili/dev/sunlight/web-client/node_modules/rsvp/dist/rsvp.js:628:16
at flush (/Users/mariogintili/dev/sunlight/web-client/node_modules/rsvp/dist/rsvp.js:2373:5)
at _combinedTickCallback (internal/process/next_tick.js:67:7)
|
Pipeline aborted
My package.json is as follows
"ember-cli-deploy": "1.0.0-beta.1",
"ember-cli-deploy-revision-data": "0.3.2",
"ember-cli-deploy-display-revisions": "0.2.2",
"ember-cli-deploy-lightning-pack": "0.6.7",
"ember-cli-deploy-ssh-tunnel": "0.2.2",
I tried upgrading the ember-cli-deploy-lighting-pack
to the latest release to date - 0.6.10
which now yields the following:
+- prepare
| |
| +- revision-data
- creating revision data using `file-hash`
- generated revision data for revision: `997780b824c374fc5ce269e95918b032`
|
+- didPrepare
|
+- fetchInitialRevisions
| |
| +- redis
- Listing initial revisions for key: `sunlight-web-client`
events.js:160
throw er; // Unhandled 'error' event
^
Error: Timed out while waiting for handshake
at Timeout._onTimeout (/Users/mariogintili/dev/sunlight/web-client/node_modules/ssh2/lib/client.js:569:19)
at ontimeout (timers.js:365:14)
at tryOnTimeout (timers.js:237:5)
at Timer.listOnTimeout (timers.js:207:5)
I can confirm no SSH keys in my setup have changed(both on my end and on our servers) and that I have a more than stable internet connection - any help is greatly appreciated. Some additional info:
ember --version
ember-cli: 2.7.0
node: 6.9.1
os: darwin x64
TypeError: this.readConfig is not a function
is getting thrown when deploying.
TypeError: this.readConfig is not a function
TypeError: this.readConfig is not a function
at Object.DeployPluginBase.extend.defaultConfig.redisDeployClient (/Users/andres/Developer/take2mfg/take2-shop/node_modules/ember-cli-deploy-lightning-pack/node_modules/ember-cli-deploy-redis/index.js:55:36)
at CoreObject.extend.readConfig (/Users/andres/Developer/take2mfg/take2-shop/node_modules/ember-cli-deploy-lightning-pack/node_modules/ember-cli-deploy-redis/node_modules/ember-cli-deploy-plugin/index.js:59:30)
at CoreObject.DeployPluginBase.extend.fetchInitialRevisions (/Users/andres/Developer/take2mfg/take2-shop/node_modules/ember-cli-deploy-lightning-pack/node_modules/ember-cli-deploy-redis/index.js:137:38)
at Object._pipeline.register.fn (/Users/andres/Developer/take2mfg/take2-shop/node_modules/ember-cli-deploy/lib/tasks/pipeline.js:93:21)
at Pipeline._notifyPipelinePluginHookExecution (/Users/andres/Developer/take2mfg/take2-shop/node_modules/ember-cli-deploy/lib/models/pipeline.js:175:19)
at tryCatch (/Users/andres/Developer/take2mfg/take2-shop/node_modules/ember-cli/node_modules/rsvp/dist/rsvp.js:538:12)
at invokeCallback (/Users/andres/Developer/take2mfg/take2-shop/node_modules/ember-cli/node_modules/rsvp/dist/rsvp.js:553:13)
at /Users/andres/Developer/take2mfg/take2-shop/node_modules/ember-cli/node_modules/rsvp/dist/rsvp.js:628:16
at flush (/Users/andres/Developer/take2mfg/take2-shop/node_modules/ember-cli/node_modules/rsvp/dist/rsvp.js:2373:5)
Appears to be related to this change ember-cli-deploy/ember-cli-deploy-plugin#15
Running a js app inside a container,
In deploy.js
I have the following simple config
ENV.redis = {
url: process.env['REDIS_URL']
}
The url in the env is REDIS_URL=redis://redis:6379/0
Apparently it will fail to parse as it will still try to connect to localhost (the default host).
To work around it I need to do the parsing myself:
const { URL } = require('url');
const redisURL = new URL(process.env['REDIS_URL'])
ENV.redis = {
host: redisURL.hostname,
port: redisURL.port,
username: redisURL.username,
password: redisURL.password
};
It will be ideal to have the parsing done by this library.
I've been attempting to pass my own redisClient
, so I could use sentinel-aware ioredis
. It looks like this won't work, and the injection is only for testing, since redisClient
needs decorations provided by this repo (upload
, activate
, etc) and it doesn't seem possible to use an alternate redis lib (since it relies on a particular method signature). Am I understanding this correctly? Anything I can do here?
Hello peeps,
I'm working on a project that makes use of this library in our CI. We basically trigger a deployment every time a build passes successfully.
Given the current design of this project - and its defaults - retrying a successful build yields an error since the deploy script will output the same manifest, causing the following:
| - Value already exists for key: zapnito-web:index:288d60899ac3cc175fcd2bbb758dc3de
|
+- didFail
Value already exists for key: zapnito-web:index:288d60899ac3cc175fcd2bbb758dc3de
undefined|
Pipeline aborted
I don't think retrying a successful build - where the code hasn't changed - should yield an error. Could we please change the behaviour on this to be a warning rather than an error?
How about sample nginx conf and some tutorial on how to set up it?
After upgrading from an older version of ember-deploy-redis I found that ember-cli-deploy-redis does not store the key prefix in the current key.
For example, assuming your project name is project-awesome
, the legacy version would store the following in the project-awesome:current
key:
project-awesome:abcdefghijklmnopqrstuvwxyz1234
The current version of ember-cli-deploy-redis stores the following in the project-awesome:current
key:
abcdefghijklmnopqrstuvwxyz1234
Is this the intended behavior?
It's not a huge deal, but I did have to implement a workaround in my rails app for this, since some of my ember apps still use the legacy redis deploy and have yet to be updated.
Instead of this:
index_key = $redis.get('project-awesome:current')
content = $redis.get(index_key)
in order to maintain compatibility I need this:
index_key = fix_redis_key('project-awesome', $redis.get('project-awesome:current'))
content = $redis.get(index_key)
private
def fix_redis_key(project_name, key)
return key if key.start_with?(project_name)
"#{project_name}:#{key}"
end
Hi,
with these plugin installed:
"ember-cli-deploy": "0.5.0-beta.2",
"ember-cli-deploy-lightning-pack": "0.2.0-beta.5",
whenever the pipeline contacts redis on Heroku I got ERR invalid DB index
. example on activation:
|
+- activate
| |
| +- redis
| - Activating revision `84549e8ecccb5d3ae31c1583f125d2dc`
Unhandled rejection Error: ERR invalid DB index
at ReplyParser.<anonymous> (/Users/masciugo/dev/quenta/node_modules/ember-cli-deploy-lightning-pack/node_modules/ember-cli-deploy-redis/node_modules/redis/index.js:317:31)
at ReplyParser.emit (events.js:107:17)
at ReplyParser.send_error (/Users/masciugo/dev/quenta/node_modules/ember-cli-deploy-lightning-pack/node_modules/ember-cli-deploy-redis/node_modules/redis/lib/parser/javascript.js:296:10)
at ReplyParser.execute (/Users/masciugo/dev/quenta/node_modules/ember-cli-deploy-lightning-pack/node_modules/ember-cli-deploy-redis/node_modules/redis/lib/parser/javascript.js:181:22)
at RedisClient.on_data (/Users/masciugo/dev/quenta/node_modules/ember-cli-deploy-lightning-pack/node_modules/ember-cli-deploy-redis/node_modules/redis/index.js:547:27)
at Socket.<anonymous> (/Users/masciugo/dev/quenta/node_modules/ember-cli-deploy-lightning-pack/node_modules/ember-cli-deploy-redis/node_modules/redis/index.js:102:14)
at Socket.emit (events.js:107:17)
at readableAddChunk (_stream_readable.js:163:16)
at Socket.Readable.push (_stream_readable.js:126:10)
at TCP.onread (net.js:538:20)
| - โ Activated revision `84549e8ecccb5d3ae31c1583f125d2dc`
|
+- didActivate
|
anything to do with this issue? It doesn't seem to have any effects on the expected outcome, though.
I might have misunderstood the way dev deploy worked, but when I 'ember deploy dev', the redis key 'app:index:current' returns nil and I only have 1 manifest_id that seems correct and it is 'default'.
wouldn't it be nice to have 'app:index:current' return 'default' ?
EDIT
using [email protected]
Similar to #25, it would be fantastic if ember-deploy-redis would also stash a checksum (or the commit SHA) and/or a build date/time (or the commit timestamp) of the index to use in the the ETag
and Last-Modified
HTTP headers. Something like:
foo:index:current => '<!DOCTYPE html>...'
foo:index:current:meta => checksum '787b3dd284fd99202adfa4e18a28126d159552c5'
timestamp 'Wed Sep 16 19:28:37 2015 +0200'
And then...
require 'sinatra'
require 'redis'
get '/' do
content_type 'text/html'
expires 3_600, :public
redis = Redis.new
index_key = redis.get("foo:current")
index_key = "foo:#{params[:index_key]}" if params[:index_key]
meta = redis.hgetall("#{index_key}:meta")
etag meta[:checksum]
last_modified meta[:timestamp]
redis.get(index_key)
end
Stashing the content in the same hash as the metadata would be ideal (to limit the Redis hits) but would break backwards compatibility.
When using yarn to manage packages it throws the following warning:
warning "[email protected]" has unmet peer dependency "redis-commands@*".
and the then-redis
maintainer has announced he is no longer maintaining then-redis
and that users should switch to node_redis
which now contains promise goodness.
So, is it possible to drop then-redis
and replace it with node_redis
?
I'm fairly new to the project, and didn't realize I need to prefix a URL option with "redis://". I know, it's a bonehead move... Any ways... The error you get is:
cleaning up...
Deploying [==>------] 33% [plugin: redis -> fetchInitialRevisions]events.js:183
throw er; // Unhandled 'error' event
^
Error: Redis connection to 127.0.0.1:6379 failed - connect ECONNREFUSED 127.0.0.1:6379
at Object._errnoException (util.js:1022:11)
at _exceptionWithHostPort (util.js:1044:20)
at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1182:14)
I think we can do a bit better than this. Perhaps checking for this prefix is advisable, or least offering a suggestion to look for this when the failure happens? I'd be more than happy to help if such a solution is desirable.
It's confusing that the plugin uses the prefix as the key name for the revisions list. One expects a "prefix" to be before something. Let's change it to prefix + ":revisions"
As well as putting the latest revision on the set of revisions, it would be great to store the json blob that ember-cli-deploy-revision-data
creates.
We would also need to remove the blob when the corresponding revision is removed from the list
My Redis config contains the following:
redis: {
allowOverwrite: true,
keyPrefix: 'zatresi',
maxRecentUploads: 100
},
When I run ember deploy prod
I get the following output:
- Deployed but did not activate revision eb65b18c84b6b6b373fa5315e6661a9b. To activate, run: ember deploy:activate prod --revision=eb65b18c84b6b6b373fa5315e6661a9b
In Redis, the key zatresi:eb65b18c84b6b6b373fa5315e6661a9b
indeed contains the HTML content of the index.html
file.
Next, I run:
ember deploy:activate prod --revision=eb65b18c84b6b6b373fa5315e6661a9b
This sets the value of they key zatresi:current
to eb65b18c84b6b6b373fa5315e6661a9b
instead of zatresi:eb65b18c84b6b6b373fa5315e6661a9b
. I believe this is a regression from a much earlier version of the addon. At some point, the behavior changed.
The documentation says:
The unique revision number for the version of the file being uploaded to Redis. The Redis key will be a combination of the keyPrefix and the revisionKey. By default this option will use either the revisionKey passed in from the command line or the revisionData.revisionKey property from the deployment context.
Default:
context.commandLineArgs.revisionKey || context.revisionData.revisionKey
I believe the default should instead be: keyPrefix + context.commandLineArgs.revisionKey || context.revisionData.revisionKey
Also, from this documentation it is not clear how I can override the revisionKey
parameter. Can I pass a function which accepts the keyPrefix
and/or context
to it?
Today, this package is using then-redis
:
ember-cli-deploy-redis/package.json
Line 39 in b299ad5
However, then-redis
is no longer supported.
I noticed this because of this warning when I yarn install
warning "ember-cli-deploy-redis > [email protected]" has unmet peer dependency "redis-commands@*".
This package should be migrated to use async behavior in node_redis
.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.