feathersjs-ecosystem / configuration Goto Github PK
View Code? Open in Web Editor NEW[MOVED] A plugin for configuring a Feathers application
Home Page: https://github.com/feathersjs/feathers
License: MIT License
[MOVED] A plugin for configuring a Feathers application
Home Page: https://github.com/feathersjs/feathers
License: MIT License
This is usually fine to be able to use environment variables in config files without the need of creating an additional file as with node-config : https://github.com/lorenwest/node-config/wiki/Environment-Variables#custom-environment-variables.
However it has a side effect that if you want to have a string value corresponding to the name of an environment variable you can't because you have no mean to prevent expansion. I found the specific case of a string value required to be TMP
but this corresponds to the TMP
environment variable, so expanded to my temporary directory path !
I would recommend to prefix environment variables with eg $
so that we have control over the expansion.
Would you accept a PR or be willing to add a index.d.ts
TypeScript declaration file for this lib? Even if the definitions are not complete yet, adding the following line to a declaration file would prevent a TS error "File '/path-to/feathers-configuration/index.d.ts' is not a module."
.
declare module 'feathers-configuration';
I added the auth config values to the config files, and for the local when I set the usernameField to use userName instead of email in run-time it replaces the usernameField value to my PC name. If I add anything else it works but not username.
local: {
entity: 'user',
usernameField: 'userName',
passwordField: 'password'
}
// becomes
local: {
entity: 'user',
usernameField: 'MyPcName',
passwordField: 'password'
}
Docs say:
require('feathers-configuration')()
I need to do
require('feathers-configuration')()()
I'd like to do
require('feathers-configuration')
just like in node-config.
In order to decouple us from Express we might want to namespace config options to app.config
instead of using app.set()
and app.get()
. The rationale:
.get
and .set
functionalityapp.get
for a route as well which is confusing.get
and .set
functionality.app.config
is a bit more explicit, especially for people that are new to Feathers/Expressapp.config
app.get
and app.set
to app.config.get
and app.config.set
except for when registering an Express get
route. This might mean implementing with the Buzzard release when the engine
adapters are more established.Related to #8. We've decided that for v2 of the generator and CLI we're going to use node-config and phase this out in it's current form.
https://github.com/lorenwest/node-config seems to be doing almost the same thing so it might make sense to use it here.
as per issue #24 we want to make this repo more focused on configuration management as a whole. When it comes to reading configs, we'll look to node-config for this rather then re-invent that functionality. We should also build out the repo to have support for more complex production configuration management storage such as vault, consul and etc so that we can not only protect secrets but also eventually support service discovery.
Configuration
classAdapter
classAdapters
should be able to read and writeAdapters
Adapters
Adapters
process.env
(default)Adapters
will be implemented with loading precedenceprocess.env.feathers
, json files, storage (vault, consul, etcd)Adapters
app.get
and app.set
feathers config get
and feathers config set
--storage
when managing a configurationprocess.env
and json filesconst feathers = require('feathers');
const configuration = require('feathers-configuration');
const etcd = require('feathers-etcd')(['http://127.0.0.1:2379']);
const vault = require('feathers-vault')(['http://127.0.0.1:8200', process.env.VAULT_TOKEN]);
feathers()
.configure(configuration()) // process.env.feathers + json files by default
.configure(etcd())
.configure(vault())
// listen to a specific storage adapter directly
vault.on('create', function(key, value) {
console.log(key, '=>', value); // /databases/mongo/production => mongodb://mongo.stack.local:27017
});
//listen across all storage adapters that support pub sub
configuration.on('create /databases/mongo/*', function(key, value) {
console.log(key, '=>', value); // /databases/mongo/production => mongodb://mongo.stack.local:27017
});
vault.create({
key:'/databases/mongo/production',
value: 'mongodb://mongo.stack.local.27107'
});
// 1. loads from process.env
// 2. loads from json files and merges
// 3. loads from etcd and merges
// 4. loads from vault and merges
// 5. listens to each storage
// 6. notifies of changes in storage
I see that NODE_ENV.json override is disabled only for development mode. Is there specific reason for that?
It's already checking for file existence, so it will be good if it just override configs from development.json if the file exists.
That will be helpful to have some configs local only by having default.json in version control and development.json ignored.
As others have mentioned in #8 we should look at using something like Vault or making a configuration service. This might be a way to re-purpose feathers-configuration. It should be more for managing config keys and secrets.
feathers-configuration could be a configuration management service that can store data wherever and just has hooks to encrypt/decrypt values. I think we have a couple options here:
Create a feathers-vault adapter. Vault can be used to read/write to many different backends like AWS IAM users, SQL DBs, Files System, etc. I would need to investigate further but @slajax might know more. Could be use like this:
const vault = require('feathers-vault');
app.use('config', vault(options));
Create our own hooks that encrypt and decrypt keys and allow you to CRUD them as well. It's possibly a little more feathersy. Could look something like this:
const service = require('feathers-mongodb');
const options = {
Model: db.collection('config')
};
app.use('/config', service(options));
app.service('config').hooks({
before: {
create: [encrypt()]
},
before: {
get: [decrypt()]
}
});
Thoughts @feathersjs/core-team?
We've been throwing a ton of stuff into our FeathersJS install and have outgrown purely static config files. Needed some flexibility and options for computed values so that derived config wasn't pushed up to the app initialization or higher layers.
I took a look at feathers-configuration and since it is using require()
to pull in the .json files, there's no reason I can think of not to support both json and node module exports.
I'm going to send over the PR. I don't see anything in these minor mods which breaks existing code; only expands the options. But let me know if I missed some detail.
Short summary of what I remember changing:
.json
requirement<env>
test file to be testing.js
instead of testing.json
(as shortest path to test both old functionality and new w/out creating more test data files).I'm writing feathers in typescript but it fails on configuration so it won't boot.
Also, I can't find any information about writing feathers in typescript or es6 so I think I'm on my own here. I just gonna follow the steps that I have for Express.
That the app boots
It fails to boot
just a normal laptop
Module versions (especially the part that's not working):
"debug": "^2.6.8",
"express": "^4.15.3",
"feathers": "^2.1.7",
**"feathers-configuration": "^0.4.1",**
"feathers-errors": "^2.9.1",
"feathers-hooks": "^2.0.2",
"feathers-hooks-common": "^3.7.2",
"feathers-knex": "^2.7.2",
"feathers-objection": "^0.6.0",
"feathers-rest": "^1.8.0",
"feathers-sequelize": "^2.3.0",
"feathers-socketio": "^2.0.0",
NodeJS version:
node 8.4.0
npm 5.3.0
yarn 0.27.5
Operating System:
windows 10
Module Loader:
import fs from 'fs';
import path from 'path';
import webpack from 'webpack';
import NodemonPlugin from 'nodemon-webpack-plugin';
import WebpackNotifierPlugin from 'webpack-notifier';
let nodeModules = {};
fs.readdirSync('node_modules')
.filter(function(x) {
return ['.bin'].indexOf(x) === -1;
})
.forEach(function(mod) {
nodeModules[mod] = 'commonjs ' + mod;
});
let webpackConfig = {
target: 'node',
entry: `./server/index.ts`,
output: {
path: (process.env.NODE_ENV === 'production') ? path.join(__dirname, 'dist') : path.join(__dirname, 'bin'),
libraryTarget: 'commonjs',
filename: 'server.js'
},
externals: nodeModules,
resolve: {
extensions: ['.ts']
},
module: {
rules: [
{ test: /\.json$/, use: 'json-loader' },
{ test: /\.tsx?$/, use: 'awesome-typescript-loader?configFileName=./tsconfig.server.json', exclude: /(node_modules|bower_components)/ }
]
},
plugins: [
new webpack.SourceMapDevToolPlugin(),
new NodemonPlugin()
]
};
export default webpackConfig;
I'm trying to setup my global app logger (winston), independant from the logging middleware.
Something like that:
logger.ts
import winston = require('winston');
import { app } from '../app';
const config = app.get('logger');
const transports = [];
if (config.console) {
transports.push(new winston.transports.Console({
colorize: true,
level: config.console.level
}));
}
export const logger = new winston.Logger({ transports });
Obviously I need the app config for this.
But on the other hand, to setup my logging middleware express-winston
, I need the logger:
app.ts
import path = require('path');
import favicon = require('serve-favicon');
import compress = require('compression');
import helmet = require('helmet');
import cors = require('cors');
import bodyParser = require('body-parser');
import feathers = require('feathers');
import configuration = require('feathers-configuration');
import hooks = require('feathers-hooks');
import rest = require('feathers-rest');
import expressWinston = require('express-winston');
import errorHandler = require('feathers-errors/handler');
import { logger } from './core/logger';
import { notFoundHandler } from './middleware/not-found-handler';
import { services } from './services';
export const app = feathers();
app.configure(configuration(path.join(__dirname, '..')));
app.use(compress())
.options('*', cors())
.use(cors())
.use(helmet())
.use(favicon( path.join(app.get('public'), 'favicon.ico') ))
.use(bodyParser.urlencoded({ extended: true }))
.use(bodyParser.json())
.use(expressWinston.logger({
winstonInstance: logger,
level: 'debug'
}))
.use('/', feathers.static(app.get('public')))
.configure(hooks())
.configure(rest())
.configure(services)
.use(expressWinston.errorLogger({
winstonInstance: logger
}))
.use(notFoundHandler())
.use(errorHandler() as any);
And boom, app.ts
needs logger and logger.ts
needs app for config. How to get out of this?
I never had similar issues with other frameworks, since usually the config is decoupled from everything else. Isn't this a major issue with your current approach?
Having the config exposed in the app is nice, but I think also having access to the config independently is also needed, for case like this, or maybe I'm missing something?
My intuition is to have default.json for running locally w/o setting any NODE_ENV.
And to run on DEV environment, we need to do NODE_ENV=development which would use the config/development.json
instead.
Hi,
It is said that feathers-configuration
is a simple wrapper on node-config. However, with node-config, if you have nested configuration, like that:
{
"Customer": {
"dbConfig": {
"host": "prod-db-server"
},
"credit": {
"initialDays": 30
}
}
}
you can access the host
attribute by using config.get('Customer.dbConfig.host')
. However, with feathers-configuration
, this dot notation do not work.
Issue related to: feathersjs-ecosystem/authentication#191.
the problem is that if you have a default.json
with
{ "facebook": { "secret": "TESTSECRET", "scope": ["email"] } }
and then another environment configuration (production.json
) with
{ "facebook": { "secret": "SECRET_FROM_ENV" } }
It blows away the scope
instead of merging it
There needs to be an escape character (most likely \
) if you do not want to convert a configuration value. E.g.
{
"env": "\\NODE_ENV",
"path": "\\./home"
}
Should set env
to the actual string NODE_ENV
and path
to ./home
Something like this doesn't work:
{...
"auth" : {
github: {
"clientID": "ENV_VARIABLE_NAME"
...
}
...
}
...
}
...
}
I think it is usefull to know which enviroment is loaded.
Example for doing something in default or development mode. (not using directly env vars in node).
@daffl what do you think about?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
π Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. πππ
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google β€οΈ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.