Giter Site home page Giter Site logo

nikaple / nest-typed-config Goto Github PK

View Code? Open in Web Editor NEW
183.0 3.0 25.0 1.92 MB

Intuitive, type-safe configuration module for Nest framework ✨

License: MIT License

JavaScript 1.19% Shell 0.28% TypeScript 98.52%
nestjs typescript configuration config configuration-management javascript nest type-safety dotenv type

nest-typed-config's Introduction

Nest-Typed-Config

Never write strings to read config again.

NPM Version Package License NPM Downloads build Coverage

Features

  • Provide a type-safe and intuitive way to config your Nest projects, just as smooth as request DTO.
  • Load your configuration with environment variables, json/yaml/toml configuration files or remote endpoints.
  • Validate your configuration with class-validator and class-transformer.
  • Provide easy to use options by default, meanwhile everything is customizable.

Installation

$ npm i --save nest-typed-config

Nest-typed-config will install the dependencies for all loaders by default. If you care about dependency size and bootstrap time, please checkout the guide to skip optional dependencies.

Inspiration

There are various popular configuration modules for Nest framework, such as the official configuration module, nestjs-config and nestjs-easyconfig. These modules can help to manage configurations, validate them, and load them through the ConfigService. But that's when type-safety is gone. For example:

// @nestjs/config, with type-casting
const dbUser = this.configService.get<string>('DATABASE_USER');
// nestjs-config, returns `any` type
const env = this.config.get('app.environment');
// nestjs-easyconfig, only string is supported
const value = this.config.get('key');

Writing type casting is a pain and hard to maintain, and it's common to use non-string configurations in real-world projects. This module aims to provide an intuitive and type-safe way to load, validate and use configurations. Just import any config model, and inject it with full TypeScript support. In a nutshell:

// config.ts
export class Config {
  @IsString()
  public readonly host!: string;

  @IsNumber()
  public readonly port!: number;
}

// app.service.ts
import { Config } from './config';

@Injectable()
export class AppService {
  constructor(private readonly config: Config) {}

  show() {
    console.log(`http://${this.config.host}:${this.config.port}`);
  }
}

Quick Start

Let's define the configuration model first. It can be nested at arbitrary depth.

// config.ts
import { Allow, ValidateNested } from 'class-validator';

// validator is omitted for simplicity
export class TableConfig {
  @Allow()
  public readonly name!: string;
}

export class DatabaseConfig {
  @Type(() => TableConfig)
  @ValidateNested()
  public readonly table!: TableConfig;
}

export class RootConfig {
  @Type(() => DatabaseConfig)
  @ValidateNested()
  public readonly database!: DatabaseConfig;
}

Then, add a configuration file such as .env.yaml under project root directory:

database:
  table:
    name: example

After creating the configuration file, import TypedConfigModule and fileLoader to load configuration from file system.

// app.module.ts
import { Module } from '@nestjs/common';
import { TypedConfigModule, fileLoader } from 'nest-typed-config';
import { AppController } from './app.controller';
import { AppService } from './app.service';
import { RootConfig } from './config';

// Register TypedConfigModule
@Module({
  imports: [
    TypedConfigModule.forRoot({
      schema: RootConfig,
      load: fileLoader(),
    }),
  ],
  providers: [AppService],
  controllers: [AppController],
})
export class AppModule {}

That's it! You can use any config or sub-config you defined as injectable services now!

// app.service.ts
import { Injectable } from '@nestjs/common';
import { RootConfig, DatabaseConfig, TableConfig } from './config';

@Injectable()
export class AppService {
  // inject any config or sub-config you like
  constructor(
    private config: RootConfig,
    private databaseConfig: DatabaseConfig,
    private tableConfig: TableConfig,
  ) {}

  // enjoy type safety!
  public show(): any {
    const out = [
      `root.name: ${this.config.name}`,
      `root.database.name: ${this.databaseConfig.name}`,
      `root.database.table.name: ${this.tableConfig.name}`,
    ].join('\n');

    return `${out}\n`;
  }
}

For a full example, please visit CodeSandbox, or our examples folder.

Using loaders

Using dotenv loader

The dotenvLoader function allows you to load configuration with dotenv, which is similar to the official configuration module. You can use this loader to load configuration from .env files or environment variables.

Example

TypedConfigModule.forRoot({
  schema: RootConfig,
  load: dotenvLoader({
    /* options */
  }),
});

Passing options

The dotenvLoader function optionally expects a DotenvLoaderOptions object as a first parameter:

export interface DotenvLoaderOptions {
  /**
   * If set, use the separator to parse environment variables to objects.
   *
   * @example
   *
   * ```bash
   * app__port=8080
   * db__host=127.0.0.1
   * db__port=3000
   * ```
   *
   * if `separator` is set to `__`, environment variables above will be parsed as:
   *
   * ```json
   * {
   *     "app": {
   *         "port": 8080
   *     },
   *     "db": {
   *         "host": "127.0.0.1",
   *         "port": 3000
   *     }
   * }
   * ```
   */
  separator?: string;

  /**
   * If set, this function will transform all environment variable keys prior to parsing.
   *
   * Be aware: If you transform multiple keys to the same value only one will remain!
   *
   * @example
   *
   * .env file: `PORT=8080` and `keyTransformer: key => key.toLowerCase()` results in `{"port": 8080}`
   *
   * @param key environment variable key
   */
  keyTransformer?: (key: string) => string;

  /**
   * If "true", environment files (`.env`) will be ignored.
   */
  ignoreEnvFile?: boolean;

  /**
   * If "true", predefined environment variables will not be validated.
   */
  ignoreEnvVars?: boolean;

  /**
   * Path to the environment file(s) to be loaded.
   */
  envFilePath?: string | string[];

  /**
   * A boolean value indicating the use of expanded variables.
   * If .env contains expanded variables, they'll only be parsed if
   * this property is set to true.
   *
   * Internally, dotenv-expand is used to expand variables.
   */
  expandVariables?: boolean;
}

Using file loader

The fileLoader function allows you to load configuration with cosmiconfig. You can use this loader to load configuration from files with various extensions, such as .json, .yaml, .toml or .js.

By default, fileLoader searches for .env.{ext} (ext = json, yaml, toml, js) configuration file starting at process.cwd(), and continues to search up the directory tree until it finds some acceptable configuration (or hits the home directory). Moreover, configuration of current environment takes precedence over general configuration (.env.development.toml is loaded instead of .env.toml when NODE_ENV=development)

Example

TypedConfigModule.forRoot({
  schema: RootConfig,
  load: fileLoader({
    /* options */
  }),
});

Passing options

The fileLoader function optionally expects a FileLoaderOptions object as a first parameter:

import { OptionsSync } from 'cosmiconfig';

export interface FileLoaderOptions extends Partial<OptionsSync> {
  /**
   * basename of config file, defaults to `.env`.
   *
   * In other words, `.env.yaml`, `.env.yml`, `.env.json`, `.env.toml`, `.env.js`
   * will be searched by default.
   */
  basename?: string;
  /**
   * Use given file directly, instead of recursively searching in directory tree.
   */
  absolutePath?: string;
  /**
   * The directory to search from, defaults to `process.cwd()`. See: https://github.com/davidtheclark/cosmiconfig#explorersearch
   */
  searchFrom?: string;
  /**
   * If "true", ignore environment variable substitution.
   * Default: true
   */
  ignoreEnvironmentVariableSubstitution?: boolean;
}

If you want to add support for other extensions, you can use loaders property provided by cosmiconfig:

TypedConfigModule.forRoot({
  schema: RootConfig,
  load: fileLoader({
    // .env.ini has the highest priority now
    loaders: {
      '.ini': iniLoader,
    },
  }),
});

Using directory loader

The directoryLoader function allows you to load configuration within a given directory.

The basename of files will be interpreted as config namespace, for example:

.
└─config
    ├── app.toml
    └── db.toml

// app.toml
foo = 1

// db.toml
bar = 1

The folder above will generate configuration as follows:

{
  "app": {
    "foo": 1
  },
  "db": {
    "bar": 1
  }
}

Example

TypedConfigModule.forRoot({
  schema: RootConfig,
  load: directoryLoader({
    directory: '/absolute/path/to/config/directory',
    /* other cosmiconfig options */
  }),
});

Passing options

The directoryLoader function optionally expects a DirectoryLoaderOptions object as a first parameter:

import { OptionsSync } from 'cosmiconfig';

export interface DirectoryLoaderOptions extends OptionsSync {
  /**
   * The directory containing all configuration files.
   */
  directory: string;
  /**
   * File regex to include.
   */
  include?: RegExp;
  /**
   * If "true", ignore environment variable substitution.
   * Default: true
   */
  ignoreEnvironmentVariableSubstitution?: boolean;
  /**
   * If "true", disallow undefined environment variables.
   * Default: true
   */
  disallowUndefinedEnvironmentVariables?: boolean;
}

If you want to add support for other extensions, you can use loaders property provided by cosmiconfig:

TypedConfigModule.forRoot({
  schema: RootConfig,
  load: directoryLoader({
    directory: '/path/to/configuration',
    // .env.ini has the highest priority now
    loaders: {
      '.ini': iniLoader,
    },
  }),
});

Using remote loader

The remoteLoader function allows you to load configuration from a remote endpoint, such as configuration center. Internally @nestjs/axios is used to perform http requests.

Example

// forRootAsync should be used when loading configuration asynchronously
TypedConfigModule.forRootAsync({
  schema: RootConfig,
  load: remoteLoader('http://localhost:8080', {
    /* options */
  }),
});

Passing options

The remoteLoader function optionally expects a RemoteLoaderOptions object as a second parameter, which accepts all axios request configuration except url.

export interface RemoteLoaderOptions extends AxiosRequestConfigWithoutUrl {
  /**
   * Config file type
   */
  type?: ((response: any) => RemoteLoaderConfigType) | RemoteLoaderConfigType;

  /**
   * A function that maps http response body to corresponding config object
   */
  mapResponse?: (config: any) => Promise<any> | any;

  /**
   * A function that determines if the request should be retried
   */
  shouldRetry?: (response: AxiosResponse) => boolean;

  /**
   * Number of retries to perform, defaults to 3
   */
  retries?: number;

  /**
   * Interval in milliseconds between each retry
   */
  retryInterval?: number;
}

You can use the mapResponse function to preprocess the server response before parsing with type, and use shouldRetry function to determine whether server response is valid or not. When server response is not valid, you can use retries and retryInterval to adjust retry strategies. For example:

/*
  Example server response:
  {
    "code": 0,
    "fileName": ".env.yaml",
    "fileType": "yaml",
    "fileContent": "database:\n    table:\n        name: example"
  }
*/

TypedConfigModule.forRootAsync({
    schema: RootConfig,
    load: remoteLoader('http://localhost:8080', {
        type: response => response.fileType,
        mapResponse: response => response.fileContent
        // retry when http status is not 200, or response code is not zero
        shouldRetry: response => response.data.code !== 0
        retries: 3,
        retryInterval: 3000
    }),
})

Using multiple loaders

Loading configuration from file system is convenient for development, but when it comes to deployment, you may need to load configuration from environment variables, especially in a dockerized environment. This can be easily achieved by providing multiple loaders. For example:

TypedConfigModule.forRoot({
  schema: RootConfig,
  // Loaders having larger index take precedence over smaller ones,
  // make sure dotenvLoader comes after fileLoader ensures that
  // environment variables always have the highest priority
  load: [
    fileLoader({
      /* options */
    }),
    dotenvLoader({
      /* options */
    }),
  ],
});

Using custom loader

If native loaders provided by nest-typed-config can't meet your needs, you can implement a custom loader. This can be achieved by providing a function which returns the configuration object synchronously or asynchronously through the load option. For example:

TypedConfigModule.forRootAsync({
  schema: RootConfig,
  load: async () => {
    return {
      host: '127.0.0.1',
      port: 3000,
    };
  },
});

Uses of variable substitutions

The ${PORT} substitution feature lets you use environment variable in some nice ways. You can also provide default values, and reference another variable in a config

If you have config file with like the below one

database:
  host: 127.0.0.1
  port: ${PORT:-12345}
  url: ${database.host}:${database:port}

And you have set environment variable for port

PORT=9000

And set ignoreEnvironmentVariableSubstitution to false in the FileLoaderOptions

load: fileLoader({
  ignoreEnvironmentVariableSubstitution: false,
}),

then fileloader will resolve ${PORT} placeholder and replace with environment variable. And you will get new config like below one

database:
  host: 127.0.0.1
  port: 9000
  url: 127.0.0.1:9000

if you won't set environment variable for port, then you will get new config like below one

database:
  host: 127.0.0.1
  port: 12345
  url: 127.0.0.1:12345

Note

when you use variable substitution the values can be string in case if you use default variable or env variable, and you need to apply transformer to class fields to get the correct type of the value.

export class Config {
  @IsString()
  public readonly host!: string;

  @IsNumber()
  @Type(() => Number)
  public readonly port!: number;

  @IsString()
  public readonly url!: string;
}

Default values

Just define your default values in config schema, and you are ready to go:

// config.ts
export class Config {
  @IsString()
  public readonly host: string = '127.0.0.1';

  @IsNumber()
  public readonly port: number = 3000;
}

Transforming the raw configuration

Environment variables are always loaded as strings, but configuration schemas are not. In such case, you can transform the raw config with normalize function:

// config.ts
export class Config {
  @IsString()
  public readonly host: string;

  @IsNumber()
  public readonly port: number;
}

// app.module.ts
TypedConfigModule.forRoot({
  schema: RootConfig,
  load: dotenvLoader(),
  normalize(config) {
    config.port = parseInt(config.port, 10);
    return config;
  },
});

Custom getters

You can define custom getters on config schema to extract common logic:

export class Config {
  @IsString()
  public readonly host: string = '127.0.0.1';

  @IsNumber()
  public readonly port: number = 3000;

  @IsString()
  public get url(): string {
    return `http://${this.host}:${this.port}`;
  }
}

Multiple Schemas

Just call TypedConfigModule.forRoot multiple times, and you're ready to go!

PS: Please do not share any class or sub-class between schemas, or Nest.js won't know which class to inject.

// config.ts
export class FooConfig {
  @IsString()
  foo!: string;
}
export class BazConfig {
  @IsString()
  baz!: string;
}

// app.module.ts
@Module({
  imports: [
    // load FooConfig from config file
    TypedConfigModule.forRoot({
      schema: FooConfig,
      load: fileLoader(),
    }),
    // load BazConfig from environment variables
    TypedConfigModule.forRoot({
      schema: BazConfig,
      load: dotenvLoader(),
    }),
  ],
})
export class AppModule {}

Custom validate function

If the default validate function doesn't suite your use case, you can provide it like in the example below:

TypedConfigModule.forRoot({
  schema: RootConfig,
  validate: (rawConfig: any) => {
    const config = plainToClass(RootConfig, rawConfig);
    const schemaErrors = validateSync(config, {
      forbidUnknownValues: true,
      whitelist: true,
    });

    if (schemaErrors.length) {
      throw new Error(TypedConfigModule.getConfigErrorMessage(schemaErrors));
    }

    return config as RootConfig;
  },
});

Using config outside Nest's IoC container (Usage in decorators)

Caution!

Using config outside Nest's IoC container will:

  1. make you struggle harder writing unit tests.
  2. force you to register TypedConfigModule synchronously using forRoot, since asynchronous configuration loading doesn't make sense under this situation.

How to

Due to the nature of JavaScript loading modules, decorators are executed before Nest's module initialization. If you want to get config value in decorators like @Controller() or @WebSocketGateway(), config module should be initialized before application bootstrap.

Suppose we need to inject routing information from the configuration, then we can define the configuration like this:

// config.ts
import { Type } from 'class-transformer';
import { ValidateNested, IsString } from 'class-validator';

export class RouteConfig {
  @IsString()
  public readonly app!: string;
}

export class RootConfig {
  @ValidateNested()
  @Type(() => RouteConfig)
  public readonly route!: RouteConfig;
}

Then create a configuration file:

route:
  app: /app

After creating the configuration file, we can initialize our ConfigModule with TypedConfigModule, and select RootConfig from ConfigModule using selectConfig method.

// config.module.ts
import { TypedConfigModule, fileLoader, selectConfig } from 'nest-typed-config';
import { RouteConfig } from './config';

export const ConfigModule = TypedConfigModule.forRoot({
  schema: RootConfig,
  load: fileLoader(),
});

export const rootConfig = selectConfig(ConfigModule, RootConfig);
export const routeConfig = selectConfig(ConfigModule, RouteConfig);

That's it! You can use rootConfig and routeConfig anywhere in your app now!

If target configuration model is marked with @Optional(), you should call selectConfig with { allowOptional: true } to allow optional configuration.

// app.controller.ts
import { Controller, Get } from '@nestjs/common';
import { AppService } from './app.service';
import { rootConfig } from './config.module';

@Controller(routeConfig.app)
export class AppController {
  constructor(private readonly appService: AppService) {}

  @Get()
  show(): void {
    return this.appService.show();
  }
}

For a full example, please visit CodeSandbox, or our examples folder.

API

Please refer to our API website for full documentation.

Changelog

Please refer to changelog.md

License

MIT.

Star History

Star History Chart

nest-typed-config's People

Contributors

akatsukilevi avatar andrey-hohlov avatar github-actions[bot] avatar hannes-kunnen avatar ilya-adnymics avatar jacquesg avatar latipun7 avatar lgtm-com[bot] avatar nikaple avatar pravin-raha avatar renovate-bot avatar renovate[bot] avatar semantic-release-bot avatar tak1n avatar techzealot avatar thejoecode avatar vsamofal avatar wagoid avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

nest-typed-config's Issues

[peerDependencies] Incompatible with nestjs version 9

I'm submitting a request to merge in some of the dependency update PRs


[ ] Regression
[ ] Bug report
[ ] Feature request
[ ] Documentation issue or request
[x] Support request

Current behavior

npm ERR! code ERESOLVE
npm ERR! ERESOLVE could not resolve
npm ERR!
npm ERR! While resolving: [email protected]
npm ERR! Found: @nestjs/[email protected]
npm ERR! node_modules/@nestjs/common
npm ERR!   @nestjs/common@"^9.0.5" from the root project
npm ERR!   peer @nestjs/common@"^8.2.3 || ^9.0.0" from @nestjs/[email protected]
npm ERR!   node_modules/@nestjs/apollo
npm ERR!     @nestjs/apollo@"^10.0.19" from the root project
npm ERR!   9 more (@nestjs/axios, @nestjs/core, @nestjs/graphql, ...)
npm ERR!
npm ERR! Could not resolve dependency:
npm ERR! peer @nestjs/common@">= 6.10.0 < 9" from [email protected]
npm ERR! node_modules/nest-typed-config
npm ERR!   nest-typed-config@"^2.4.1" from the root project

Expected behavior

We should expect the major version releases of nestjs to be included in the perrDependency tree

Environment


Nest version: 9.0.5
 
For Tooling issues:
- Node version: 16.15.1
- Platform:  Mac

Request to update readme / docs to use ValidateNested instead of Allow where it makes sense

I'm submitting a...


[ ] Regression 
[ ] Bug report
[ ] Feature request
[x ] Documentation issue or request
[ ] Support request

Current behavior

Nested Validation requires attribute that is not mentioned in the docs.
@ValidateNested()

Expected behavior

Consider replacing @Allow() with @ValidateNested() in the examples where it makes sense.

I had to dig through the tests to find an example of this class being used.

I realize this is a class-validator package feature, but it is likely that many will be using
both these packages for the first time.

Minimal reproduction of the problem with instructions

Was confused why DatabaseConfig url was not being validated:


export class DatabaseConfig {
  @IsString()
  public readonly url!: string;
}

export class RootConfig {
  @Type(() => DatabaseConfig)
  @Allow()
  public readonly database!: DatabaseConfig;
}

What is the motivation / use case for changing the behavior?

Save others from frustration of figuring out why validation only seems to run on the RootConfig and
not on any of its properties

Just switched Allow with ValidateNested here and it works:


export class DatabaseConfig {
  @IsString()
  public readonly url!: string;
}

export class RootConfig {
  @Type(() => DatabaseConfig)
  @ValidateNested()
  public readonly database!: DatabaseConfig;
}

Environment

Doc only

Unable to bundle `nest-typed-config` with webpack

I'm submitting a...

  • Regression
  • Bug report
  • Feature request
  • Documentation issue or request
  • Support request

Current behavior

The app fails to start when bundling with webpack, even if you exclude class-validator from bundling.

Webpack copies the https://github.com/Nikaple/nest-typed-config/blob/main/lib/utils/imports.util.ts directly into the bundle, except it tries to replace the require call but can't resolve the dependency, so instead replaces it with an empty context, which causes the requireFromRootNodeModules to throw an exception when starting.

/***/ }),
/* 1368 */
/***/ ((__unused_webpack_module, exports, __webpack_require__) => {

"use strict";

Object.defineProperty(exports, "__esModule", ({ value: true }));
exports.plainToClass = exports.validateSync = void 0;
const requireFromRootNodeModules = (moduleName) => {
    const modulePath = __webpack_require__(1369).resolve(moduleName, { paths: ['../..', '.'] });
    // eslint-disable-next-line @typescript-eslint/no-var-requires
    return __webpack_require__(1369)(modulePath);
};
exports.validateSync = requireFromRootNodeModules('class-validator').validateSync;
exports.plainToClass = requireFromRootNodeModules('class-transformer').plainToClass;
//# sourceMappingURL=imports.util.js.map

/***/ }),
/* 1369 */
/***/ ((module) => {

function webpackEmptyContext(req) {
	var e = new Error("Cannot find module '" + req + "'");
	e.code = 'MODULE_NOT_FOUND';
	throw e;
}
webpackEmptyContext.keys = () => ([]);
webpackEmptyContext.resolve = webpackEmptyContext;
webpackEmptyContext.id = 1369;
module.exports = webpackEmptyContext;

I created a basic patch that looks to fix the issue

diff --git a/dist/loader/dotenv-loader.js b/dist/loader/dotenv-loader.js
index 2bd4f53600790a2626f6469070c8c20f880ea6bf..142087986e787d65b4b1f5d269e435d0394c2398 100644
--- a/dist/loader/dotenv-loader.js
+++ b/dist/loader/dotenv-loader.js
@@ -40,7 +40,7 @@ const debug_util_1 = require("../utils/debug.util");
 let dotenv;
 let dotenvExpand;
 const loadEnvFile = (options) => {
-    dotenv = (0, load_package_util_1.loadPackage)('dotenv', 'dotenvLoader');
+    dotenv = require('dotenv');
     const envFilePaths = Array.isArray(options.envFilePath)
         ? options.envFilePath
         : [options.envFilePath || (0, path_1.resolve)(process.cwd(), '.env')];
diff --git a/dist/utils/imports.util.js b/dist/utils/imports.util.js
index 6ba2c6d4bca3e4ecc0003881494f2c21452c7a8b..cda6abecb57a8b2fb4e0e31222318a714a166ba9 100644
--- a/dist/utils/imports.util.js
+++ b/dist/utils/imports.util.js
@@ -6,6 +6,16 @@ const requireFromRootNodeModules = (moduleName) => {
     // eslint-disable-next-line @typescript-eslint/no-var-requires
     return require(modulePath);
 };
-exports.validateSync = requireFromRootNodeModules('class-validator').validateSync;
-exports.plainToClass = requireFromRootNodeModules('class-transformer').plainToClass;
+
+try {
+    exports.validateSync = requireFromRootNodeModules('class-validator').validateSync;
+} catch {
+    exports.validateSync = require('class-validator').validateSync;
+}
+
+try {
+    exports.plainToClass = requireFromRootNodeModules('class-transformer').plainToClass;
+} catch {
+    exports.plainToClass = require('class-transformer').plainToClass;
+}
 //# sourceMappingURL=imports.util.js.map

Besides the requireFromRootNodeModules, the loadPackage function also causes issues with webpacks resolution, while it's a convenient shorthand i'd suggest inlining it instead to avoid the issue.

try {
  dotenv = require('dotenv');
} catch (e) {
  console.error(MISSING_REQUIRED_DEPENDENCY('dotenv', 'dotenvLoader'));
  process.exit(1);
}

You could still have some sort of utility function if you want, like

dotenv = loadPackage('dotenv', 'dotenvLoader', () => require('dotenv'));

const MISSING_REQUIRED_DEPENDENCY = (name: string, reason: string) =>
  `The "${name}" package is missing. Please, make sure to install this library ($ npm install ${name}) to take advantage of ${reason}.`;

export function loadPackage(packageName: string, context: string, requireFn): any {
  try {
    return requireFn()
  } catch (e) {
    console.error(MISSING_REQUIRED_DEPENDENCY(packageName, context));
    process.exit(1);
  }
}

Expected behavior

Support being bundled

Minimal reproduction of the problem with instructions

Set externals: [] in your webpack config

What is the motivation / use case for changing the behavior?

Environment


Nest version: 10.3.1
Nest-Typed-Config version: 2.9.2

 
For Tooling issues:
- Node version: 20  
- Platform: Windows 

Others:

Use arbitrary ENV variable for file loading

I'm submitting a...

  • Regression
  • Bug report
  • Feature request
  • Documentation issue or request
  • Support request

Current behavior

I would like to use fileLoader but NODE_ENV is hardcoded in its behaviour now (https://github.com/Nikaple/nest-typed-config/blob/main/lib/loader/file-loader.ts#L66)

Expected behavior

Add ability to use arbitrary ENV variable when loading config files. The default value for it is NODE_ENV. But overall NODE_ENV isn't designed for specifying deployment env rather the application modes (https://stackoverflow.com/a/53368078)

Minimal reproduction of the problem with instructions

What is the motivation / use case for changing the behavior?

Environment


Nest version: X.Y.Z
Nest-Typed-Config version: X.Y.Z

 
For Tooling issues:
- Node version: XX  
- Platform:  

Others:

Different throw exception behaviour on config syntax error when an array of loaders is given

I'm submitting a...

  • Regression
  • Bug report
  • Feature request
  • Documentation issue or request
  • Support request

Current behavior

When there is a syntax error in an config file ( I tested .yaml ), using a single loader throws and error. An array of loaders does not.

Issue can be traced here.

Expected behavior

The behaviour to be configurable or at least consistent.

Minimal reproduction of the problem with instructions

Using config:

with:
   an_error
   inside: the file

Throws:

export const ConfigModule = TypedConfigModule.forRoot({
  schema: RootConfig,
  load: fileLoader()
});

Does not throw:

export const ConfigModule = TypedConfigModule.forRoot({
  schema: RootConfig,
  load: [fileLoader()]
});

What is the motivation / use case for changing the behavior?

Environment


Nest version: 16.17.0
Nest-Typed-Config version: 2.7.0

 
For Tooling issues:
- Node version: XX  
- Platform:  

Others:

Nested configs won't initialize if no environment variable of that nested config is set

I'm submitting a...


[ ] Regression
[x] Bug report
[ ] Feature request
[ ] Documentation issue or request
[ ] Support request

Current behavior

If using nested configs, nest-typed-config will not initialize a nested config when no environment variable was set for this config, even though default values for this configuration are present.

Output when starting (for the concrete ExampleConfig, see the example below):

Nest can't resolve dependencies of the AppService (?). Please make sure that the argument ExampleConfig at index [0] is available in the AppModule context.

It looks like when no environment variable for a nested config is set, the nested config will not be initialized.
When I set an environment variable that is contained inside the nested config, everything works just fine.

Expected behavior

The nested config should be available, even though no environment variables for this config were set.

Minimal reproduction of the problem with instructions

config.ts:

import { Type } from 'class-transformer';
import { IsInt, Min, ValidateNested } from 'class-validator';

export class ExampleConfig {
    @IsInt()
    @Type(() => Number)
    @Min(0)
    public readonly helloWorld: number = 1337;

    @IsInt()
    @Type(() => Number)
    @Min(0)
    public readonly justANumber: number = 42;
}

export class RootConfig {
    @ValidateNested()
    @Type(() => ExampleConfig)
    public readonly example!: ExampleConfig;
}

app.module.ts:

import { Global, Module } from '@nestjs/common';
import { dotenvLoader, TypedConfigModule } from 'nest-typed-config';
import { RootConfig } from './config';
import { AppService } from './app.service';

@Global()
@Module({
    imports: [
        TypedConfigModule.forRoot({
            schema: RootConfig,
            load: dotenvLoader({
                separator: '_',
            })
        })
    ],
    providers: [
        AppService
    ]
})
export class AppModule {}

app.service.ts:

import { Injectable } from '@nestjs/common';
import { ExampleConfig } from '../config';

@Injectable()
export class AppService {
    constructor(
        private readonly exampleConfig: ExampleConfig
    ) {}
}

Environment


Nest version: 9.0.11
 
For Tooling issues:
- Node version: v18.9.0
- Platform:  Mac

Others:
Package manager: Yarn

keep comments in declaration file

I'm submitting a...


[ ] Regression 
[ ] Bug report
[x] Feature request
[ ] Documentation issue or request
[ ] Support request

Current behavior

No documentation when importing from nest-typed-config, removeComments should be set to false in tsconfig.json

Expected behavior

Minimal reproduction of the problem with instructions

What is the motivation / use case for changing the behavior?

Error trying to start project with nest typed config when using class-validator

I'm submitting a...

  • Regression
  • Bug report
  • Feature request
  • Documentation issue or request
  • Support request

Current behavior

When I try to add the nest typed config as a dependency I get this error please verify that package.json has a valid main entry.
The issue I think mainly has to do with the class-validator dependency but I am not quite sure.
I can explain in more detail if needed

Error: Cannot find module '/home/almedin/Desktop/rekog/provatar/node_modules/.pnpm/@[email protected]_@[email protected]_@[email protected]_@nestjs+core@9_nhrtj2kto3prduyeup3bce5y4q/node_modules/graphql-subscriptions/dist/index.js'. Please verify that the package.json has a valid "main" entry
    at tryPackage (node:internal/modules/cjs/loader:444:19)
    at Function.Module._findPath (node:internal/modules/cjs/loader:715:18)
    at Function.Module._resolveFilename (node:internal/modules/cjs/loader:1130:27)
    at Function.Module._load (node:internal/modules/cjs/loader:985:27)
    at Module.require (node:internal/modules/cjs/loader:1235:19)
    at require (node:internal/modules/helpers:176:18)
    at Object.<anonymous> (/home/almedin/Desktop/rekog/provatar/node_modules/.pnpm/@[email protected]_@[email protected]_@[email protected]_@nestjs+core@9_nhrtj2kto3prduyeup3bce5y4q/packages/query-graphql/src/subscription/index.ts:2:1)
    at Module._compile (node:internal/modules/cjs/loader:1376:14)
    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1435:10)
    at Module.load (node:internal/modules/cjs/loader:1207:32)
    ```

## Expected behavior
I should be able to use the class-validator and start the project with no errors even when im using class-validator


<!-- Describe what the desired behavior would be. -->
I should be able to start the project when i

## Minimal reproduction of the problem with instructions

<!-- Please share a repo, a gist, or step-by-step instructions. -->

## What is the motivation / use case for changing the behavior?

<!-- Describe the motivation or the concrete use case. -->

## Environment

<pre><code>
Nest version: X.Y.Z
Nest-Typed-Config version: X.Y.Z
<!-- Check whether this is still an issue in the most recent Nest version -->
 
For Tooling issues:
- Node version: XX  <!-- run `node --version` -->
- Platform:  <!-- Mac, Linux, Windows -->

Others:
<!-- Anything else relevant?  Operating system version, IDE, package manager, ... -->
</code></pre>

Default values are overriding loaders?

I'm submitting a...


[ ] Regression 
[X] Bug report
[ ] Feature request
[ ] Documentation issue or request
[ ] Support request

Current behavior

Default values seem to override loaders.

Expected behavior

I would expect loaders to always override matching default values.

NOTE: This is my first time using this library, so maybe I'm missing something simple?

Minimal reproduction of the problem with instructions

import {
  TypedConfigModule,
  dotenvLoader,
  selectConfig,
} from 'nest-typed-config';
import { IsString, IsNumber } from 'class-validator';
import { mapKeys, camelCase } from 'lodash';

export class Config {
  @IsString()
  readonly nodeEnv: string = 'test';

  @IsString()
  readonly host: string = '127.0.0.1';

  @IsNumber()
  readonly port: number = 50051;

  @IsString()
  get url(): string {
    return `http://${this.host}:${this.port}`;
  }

  @IsString()
  get protoPath(): string {
    return this.nodeEnv === 'production' ? 'proto' : '../../../libs/proto';
  }
}

export const ConfigModule = TypedConfigModule.forRoot({
  schema: Config,
  load: [
    async () => {
      const rawEnv = dotenvLoader()();
      const mapped = mapKeys(rawEnv, (value, key) => camelCase(key));
      console.log(mapped); // nodeEnv = 'development'
      return mapped;
    },
  ],
});

export const appConfig = selectConfig(ConfigModule, Config);

console.log(appConfig); // nodeEnv = 'test' (wrong? shouldn't the loader have precedence?)

What is the motivation / use case for changing the behavior?

Trying to do a simple remapping of ENV variables

Environment


Nest version: 13.9.6

 
For Tooling issues:
- Node version: v16.14.2  
- Platform: macOS 

Others:

Does .env.yml support environment variables?

For an example:
I have env variable:-> DB_PORT=9002

.env.yml
----------------
db:
  port: ${DB_PORT}

Will this replace ${DB_PORT} with actual value 9002

Is there a way i can achieve this?

How to: async subset config sub modules?

I'm submitting a...

  • Regression
  • Bug report
  • Feature request
  • Documentation issue or request
  • Support request

Desired behavior

This might be a general nestjs question, but I figured I would ask here as this is probably a use case others may have. Basically, I am developing an App -> Lib relationship with strict config checking (to prevent developers missing setting an environment variable).

The requirement here is that the App needs to fetch a bunch of variables from external source (AWS Secret Manager). This works fine and the Config is type checked after loading. However, I have a library that depends on some of the configs (a subset). I want there to be validation on the library's module as well. But I can't figure out how to get the library to only load after the App has loaded.. I'm getting undefined values for my config values within the library getting booted up. See illustration below:

AppModule

import { getSecret } from '@aws-lambda-powertools/parameters/secrets'
import { Config } from 'app/config'
/* 
  public readonly VAR_1!: string
  public readonly VAR_2!: string
  public readonly VAR_3!: string
  public readonly VAR_4!: string
*/

@Module({
  imports: [
    TypedConfigModule.forRootAsync({
      isGlobal: true,
      schema: Config,
      load: [
        dotenvLoader(),
        async () => {
          const secret = await getSecret('my-secret', {
            transform: 'json',
          })
          return secret
        },
      ],
    }),
    ServiceModule.registerAsync({
      inject: [Config],
    }),
  ],
  providers: [AppService],
})
export class AppModule {}

and then in my lib (ServiceModule):

import { HttpModule, HttpModuleAsyncOptions, HttpService } from '@nestjs/axios'
import { Config } from 'lib/service/config'
/* 
  public readonly VAR_3!: string
  public readonly VAR_4!: string
*/

export const SERVICE_API_REST = 'SERVICE_API_REST'

@Module({})
export class ServiceModule {
  static registerAsync(options: any): DynamicModule {
    return {
      module: CoreSVCSModule,
      imports: [
        HttpModule.registerAsync({
          inject: [Config],
          useFactory: async (config: Config) => {
            return {
              url: config.VAR_3,
              headers: {
                'Content-Type': 'application/json',
                'api-key': config.VAR_4,
              },
            }
          },
        } as HttpModuleAsyncOptions),
      ],
      providers: [
        {
          provide: SERVICE_API_REST,
          inject: [HttpService],
          useFactory: (client: HttpService) => client,
        },
      ],
      exports: [SERVICE_API_REST],
    }
  }
}

How can I change this to work as desired?

What is the motivation / use case for changing the behavior?

Illustration of developing libraries that share configs from a parent app, enforcing the same checks and informing developers which configs are needed from parent.

Environment


Nest version: 10
Nest-Typed-Config version: 2.6.0

For Tooling issues:
- Node version: 20
- Platform:  Mac

Others:
nx monorepo

Provide `semantic-release` configuration to respect `main` branch

I'm submitting a...


[x] Regression 
[ ] Bug report
[ ] Feature request
[ ] Documentation issue or request
[ ] Support request

Current behavior

The release workflows kind of has false positive. The CI still passes because it's not really an error, but the release not triggered.

Here's the logs:
> semantic-release

[10:01:31 AM] [semantic-release] › ℹ  Running semantic-release version 18.0.0
[10:01:31 AM] [semantic-release] › ✔  Loaded plugin "verifyConditions" from "@semantic-release/npm"
[10:01:31 AM] [semantic-release] › ✔  Loaded plugin "verifyConditions" from "@semantic-release/github"
[10:01:31 AM] [semantic-release] › ✔  Loaded plugin "verifyConditions" from "@semantic-release/git"
[10:01:31 AM] [semantic-release] › ✔  Loaded plugin "analyzeCommits" from "@semantic-release/commit-analyzer"
[10:01:31 AM] [semantic-release] › ✔  Loaded plugin "generateNotes" from "@semantic-release/release-notes-generator"
[10:01:31 AM] [semantic-release] › ✔  Loaded plugin "prepare" from "@semantic-release/npm"
[10:01:31 AM] [semantic-release] › ✔  Loaded plugin "prepare" from "@semantic-release/git"
[10:01:31 AM] [semantic-release] › ✔  Loaded plugin "publish" from "@semantic-release/npm"
[10:01:31 AM] [semantic-release] › ✔  Loaded plugin "publish" from "@semantic-release/github"
[10:01:31 AM] [semantic-release] › ✔  Loaded plugin "addChannel" from "@semantic-release/npm"
[10:01:31 AM] [semantic-release] › ✔  Loaded plugin "addChannel" from "@semantic-release/github"
[10:01:31 AM] [semantic-release] › ✔  Loaded plugin "success" from "@semantic-release/github"
[10:01:31 AM] [semantic-release] › ✔  Loaded plugin "fail" from "@semantic-release/github"
[10:01:35 AM] [semantic-release] › ℹ  This test run was triggered on the branch main, while semantic-release is configured to only publish from next, alpha, therefore a new version won’t be published.
see: semantic-release/semantic-release#1581

semantic-release not configured to use main branch by default, you need to configure the configs 😉

I have opinionated shareable config too if you don't mind. It's configured to use main branch too alongside with next, alpha, beta, and maintenance version branch (ex: 2.x). It enforces the conventional commits specs and produced this kind of release notes and changelogs.
example release notes

Expected behavior

No CI error, and the package published to NPM successfully.

What is the motivation / use case for changing the behavior?

New commits not published automatically.

Default value is ignored if property is undefined

I'm submitting a...


[ ] Regression 
[x] Bug report
[ ] Feature request
[ ] Documentation issue or request
[ ] Support request

We use environment variables from .env file for local development and set them in deployment settings in cloud.
Config properties are in camelCase, but ALL_CAPS is used for environment variables.
Some config options are optional and have default values.

To support this requirements we use dotenvLoader to load .env and normalize function to map values.

export class Config {
  @IsInt()
  readonly propertyWithDefaultValue: number = 4321;
}
TypedConfigModule.forRoot({
  schema: Config,
  load: dotenvLoader(),
  normalize(config) {
    return {
      propertyWithDefaultValue: config.UNDEFINED_ENV_PROPERTY,
    };
  },
}),

Current behavior

It ignores default value and applies undefined.

Expected behavior

Use default value if value is undefined.

Minimal reproduction of the problem with instructions

exposeDefaultValues: true in plainToClass method of class-transformer makes it works.

See PR with fix and test for this case.

New Release for NestJS 10 support

I'm submitting a

  • Regression
  • Bug report
  • Feature request
  • Documentation issue or request
  • Support request

Current behavior

Current 2.6.0 release requires NestJS < 10 as a peer dependency.

npm ERR! Could not resolve dependency:
npm ERR! peer @nestjs/common@">= 6.10.0 < 10" from [email protected]
npm ERR! node_modules/nest-typed-config
npm ERR!   nest-typed-config@"^2.6.0" from the root project
npm ERR! 
npm ERR! Conflicting peer dependency: @nestjs/[email protected]
npm ERR! node_modules/@nestjs/common
npm ERR!   peer @nestjs/common@">= 6.10.0 < 10" from [email protected]
npm ERR!   node_modules/nest-typed-config
npm ERR!     nest-typed-config@"^2.6.0" from the root project

Expected behavior

New Release with NestJS 10 support. Main already has it fixed with the following pull request:

https://github.com/Nikaple/nest-typed-config/pull/388/files

External Configuration "Validity" Check.

I'm submitting a...


[ ] Regression 
[ ] Bug report
[x] Feature request
[ ] Documentation issue or request
[ ] Support request

Expected behavior

Would it be possible to add a capability for a runtime check of the configuration? For example:

  • If I have a database configuration I'd like to test a successful connection to it.
  • For an external API I'd like to hit a simple endpoint.

Add example using multiple environments

I'm submitting a...


[ ] Regression 
[ ] Bug report
[ ] Feature request
[x] Documentation issue or request
[ ] Support request

Current behavior

Examples on how to use this package with multiple environments are missing.

Expected behavior

An example to show how this package can be use to load the right .env file

My use case:

.
├── src
│   └── ...
└── env
    ├── .env.development
    ├── .env.production
    └── .env.staging

each .env must follow the same EnvironmentVariables interface. The one that will be loaded is defined by process.env.NODE_ENV value. Also, I have an application config and database config that will use the env. vars.

interface EnvironmentVariables {
  DB_PORT: number
  APP_PORT: number
}

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Rate-Limited

These updates are currently rate-limited. Click on a checkbox below to force their creation now.

  • chore(deps): update dependency @nestjs/axios to v3.0.2
  • chore(deps): update dependency @types/express to v4.17.21
  • chore(deps): update dependency @types/jest to v29.5.12
  • chore(deps): update dependency @types/lodash.frompairs to v4.0.9
  • chore(deps): update dependency @types/lodash.merge to v4.6.9
  • chore(deps): update dependency @types/set-value to v4.0.3
  • chore(deps): update dependency parse-json to v7.1.1
  • chore(deps): update dependency rimraf to v5.0.5
  • chore(deps): update dependency rxjs to v7.8.1
  • chore(deps): update dependency rxjs to v7.8.1
  • chore(deps): update dependency semantic-release to v22.0.12
  • chore(deps): update dependency ts-jest to v29.1.2
  • chore(deps): update dependency typedoc to v0.25.13
  • fix(deps): update dependency class-validator to v0.14.1
  • chore(deps): update dependency @commitlint/cli to v17.8.1
  • chore(deps): update dependency axios to v0.28.1
  • chore(deps): update dependency cosmiconfig to v8.3.6
  • chore(deps): update dependency dotenv to v16.4.5
  • chore(deps): update dependency dotenv to v16.4.5
  • chore(deps): update dependency eslint-config-prettier to v9.1.0
  • chore(deps): update dependency eslint-plugin-import to v2.29.1
  • chore(deps): update dependency reflect-metadata to v0.2.2
  • chore(deps): update dependency typescript to v5.4.5
  • chore(deps): update dependency yaml to v2.4.2
  • chore(deps): update dependency yaml to v2.4.2
  • chore(deps): update nest monorepo to v10.3.8 (@nestjs/common, @nestjs/core, @nestjs/platform-express, @nestjs/testing)
  • chore(deps): update actions/setup-node action to v4
  • chore(deps): update dependency @commitlint/cli to v19
  • chore(deps): update dependency axios to v1.6.8
  • chore(deps): update dependency cosmiconfig to v9
  • chore(deps): update dependency cosmiconfig to v9
  • chore(deps): update dependency dotenv-expand to v11
  • chore(deps): update dependency dotenv-expand to v11
  • chore(deps): update dependency eslint to v9
  • chore(deps): update dependency husky to v9
  • chore(deps): update dependency lint-staged to v15
  • chore(deps): update dependency node to v20 (node, @types/node)
  • chore(deps): update dependency parse-json to v8
  • chore(deps): update dependency parse-json to v8
  • chore(deps): update dependency semantic-release to v23
  • chore(deps): update github/codeql-action action to v3
  • chore(deps): update pnpm to v9
  • 🔐 Create all rate-limited PRs at once 🔐

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

Ignored or Blocked

These are blocked by an existing closed PR and will not be recreated unless you click a checkbox below.

Detected dependencies

github-actions
.github/workflows/build.yml
  • actions/checkout v3
  • pnpm/action-setup v2
  • actions/setup-node v3
  • coverallsapp/github-action v2.2.0
.github/workflows/codeql.yml
  • actions/checkout v3
  • github/codeql-action v2
  • github/codeql-action v2
  • github/codeql-action v2
.github/workflows/doc.yml
  • actions/checkout v3
  • pnpm/action-setup v2
  • actions/setup-node v3
  • crazy-max/ghaction-github-pages v3.1.0
.github/workflows/release.yml
  • actions/checkout v3
  • pnpm/action-setup v2
  • actions/setup-node v3
  • coverallsapp/github-action v2.2.0
npm
package.json
  • chalk 4.1.2
  • class-transformer 0.5.1
  • class-validator ^0.14.0
  • debug 4.3.4
  • lodash.frompairs 4.0.1
  • lodash.merge 4.6.2
  • set-value ^4.1.0
  • @commitlint/cli 17.7.1
  • @iarna/toml 2.2.5
  • @latipun7/commitlintrc 1.1.3
  • @latipun7/releaserc ^2.1.0
  • @nestjs/axios 3.0.0
  • @nestjs/cli 10.1.17
  • @nestjs/common 10.2.5
  • @nestjs/core 10.2.5
  • @nestjs/platform-express 10.2.5
  • @nestjs/testing 10.2.5
  • @types/debug 4.1.8
  • @types/express 4.17.17
  • @types/jest 29.5.5
  • @types/lodash.frompairs 4.0.7
  • @types/lodash.merge 4.6.7
  • @types/node 18.17.17
  • @types/set-value ^4.0.1
  • @typescript-eslint/eslint-plugin 5.62.0
  • @typescript-eslint/parser 5.62.0
  • axios 1.5.0
  • cosmiconfig 8.3.6
  • dotenv 16.3.1
  • dotenv-expand 10.0.0
  • eslint 7.32.0
  • eslint-config-prettier 9.0.0
  • eslint-plugin-import 2.28.1
  • eslint-plugin-prettier 4.2.1
  • husky 8.0.3
  • jest 29.7.0
  • lint-staged 14.0.1
  • parse-json 7.1.0
  • prettier 2.8.8
  • reflect-metadata 0.1.13
  • rimraf 5.0.1
  • rxjs 7.8.1
  • semantic-release ^22.0.0
  • ts-jest 29.1.1
  • typedoc 0.25.1
  • typescript 5.2.2
  • yaml 2.3.2
  • @iarna/toml >= 2.2.5
  • @nestjs/axios >= 0.1.0
  • cosmiconfig >= 8.0.0
  • dotenv >= 16.0.0
  • dotenv-expand >= 10.0.0
  • parse-json >= 5.2.0
  • yaml >= 1.10.2
  • @nestjs/common >= 6.10.0 < 11
  • reflect-metadata >= 0.1.12 < 0.3
  • rxjs >= 6.0.0 < 8
  • pnpm 7.33.1
  • rxjs 7.8.0
  • axios 0.27.2
nvm
.nvmrc
  • node v18

  • Check this box to trigger a request for Renovate to run again on this repository

Add ability to specify a default value for environment variable in the yaml file

  • Feature request

Current behavior

right now if you have such a config yaml file for example:

logs:
  colorize: ${LOGS_COLORIZE}

and have a default value in the class

export default class LoggerConfig {
  @IsBoolean()
  @Allow()
  colorize = true;
}

you will get an error when the app is starting

Error: Environment variable is not set for variable name: 'LOGS_COLORIZE'

Expected behavior

It will be nice to have the ability to specify a default value in the file itself. Like this:

logs:
  colorize: ${LOGS_COLORIZE:true}

What is the motivation/use case for changing the behavior?

it just will be convenient, I've seen this behavior in all config libs in java and .net.

Expanding env variables is not working anymore in v2.5.1

I'm submitting a...


[X] Regression 
[ ] Bug report
[ ] Feature request
[ ] Documentation issue or request
[ ] Support request

Current behavior

Seems that expanding env variables is not working anymore in v2.5.1.
Setting the expandVariables: true in module config for dotenvLoader gives error

config = dotenvExpand({ parsed: config }).parsed!;
                 ^
TypeError: dotenvExpand is not a function

Reverting to v2.4.7 helped solving the issue.

Expected behavior

Expanding env variables works as described in documentation.

Minimal reproduction of the problem with instructions

This should be enough to reproduce. Create a .env file with following contents:

foo=123
bar=${foo}456

Create a simple config file:

// config.ts
export class AppConfig {
  foo: string;
  bar: string;
}

In your app.module.ts file configure the NestTypedConfig:

// app.module.ts
import { TypedConfigModule, dotenvLoader } from 'nest-typed-config';
import { AppConfig } from './config';
... // other imports from nestjs

@Module({
  imports: [
    TypedConfigModule.forRoot({
      schema: AppConfig,
      load: dotenvLoader({
        expandVariables: true,
      }),
    })
  ],
})
export class AppModule {}

Start the project and the error will occur.

What is the motivation / use case for changing the behavior?

Environment


Nest version: 9.2.0
Nest-Typed-Config version: 2.5.1

 
For Tooling issues:
- Node version: 18.14.1  
- Platform:  Ubuntu

Others:

@ValidateNested doesn't work without custom validate function

I'm submitting a...


[ ] Regression 
[x] Bug report
[ ] Feature request
[ ] Documentation issue or request
[ ] Support request

Current behavior

Originally posted by @imkh in #148 (comment)

  1. For some reason, having a nested configuration in the RootConfig class with the @ValidateNested() decorator doesn't work. You must add the custom validate function shown in the README. It seems like someone else already ran into this problem in this issue. The README of class-validator does mention that nested object must be an instance of a class, otherwise @ValidateNested won't know what class is target of validation. so that might be the reason?

Expected behavior

As explained in your comment here, "there should be no difference with builtin validate function and custom validate function.".

@ValidateNested() should work without the need to implement a custom validate function (especially since this decorator will be used almost always if you have nested config classes, otherwise it defeats the purpose of this feature).

Minimal reproduction of the problem with instructions

Minimum reproduction repo here: https://github.com/imkh/nest-typed-config-issue-149

I wasn't able to reproduce the bug in the basic example of this repo:

$ git clone [email protected]:Nikaple/nest-typed-config.git
$ cd nest-typed-config.git
$ cd examples/basic
$ npm install @nestjs/core @nestjs/platform-express # The example wouldn't start without installing those 2 dependencies
$ npm run start # Go to http://127.0.0.1:<port>, nested config seem to work as expected

What is the motivation / use case for changing the behavior?

/

Environment


Nest version:
@nestjs/cli 8.2.1
@nestjs/core 8.3.1
(both latest versions at the time of writing)

 
For Tooling issues:
- Node version: 16.13.2 
- Platform: Mac 

Others:

Automatically assign environments from `.env` file to `process.env`

I'm submitting a...


[ ] Regression 
[ ] Bug report
[x] Feature request
[ ] Documentation issue or request
[ ] Support request

Current behavior

imports: [TypeOrmModule.forRoot()] (without passing a configuration object to forRoot()) is breaking even thought I specify configuration in environment variables as .env file.

Expected behavior

Specifying TypeORM configuration as Environment Variables as .env file should pass and logged in to database with only imports: [TypeOrmModule.forRoot()] (without passing a configuration object to forRoot()).

Minimal reproduction of the problem with instructions

What is the motivation / use case for changing the behavior?

TypeORM accept environment variables as configuration so it can't be used with dependency injection.

Environment


Nest version: 8.1.2

 
For Tooling issues:
- Node version: 16.13.0  
- Platform:  

Others:

Load yml with dynamic keys

I'm submitting a...


[ ] Regression 
[ ] Bug report
[ ] Feature request
[ ] Documentation issue or request
[X] Support request

I am trying to load several quite complex yml files with nest-typed-config
For that reason, I am using the directoryLoader. The yml files are similiar to docker-compose.yml files.
There are section names, that are dynamic and not known to me. The user can specify these names.

The files are looking like this

test1:
  type: A
  endPoint: endpoint1.com
  port: 9000
  anyProp: 'abc'

test2:
  type: B
  endPoint: endpoint2.com
  port: 443
  path: /path/to/anything/

Depending on a type property, each section can have different properties.
But of course, same type means same properties.

My RootConfig looks like

export class AConfig{
    @Allow()
    type: 'A'

    @Allow()
    @IsUrl()
    endPoint: string

    @Allow()
    @IsNumber()
    port: number

    @Allow()
    @IsNotEmpty()
    anyProp: string
}

export class RootConfig {
    public readonly endpoints!: {
        [key: string]: AConfig | BConfig
    }
};

I see, that the yml files are loaded, but the validation fails.
Is there a easy way to achieve that?

I get this exception:

- config undefined does not match the following rules:
    - unknownValue: an unknown value was passed to the validate function, current config is `undefined`

at Function.validateWithClassValidator (/usr/src/app/node_modules/nest-typed-config/lib/typed-config.module.ts:135:13)
at Function.getDynamicModule (/usr/src/app/node_modules/nest-typed-config/lib/typed-config.module.ts:52:20)
at Function.forRoot (/usr/src/app/node_modules/nest-typed-config/lib/typed-config.module.ts:23:17)

Build failed with version 2.5.0

I'm submitting a...


[ ] Regression 
[x] Bug report
[ ] Feature request
[ ] Documentation issue or request
[ ] Support request

Current behavior

src/app.module.ts:3:10 - error TS2305: Module '"nest-typed-config"' has no exported member 'TypedConfigModule'.

3 import { TypedConfigModule } from 'nest-typed-config';
           ~~~~~~~~~~~~~~~~~
src/config/index.ts:2:10 - error TS2305: Module '"nest-typed-config"' has no exported member 'TypedConfigModuleOptions'.

2 import { TypedConfigModuleOptions, fileLoader } from 'nest-typed-config';
           ~~~~~~~~~~~~~~~~~~~~~~~~
src/config/index.ts:2:36 - error TS2305: Module '"nest-typed-config"' has no exported member 'fileLoader'.

2 import { TypedConfigModuleOptions, fileLoader } from 'nest-typed-config';

Expected behavior

Minimal reproduction of the problem with instructions

What is the motivation / use case for changing the behavior?

Environment


Nest version: X.Y.Z

 
For Tooling issues:
- Node version: XX  
- Platform:  

Others:

env var substitutions break if the variable isn't set

I'm submitting a...


[ ] Regression 
[X] Bug report
[ ] Feature request
[ ] Documentation issue or request
[ ] Support request

Current behavior

If you use Environment Variable Substitution by setting ignoreEnvironmentVariableSubstitution to false and the environment variable is not set it throws an error crashing the app.

 Info  Webpack is building your sources...

webpack 5.73.0 compiled successfully in 416 ms
Type-checking in progress...

project/node_modules/nest-typed-config/lib/loader/file-loader.ts:90
      throw new Error(
            ^
Error: Environment variable is not set for variable name: 'NODE_ENV'
    at replace (project/node_modules/nest-typed-config/lib/loader/file-loader.ts:90:13)
    at String.replace (<anonymous>)
    at placeholderResolver (project/node_modules/nest-typed-config/lib/loader/file-loader.ts:100:19)
    at project/node_modules/nest-typed-config/lib/loader/file-loader.ts:147:30
    at Function.getRawConfig (project/node_modules/nest-typed-config/lib/typed-config.module.ts:76:12)
    at Function.forRoot (project/node_modules/nest-typed-config/lib/typed-config.module.ts:21:28)
    at Object../src/app.module.ts (project/dist/main.js:79:51)
    at __webpack_require__ (project/dist/main.js:937:42)
    at Object../src/main.ts (project/dist/main.js:700:22)
    at __webpack_require__ (project/dist/main.js:937:42)
No errors found.

Expected behavior

returning an undefined without throwing would be fantastic

Minimal reproduction of the problem with instructions

# .env.yml
testEnvs: ${NODE_ENV}
// in config schema
import { IsOptional, IsString } from 'class-validator';

export class Config {
  @IsString()
  @IsOptional()
  public readonly testEnvs?: string;
}
// in  app.module.ts module imports
TypedConfigModule.forRoot({
  schema: Config,
  load: fileLoader({
    ignoreEnvironmentVariableSubstitution: false,
  }),
  isGlobal: true,
}),

What is the motivation / use case for changing the behavior?

It would be better to respect the config schema, and allow optional fields to be treated as optional.

Environment


Nest version: 9.1.1

 
For Tooling issues:
- Node version: v18.8.0  
- Platform:  Linux Ubuntu

Others:

Environment Variable Substitution Not Working

I'm submitting a...

  • Regression
  • Bug report
  • Feature request
  • Documentation issue or request
  • Support request

Current behavior

Environment variable substitution defined in .env doesn't seem to be substituted into .env.yaml even when ignoreEnvironmentVariableSubstitution: false for file loader. Config file is defined below

// .env
PORT=9000
LOADER=fileLoader
app:
  somePort: ${PORT:-12345}
  envLoader: ${LOADER}

The current output is

app: 
    somePort: 12345
    envLoader: undefined

Expected behavior

Expected output

app:
    somePort: 9000
    envLoader: fileLoader

Minimal reproduction of the problem with instructions

https://gist.github.com/Nathan-Bernardo/bcb98b2215571223299209b9088489f2

Environment


Nest version: 8.0.0
Nest-Typed-Config version: 2.9.1

 
For Tooling issues:
- Node version: 16.18.2  
- Platform: Ubuntu 18.04

Matching between environment variables & config file properties

I'm submitting a...


[ ] Regression 
[ ] Bug report
[x] Feature request
[ ] Documentation issue or request
[ ] Support request

I'm trying to replicate the same configuration capabilities as large open-source projects such as GitLab, Sourcegraph, or Kibana. In those projects, you have the ability to specify configuration via a config file (gitlab.yml, sourcegraph.json, kibana.yml) or an equivalent environment variable (with env variables having priority over the config file).

GitLab Sourcegraph Kibana
gitlab.yml admin/config/site.schema.json kibana.yml
Env variables Env variables Env variables

So far so good, by following the README, I was able to get to the same setup. The last thing I'm unable to replicate is the ability to have an env variable name that is different from a YML property, but still have the two match and reference the same configuration property. The main reason is that env variables are typically written in uppercase with underscores (SERVER_NAME) while YML or JSON properties are usually in lowercase & camelCase (serverName or server.name).

For example with Kibana:

There could be a few ways to implement something like this:

  1. some kind of "middleware" that would translate serverName to SERVER_NAME or database.url to DATABASE_URL
  2. not sure if possible: export a custom decorator to add on top of the property in the Config class
// config.ts
export class Config {
  @EnvVariable('SERVER_NAME')
  public readonly serverName!: string;
}

The second solution is inspired by something I was able to do with a Golang config library: https://github.com/gravitational/configure

type Config struct {
    StringVar   string              `env:"STRING_VAR" cli:"string-var" yaml:"string_var"`
    BoolVar     bool                `env:"BOOL_VAR" cli:"bool_var" yaml:"bool_var"`
    IntVar      int                 `env:"INT_VAR" cli:"int_var" yaml:"int_var"`
    HexVar      hexType             `env:"HEX_VAR" cli:"hex_var" yaml:"hex_var"`
    MapVar      map[string]string   `env:"MAP_VAR" cli:"map_var" yaml:"map_var,flow"`
    SliceMapVar []map[string]string `env:"SLICE_MAP_VAR" cli:"slice_var" yaml:"slice_var,flow"`
}

An other Golang config library (https://github.com/peterbourgon/ff) opted for the first solution:

Additionally, the example will look in the environment for variables with a MY_PROGRAM prefix. Flag names
are capitalized, and separator characters are converted to underscores. In this case, for example,
MY_PROGRAM_LISTEN_ADDR would match to listen-addr.

Webpack builds in monorepos are broken

I'm submitting a...


[ ] Regression 
[x] Bug report
[ ] Feature request
[ ] Documentation issue or request
[ ] Support request

Current behavior

Using this nest-typed-config in nestjs monorepo, which are built with web pack results. In class-validator library not being found due to dynamic loading. Previous version exhibited similar behaviour for dotenv, but it was easier to work around by providing own loader.

Expected behavior

Webpack builds should not fail.

Environment


Nest version: 9.0.0

 
For Tooling issues:
- Node version: 16  

Others:

v2.5.0 has a failing build

I'm submitting a...


[x] Regression 
[ ] Bug report
[ ] Feature request
[ ] Documentation issue or request
[ ] Support request

Current behavior

Installing v2.4.7 does include the dist.
Installing v2.5.0 does not include the dist.

Expected behavior

The dist folder to be included in the package, so the index.ts can export everything from it.

Minimal reproduction of the problem with instructions

  1. Run npm init
  2. Run npm install nest-typed-config
  3. See that the dist folder is not present.

What is the motivation / use case for changing the behavior?

The package is currently unusable.

Environment


Nest version: N/A
 
For Tooling issues:
- Node version: 16.16.0 
- Platform: Windows

Others:

This happens using both yarn and npm.

On demand loading native loaders

I'm submitting a...


[ ] Regression 
[ ] Bug report
[x] Feature request
[ ] Documentation issue or request
[ ] Support request

that introduces breaking changes

Current behavior

There are couple optional features that requires some 3rd-party package to work, and these packages are always loaded (due to the following barrel file)

export * from './file-loader';
export * from './remote-loader';
export * from './dotenv-loader';
export * from './directory-loader';

Expected behavior

Only load the dependencies used by the loader(s) that will be used in the project.

So, instead of
import { TypedConfigModule, fileLoader } from 'nest-typed-config';
we must do this:

import { TypedConfigModule } from 'nest-typed-config';
import { fileLoader } from 'nest-typed-config/file-loader';
// or (both working in the same way)
import fileLoader from 'nest-typed-config/file-loader';

BONUS: only install dependencies that will be used in the end. How? moving hard-dependencies to optional dependencies instead.

"@iarna/toml": "^2.2.5",

"axios": "^0.21.1",

"dotenv": "^9.0.0",
"dotenv-expand": "^5.1.0",

"yaml": "^1.10.2"

What is the motivation / use case for changing the behavior?

Let's say I want to use only the dotenv loader (or my own custom loader) in my Nodejs project, thus I don't need to load (and install) all other loaders and their dependencies.

Lazy-loading will decrease the bootstrap time and the bundle size of this package (if you choose to move few prod deps. to optional deps.).

optionalDependencies should be peerDependencies

I'm submitting a...

  • Regression
  • Bug report
  • Feature request
  • Documentation issue or request
  • Support request

Current behavior

Installing nest-typed-config installs the following optional dependencies:

  • @iarna/toml
  • @nestjs/axios
  • cosmiconfig
  • dotenv
  • dotenv-expand
  • parse-json
  • yaml

While it is documented to use npm install --no-optional, this is not supported in other package managers and is not the actual use case for optionalDependencies.

npm

If a dependency can be used, but you would like npm to proceed if it cannot be found or fails to install, then you may put it in the optionalDependencies object.
https://docs.npmjs.com/cli/v9/configuring-npm/package-json?v=true#optionaldependencies

yarn

Similar to the dependencies field, except that these entries will not be required to build properly should they have any build script. Note that such dependencies must always be resolvable (otherwise we couldn't store it in the lockfile, which could lead to non-reproducible installs), but those which list os / cpu / libc fields will not be fetched unless they match the current system architecture.

This field is usually not what you're looking for, unless you depend on the fsevents package. If you need a package to be required only when a specific feature is used then use an optional peer dependency. Your users will have to satisfy it should they use the feature, but it won't cause the build errors to be silently swallowed when the feature is needed.
https://yarnpkg.com/configuration/manifest#optionalDependencies

Expected behavior

The dependencies should instead be peerDependencies with peerDependenciesMeta setting optional to true.

What is the motivation / use case for changing the behavior?

Ideally, unused dependencies are not included when installing a package.

Environment


Nest version: 9.0.11
Nest-Typed-Config version: 2.5.2

 
For Tooling issues:
- Node version: v18.16.0
- Platform: Linux

Others:
Package manager: [email protected] and [email protected]

Add common production environment key/value parsing example

I'm submitting a...

  • Regression
  • Bug report
  • Feature request
  • Documentation issue or request
  • Support request

Current behavior

Environment keys aren't properly cased or camel-cased as is typical when writing the code.

Expected behavior

It should be documented how to perform common key transforms to coerce and transform the key and value as needed for loading an app in dev mode using a yaml file and/or in production using environment variables.

Minimal reproduction of the problem with instructions

Use multiple loaders per https://github.com/Nikaple/nest-typed-config#using-multiple-loaders

Note that if one has a config like:

export class DatabaseConfig {
  @IsNumber()
  public readonly port!: number;

  @IsString()
  public readonly userName: string;
}

Passing the right data in via a config file works easily, but getting the dotenv loader to properly deal with environment variables is not as well documented.

What is the motivation / use case for changing the behavior?

The easiest fix in the documentation is to note in a single spot how to handle what I believe is a relatively common pattern of local config files vs. production environment variable usage. Taking the above example, one needs to make a few tweaks to make that schema work for environment variables like so:

export class DatabaseConfig {
  @IsNumber()
  @Type(() => Number)
  public readonly port!: number;

  @IsString()
  public readonly userName: string;
}

This is documented in the README under variable substitution, but I missed it the first few times trying to find the common handling for environment variables. The section here documenting transforming the variable manually also lends itself to confusing the ideal path:
https://github.com/Nikaple/nest-typed-config#transforming-the-raw-configuration

The last bit missing above is how to get an environment variable into the key format of userName as shown above. I used a key transform like this to solve that generically in a way that'd work for all keys when nesting:

    TypedConfigModule.forRoot({
      schema: RootConfig,
      load: [
        fileLoader(),
        dotenvLoader({
          separator: '__',
          keyTransformer: (key) =>
            key
              .toLowerCase()
              .replace(/(?<!_)_([a-z])/g, (_, p1) => p1.toUpperCase()),
        }),
      ],
    }),

This way DATABASE__USER_NAME=testuser is properly tranlated to the database config of userName.

I think having an example of how to handle the key transform for common camel-cased property names along with the recommendation of using the @Type based value transform would be beneficial to many.

Complex types are not preserved when transformed in `normalize`

I'm submitting a...

[ ] Regression
[x] Bug report
[ ] Feature request
[ ] Documentation issue or request
[ ] Support request

Current behavior

Simple transformation in the normalize function work as expected, but converting them to other datatypes (for eg Uint8Array) does not work.

Expected behavior

The types and values should be preserved.

Minimal reproduction of the problem with instructions

Stackblitz: https://stackblitz.com/edit/nestjs-starter-demo-uus4oz?file=src/config.ts

In this example, I convert TEST into an Uint8Array. When I print that inside the function itself, I get the expected output. But doing so in a service does not give the correct output.

image

Environment

All environments

Bug/Feature request: only load environment variables that are specified in the config file

I'm submitting a...


[ ] Regression 
[x] Bug report
[x] Feature request
[ ] Documentation issue or request
[ ] Support request

Current behavior

Using the env loader using only environment variables, the variables not specified in the config file still get added to the config object.

Expected behavior

To only have the environment variables in the config that are specified as class fields.

Minimal reproduction of the problem with instructions

  1. Use the envloader in app.module.ts:
imports: [
  TypedConfigModule.forRoot({
    schema: Config,
    load: dotenvLoader({
      ignoreEnvFile: true,
      ignoreEnvVars: false,
    }),
  }),
  ...
],
  1. Add a config file:
export default class Config {
  @IsNumber()
  public readonly FOO: number;

  @IsString()
  public readonly BAR: string;
}
  1. Add environment variables for FOO, BAR, and BAZ.
  2. Find that the resulting config has FOO, BAR, as well as BAZ.

What is the motivation / use case for changing the behavior?

During development, this behaviour clutters the config a lot. Most programmers have tons of environment variables on different machines, making it hard to keep track of what configs we actually want during code.
This also makes mistakes easy to miss, as one might try to use variable BAZ during development, which would work just fine as it is put in the config object, without it actually being specified in the config file.

Environment


Nest version: 8.2.1

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.