japgolly/clear-config

Scala FP configuration library with a focus on runtime clarity

clear-config's Issues

ConsumerFn defaults don't appear in the report

  def poolConfig: ConfigDef[JedisPoolConfig => Unit] =
    ConfigDef.consumerFn[JedisPoolConfig](
        _.get("BLOCK_WHEN_EXHAUSTED", _.setBlockWhenExhausted),
        _.get("EVICTION_POLICY_CLASS_NAME", _.setEvictionPolicyClassName),
        _.getOrUse("FAIRNESS", _.setFairness)(true), // <------------------ look
        _.get("JMX_ENABLED", _.setJmxEnabled),
        _.get("JMX_NAME_BASE", _.setJmxNameBase),
        _.get("JMX_NAME_PREFIX", _.setJmxNamePrefix),
        _.get("LIFO", _.setLifo),
        _.get("MAX_IDLE", _.setMaxIdle),
        _.get("MAX_TOTAL", _.setMaxTotal),
        _.get("MAX_WAIT_MILLIS", _.setMaxWaitMillis),
        _.get("MIN_EVICTABLE_IDLE_TIME_MILLIS", _.setMinEvictableIdleTimeMillis),
        _.getOrUse("MIN_IDLE", _.setMinIdle)(2), // <------------------ look
        _.get("NUM_TESTS_PER_EVICTION_RUN", _.setNumTestsPerEvictionRun),
        _.get("SOFT_MIN_EVICTABLE_IDLE_TIME_MILLIS", _.setSoftMinEvictableIdleTimeMillis),
        _.get("TEST_ON_BORROW", _.setTestOnBorrow),
        _.get("TEST_ON_CREATE", _.setTestOnCreate),
        _.get("TEST_ON_RETURN", _.setTestOnReturn),
        _.get("TEST_WHILE_IDLE", _.setTestWhileIdle),
        _.get("TIME_BETWEEN_EVICTION_RUNS_MILLIS", _.setTimeBetweenEvictionRunsMillis)
      )
      .withPrefix("POOL_")

and the report

Used keys (59):
+------------------------------------------------------+----------+------------------+
| Key                                                  | Fake env | Default          |
+------------------------------------------------------+----------+------------------+
| CACHE_REDIS_POOL_BLOCK_WHEN_EXHAUSTED                |          |                  |
| CACHE_REDIS_POOL_EVICTION_POLICY_CLASS_NAME          |          |                  |
| CACHE_REDIS_POOL_FAIRNESS                            |          |                  |
| CACHE_REDIS_POOL_JMX_ENABLED                         |          |                  |
| CACHE_REDIS_POOL_JMX_NAME_BASE                       |          |                  |
| CACHE_REDIS_POOL_JMX_NAME_PREFIX                     |          |                  |
| CACHE_REDIS_POOL_LIFO                                |          |                  |
| CACHE_REDIS_POOL_MAX_IDLE                            |          |                  |
| CACHE_REDIS_POOL_MAX_TOTAL                           |          |                  |
| CACHE_REDIS_POOL_MAX_WAIT_MILLIS                     |          |                  |
| CACHE_REDIS_POOL_MIN_EVICTABLE_IDLE_TIME_MILLIS      |          |                  |
| CACHE_REDIS_POOL_MIN_IDLE                            |          |                  |
| CACHE_REDIS_POOL_NUM_TESTS_PER_EVICTION_RUN          |          |                  |
| CACHE_REDIS_POOL_SOFT_MIN_EVICTABLE_IDLE_TIME_MILLIS |          |                  |
| CACHE_REDIS_POOL_TEST_ON_BORROW                      |          |                  |
| CACHE_REDIS_POOL_TEST_ON_CREATE                      |          |                  |
| CACHE_REDIS_POOL_TEST_ON_RETURN                      |          |                  |
| CACHE_REDIS_POOL_TEST_WHILE_IDLE                     |          |                  |
| CACHE_REDIS_POOL_TIME_BETWEEN_EVICTION_RUNS_MILLIS   |          |                  |

Support for HOCON (typesafe config) backend?

I didn't see an example in the README - is this something that could be added, perhaps as a separate module?

Add conclusive column

Support running from node and reading env

Add ConfigValueParsers for UR{I,L}

Add ConfigValueParser for regex

Migrate to cats by default

I saw the note in the README - opening this as a ticket for people to track if they care.

If you're interested in getting a hand for it, I might be able to find some time.

Reduce common boilerplate

I'm always doing the same thing. Make this easier out-of-the-box:

  def sources: ConfigSources[IO] =
    // Highest pri
    ConfigSource.environment[IO].expandInlineProperties("APP_PROPS").mapKeyQueries(acceptExternalKeyFormat) >
      ConfigSource.propFileOnClasspath[IO]("app.properties", optional = true) >
      ConfigSource.system[IO]
  // Lowest pri

  private def acceptExternalKeyFormat(k: ConfigKey): List[ConfigKey] =
    k :: k.map(_.toUpperCase.replace('.', '_')) :: Nil

  def load: IO[Option[AppConfig]] =
    config
      .withReport
      .run(sources)
      .map { result =>
        result.toEither match {
          case Right((config, report)) =>
            if (config.showUnusedConfig)
              logger.info("Config report\n" + report.full)
            else
              logger.info("Config report\n" + report.sources + "\n\n" + report.used)
            Some(config)
          case Left(err) =>
            logger.error("Invalid config:\n" + err)
            None
        }
      }

Clarify in report that obfuscated values are obfuscated

Poor Tostein...

`.option` is too intense

It will fail if one value is provided but another needed value is not. I want a version that just returns None without failing

Make Found("") distinguishable from NotFound in report

Probably just wrap /^"\s*"$/ in quotes

Exclude unused from report with a boolean

Make this easier

            if (config.showUnusedConfig)
              logger.info("Config report\n" + report.full)
            else
              logger.info("Config report\n" + report.sources + "\n\n" + report.used)

Feedback: Transformation / upgrades.

This seems like a good utility, but it seems far from being a complete solution.

Is there plans to add migrations / upgrades / in-app mutation/setting of config values?

So hard to just exclude sources from reports

val exclude = List(SourceName.environment, SourceName.system)

val configReport = configReport0
  .withUnusedSettings(
    _.withColFilter(_ && ConfigReport.ColFilter.excludeSources(exclude: _*))
      .withRowFilter(_ && ConfigReport.RowFilter((_, m) => m.keys.exists(!exclude.contains(_)))))
      // .withRowFilter(_ && ConfigReport.RowFilter.excludeEmpty)) // TODO Fix microlibs

println(configReport.reportUsed)

// TODO This is a temporary hack - need a way to know if empty after filter-application
if (configReport.reportUnused.length > 23)
  println("Unused config:\n" + configReport.reportUnused)

Obscure by default: credential

Export Report

Create `ConfigDef[Unit]` from `logback.xml`

No more manually doing the following:

ConfigDef.getOrUse[String]("LOG_APPENDER", "DEV") *>
      ConfigDef.getOrUse[String]("LOG_LEVEL_ROOT", "INFO") *>
      ConfigDef.getOrUse[String]("LOG_LEVEL_SQL", "INFO") *>
      ConfigDef.unit

Support list properties

eg:

A_1=a
A_2=b
// becomes
List("a", "b")

A.1.A = 10
A.1.B = 11
A.2.A = 20
A.2.B = 21
// becomes
List( X(10,11), x(20,21))

(Key,Int) => Key?
(Key,Int) => Suffix?

ConfigDef#map ambiguous due to applicative syntax

ConfigParser should just take a String

Look at this travesty:

      val cfg = RestServiceConfig.configParserHostLookup.parse(
        japgolly.clearconfig.internals.Lookup.Found(

Add `ConfigValueParser[UUID]`

Config report doesn't connect name-adjusted keys together

For example:

  val port = ConfigDef.getOrUse[Port]("server.port", port"8085")
  def configSources[F[_]: Applicative]: ConfigSources[F] = {
    // Allow property names like `foo.bar` to map to environment variables like `FOO_BAR`
    // Taken from https://github.com/japgolly/clear-config/issues/132
    def propertyToEnv(k: ConfigKey): List[ConfigKey] =
      k :: k.map(_.toUpperCase.replace('.', '_')) :: Nil

    ConfigSource.environment[F].mapKeyQueries(propertyToEnv) >
      ConfigSource.propFile[F]("deploy.properties", optional = true)
      ConfigSource.system[F]
  }
  
  def readConfig[F[_]: MonadThrow](log: Logger[F]): F[MainConfig] =
    config.withReport
      .run(configSources[F])
      .flatMap(_.toEither.leftMap(e => new Exception(e) with NoStackTrace {}).liftTo[F])
      .flatMap { case (cfg, report) =>
        // Don't bother reporting unused environment variables, or "system" settings
        val reportMessage = report
          .mapUnused(_.withoutSources(ConfigSourceName.environment, ConfigSourceName.system))
          .full

        log
          .info(reportMessage)
          .as(cfg)
      }

Results in a config report like this:

Used keys (5):
+-----------------+------+-----------------------------------------------------------------+---------+
| Key             | Env  | /Users/gavin/blahblah/blahblahblahblahblah/local-dev.properties | Default |
+-----------------+------+-----------------------------------------------------------------+---------+
| SERVER_PORT     | 8089 |                                                                 |         |
| server.port     |      |                                                                 | 8085    |
+-----------------+------+-----------------------------------------------------------------+---------+

Default values are rendered as `(_: A).toString` instead of the string value

Add `ConfigValueParser[URI]`

Restore Config#withCaseInsensitiveKeys

Add ConfigValueParser.oneOf[A](f: (String, A)*)

And provide a sensible error message showing all possible options.

Also have oneOf[A](preProcess: String => String, f: (String, A)*)

Rename `valueDisplay0`

Report.Settings.default.copy(valueDisplay0 = valueDisplay, maxValueLen = Some(100))