aspnet/configuration

[Archived] Interfaces and providers for accessing configuration files. Project moved to https://github.com/aspnet/Extensions

configuration's Issues

Enumeration of distinct children

GIven an IConfiguration, and a prefix, and a delimiter, return the distinct values that occur after the prefix and before the delimiter

Files are opened with improper access mode in all file-based config sources

Add more formats supported by CommandLineConfigurationSource

Previous discussions can be found at #30 .

Hi @lodejard ,what do you think about @Eilon 's follow up in #30 regarding the formats we will support?

Fix naming for Json configuration provider

The name should be Microsoft.AspNet.ConfigurationModel.Json

The extension method should be in the Microsoft.AspNet.ConfigurationModel namespace

Add logic for reading environment variables from Azure WebSites

If the user adds a SQL Server connectionString called "DefaultConnection" in Azure WebSites, the connectionString flows as environment variable "SQLAZURECONNSTR_DefaultConnection".
The config source for environment variables should be able to read this Key "DefaultConnection" correctly

Make all interfaces assembly neutral

Add json config file format

{
   "name" : "test",
   "address": { 
        "street" : "Something street",
        "zipcode": "12345"
   }
}

I should be able to access this via the config API:

var config = new Configuration();
config.AddJsonFile(@"c:\something\myconfig.json");

var name = config.Get("name");
Assert.Equal("test", name);

var address = config.Get("address:street");
Assert.Equal("Something street", address);

parent #13

Create environment and file based configuration sources

Needed for simple user configuration scenarios, these will be the minimum we need to ship.

Share Config files between different projects

Today you need to duplicate your config if your config is shared between a web application project, console project and class library.
Is there a way we can make it easier for you to share configs between different projects in your solution?
A classic case is sharing connectionstrings

IniFileConfigurationSource.Commit() should remove emptied section headers if later we can remove keys

parent #7

1. Do we need Remove(key) operation on Configuration & config sources?
2. When doing Commit(), do we want to remove an empty section header, which is emptied by a Remove(key) operation? For example: call Remove("Foo:Bar") on the following config source

[Foo]
Bar = Value

1. How about section headers that users leave empty deliberately? For example:

[XYZ]
[Foo]
Bar = Value

Tracking: Options PR feedback

1.The default implementation for option accessor and option setups seems to prevent scoped option setups from working. I believe these more dynamic scenarios are important and not necessarily exclusive to data.
2.I think it would be good to have a nice pattern to register option setups that override or not override other option setups, i.e. the issue of specifying the "Order" of the IOptionsSetup. Hao proposed using a flag, which I am ok with trying. Not sure it will make it to this checking though.
3.The implementation of binding to keys from configuration to matching properties on TOptions only work with core and desktop .NET and I haven't got a good answer on whether this can be addressed.

Block access to config files so that they cannot be served on the web

Today Config.json lives in the root of your app. In ASP.NET we used a handler that used to block access to web.config
How do we block access to config.json so it is not servable on the web?

Need a way to store encrypted values in configuration

System.Web supports encrypted configuration as seen in this MSDN article.

We should consider having some solution to this problem in ASP.NET/EF vNext.

Add resource strings for exception messages

Add functional tests for Configuration repo

Changing env variables at runtime would require web server restart

In the current tooling behavior, if there is an addition/update to env variables VS has to be restarted for them to be picked up. In case of runtime, if the env variables are edited, should the web server be restarted? If yes, can we consider not doing this as I feel restarting just for a change in env variable might have implications

Environment source needs prefix value

A given prefix value would be required for the environment variable to match. Removed from actual key that is requested from configuration.

e.g: Configuration.Add(new EnvironmentSource(prefix));

Config sources convert dots to colons in config keys

Not sure why we do this kind of normalization. It took quite a while to debug this when porting a customer application from MVC 5 using web.config to using MVC 6 with config.json.

Repro:

Have this config.json file:

{
    "foo.bar":  "foo.bar value"
}

And this MVC controller:

public IActionResult Index()
{
    var config = new Configuration().AddJsonFile("config.json");
    var configData = new StringBuilder();
    foreach (var key in new[] { "foo.bar", "foo:bar" })
    {
        configData.AppendFormat("[{0}] = {1}<br/>", key, config[key]);
    }
    return Content(configData.ToString());
}

And run it.

Note that the foo.bar key has no value, whereas the foo:bar key does (even though that's nowhere in my config file). Very confusing.

Review Options ReadProperties error handling

Currently we just silently ignore any errors

Poco Deserialization?

Should config be able to deserialize a poco stored in a config file?

I have gotten around it with the following:

        private static T Apply<T>(T model, string prefix, IConfiguration config) {
            var keys = config.GetSubKeys(prefix);
            foreach (var k in keys) {
                ChangeValue(model, k.Key, config.Get(Setting(prefix, k.Key)));
            }

            return model;
        }

        private static string Setting(params string[] parts) {
            return string.Join(":", parts);
        }

        private static bool ChangeValue(object model, string propertyName, object value) {
            var t = model.GetType();
            try {
                t.GetProperty(propertyName).SetValue(model, value);
            } catch (Exception ex) {
                //throw new InvalidOperationException("Cannot set property: " + propertyName, ex);
                return false;
            }
            return true;
        }

So I think yes, yes it should but before I fork, branch, and PR I figured I should post an issue and see what the feedback is.

Browsing to project k website deployed to azure shows error

I deployed bug tracker to azure through FTP and I see following error

Helios failed to start.  

The ApplicationStart method threw an exception. Detailed exception information is below. 

System.ArgumentException: An item with the same key has already been added. 
at System.ThrowHelper.ThrowArgumentException(ExceptionResource resource) 
at System.Collections.Generic.Dictionary`2.Insert(TKey key, TValue value, Boolean add) 
at System.Linq.Enumerable.ToDictionary[TSource,TKey,TElement](IEnumerable`1 source, Func`2 keySelector, Func`2 elementSelector, IEqualityComparer`1 comparer) 
at Microsoft.AspNet.ConfigurationModel.Sources.EnvironmentVariablesConfigurationSource.Load(IDictionary envVariables) 
at Microsoft.AspNet.ConfigurationModel.Sources.EnvironmentVariablesConfigurationSource.Load() 
at Microsoft.AspNet.ConfigurationModel.Configuration.Add(IConfigurationSource configurationSource) 
at Microsoft.AspNet.ConfigurationModel.ConfigurationExtensions.AddEnvironmentVariables(IConfigurationSourceContainer configuration) 
at Microsoft.AspNet.Loader.IIS.KlrHttpApplication.ApplicationStart(IHttpApplication application) in G:\Helios\src\Microsoft.AspNet.Loader.IIS\KlrHttpApplication.cs:line 66 
at Microsoft.AspNet.Loader.IIS.HttpApplicationBase.InvokeApplicationStart(IHttpApplication application) in G:\Helios\src\Microsoft.AspNet.Loader.IIS\HttpApplicationBase.cs:line 27 

<!---
@huboard:{"order":6.1875,"custom_state":"ready"}
-->

Configuration no-ops with no sources

Setting a value in configuration with no sources has no effect. If this is by design then consider throwing instead of doing nothing.

var configuration = new Configuration();
configuration.Set("Hello", "World");

// Nope
configuration.Get("Hello")

Move PathResolver elsewhere

This is generally useful code that should go elsewhere

IniFileConfigurationSource.Commit() cannot handle empty value

Assume we have an ini file config.ini

; No value for this key
DefaultKey=

Then we do

var config = new IniFileConfigurationSource("config.ini");
config.Set("DefaultKey", "Value");
config.Commit();

The last line throws an exception because empty value is not handled properly. The expected behavior is that config.ini file is updated to

; No value for this key
DefaultKey=Value

There is no way to reference a compiled .dll

The current KRE that works with the CTP only references the first .dll in a nuget package. This has been fixed but as a workaround I decided to just add a direct reference to the .dll and it appears that there is no way to reference a compiled .dll using the new project.json

Consider using ITypeActivator for OptionsAccessor

Commit() shoudn't throw when finds newly added key-value pairs in config source

We have a config.ini file

Key=Value

The following code throws an exception saying that a new key is found:

var config = new IniFileConfigurationSource("config.ini");
config.Load();
config.Set("NewKey", "NewValue");  // A new key-value pair is added
config.Commit();  // throws exception

Instead of throwing, Commit() should generate a file with following content:

Key=Value
NewKey=NewValue

It should be true for IniFileConfigurationSource, XmlConfigurationFile and JsonConfigurationFile

Microsoft.Framework.ConfigurationModel.FunctionalTests doesn't run during build

The project.json file needs to include a configurations section. This would be fixed by aspnet/Universe#80

Design and implement key retrieval based on a prefix

Configuration navigation model includes idea of getting an IConfiguration which has its key space prefixed. Subsequent operations all pass to the original root object, but with the prefix prepended to any key strings. Further navigation are all based from the level being operated on.

Load() of file-based configuration source should create the file if it doesn't exist

Some users report that they should be able to do something like

var config = new Configuration();
config.AddIniFile("C:\new_file.ini");
config.Set("key", "value");
config.Commit();

when they want to create a new ini file.
However, when they call Configuration.Add(file), it always tries to load the content of given file. So if C:\new_file.ini doesn't exist, config.AddIniFile("C:\new_file.ini"); throws an exception saying the file doesn't exist and cannot be loaded.

Consider exposing Order defaults for OptionsSetup

Maybe either via constants, or public enum?

IConfiguration should support indexer[]

config[x] = y;
var z = config[x];

Aught to work exactly like

config.Set(x, y);
var z = config.Get(x);

Investigate OptionsAccessor null out setups without creating race condition

Create OptionsModel assembly

Add a new assembly inside of the Configuration Repo called Microsoft.Framework.OptionsModel and move all of the options into this assembly
• Hosting would depend on this assembly to add OptionsServices.DefaultServices as part of app.UseServices() (lifetime of open generic is singleton)
• Add a new extension method T Get where T : IConvertable to Configuration
• Add ConfigOptionsSetup which takes IConfiguration and is registered as an open generic
• Add ConfigSetup method which registers an IConfigOptionsSetup
• Add static helper method Bind(IConfiguration, object) to OptionsServices
• Keep default options orders as internal constants for now (Configuration = -1000, Default = 0)
• Support recursive binding so IdentityOptions can just use ConfigOptions directly (do not support lists/dictionaries for now, but its possible to add later)

Add more file formats for configuration

We should add a few more formats as separate projects

• Microsoft.AspNet.Configuration.Json
• Microsofot.AspNet.Configuration.Xml

YAML Support

In addition to INI, XML & JSON, we should consider supporting YAML files.

Make IConfiguration interface assembly neutral

crypting part of the configuration

Hi,
the actual configuration architecture is interesting but so far I see that a interesting Feature of the actual asp.net Runtime is missing, the feature to crypt part of the configuration.
this Feature is quite useful,. because sometimes I need to place in the configuration sensible data, as POS authentication credentials, but I would like to make safe the data in the case of unauthorized access.
How can I handle this?

Keys should be case insensitive

In config reading the Keys should be case insensitive.

config.json should support /* */ and // comments

Currently config.json supports /* */ but not // comments. It should support both. We've heard from customers already who have tried to use // and run into issues.

config.json

{
    "Data": {
        "DefaultConnection": { 
            "ConnectionString": "Server=(localdb)\\mssqllocaldb;Database=aspnetvnext-23e8c92e-86d5-4b83-8f2a-0b3cf1df7f6a;Trusted_Connection=True;MultipleActiveResultSets=true" 
        }
    }

    /*
        Some comment here
        */

    // some other comment here
}

Here is the exception

Helios failed to start.

The ApplicationStart method threw an exception. Detailed exception information is below.

System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> Newtonsoft.Json.JsonReaderException: Error parsing comment. Expected: *, got /. Path 'Data', line 12, position 6.
at Newtonsoft.Json.JsonTextReader.ParseComment()
at Newtonsoft.Json.JsonTextReader.ParsePostValue()
at Newtonsoft.Json.JsonTextReader.ReadInternal()
at Newtonsoft.Json.JsonTextReader.Read()
at Microsoft.Framework.ConfigurationModel.JsonConfigurationSource.SkipComments(JsonReader reader)
at Microsoft.Framework.ConfigurationModel.JsonConfigurationSource.Load(Stream stream)
at Microsoft.Framework.ConfigurationModel.JsonConfigurationSource.Load()
at Microsoft.Framework.ConfigurationModel.Configuration.Add(IConfigurationSource configurationSource)
at Microsoft.Framework.ConfigurationModel.JsonConfigurationExtension.AddJsonFile(IConfigurationSourceContainer configuration, String path)
at WebApplication2.Startup.Configure(IBuilder app) in d:\users\sayedha\documents\visual studio 14\Projects\WebApplication2\WebApplication2\Startup.cs:line 24
--- End of inner exception stack trace ---
at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor)
at System.Reflection.RuntimeMethodInfo.UnsafeInvokeInternal(Object obj, Object[] parameters, Object[] arguments)
at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
at System.Reflection.MethodBase.Invoke(Object obj, Object[] parameters)
at Microsoft.AspNet.Hosting.Startup.StartupLoader.<>c__DisplayClass0.b__4(IBuilder builder)
at Microsoft.AspNet.Loader.IIS.KlrHttpApplication.HeliosStartupLoaderProvider.WrappingStartupLoader.<>c__DisplayClass0.b__2(IBuilder builder)
at Microsoft.AspNet.Hosting.HostingEngine.EnsureApplicationDelegate(HostingContext context)
at Microsoft.AspNet.Hosting.HostingEngine.Start(HostingContext context)
at Microsoft.AspNet.Loader.IIS.KlrHttpApplication.ApplicationStart(IHttpApplication application)
at Microsoft.AspNet.Loader.IIS.HttpApplicationBase.InvokeApplicationStart(IHttpApplication application)

Configuration model binder

The ability to map a strongly typed model from a configuration.

Duplicate key exception when trying to override existing command line option

The scenario here is that I have a project.json with a command that specifies a webserver:

{
    "dependencies": {
        "Microsoft.AspNet.Http": "0.1-alpha-*",
        "Nowin.vNext": ""
    },
    "commands": {
        "web": "Microsoft.AspNet.Hosting --server Nowin.vNext"
    }
}

Now I want to be able to override that --server value without editing the file and I don't want to use an environment variable.

k web --server firefly

Fails with the following exception:

System.FormatException: A duplicate key 'server' was found.
   at Microsoft.Framework.ConfigurationModel.CommandLineConfigurationSource.Load()
   at Microsoft.Framework.ConfigurationModel.Configuration.Add(IConfigurationSource configurationSource)
   at Microsoft.Framework.ConfigurationModel.ConfigurationExtensions.AddCommandLine(IConfigurationSourceContainer config
uration, String[] args)

I think this should just override the previous option instead of exploding.

Changes to config files should not require an apprestart for changes to be picked up

Changes to the Config.json file don’t getting picked up automatically and one needs to restart application explicitly. We should consider adding support for this

IConfiguration change notification

Some sources may observe alterations to the configuration and raise a notification that key values of changed. I could then reconcile the changes with the values I have been using.

Add xml configuration file format

parent #13

Commit() shoudn't throw when finds newly added key-value pairs in original file

See the bug described in #90
If we think it in the other way, we find another possible bug:
We have a config.ini file

Key=Value

Then we do

var config = new IniFileConfigurationSource("config.ini");
config.Load();
config.Set("Key", "NewValue");  // Update the value

Before we commit the change back to file, config.ini file is externally modified to

Key=Value
Key2=Value2

Then if we do config.Commit(), current implementation throws an exception saying that a new key is found in original config file.
Another possible approach is, instead of throwing, Commit() should generate a file with following content:

Key=NewValue
Key2=Value2

That is, although Key2=Value2 doesn't exist in memory (config source), because it appears in original config file, we preserve it in the result file. It gives developers more flexibility.

The drawback of approach above is: the result file of Commit() is not an exact copy of what you have in memory and it might be counterintuitive.

@Eilon @loudej which approach do you guys prefer?

VS is not monitoring config json file change, we might need a web.config behavior.

In old web way, if web.config changed, IIS will automatically detect the changes and restart the website.

With current Dev14 CTP behavior, if I change a setting file that has my settings, I need to restart everything during development, since VS is not monitoring the change. This is not as convinient as web.config change detection, and will cause extra step to restart server if hosting the web on a server.

I think we need to create a default setting file name and monitor it so if it changes, we'll restart the web hosting.

This is originally posted in tooling: aspnet/Tooling#78 (comment), David Fowler suggested to post here.

May relate to #6

Design and implement IConfiguration.Set() and Commit()

Does Set(value); call set on every IConfiguration or just the first writable one?

Calling Set changes a value without committing until Save is called.

".AddCommandLine" should probably require string[] args to be passed in

I'm not sure if going to the process and reading all the args will work well. It will probably pick up things that were passed to klr and apphost that shouldn't be seen as app switches.

[Feature request] Add a TryGet<type> method that returns type instead of string

The TryGet method returns on type string only. I believe the point of having a TryGet is to get the value without throwing an exception. This method can be extended to return strongly typed values instead of string. It can return null in case the value cannot be cast.