Giter Site home page Giter Site logo

configremedy's Introduction

ConfigRemedy / Configuratron (TBD)

Continuous Delivery, hosting in the cloud and enterprise software development tease out some interesting problems related to configuration management. Configuratron tries to mitigate them.

Configuratron lets you take back control of your configuration

TODO: Selling points here TODO: Explain the problems

One of the progressive ideas we propose is to differentiate between environment and application configuration variables. We aim to extend what Octopus Deploy already provides. We believe that Octopus Deploy is very well suited to handle and maintain environment variables. Environment variables are very often related to server names, database connection strings etc. Application variables, on the other hand, are variables that change the way to application behaves. E.g. ReplyToAddress, EnableSendingOutsideDomain, UseLegacyCalculation etc. In many cases, you will have different values for settings in different environments.

Goals

  • Easy to install and deploy
  • Easy to use
  • Extendable
  • Flexible
  • Secure
  • TODO: Refine

Inspired by Escape

configremedy's People

Contributors

tnicolaysen avatar evgenyb avatar

Stargazers

Sirar Salih avatar

Watchers

 avatar James Cloos avatar

configremedy's Issues

Installation strategy and Nancy vs Katana

As I see it there are 2 possible ways how user can install Configuratron to his environment.

  1. Install it as WIndows Service
  2. Host it under existing IIS

Both Nancy and Katana can be hosted as self-hosted windows service (for instance with TopShelf) or hosted under IIS.

Technically it's possible to organise projects like

  • core.api
  • web.host
  • self.host

where core.api will contain either apicontrollers or nancy modules and *.host projects will be a placeholders for core.api + infrastructure code.

So I want to challenge us with the following 2 questions:

  1. Will it add any value to allow user to choose if he wants to deploy Configuratron under IIS vs. as Windows Service? Or is it something that will just overcomplicate overall system architecture / installation wizard / testing?
  2. Should we use nancy or katana for api implementation? We should make sort of Pros / Cons analysis and compare those 2 frameworks.

Support OWIN authentication

Windows Authentication

As a user of the Configuratron
Given that it's installed in intranet with active directory
I should be able to get access to the system with my windows identity
So I don't need to type my user and password

SSO - Gmail, Facebook, Twitter

As a user of the Configuratron
I should be able to get access to the system with my gmail/facebook/twitter etc... accounts

Since OWIN pipeline becomes the default, we should consider to use Katana for authentication.

Find out which settings can be turned into constants

As a developer
I want to know which configuration properties are never changed
So that I can turn them into constants
And simplify configuration mangement

Background
Due to long release cycles developers have (been forced to) put more than necessary into configuration files. This has in many cases lead to a situation where large applications have hundreds of configuration properties. With such an amount of run-time configurability, you are in risk of breaking the application by accidentally setting a property value to an value that is not supported. I.e. enum name, arithmetic overflow or wrong file paths.

Details
A simple implementation would be to scan the history of the configuration settings and see when the last change was done. A "suggestion list" can be provided where you list out values that haven't been changed for 12 months. As a bonus, you could get this as a block of code you can copy/paste into your app. E.g.

// ---
private const int PropertyThatHaventChanged = 42;
private const string ApplicationName = "I will never change";
// ---

Wix installer and how to test install packages

Since the end product will be installation package(s), we need to add Wix project(s) and "deploy" packages with Octopus. The interesting aspect here will be how to test packages.

Ideally we should test installation / upgrade process for different combination of the environments:

  • WIn7 32/64 bit
  • WIn8 32/64 bit
  • WindowsServer 2003 32/64 bit
  • WindowsServer 2008 32/64 bit
  • WindowsServer 2012 32/64 bit
    etc...

There are at least 2 options:

  1. Automated tests. That is - provision new VM with specified OS -> install/upgrade package -> run tests that will confirm that installation went well or failed...
  2. Let the community test it :)

Track usage

As a user of the configuration system
I want to know which client consumed a given configuration setting
So that I can find unused settings
And know which application will need updates if I do big changes

Details
If the system propagates events like SettingRead (settingKey, application, environment, clientIdentity), a designated component could populate a usage property on the application document.

In the UI it would in a simple view just show Last used: 2014-07-08 14:38 by Some App. Clicking this would reveal details about usage. One way to track usage is to list up last usage by client. E.g.

Client ID Last read
Some app 2014-07-08 14:38
Other app 2013-01-03 03:20
Wierd app 2010-02-03 06:32 โš ๏ธ

Installation process

Some issues to discuss:

As part of installation and post-install customisation process we need to get the following information from the user:

  • installation destination folder
  • application port
  • administrator username / password (or just username if running in intranet with AD)

Installation folder will be covered by the installer (either Wix or others). Now, when it comes to port, host, DbPath, LogsPath and administrator credentials there are at least 2 possible ways to handle it:

  1. Web-based management - when application starts for the first time it asks user for all necessary information.
  2. Desktop-based management - we provide management application, where user can customize all settings.

There are pros and cons with both solutions, so I just want us to write our thoughts here.

Web-based management
Pros:

  • no desktop application
  • no need to remotely login to the server
  • less deployment units to maintain/update/patch
  • easy access with web browser from any PC within same network

Cons:

  • additional start-up logic in web application
  • if settings are stored in windows service .config file, changing .config file from service itself sounds like a smell for me...
  • can't get access to management functionality if hosting service is down

Desktop-based management
Pros:

  • separation of responsibilities -> api and web do their job and so does management application
  • event if hosting service is down, admin still can run management app

Cons:

  • admin has to remotely login to the server to get access to Management app
  • additional development unit
  • additional deployment unit

Examples:

TeamCity - web based
Octopus - desktop based

Keep an audit log

As a user of the configuration system
I want to see who changed what
So that I can review this in case of any problem

(WIP)

As an official auditer auditing an enterprise
I want to see proof of what changes has been made to certain configuration properties
So that I can perform an audit

Details

If the system propagates events like SettingCreated( ...), SettingDeleted(...), SettingChanged(...) a component can subscribe to these and keep a proper audit log.

Proposed format for audit log:

public struct SettingHistory
{
    public string Who { get; set; }
    public ChangeType What { get; set; }
    public DateTime When { get; set; }
    public string Where{ get; set; }
    public string From { get; set; }
    public string To { get; set; }
}

[Mockups] - environments page

Show environments as a list
image

User can

  • edit existing environment
  • add application to the environment
  • add setting to the environment level settings

Client library for .NET

As a developer
I want to integrate ConfigRemedy in my codebase
So that I can use it

Idea 1: Reflection-magic

All the user needs to do is create a interface without any implementation, and the library will create a proxy at run-time. This proxy will connect to the REST API and get the keys.

The benefits with this solution is that we get type-support, that it's easy to find usage in code and it's unobtrusive (transparent that there'a a library doing all the hard work).

In order to create such a proxy, we could use a simple factory. I.e.

ConfigRemedy.GetInstance<IApplicationSettings>();

// Should be baked into a IoC-framework like Castle Windsor.
// I imagine it can be registered in the container like so:
container.WithConfigRemedyFacility();
// and then when it tries to resolve a component implementing IProvideConfiguration
// it would create a factory.

The goal is to be as unobtrusive as possible and have all the wiring "outside" of the application.

Here is an example of what one interface could look like:

[ApplicationName("MyFantasticApp")]
public interface IApplicationSettings : IProvideConfiguration
{
  int SomeProperty { get; } 
  string AnotherProperty { get; }

  [Key("differentKey")]
  string NameThatHasMoreMeaning { get; }

  [DateFormat("yyyy-MM-dd")] // same as in .NET (can be avoided if dates are normalized in DB)
  DateTime ADifficultProperty { get; }
}

So, what about special conventions in string names that are not valid C# property name?
You might see properties like <add key="aspnet:FormsAuthReturnUrlVar" value="" />
Well, it's not a problem. It should still be possible to reference such variables by using the attributes shown in the example above, but it should be possible to use it without attributes.
My initial take on this is to support the namespace by replacing : with _ leaving the interface property named something like string Aspnet_FormsAuthReturnUrlVar { get; }

Then, what about <add key="Some.Scoped.Variable-WithWierdChars" value="" />. Again, the fallback to the attribute is viable here, but another method is applying a canonicalization scheme. In the case above, it would match a query for the string Some.Scoped.Variable-WithWierdChars as well as the interface property Some_Scoped_Varialbe_WithWierdChars.
Note: Property names would need to be validated against C#'s rules on what is a valid property name. Based on the language spec, there's a lot of valid choices, but it can probably be simplified to allowing something like the regex I've created on this page.

As a bonus, the user could copy/paste the appSettings section into the web application and get a suggested interface he then can copy/paste back into his code and use immediately. The main effort would be to replace the code that calls ConfigurationManager directly, but hey, search and replace works :)

Idea 2: Shim ConfigurationManager

It might be possible to replace ConfigurationManager.AppSettings Property using Microsoft Fakes or something else.

The benefit with this approach is that you could migrate your existing App.config into ConfigRemedy without doing any code changes. It would work as a drop-in replacement.

The way I imagine it to work is to intercept the "Get" method and redirect the call to the configuration system. You would off-course have two variables present in the appSettings for this to work.
"ConfigRemedy.BaseUrl" = "https://configsystem:8080/"
"ConfigRemedy.Environment" = "DEV"
(These two environment-specific variables should be replace during the deploy process. Octopus Deploy is excellent at handling this kind of job.)

If the client code calls ConfigurationManager.AppSettings["SomeProperty"] it would do a synchronous request to the API and retrieve this value (as well as caching it). Assuming the network is set up properly, this should be a very quick operation. The configuration values already present in the App.config would be ignored, and should be replace to "(set in ConfigRemedy)" or something similar.

This might be a good first step.

A bonus would be to create a feature on the client page which would parse the appSettings section and create those settings.

Integration with Octopus

As a developer
I want to import / synchronise environments and projects between Octopus and Configuratron
So that I can simplify first time initialisation
And simplify maintenance

This is not high priority task, but it will be very practical to integrate with Octopus. It can be as simple as import all / selected environments, projects from Octopus into Configuratron or we can do it more advanced and implement synchronisation routine that will synchronise environments between both.

Lets imagine that both Octopus and Configuratron are up and running and we need to add new environment. Then the logical flow will be as follow:

Add new environment in Octopus -> Import / Synchronise environment to Configuratron

There should be way to setup Octopus API url and API key and specify other parameters like Scope etc...

Command line executor

As a developer
I should be able to execute Configuratron's tasks from command line tool
So it will allow me to script my configuration related tasks

@tnicolaysen:
I've had this idea that the command-line tooling should be dead-simple.
I'm inspired by the way UNIX tools are built. By that I mean piping and composition.
I imagine something like this:

> Diff-Environments -Source DEV -Target PROD -Server http://config-server/ | Update-Environment

In this example, the response from Diff-Environments would be in JSON format and contain the properties that have been added, changed and deleted. The Update-Environment command will then take that as an input and update the target environment. I think the params can be piped to the next commandlet.

Having this philosophy with small specialized commandlets, we enable a very flexible and simple platform. These can be aggregated and simplified over time.

Many of these tools will just invoke an action on the REST API. E.g.GET http://config-server/environments/diff?source=prod&target=prod.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.