rust-cli / confy Goto Github PK
View Code? Open in Web Editor NEW๐ Zero-boilerplate configuration management in Rust
License: Other
๐ Zero-boilerplate configuration management in Rust
License: Other
Thank you for this little, but very useful crate.
#12 fixes a blocker: confy should never panic confronted to erroneous configuration files.
Unfortunately version 0.3.1, the last published on crates.io, comes without bug-fix and panics!
As a result my crate getreu/tp-note: fast note-taking with templates and filename synchronization also panics. This is a blocker for Windows user!
Please release immediately when a bug-fix is available and tested. I tested your last commit 5a58388 and it works well.
Hi,
https://docs.rs/confy/0.3.1/src/confy/lib.rs.html#27 points to the non-existing https://docs.rs/crates/serde
, but should link to the correct page (https://docs.rs/serde) since commit 40d9dfdf266fda310a52bf6c57798f0d71ae742e. I'm not sure what's wrong here, deployment wrong or something. I would have opened a PR but I am not sure there's anything to fix in the docs themselves.
Best,
Lexi
Hello
Unfortunately confy::load(...)
fails with SerializeTomlError(ValueAfterTable)
error if there is something defined after a HashMap
in the configuration struct
. For example, my config struct
is like this does not work:
#[derive(Debug, Serialize, Deserialize)]
struct Config {
title: String,
api_key: String,
updated: String,
centers: std::collections::HashMap<String,String>,
tasks: Vec<String>,
}
I got it working by swapping the order of centers
HashMap and tasks
Vector, but I'd rather keep then in the order... Is there any way to make this happen, or is this problem due to how TOML works?
Is there any plan to make a release? Things like #8 are fixed but not on crates.io yet, which IMHO makes this great library unusable.
confy not working properly in mac os , working fine in linux
pub fn load_path_or_else<T, F>(
path: impl AsRef<Path>,
op: F
) -> Result<T, ConfyError>
where
T: DeserializeOwned + Serialize,
F: FnOnce() -> T,
{ }
Add the function so that user can pass a closure, which could return a custom value if the file doesn't exist or is empty.
Just spotted in the documentation for the store
functionality:
let my_cfg = MyConf {};
confy::store(my_cfg)?;
I reckon should be:
let my_cfg = MyConf {};
confy::store("my-app-name", my_cfg)?;
Just a small detail, but certainly worth clearing up!
I've noticed that when confy tries to write to a file path, it overwrites the file in-place:
Lines 281 to 287 in cd69322
In my experience (though not with apps using confy), overwriting files in-place causes problems. One possible issue (for files shared between many apps) is that either other applications will be able to read half-written contents, or the file will be locked and unreadable for the duration of the write. A more serious integrity issue is that if the application terminates during the file write (either from panicking, or from being terminated in taskmgr or Ctrl-C'd), it will result in a corrupted file being written to disk, with both the old and new file data being lost.
As an example of what can go wrong, a Python app I wrote (corrscope) saves config files directly to disk (since I didn't know better at the time), overwriting the previous file. Every few months I get a report from a user who can't open the app because the config file is corrupt and fails to load.
Confy's store_path()
first opens the file, then calls either toml::to_string_pretty
or serde_yaml::to_string
(both of which are fallible and return Result<...>
), and use the ?
operator on the Result
. If the function called fails, then store_path()
exits, but the file on the disk has already been truncated and the damage is done. (I'm not sure if TOML or YAML serialization is likely to fail, either reliably or on edge cases, in unfinished programs on developer machines or in released applications on user machines.)
In corrscope, I suspect a similar event can occur if the serializer is left in a corrupted state. (I use a persistent global YAML serializer object for both documents and settings, since I hadn't learned Rust at the time, which would've taught me to avoid shared mutability.)
On top of this, there's the risk of getting terminated or Ctrl-C'd during the serialization operation (or if the serialization hangs and the app needs to be killed). I'm not sure if a system crash after the app finishes writing can lead to corruption, in various filesystems and journaling modes.
A common strategy is to write to a temporary file and rename it on top of the original. While imperfect (may not always preserve permissions or owners), it's the least bad strategy I'm aware of for saving generic textual config files. (An alternative, advocated by danluu, is to not rename on top of the original file, but use an external log file for crash recovery. However, this introduces complexity in the app, and is unworkable if other apps expect plaintext config files and don't understand how to recover from log files.)
Additionally if you want to recover from system crashes as well, you need to fsync the temporary file before renaming it to the original filename. If you don't do so, it's possible that the temporary file gets renamed and overwrites the original file (and the rename gets flushed to disk), but the system crashes before the temporary file's contents are written to disk.
Yet another level of protection is fsyncing the directory to ensure that if the system crashes after renaming the temporary file over the old file, the rename can't be reverted. However this is slower than not fsyncing the directory, is not necessary to prevent corruption on Linux, and is not possible on Windows.
https://github.com/untitaker/rust-atomicwrites is a crate that performs atomic file writing. However it performs a directory fsync on Linux, which I think is unnecessary, slow, and inconsistent between platforms. I would rewrite or vendor the crate without performing a directory fsync.
Is there any chance to create a load_or_die feature such that application don't start if not able to find configuration file? I'd be happy to implement this given the opportunity.
We should be on the latest stable Rust edition, currently we are on 2018 when 2021 is available.
Hello,
The repository contains one LICENSE file but the Cargo.toml file lists both MIT/X11 and Apache-2.0
Could you please clarify the licensing?
I see a lot of Rust Crates being stuck in the 0.X release cycle.
What is missing before it can be declared a version 1.0.0?
Do you need community help to complete it?
Just a question.
Hi! Let's release 0.5.0!
Confy 0.3.1
Steps:
key: 'value
otherkey: 'value'
let cfg: Config = match confy::load("appname") {
Ok(_) => {
println!("Done!");
std::process::exit(exitcode::OK);
}
Err(e) => {
eprintln!("Error: {}", e);
std::process::exit(exitcode::CONFIG);
}
}
Expected:
The program exits cleanly with exitcode::CONFIG error code, after printing Error: etc
.
Actual Result:
`thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Error { inner: ErrorInner { kind: NewlineInString, line: Some(0), col: 14, message: "", key: [] } }', libcore/result.rs:1009:5
stack backtrace:
0: std::sys::unix::backtrace::tracing::imp::unwind_backtrace
1: std::sys_common::backtrace::_print
2: std::panicking::default_hook::{{closure}}
3: std::panicking::default_hook
4: std::panicking::rust_panic_with_hook
5: std::panicking::continue_panic_fmt
6: rust_begin_unwind
7: core::panicking::panic_fmt
8: core::result::unwrap_failed
9: confy::load
10: ren::main
11: std::rt::lang_start::{{closure}}
12: std::panicking::try::do_call
13: __rust_maybe_catch_panic
14: std::rt::lang_start_internal
15: main
16: __libc_start_main
17: _start
For a crate like this, I think it's super important to lay out goals and non-goals. Let's figure out what they are!
Some sources of inspiration for goals and non-goals:
Since 0.4.0 it seems an old error has come back whereby if a configuration file is missing a field it gives an Error during load_config.
confy::load("name")
crashes withBadConfigDirectoryStr
Version 0.4.0
(The Discord chat room doesn't seem to load, so I'm asking here)
In the design philosophy of confy, is the developer allowed (or encouraged) to edit the config file manually?
I'm asking because based on the documentation, the creation of a new config file is done by:
Default
trait on the config structEditing the config is done by confy::store()
. So the creation and update of the config are done through Rust code.
So if I understand correctly, this crate is used to persist configurations between different executions. But not as a config file solution similar to dotenv or rc files (e.g. .bashrc
, .vimrc
), where people write a configuration file manually so they can override the program's behavior without re-compiling the code?
If the user is encouraged to directly edit the config file instead of using the confy::store()
function, then I believe we need to put more thoughts into how a first-time user can create a new config file. Maybe the developer needs to implement a separate "first-time configuration wizard" CLI that help the user choose the configuration? It would be great to clarify the common use cases and patterns in the documentation besides the two core APIs.
In my properties i have the following enum for a datetime. Which uses camelcase
pub enum DateTimeFormat {
Rfc3339,
Rfc2822,
Epochms
}
My application toml file sets the property as such
datetime_format = 'Rfc3339'
If I use non cap
datetime_format = 'rfc3339'
it fails at
confy::load_path(inpath)
Is there an acceptable way of rectifying this in a deserializer
It feels weird that I should need to implement a feature with my CLI application to display the path to the configuration file, when all I need to know is the default location for that file to delete it locally for testing. I get it, help oneself, but you could save a lot of people a lot of searching / unnecessary code by just making a small table in the readme for the main OSes supported in Rust (windows, macos, linux, wasm?).
Note this isn't a judgement on the quality of this project, so please realize I just want it to be more accessible to folks newer to using Rust for CLI development.
Hello,
I've recently come across this crate and am quite a fan! With that being said, during my use of the crate, I discovered that I couldn't use the ?
operator in the context of a function returning Result<(), Box<dyn std::error::Error>>
. This seems to have become a somewhat idiomatic way of dealing with errors in Rust.
The failure
crate, while certainly having its upsides, has had its use in libraries discouraged (due to a lack of stabilization) and the standard library has been gradually incorporating fixes that the crate was intended to provide.
Finally, failure pulls in backtrace
as a dependency, which breaks cross compilation from Linux -> macOS, which is what has brought me here. I'd suggest replacing the failure
crate with the std::error
implementation and would be more than happy to make a pull request.
Let me know what you think!
The general idea behind this feature is the following. Take a configuration file:
my_option = "foo"
and a CLI which also wants to provide the option my_option
which, if provided overrides the config state and if isn't, just uses the default provided by the configuration.
clap.rs
directly, structopt
or thunder
so a more generic approach is generally goodResult is None when test.toml is blank. I wish to get as below, how can I do it?
Some(
Test {
sub: Some(
[
Data {
id: 2,
name: "3",
},
Data {
id: 4,
name: "5",
},
],
),
},
)
my code is here:
testconfig.rs:
use serde::{Deserialize, Serialize};
#[derive(Debug, Serialize, Deserialize)]
pub struct Data {
pub id: usize,
pub name: String,
}
impl Default for Data {
fn default() -> Self {
Self {
id: 1,
name: "test1".to_string(),
}
}
}
#[derive(Debug, Serialize, Deserialize)]
pub struct Test {
#[allow(dead_code)]
pub sub: Option<Vec<Data>>,
}
impl Default for Test {
fn default() -> Self {
Self { sub: Some(vec![Data{id:2,name: "3".to_string()}, Data{id:4, name: "5".to_string()}]) }
}
}
#[derive(Debug, Serialize, Deserialize)]
pub struct TestGlobalConfig {
#[allow(dead_code)]
pub test: Option<Test>,
}
impl Default for TestGlobalConfig {
fn default() -> Self {
Self { test: Some(Test::default()) }
}
}
main.rs:
use std::sync::RwLock;
pub mod testconfig;
use crate::testconfig::TestGlobalConfig;
lazy_static::lazy_static! {
static ref GLOBAL: RwLock<TestGlobalConfig> ={
let path = concat!(env!("CARGO_MANIFEST_DIR"),"/test.toml");
if let Ok(global_config) = confy::load_path::<TestGlobalConfig>(path) {
RwLock::new(global_config)
} else {
panic!("load config file {path:?} error!");
}
};
}
fn main() {
let binding = GLOBAL.read().unwrap();
let settings = &binding.test;
println!("{:#?}", settings);
}
Please release more often. Even a few smaller improvements can make a big difference, according to what the user needs.
For example: I am desperately waiting for
pub fn get_configuration_file_path(name: &str) -> Result<PathBuf, ConfyError>
.
The commit 83c3c22 could be easily shipped as intermediate version.
confy requires serde for serialization and deserialization and the documentaion is not clear about this. Also confy will fail in file is missing my_app
so Default trait was also added.
Add confy and serde to your cargo.toml
confy = "0.4.0"
serde = { version = "1.0", features = ["derive"] }
Add Default trait to generate a file with default values if missing.
use serde::{Serialize, Deserialize};
#[derive(Default, Debug, Serialize, Deserialize)]
struct MyConfig {
name: String,
comfy: bool,
foo: i64,
}
fn main() -> Result<(), io::Error> {
let cfg: MyConfig = confy::load("my_app")?;
println!("{:#?}", cfg);
Ok(())
}
@nyanpasu64 mentioned #48 that we maybe should run CI also on other images than ubuntu as well.
What'd you think?
Comments can make config files more user-friendly by giving hints about possible values and where more info can be found.
When config files get large, sections are useful to break it up so that things are easier to find.
Thanks for making this, confy made my life much much easier today.
Should it be possible to have a global config that is overwritten by a local/ user config, etc. And how would be the best way to implement this?
Suggested by #2 (comment)
Are there any opinions on adding JSON as a serialization format? My project uses JSON because it requires interop between multiple components, and JSON's tooling is very strong and ubiquitous. There's also value in how restrictive the data model is. While I want to use confy, I can't justify adding another serialization format (and all the tooling requirements and conversion ambiguity that come with it) to my project.
I see that YAML is already a config option. Is there desire for adding one for JSON? If so, I would be happy to take it on and make a PR following the same general structure as the YAML feature. I understand there are arguments against JSON as a configuration format however (lack of comments for example) and don't want to jump in adding it if the maintainers have no desire for the feature.
First of all, I love the convenience that this package provides but I ran into a problem that might have a simple solution.
I want to have multiple config files per project. For example, I would like to store/load the following files on Linux:
~/.config/app-name/config1.toml
~/.config/app-name/config2.toml
~/.config/app-name/config3.toml
With confy's current functionality I can only generate a ~/.config/app-name/app-name.toml
file but I'd like to have multiple configs per project.
I've got a quick solution working for me on a fork but it would be nice if this kind of functionality could be in the official package.
The directories crate is maintained again while the crate that was supposed to be the continuation of directories is now unmaintained.
Confy seems to work fine as long as you're adding information to the config file, but as soon as you remove anything from the config, confy breaks. For example, consider the following code:
extern crate confy;
#[macro_use] extern crate serde_derive;
use std::default::{Default};
#[derive(Serialize, Deserialize)]
struct Config {
lines: Vec<i32>
}
impl Default for Config {
fn default() -> Self {
Self {
lines: vec![1, 2, 3]
}
}
}
fn main() -> Result<(), std::io::Error> {
let mut config: Config = confy::load("confy-bug")?;
confy::store("confy-bug", config)
}
If you run the above code you will produce the following output as expected:
lines = [
1,
2,
3,
]
however if you add the following:
fn main() -> Result<(), std::io::Error> {
let mut config: Config = confy::load("confy-bug")?;
+ config.lines.clear();
confy::store("confy-bug", config)
}
You will instead end up with this syntactically incorrect file:
lines = []
1,
2,
3,
]
It seems that confy is just writing on top of the old file without cleaning up any excess lines left over from the last write.
I was able to fix the issue locally by making the following changes:
diff --git a/src/lib.rs b/src/lib.rs
index f468e89..cb3e03b 100644
--- a/src/lib.rs
+++ b/src/lib.rs
@@ -143,7 +143,7 @@ pub fn store<T: Serialize>(name: &str, cfg: T) -> Result<(), IoError> {
].iter()
.collect();
- let mut f = OpenOptions::new().write(true).create(true).open(path)?;
+ let mut f = OpenOptions::new().write(true).create(true).truncate(true).open(path)?;
let s = toml::to_string_pretty(&cfg).unwrap();
f.write_all(s.as_bytes())?;
Ok(())
User should know which should file should they edit to change the program behaviour
Hello! When making changes to my configuration, how can I "migrate" from one configuration structure to another?
mode = "Standard"
address = "127.0.0.1:53"
tor = true
mode = "Standard"
address = "127.0.0.1:53"
[tor]
enabled = true
address = "127.0.0.1:9050"
Here's a repo that contains the basic code: https://github.com/chris9740/confy-migrate
add a new pub fn and remove config
I'm using confy 0.3.1, and it seems that I am unable to remove items from a vec in my config (adding items works fine). But when I was trying to build a simple example to demonstrate the issue, I found a very confusing one.
Here's my code:
use serde::{Deserialize, Serialize};
#[derive(Debug, Serialize, Deserialize)]
struct MyItem {
name: String,
}
#[derive(Debug, Serialize, Deserialize)]
struct MyConfig {
items: Vec<MyItem>,
}
/// `MyConfig` implements `Default`
impl ::std::default::Default for MyConfig {
fn default() -> Self {
Self {
items: vec![MyItem { name: "foo".into() }, MyItem { name: "bar".into() }],
}
}
}
fn main() -> Result<(), ::std::io::Error> {
let mut cfg: MyConfig = confy::load("foobar")?;
println!("{:?}", cfg);
cfg.items.remove(0);
println!("{:?}", cfg);
confy::store("foobar", cfg)?;
Ok(())
}
This is what I get in the config file after running it (any number of times):
[[items]]
name = 'bar'
[[items]]
name = 'bar'
I expected one item after the first run and zero after subsequent runs. And the default vec has two different items (foo
and bar
) whereas the toml file have the same item twice.
Hello, I'm using this crate in one project. The latest version of the crate depends on directories 4.0, but version 5.0.1 is the current version. Any thoughts on updating this dependency and other dependencies and publishing a new version of confy? I can send a PR with the dependency update and handle any API changes that might happen.
The unwrap
call unwrapping toml::from_str
's output in line 105 of /src/lib.rs
(currently in master 7115b65 and 0.3.1 95b1aac) does that:
Ok(mut cfg) => Ok(toml::from_str(&cfg.get_string().unwrap()).unwrap()),
Will it be reasonable to create an enum ConfyError { BadTomlData }
so I could return it in the load function at the point of toml::from_str
's failure?
It would change load's type signature into a more general one or a totally different one if we return both IoError
s and ConfyError
as Error
trait values, I am not sure, so it might not be a good idea.
confy depends on a quite old version of directories.
In Cargo.toml the directories version is specified as
directories = "^2.0"
This is equivalent to "2.0"
which means >=2.0.0, <3.0.0
Please change it to "4.0"
The directories
crate has a breaking change when upgrading to version 3 (specifically macOS change). https://github.com/dirs-dev/directories-rs#3
I did not check more, but I will before making a PR.
I want to start using confy
in my project, but I can't figure out how to include a manually editable configuration file?
From my understanding of the practical example in the readme, it is loading a .toml file named "app-name" out as the values. I try to include a app-name.toml
in the root of my project, but it does not work.
Is it possible to do this?
Here's the output I'm getting:
thread '<unnamed>' panicked at 'Config read error: BadTomlData(Error { inner: ErrorInner { kind: Custom, line: Some(0), col: 0, at: Some(0), message: "missing field
secret_key", key: [] } })', src/utils/config.rs:25:36
From this code:
#[derive(serde::Serialize, serde::Deserialize)]
pub struct Config {
pub secret_key: String,
pub test_attr: u8,
}
impl ::std::default::Default for Config {
fn default() -> Self {
Self {
test_attr: 0,
secret_key: "secret".into(),
}
}
}
pub fn load_config() -> Config {
confy::load("app-name").expect("Config read error")
}
In my project root folder app-name.toml
alongside cargo.toml
:
secret_key = "something"
test_attr = 123
Is this a feature request? I dont know.
Not sure if I'm just being stupid, but it looks like when you add config values after the original toml file has been created it errors with Bad TOML data
. I would expect it to just use the specified default trait and use that instead.
This is more a question/suggestion than an issue/bug report, since I cannot seem to find an explanation on this behaviour.
Assuming there is an AppConfig
struct in place that contains of a handful values and there is a default implementation for those values in place as well. Then this config is being loaded on the very first launch and presumably stored on exit.
When the AppConfig
struct changes, however, and for example a new value was introduced, it looks like confy simply refuses to load the configuration altogether, even though a default for that new value was specified:
Error: BadTomlData(Error { inner: ErrorInner { kind: Custom, line: Some(0), col: 0, at: Some(0), message: "missing field `api_endpoint`", key: [] } })
Would it be possible/make sense to "merge" the existing config and the defaults for value that does not exist in the .toml yet, in order to allow loading the configuration?
This would simplify the main API and allow more configuration options without blowing up the main code body.
Suggested by #2 (review)
I use confy to store float values in toml. It was working fine until recently (a month ago?) instead of storing something like
value = 0.0
it is stored like
value = --0.0
and of course after restart confy returns an error on this value.
I understand that this is probably a bug in the toml crate (?) which has likely got a minor update which breaks confy, but I am not sure how to report the issue there.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.