go-gcfg / gcfg Goto Github PK
View Code? Open in Web Editor NEWread INI-style configuration files into Go structs; supports user-defined types and subsections
Home Page: https://gopkg.in/gcfg.v1
License: Other
read INI-style configuration files into Go structs; supports user-defined types and subsections
Home Page: https://gopkg.in/gcfg.v1
License: Other
The go-git
project uses a forked version gcfg
to parse Git .config
files. We have recently hit a case that was not handled in a way that is compatible with the parser in Git itself - values that use '\' escaping without quotes around it:
[remote "origin"]
url = E:\\Git\\emclient\\emclient.git
I don't know whether that is actually a standard for INI files or not, but in go-git
it has to be handled for compatibility reasons. I have submitted a PR at src-d#1, so you can take a look.
We use this library to parse our config for please
One thing I'm trying to figure out is updating config automatically for our users. For example, if they want to add language support to their project, I want to be able to plz init java
and that will add:
[java]
javac = /opt/java/bin/javac
I was thinking it wouldn't be too crazy if we maintain the original token stream. We can detect any changes/new non-zero entries in the struct, and append these to the first instance of the relevant section (i.e. [java]
).
I just thought I would open up an issue here and see if this is something you'd be interested in having upstream.
package main
import (
"fmt"
"gopkg.in/gcfg.v1"
)
type Config struct {
Settings struct {
Enabled bool
}
}
func main() {
var config Config
err := gcfg.ReadStringInto(&config, "[Settings]\nEnabled = xxx\n")
fmt.Printf("read gave: %#v; %s\n", config, err)
}
$ go run parse.go
read gave: main.Config{Settings:struct { Enabled bool }{Enabled:false}}; failed to parse bool `xxx`
The error message should include information identifying the section and field that caused the error. In this case, maybe:
read gave: main.Config{Settings:struct { Enabled bool }{Enabled:false}}; failed to parse bool `xxx` into Settings.Enabled
Some time ago I forked this project into src-d/gcfg. The goal of the fork was using this library to parse git's config files in src-d/go-git.
The main change is this one: src-d@bdb40b4
It adds a new function to parse a file using a custom callback. This callback can be used to implement deserialization to custom structures or just other kind of deserialization logic.
Is this something that you would consider adding? If that is the case, it'd be great to read your feedback and I'd be willing to make a PR.
LICENSE
file is rather confusing as it mention two different licenses without scope where a particular license applies.
If some files are under the terms of a particular license then it will be best to add relevant copyright headers to all files.
If both licenses apply at the same time then perhaps only BSD-3-clause (most restricted) should be mentioned.
Thanks.
As noted in the error handling documentation, it is reasonable for a programmer error— code that is trying to use gcfg in a way that cannot possibly work, like passing something other than a struct pointer to ReadInto
— to trigger a panic. And the documentation also specifies that the top-level config struct must have fields that are either structs or maps of strings to struct pointers, so declaring some other kind of field there is a programmer error.
However, the way the latter requirement is currently enforced makes the panic data-dependent: that is, you could easily write incorrect code that will be undetected in some cases, and panic in other cases, depending on what's in the config file.
For instance:
type Config struct {
Bar Bar
Baz int
}
type Foo struct {
Field string
}
In this example, the Baz
field is something I should not have put there— at least, it's not valid as far as gcfg is concerned. Perhaps I meant for it to be private and ignored by gcfg, but I accidentally exported it. Or maybe I didn't read the documentation. Either way, assuming the Baz
was not really supposed to be part of the configuration schema, I might normally be running the program with a config file like this—
[Bar]
Field = x
—and it would work just fine. Then one day someone makes a typo in the config file—
[Baz]
Field = x
—and the program panics.
While this is still the programmer's fault, I think gcfg's current behavior— only checking that the type is valid if the section name is actually used— makes it too easy to miss errors like this, and end up with code that is vulnerable to panicking only for certain erroneous inputs.
If it is an absolute rule that all exported fields in the target struct must be either structs or map[string]*struct, then I think it would be reasonable and desirable for gcfg to check the fields ahead of time and immediately panic if they're wrong. I'd like to be able to assume that if I've written my code correctly in this regard, it won't panic for any input data, but if I've made this kind of mistake, it won't not panic just because I happened to test with valid data.
I'm using https://github.com/gonicus/gofaxip which uses this ini-file parser.
If I edit the gofax.conf
file via git config
it's possible that gofaxip doesn't start.
The command I used to edit gofax.conf
was:
git config --file=/etc/gofax.conf freeswitch.header '"cheezburger plz"'
the error that is thrown from https://github.com/gonicus/gofaxip/blob/v1.1-1/gofaxlib/config.go#L68 is:
config.go:68: Config: /etc/gofax.conf:13:11: unquoted '\' must be followed by new line
The configuration file for the Amazon SDK for go contains keys which contain underscores. This is reported as an illegal character by gcfg.
Some documentation on the SDK configuration file is here: https://github.com/aws/aws-sdk-go/wiki/configuring-sdk#creating-the-credentials-file
Is there a way to import ini files which contain underscores?
Hello there,
do you plan to also migrate the feature branch user_vars
from code.google.com to this repository?
I'm currently depending on this feature branch for a project.
A merge of the feature branch to master would be even better! :)
When using a simple section configuration, it is easy to assign default values prior to parsing with gcfg:
type Config struct {
Section struct {
ThatThing bool
}
}
cfg := Config{}
cfg.Section.ThatThing = true
err := gcfg.ReadStringInto(&cfg, "")
This technique is not possible when using subsections, as the value object is constructed during parsing (the subsection names are not known ahead of time).
Workaround: Parse the input twice. Once to identify sub-section names. Use those subsection names to construct a map[string]*SubSectionConfig
with defaults. Then parse the input document a second time using that object with proper defaults.
// example of defaults with subsections
func main() {
CONFIG_DOCUMENT := `
[Rainbows "morning"]
[Rainbows "evening"]
Color = rose
`
type SubSectionConfig struct {
Color string
}
type Config struct {
Rainbows map[string]*SubSectionConfig
}
printRainbows := func(name string, cfg Config) {
for k, v := range cfg.Rainbows {
log.Printf("%s.Rainbows[%s] = '%s'", name, k, v.Color)
}
}
first := Config{}
second := Config{
Rainbows: make(map[string]*SubSectionConfig),
}
gcfg.ReadStringInto(&first, CONFIG_DOCUMENT)
printRainbows("first", first)
for k, _ := range first.Rainbows {
second.Rainbows[k] = &SubSectionConfig{
Color: "black", // default color.
}
}
gcfg.ReadStringInto(&second, CONFIG_DOCUMENT)
printRainbows("second", second)
}
The result of this second parsing uses the default value for sections which do not override its value.
It would be nice if there were some way of initializing subsection structs without relying on this double-parsing workaround.
Fedora Rawhide with Go 1.13 beta 1:
Testing in: /builddir/build/BUILD/gcfg-1.2.3/_build/src
PATH: /builddir/build/BUILD/gcfg-1.2.3/_build/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/sbin
GOPATH: /builddir/build/BUILD/gcfg-1.2.3/_build:/usr/share/gocode
GO111MODULE: off
command: go test -buildmode pie -compiler gc -ldflags "-X gopkg.in/gcfg.v1/version=1.2.3 -extldflags '-Wl,-z,relro -Wl,--as-needed -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld '"
testing: gopkg.in/gcfg.v1
gopkg.in/gcfg.v1
PASS
ok gopkg.in/gcfg.v1 0.013s
gopkg.in/gcfg.v1/scanner
PASS
ok gopkg.in/gcfg.v1/scanner 0.004s
gopkg.in/gcfg.v1/token
PASS
ok gopkg.in/gcfg.v1/token 0.037s
gopkg.in/gcfg.v1/types
--- FAIL: TestParseInt (0.00s)
int_test.go:63: ParseInt(int, "0", IntMode(Dec)): pass; got 0, error <nil>
int_test.go:63: ParseInt(int, "10", IntMode(Dec)): pass; got 10, error <nil>
int_test.go:63: ParseInt(int, "-10", IntMode(Dec)): pass; got -10, error <nil>
int_test.go:63: ParseInt(int, "x", IntMode(Dec)): pass; got 0, error failed to parse "x" as int: expected integer
int_test.go:63: ParseInt(int, "0xa", IntMode(Hex)): pass; got 10, error <nil>
int_test.go:63: ParseInt(int, "a", IntMode(Hex)): pass; got 10, error <nil>
int_test.go:63: ParseInt(int, "10", IntMode(Hex)): pass; got 16, error <nil>
int_test.go:63: ParseInt(int, "-0xa", IntMode(Hex)): pass; got -10, error <nil>
int_test.go:54: ParseInt(int, "0x", IntMode(Hex)): fail; got error failed to parse "0x" as int: strconv.ParseInt: parsing "0x": invalid syntax, want ok
int_test.go:54: ParseInt(int, "-0x", IntMode(Hex)): fail; got error failed to parse "-0x" as int: strconv.ParseInt: parsing "-0x": invalid syntax, want ok
int_test.go:63: ParseInt(int, "-a", IntMode(Hex)): pass; got -10, error <nil>
int_test.go:63: ParseInt(int, "-10", IntMode(Hex)): pass; got -16, error <nil>
int_test.go:63: ParseInt(int, "x", IntMode(Hex)): pass; got 0, error failed to parse "x" as int: expected integer
int_test.go:63: ParseInt(int, "10", IntMode(Oct)): pass; got 8, error <nil>
int_test.go:63: ParseInt(int, "010", IntMode(Oct)): pass; got 8, error <nil>
int_test.go:63: ParseInt(int, "-10", IntMode(Oct)): pass; got -8, error <nil>
int_test.go:63: ParseInt(int, "-010", IntMode(Oct)): pass; got -8, error <nil>
int_test.go:63: ParseInt(int, "10", IntMode(Dec|Hex)): pass; got 10, error <nil>
int_test.go:63: ParseInt(int, "010", IntMode(Dec|Hex)): pass; got 10, error <nil>
int_test.go:63: ParseInt(int, "0x10", IntMode(Dec|Hex)): pass; got 16, error <nil>
int_test.go:63: ParseInt(int, "10", IntMode(Dec|Oct)): pass; got 10, error <nil>
int_test.go:63: ParseInt(int, "010", IntMode(Dec|Oct)): pass; got 8, error <nil>
int_test.go:63: ParseInt(int, "0x10", IntMode(Dec|Oct)): pass; got 0, error failed to parse "0x10" as int: extra characters "x10"
int_test.go:63: ParseInt(int, "10", IntMode(Hex|Oct)): pass; got 0, error ambiguous integer value; must include '0' prefix
int_test.go:63: ParseInt(int, "010", IntMode(Hex|Oct)): pass; got 8, error <nil>
int_test.go:63: ParseInt(int, "0x10", IntMode(Hex|Oct)): pass; got 16, error <nil>
int_test.go:63: ParseInt(int, "10", IntMode(Dec|Hex|Oct)): pass; got 10, error <nil>
int_test.go:63: ParseInt(int, "010", IntMode(Dec|Hex|Oct)): pass; got 8, error <nil>
int_test.go:63: ParseInt(int, "0x10", IntMode(Dec|Hex|Oct)): pass; got 16, error <nil>
--- FAIL: TestScanFully (0.00s)
scan_test.go:32: ScanFully(*int, "a", 'v') = failed to parse "a" as int: expected integer; *ptr==0
scan_test.go:23: ScanFully(*int, "0x", 'v'): want ok, got error failed to parse "0x" as int: strconv.ParseInt: parsing "0x": invalid syntax
scan_test.go:32: ScanFully(*int, "0x", 'd') = failed to parse "0x" as int: extra characters "x"; *ptr==0
FAIL
exit status 1
FAIL gopkg.in/gcfg.v1/types 0.004s
It seems strconv.ParseInt does not accept 0x as valid anymore.
panic: ./config/test.ini:5:4: illegal character U+005F '_'
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.