eugeny/reconfigure

Config-file-to-Python mapping library (ORM).

Reconfigure - Python object mapping for config files

Browse API on SourceGraph

Quickstart

>>> from reconfigure.configs import FSTabConfig 
>>> from reconfigure.items.fstab import FilesystemData
>>> 
>>> config = FSTabConfig(path='/etc/fstab')
>>> config.load()
>>> print config.tree
{
    "filesystems": [
        {
            "passno": "0", 
            "device": "proc", 
            "mountpoint": "/proc", 
            "freq": "0", 
            "type": "proc", 
            "options": "nodev,noexec,nosuid"
        }, 
        {
            "passno": "1", 
            "device": "UUID=dfccef1e-d46c-45b8-969d-51391898c55e", 
            "mountpoint": "/", 
            "freq": "0", 
            "type": "ext4", 
            "options": "errors=remount-ro"
        }
    ]
}
>>> tmpfs = FilesystemData()
>>> tmpfs.mountpoint = '/srv/cache'
>>> tmpfs.type = 'tmpfs'
>>> tmpfs.device = 'none'
>>> config.tree.filesystems.append(tmpfs)
>>> config.save()
>>> quit()
$ cat /etc/fstab
proc    /proc   proc    nodev,noexec,nosuid     0       0
UUID=dfccef1e-d46c-45b8-969d-51391898c55e / ext4 errors=remount-ro 0 1
none    /srv/cache      tmpfs   none    0       0

This is actually a shortcut to:

>>> from reconfigure.parsers import SSVParser
>>> from reconfigure.builders import BoundBuilder
>>> from reconfigure.items.fstab import FSTabData
>>> content = open('/etc/fstab').read()
>>> syntax_tree = SSVParser().parse(content)
>>> syntax_tree
<reconfigure.nodes.RootNode object at 0x7f1319eeec50>
>>> print syntax_tree
(None)
        (line)
                (token)
                        value = proc
                (token)
                        value = /proc
                (token)
                        value = proc
                (token)
                        value = nodev,noexec,nosuid
                (token)
                        value = 0
                (token)
                        value = 0
        (line)
                (token)
                        value = UUID=83810b56-ef4b-44de-85c8-58dc589aef48
                (token)
                        value = /
                (token)
                        value = ext4
                (token)
                        value = errors=remount-ro
                (token)
                        value = 0
                (token)
                        value = 1

>>> builder = BoundBuilder(FSTabData)
>>> data_tree = builder.build(syntax_tree)
>>> print data_tree
{
    "filesystems": [
        {
            "passno": "0", 
            "device": "proc", 
            "mountpoint": "/proc", 
            "freq": "0", 
            "type": "proc", 
            "options": "nodev,noexec,nosuid"
        }, 
        {
            "passno": "1", 
            "device": "UUID=83810b56-ef4b-44de-85c8-58dc589aef48", 
            "mountpoint": "/", 
            "freq": "0", 
            "type": "ext4", 
            "options": "errors=remount-ro"
        }
    ]
}

Parsers and builders can be paired in almost any possible combination.

Reconfigure can be easily extended with your own parsers and builders - read the docs!

Supported configs:

• Ajenti (ajenti)
• BIND9 DNS (bind9)
• Crontabs (crontab)
• Samba CTDB (ctdb)
• ISC DHCPD / uDHCPD (dhcpd)
• NFS /etc/exports (exports)
• /etc/fstab (fstab)
• /etc/group (group)
• /etc/hosts (hosts)
• iptables-save dump (iptables)
• Netatalk afp.conf (netatalk)
• NSD DNS (nsd)
• /etc/passwd (passwd)
• /etc/resolv.conf (resolv)
• Samba (samba)
• Squid 3 (squid)
• Supervisord (supervisor)

Included parsers:

• BIND9 config (bind9)
• Crontab (crontab)
• NFS Exports (exports)
• .ini (ini)
• iptables-save (iptables)
• nginx-like (nginx)
• squid (squid)
• nsd (nsd)
• CSV-like space-separated values (ssv)
• JSON (jsonparser)

reconfigure's Issues

Missing LICENSE file

What license is this package falls under?

GitHub lacks any tags after 0.1.39

Please push the tags :)

Thanks!

has some doc about resolv.ResolvConfig?

has some doc about resolv.ResolvConfig,i found no docs about the nameserver setting ,someone can help?

Enable Sourcegraph

README.rst links to Sourcegraph, but it isn't enabled for this repo. You can enable it at
https://sourcegraph.com/github.com/Eugeny/reconfigure.

Need license information

Hi,

I'm trying to package ajenti and reconfigure for Mageia, but there doesn't seem to be a license for reconfigure. Can you update pypi and github with the license that this is distributed under. Thanks.

Suggestion

Use https://github.com/LavirtheWhiolet/self-bootstrap.

Support nginx multiline directive

Right now, directive such as :

    gzip_types
        application/atom+xml
        application/javascript
        application/json
        application/ld+json
        application/manifest+json
        application/rss+xml
        application/vnd.geo+json
        application/vnd.ms-fontobject
        application/x-font-ttf
        application/x-web-app-manifest+json
        application/xhtml+xml
        application/xml
        application/xml+rss
        font/opentype
        image/bmp
        image/svg+xml
        image/x-icon
        text/cache-manifest
        text/css
        text/event-stream
        text/javascript
        text/plain
        text/vcard
        text/vnd.rim.location.xloc
        text/vtt
        text/xml
        text/x-component
        text/x-cross-domain-policy;

makes reconfigure fail on parsing.
This is valid syntax (even if I'm unable to find an official description of the syntax), so it should be parsed as such.

I didn't find (yet?) a clean way to parse it with the current implementation.

Python 3 support

Here's a patch to improve Python3 support: python3.patch

Javascript version?

Awesome idea :-) Is there a Javascript version or you plan to port it? It would be useful for NodeOS :-)

Bind9 missing new 'allow-query-on' keyword parser

Bind 9.11 introduced a new keyword allow-query-on that looks like:

allow-query-on { 
 external_bastion_ip_acl; // that public DNS server
 external_downstream_nameservers_acl;
 };

It resulted in a parse error of:

Traceback (most recent call last):
  File "./bind.py", line 24, in <module>
    nodes = includer.compose(NAMED_CONF, node_tree)
  File "/home/steve/.local/lib/python3.5/site-packages/reconfigure/includers/auto.py", line 13, in compose
    self.compose_rec(origin, origin, tree)
  File "/home/steve/.local/lib/python3.5/site-packages/reconfigure/includers/auto.py", line 36, in compose_rec
    subtree = self.parser.parse(content)
  File "/home/steve/.local/lib/python3.5/site-packages/reconfigure/parsers/nginx.py", line 26, in parse
    raise Exception('Invalid tokens: %s. Tokens: %s' % (remainder, tokens))
Exception: Invalid tokens: allow-query-on { 
 external_bastion_ip_acl; // that public DNS server
 external_downstream_nameservers_acl;
 };

Let me know what I can do.

Infinite loop when encountering exclamation mark symbol firstly for BIND9Parser

When encountering the following content:

    allow-transfer {
        trusted_residential_network_acl; // allow XFER from homenet

        external_downstream_nameservers_acl;
        key ddns-sha256-arca-a-key; // TODO tighten this down

        !{ !localhost; any; };
        key master-to-slave-key; // only localhost can use key
        localhost; // not so useful for unsecured RNDC uses
        };

The infinite loop resides at the line:

    !{ !localhost; any; };

Apparently, the exclamation mark is tripping that up when pressing Ctrl-C, as traceback shows:

(Pdb) bt
  /usr/lib/python3.5/runpy.py(193)_run_module_as_main()
-> "__main__", mod_spec)
  /usr/lib/python3.5/runpy.py(85)_run_code()
-> exec(code, run_globals)
  /usr/lib/python3.5/pdb.py(1692)<module>()
-> pdb.main()
  /usr/lib/python3.5/pdb.py(1665)main()
-> pdb._runscript(mainpyfile)
  /usr/lib/python3.5/pdb.py(1546)_runscript()
-> self.run(statement)
  /usr/lib/python3.5/bdb.py(431)run()
-> exec(cmd, globals, locals)
  <string>(1)<module>()
  /home/steve/work/python/reconfigure/test/bind.py(4)<module>()
-> import reconfigure
  /home/steve/.local/lib/python3.5/site-packages/reconfigure-0.1.81-py3.5.egg/reconfigure/includers/auto.py(13)compose()
-> self.compose_rec(origin, origin, tree)
  /home/steve/.local/lib/python3.5/site-packages/reconfigure-0.1.81-py3.5.egg/reconfigure/includers/auto.py(38)compose_rec()
-> self.compose_rec(root, file, subtree)
  /home/steve/.local/lib/python3.5/site-packages/reconfigure-0.1.81-py3.5.egg/reconfigure/includers/auto.py(38)compose_rec()
-> self.compose_rec(root, file, subtree)
  /home/steve/.local/lib/python3.5/site-packages/reconfigure-0.1.81-py3.5.egg/reconfigure/includers/auto.py(20)compose_rec()
-> self.compose_rec(root, origin, child)
  /home/steve/.local/lib/python3.5/site-packages/reconfigure-0.1.81-py3.5.egg/reconfigure/includers/auto.py(36)compose_rec()
-> subtree = self.parser.parse(content)
> /home/steve/.local/lib/python3.5/site-packages/reconfigure-0.1.81-py3.5.egg/reconfigure/parsers/nginx.py(24)parse()
-> tokens, remainder = scanner.scan(' '.join(filter(None, content.split(' '))))
  /usr/lib/python3.5/re.py(367)scan()
-> m = match()

The loop is not stuck in the NginxParser(BaseParser), but in the following function:

tokens, remainder = scanner.scan(' '.join(filter(None, content.split(' '))))

According to the traceback, it is stuck in match():

/usr/lib/python3.5/re.py(367)scan()

Probably a bad regex. I realized that NginxParser is being used. I do think that tokens can be enhanced to allow exclamation mark.

reconfigure/reconfigure/parsers/nginx.py

Line 24 in 2b8729a

tokens, remainder = scanner.scan(' '.join(filter(None, content.split(' '))))

Two fails on openSUSE in CrontabConfigTest

I have this packaged for openSUSE (https://build.opensuse.org/package/show/home:jayvdb:py-new/python-reconfigure), but there are two odd fails, and it would be nice to fix them if possible before submitting to the main package collection. Recording them now. I'll dig into them later, but would be most grateful if there are any hints obvious to other users that I havent seen yet.

It occurs locally and in RPM created VM. Occurs on Python 2.7 and Python 3.8.

/usr/bin/python3 -m nose
.....F.........................F..................
======================================================================
FAIL: test_config (reconfigure.tests.configs.crontab_tests.CrontabConfigTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/abuild/rpmbuild/BUILD/reconfigure-0.1.82/reconfigure/tests/configs/base_test.py", line 35, in test_config
    self.assertEquals(
AssertionError: Lists differ: ['#co[69 chars]l', '1', '*', '0', '1', '2', 'date', '-s', 'NAME', '=', 'TEST'] != ['#co[69 chars]l', '1', '*', '0', '1', '2', 'date', '-s', 'NAME=TEST']

First differing element 18:
'NAME'
'NAME=TEST'

First list contains 2 additional elements.
First extra element 19:
'='

  ['#comment',
   'line',
   '*',
   '*',
   '*',
   '*',
   '*',
   'date',
   '@reboot',
   'ls',
   '-al',
   '1',
   '*',
   '0',
   '1',
   '2',
   'date',
   '-s',
-  'NAME',
-  '=',
-  'TEST']
+  'NAME=TEST']
?   +++++


======================================================================
FAIL: test_stringify (reconfigure.tests.parsers.crontab_tests.CrontabParserTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/abuild/rpmbuild/BUILD/reconfigure-0.1.82/reconfigure/tests/parsers/base_test.py", line 30, in test_stringify
    self.assertEquals(a.split(), b.split())
AssertionError: Lists differ: ['#co[69 chars]l', '1', '*', '0', '1', '2', 'date', '-s', 'NAME', '=', 'TEST'] != ['#co[69 chars]l', '1', '*', '0', '1', '2', 'date', '-s', 'NAME=TEST']

First differing element 18:
'NAME'
'NAME=TEST'

First list contains 2 additional elements.
First extra element 19:
'='

  ['#comment',
   'line',
   '*',
   '*',
   '*',
   '*',
   '*',
   'date',
   '@reboot',
   'ls',
   '-al',
   '1',
   '*',
   '0',
   '1',
   '2',
   'date',
   '-s',
-  'NAME',
-  '=',
-  'TEST']
+  'NAME=TEST']
?   +++++

-------------------- >> begin captured stdout << ---------------------
SOURCE: #comment line
* * * * * date
@reboot ls -al
1 * 0 1 2 date -s
NAME = TEST

GENERATED: #comment line
* * * * * date
@reboot ls -al
1 * 0 1 2 date -s
NAME=TEST

--------------------- >> end captured stdout << ----------------------

----------------------------------------------------------------------
Ran 50 tests in 0.241s

FAILED (failures=2)