1and1 / dim Goto Github PK

DNS and IP Management

License: MIT License

Shell 0.25% Dockerfile 0.03% Python 24.78% Makefile 0.47% Java 46.71% HTML 0.17% Perl 16.40% Raku 0.35% JavaScript 1.21% Vue 4.62% CSS 0.03% Less 2.48% SCSS 2.51%

dim's Introduction

DIM - DNS and IP Management (and also DHCP)

DIM can be used as IP Management for a companies whole IP address space (e.g. RFC1918, public IPv4, ULA IPv6, public IPv6 (GUA), Multicast IPs, ...).

DIM can be used to manage all DNS reverse entries for all IP address space.

DIM allows to document subnets with their vlan id and gateway in a way that this information can be reused for automatic IP configuration on devices.

DIM simplifies the steps "mark ip as used, create forward record, create reverse entry, reload changed zones" to a single line in your preferred shell.

DIM provides an API to allow products to consume and return single IPv4 addresses or whole /64 or /56 prefixes for IPv6.

Quickstart / Tutorial

Download VM (Documentation how the VM was created). The VM is preconfigured including PowerDNS and PowerDNS recursor so that you can immediately check whether your commands had effects.

Read Tutorial to see how DIM can be used to document Prefixes and manage DNS Records.

Docker

not yet available. Pull requests welcome.

Future

There is an effort going on to rewrite the middleware in go. It is planed to be a drop-in replacement. Main Goals:

replace MySQL with PostgreSQL
do not use an ORM
remove properties tables, use jsonb field instead
put more logic in the database to avoid transfering large datasets to the application code
introduce generic log for all actions
get rid of the global lock for transaction synchronization

A link to the project will be added once the developer declares it to be ready for the public.

dim's People

Contributors

Stargazers

Watchers

dim's Issues

analyze whether simplejson, argparse can be removed from ndcli

--no-default-reserve does not work as expected

ndcli modify pool POOL add subnet 100.64.208.0/21 --no-default-reserve does actually mark 100.64.208.0 as reserved

[ndcli] need el7 packages for arsys

[L3D] set a default layer3domain in .ndclirc

To avoid giving layer3domain always on the command line set a default in .ndclirc

[ndcli] use framework for commandline interface

There is a lot of code dedicated to creating the command line, usage help and shell completion. I think this should be replace by a framework like https://pypi.org/project/click/

understand and fix "'dim.zoneview.zone_id' isn't in GROUP BY" when running ndcli list zones

running ndcli list zones on my installation fails with the following stacktrace of the middleware code

Sep 24 10:52:32 ds11.bs.wzt.de.k2.ai manage_dim[1484649]: INFO  87f31767724bd368 - miesi called zone_list(profile=False, fields=True, limit=11)
Sep 24 10:52:32 ds11.bs.wzt.de.k2.ai manage_dim[1484649]: INFO  87f31767724bd368 - 0.017 for zone_list(profile=False, fields=True, limit=11)
Sep 24 10:52:32 ds11.bs.wzt.de.k2.ai manage_dim[1484649]: ERROR 87f31767724bd368 - (MySQLdb._exceptions.OperationalError) (1055, "'dim.zoneview.zone_id' isn't in GROUP BY")
Sep 24 10:52:32 ds11.bs.wzt.de.k2.ai manage_dim[1484649]: [SQL: SELECT zone.name AS name, anon_1.views AS anon_1_views, coalesce(anon_2.zone_groups, %s) AS zone_groups, %s AS can_create_rr, %s AS can_delete_rr
Sep 24 10:52:32 ds11.bs.wzt.de.k2.ai manage_dim[1484649]: FROM zone LEFT OUTER JOIN (SELECT zoneview.zone_id AS zone_id, count(%s) AS views
Sep 24 10:52:32 ds11.bs.wzt.de.k2.ai manage_dim[1484649]: FROM zoneview GROUP BY zoneview.id) AS anon_1 ON zone.id = anon_1.zone_id LEFT OUTER JOIN (SELECT zone.id AS zone_id, zoneview.id AS id, count(%s) AS zone_groups
Sep 24 10:52:32 ds11.bs.wzt.de.k2.ai manage_dim[1484649]: FROM zone INNER JOIN zoneview ON zone.id = zoneview.zone_id INNER JOIN zonegroupzoneview AS zonegroupzoneview_1 ON zoneview.id = zonegroupzoneview_1.zoneview_id INNER JOIN zoneg>
Sep 24 10:52:32 ds11.bs.wzt.de.k2.ai manage_dim[1484649]: WHERE zone.name LIKE %s AND zone.profile = false ORDER BY name
Sep 24 10:52:32 ds11.bs.wzt.de.k2.ai manage_dim[1484649]:  LIMIT %s]
Sep 24 10:52:32 ds11.bs.wzt.de.k2.ai manage_dim[1484649]: [parameters: (0, 1, 1, '*', '*', '%', 11)]
Sep 24 10:52:32 ds11.bs.wzt.de.k2.ai manage_dim[1484649]: (Background on this error at: http://sqlalche.me/e/e3q8)
Sep 24 10:52:32 ds11.bs.wzt.de.k2.ai manage_dim[1484649]: Traceback (most recent call last):
Sep 24 10:52:32 ds11.bs.wzt.de.k2.ai manage_dim[1484649]:   File "/usr/lib64/python3.6/site-packages/sqlalchemy/engine/base.py", line 1244, in _execute_context
Sep 24 10:52:32 ds11.bs.wzt.de.k2.ai manage_dim[1484649]:     cursor, statement, parameters, context
Sep 24 10:52:32 ds11.bs.wzt.de.k2.ai manage_dim[1484649]:   File "/usr/lib64/python3.6/site-packages/sqlalchemy/engine/default.py", line 552, in do_execute
Sep 24 10:52:32 ds11.bs.wzt.de.k2.ai manage_dim[1484649]:     cursor.execute(statement, parameters)
Sep 24 10:52:32 ds11.bs.wzt.de.k2.ai manage_dim[1484649]:   File "/usr/lib64/python3.6/site-packages/MySQLdb/cursors.py", line 209, in execute
Sep 24 10:52:32 ds11.bs.wzt.de.k2.ai manage_dim[1484649]:     res = self._query(query)
Sep 24 10:52:32 ds11.bs.wzt.de.k2.ai manage_dim[1484649]:   File "/usr/lib64/python3.6/site-packages/MySQLdb/cursors.py", line 315, in _query
Sep 24 10:52:32 ds11.bs.wzt.de.k2.ai manage_dim[1484649]:     db.query(q)
Sep 24 10:52:32 ds11.bs.wzt.de.k2.ai manage_dim[1484649]:   File "/usr/lib64/python3.6/site-packages/MySQLdb/connections.py", line 239, in query
Sep 24 10:52:32 ds11.bs.wzt.de.k2.ai manage_dim[1484649]:     _mysql.connection.query(self, query)
Sep 24 10:52:32 ds11.bs.wzt.de.k2.ai manage_dim[1484649]: MySQLdb._exceptions.OperationalError: (1055, "'dim.zoneview.zone_id' isn't in GROUP BY")
Sep 24 10:52:32 ds11.bs.wzt.de.k2.ai manage_dim[1484649]: The above exception was the direct cause of the following exception:
Sep 24 10:52:32 ds11.bs.wzt.de.k2.ai manage_dim[1484649]: Traceback (most recent call last):
Sep 24 10:52:32 ds11.bs.wzt.de.k2.ai manage_dim[1484649]:   File "/srv/http/dim/dim/jsonrpc.py", line 188, in jsonrpc_handler
Sep 24 10:52:32 ds11.bs.wzt.de.k2.ai manage_dim[1484649]:     return json.dumps(dict(result=method(*args, **kwargs), **json_response),
Sep 24 10:52:32 ds11.bs.wzt.de.k2.ai manage_dim[1484649]:   File "/srv/http/dim/dim/transaction.py", line 25, in wrapper
Sep 24 10:52:32 ds11.bs.wzt.de.k2.ai manage_dim[1484649]:     return f(*args, **kwargs)
Sep 24 10:52:32 ds11.bs.wzt.de.k2.ai manage_dim[1484649]:   File "/srv/http/dim/dim/transaction.py", line 42, in wrapper
Sep 24 10:52:32 ds11.bs.wzt.de.k2.ai manage_dim[1484649]:     return f(*args, **kwargs)
Sep 24 10:52:32 ds11.bs.wzt.de.k2.ai manage_dim[1484649]:   File "/srv/http/dim/dim/rpc.py", line 1464, in zone_list
Sep 24 10:52:32 ds11.bs.wzt.de.k2.ai manage_dim[1484649]:     zones = zones[offset:offset + int(limit)]
Sep 24 10:52:32 ds11.bs.wzt.de.k2.ai manage_dim[1484649]:   File "/usr/lib64/python3.6/site-packages/sqlalchemy/orm/query.py", line 3007, in __getitem__
Sep 24 10:52:32 ds11.bs.wzt.de.k2.ai manage_dim[1484649]:     return list(res)
Sep 24 10:52:32 ds11.bs.wzt.de.k2.ai manage_dim[1484649]:   File "/usr/lib64/python3.6/site-packages/sqlalchemy/orm/query.py", line 3317, in __iter__
Sep 24 10:52:32 ds11.bs.wzt.de.k2.ai manage_dim[1484649]:     return self._execute_and_instances(context)
Sep 24 10:52:32 ds11.bs.wzt.de.k2.ai manage_dim[1484649]:   File "/usr/lib64/python3.6/site-packages/sqlalchemy/orm/query.py", line 3342, in _execute_and_instances
Sep 24 10:52:32 ds11.bs.wzt.de.k2.ai manage_dim[1484649]:     result = conn.execute(querycontext.statement, self._params)
Sep 24 10:52:32 ds11.bs.wzt.de.k2.ai manage_dim[1484649]:   File "/usr/lib64/python3.6/site-packages/sqlalchemy/engine/base.py", line 988, in execute
Sep 24 10:52:32 ds11.bs.wzt.de.k2.ai manage_dim[1484649]:     return meth(self, multiparams, params)
Sep 24 10:52:32 ds11.bs.wzt.de.k2.ai manage_dim[1484649]:   File "/usr/lib64/python3.6/site-packages/sqlalchemy/sql/elements.py", line 287, in _execute_on_connection
Sep 24 10:52:32 ds11.bs.wzt.de.k2.ai manage_dim[1484649]:     return connection._execute_clauseelement(self, multiparams, params)
Sep 24 10:52:32 ds11.bs.wzt.de.k2.ai manage_dim[1484649]:   File "/usr/lib64/python3.6/site-packages/sqlalchemy/engine/base.py", line 1107, in _execute_clauseelement
Sep 24 10:52:32 ds11.bs.wzt.de.k2.ai manage_dim[1484649]:     distilled_params,
Sep 24 10:52:32 ds11.bs.wzt.de.k2.ai manage_dim[1484649]:   File "/usr/lib64/python3.6/site-packages/sqlalchemy/engine/base.py", line 1248, in _execute_context
Sep 24 10:52:32 ds11.bs.wzt.de.k2.ai manage_dim[1484649]:     e, statement, parameters, cursor, context
Sep 24 10:52:32 ds11.bs.wzt.de.k2.ai manage_dim[1484649]:   File "/usr/lib64/python3.6/site-packages/sqlalchemy/engine/base.py", line 1466, in _handle_dbapi_exception
Sep 24 10:52:32 ds11.bs.wzt.de.k2.ai manage_dim[1484649]:     util.raise_from_cause(sqlalchemy_exception, exc_info)
Sep 24 10:52:32 ds11.bs.wzt.de.k2.ai manage_dim[1484649]:   File "/usr/lib64/python3.6/site-packages/sqlalchemy/util/compat.py", line 383, in raise_from_cause
Sep 24 10:52:32 ds11.bs.wzt.de.k2.ai manage_dim[1484649]:     reraise(type(exception), exception, tb=exc_tb, cause=cause)
Sep 24 10:52:32 ds11.bs.wzt.de.k2.ai manage_dim[1484649]:   File "/usr/lib64/python3.6/site-packages/sqlalchemy/util/compat.py", line 128, in reraise
Sep 24 10:52:32 ds11.bs.wzt.de.k2.ai manage_dim[1484649]:     raise value.with_traceback(tb)
Sep 24 10:52:32 ds11.bs.wzt.de.k2.ai manage_dim[1484649]:   File "/usr/lib64/python3.6/site-packages/sqlalchemy/engine/base.py", line 1244, in _execute_context
Sep 24 10:52:32 ds11.bs.wzt.de.k2.ai manage_dim[1484649]:     cursor, statement, parameters, context
Sep 24 10:52:32 ds11.bs.wzt.de.k2.ai manage_dim[1484649]:   File "/usr/lib64/python3.6/site-packages/sqlalchemy/engine/default.py", line 552, in do_execute
Sep 24 10:52:32 ds11.bs.wzt.de.k2.ai manage_dim[1484649]:     cursor.execute(statement, parameters)
Sep 24 10:52:32 ds11.bs.wzt.de.k2.ai manage_dim[1484649]:   File "/usr/lib64/python3.6/site-packages/MySQLdb/cursors.py", line 209, in execute
Sep 24 10:52:32 ds11.bs.wzt.de.k2.ai manage_dim[1484649]:     res = self._query(query)
Sep 24 10:52:32 ds11.bs.wzt.de.k2.ai manage_dim[1484649]:   File "/usr/lib64/python3.6/site-packages/MySQLdb/cursors.py", line 315, in _query
Sep 24 10:52:32 ds11.bs.wzt.de.k2.ai manage_dim[1484649]:     db.query(q)
Sep 24 10:52:32 ds11.bs.wzt.de.k2.ai manage_dim[1484649]:   File "/usr/lib64/python3.6/site-packages/MySQLdb/connections.py", line 239, in query
Sep 24 10:52:32 ds11.bs.wzt.de.k2.ai manage_dim[1484649]:     _mysql.connection.query(self, query)
Sep 24 10:52:32 ds11.bs.wzt.de.k2.ai manage_dim[1484649]: sqlalchemy.exc.OperationalError: (MySQLdb._exceptions.OperationalError) (1055, "'dim.zoneview.zone_id' isn't in GROUP BY")
Sep 24 10:52:32 ds11.bs.wzt.de.k2.ai manage_dim[1484649]: [SQL: SELECT zone.name AS name, anon_1.views AS anon_1_views, coalesce(anon_2.zone_groups, %s) AS zone_groups, %s AS can_create_rr, %s AS can_delete_rr
Sep 24 10:52:32 ds11.bs.wzt.de.k2.ai manage_dim[1484649]: FROM zone LEFT OUTER JOIN (SELECT zoneview.zone_id AS zone_id, count(%s) AS views
Sep 24 10:52:32 ds11.bs.wzt.de.k2.ai manage_dim[1484649]: FROM zoneview GROUP BY zoneview.id) AS anon_1 ON zone.id = anon_1.zone_id LEFT OUTER JOIN (SELECT zone.id AS zone_id, zoneview.id AS id, count(%s) AS zone_groups
Sep 24 10:52:32 ds11.bs.wzt.de.k2.ai manage_dim[1484649]: FROM zone INNER JOIN zoneview ON zone.id = zoneview.zone_id INNER JOIN zonegroupzoneview AS zonegroupzoneview_1 ON zoneview.id = zonegroupzoneview_1.zoneview_id INNER JOIN zoneg>
Sep 24 10:52:32 ds11.bs.wzt.de.k2.ai manage_dim[1484649]: WHERE zone.name LIKE %s AND zone.profile = false ORDER BY name
Sep 24 10:52:32 ds11.bs.wzt.de.k2.ai manage_dim[1484649]:  LIMIT %s]
Sep 24 10:52:32 ds11.bs.wzt.de.k2.ai manage_dim[1484649]: [parameters: (0, 1, 1, '*', '*', '%', 11)]
Sep 24 10:52:32 ds11.bs.wzt.de.k2.ai manage_dim[1484649]: (Background on this error at: http://sqlalche.me/e/e3q8)

To reproduce this problem it is probably needed to have a zone with at minimum 2 views. Something like

ndcli create zone example.com
ndcli modify zone example.com add view internal

should do the trick.

If not, the zone views need to be connected to outputs

ndcli create output one plugin pdns ...
ndcli create output two plugin pdns ...

ndcli create zone-group internal
ndcli create zone-group public

ndcli modify zone-group internal add zone example.com view internal
ndcli modify zone-group public add zone example.com view default

ndcli modify output one add zone-group internal
ndcli modify output two add zone-group public

[history] history zone-groups empty

[L3D] set type ndcli modify layer3domain default set type global-table

make the type of a layer3domain changeable

ndcli modify layer3domain set type

[pdns-output] get testsuite running

Feature Request: CAS alternative for dim-web

It would be nice to use dim-web with OIDC or SAML

Can not find python3-dimclient rpm

Can not find anywhere python3-dimclient rpm required by python3-ndcli-4.0.1-1.el8.x86_64.rpm

Would like to have ndcli installed in a centos 8 (rocky 8 , actually) host.

.github/workflows/release_dimclient.yml suggests it should be part of the release, but it is not there.

$ sudo dnf install python3-ndcli-4.0.1-1.el8.x86_64.rpm
Last metadata expiration check: 2:59:04 ago on mié 29 sep 2021 04:54:50 UTC.
Error:
Problem: conflicting requests

nothing provides python3-dimclient needed by python3-ndcli-4.0.1-1.el8.x86_64
(try to add '--skip-broken' to skip uninstallable packages or '--nobest' to use not only best candidate packages)

ndcli import rev-zone does not work

dig AXFR 192.17.217.in-addr.arpa. @avalon.iks-jena.de. | ndcli import rev-zone

should work

Now that allocate right means only allocate we need a separate deallocate, free or delete right to allow people to clean up but not allocate ips. Just like create_rr and delete_rr right.

[ndcli] notify user of missing .ndclirc

notify user of missing .ndclirc when -s <URL> not in effect.

[L3D] add table types global-table, local-table

Then we would have
vrf
global-table
local-table

[packaging] package dim-bind

el7/el8/rc33/debian

systemd unit file which can be instantiated to ease spilt horizon setups

Readme should include note that not only bind is supported but also nsd and other bind file reading daemons.

Readme should contain note that zone signing is not supported with bind.

add vxlan stories

[dim] fix sqlalchemy usage to produce statements which work with stricter MySQL sql_mode

target sql mode

sql_mode                                 = 'REAL_AS_FLOAT,PIPES_AS_CONCAT,ANSI_QUOTES,IGNORE_SPACE,ONLY_FULL_GROUP_BY,ANSI,STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION'

[L3D] deleteing a layer3domain throws an unexplained error

when deleting a layer3domain, the following error can be thrown

$ ndcli create layer3domain mlan type vrf rd 1:1
$ ndcli create pool somepool layer3domain mlan
$ ndcli delete layer3domain mlan
ERROR - (_mysql_exceptions.IntegrityError) (1451, 'Cannot delete or update a parent row: a foreign key constraint fails ("netdot"."ippool", CONSTRAINT "ippool_ibfk_3" FOREIGN KEY ("layer3domain_id") REFERENCES "layer3domain" ("id"))') [SQL: u'DELETE FROM layer3domain WHERE layer3domain.id = %s'] [parameters: (3L,)]

ndcli should display an error message like

$ ndcli delete layer3domain mlan
ERROR - layer3domain still has pools: somepool

[ndcli][fc] check .bashrc provides autocomplete

[ndcli] [list containers] missing /32

From ndcli list containers

   74.208.191.168/30 (Available)
    74.208.191.173 (Static)
    74.208.191.174 (Static)
    74.208.191.175 (Available)
    74.208.191.176/28 (Available)

74.208.191.172 needs to appear

write testcase
fix code
release

incorrect pool statistics when assignmentsize is in effect

ndcli list pool simply calculates the number of available assignments/delegations and the subtracts the number of configured delegations. It is not taking into account that free space between delegations might be blocked by static ips.

some ndcli commands are not respecting dry-run parameter

It appears ndcli does not respect the -n/--dry-run parameter for some commands.
e.g.
ndcli create user-group supports the parameter according to the man page, but does not wrap it into a transaction

$ ndcli create user-group 'Dep_Foobar' 'Dep_Foobar' -d -n
INFO - Dryrun mode, no data will be modified
DEBUG - Dim server URL: ***
DEBUG - Username: ***
DEBUG - dim call: department_number('Dep_Foobar')
DEBUG - time taken: 0.019
DEBUG - dim result: 21239255
DEBUG - dim call: group_create('Dep_Foobar', {'department_number': 21239255})
DEBUG - time taken: 0.032
DEBUG - dim result: None
$ ndcli create user-group 'Dep_Foobar 'Dep_Foobar' -d -n
INFO - Dryrun mode, no data will be modified
DEBUG - Dim server URL: ***
DEBUG - Username: ***
DEBUG - dim call: department_number('Dep_Foobar')
DEBUG - time taken: 0.025
DEBUG - dim result: 21239255
DEBUG - dim call: group_create('Dep_Foobar', {'department_number': 21239255})
DEBUG - time taken: 0.027
DEBUG - dim error: {'code': 20, 'message': 'A group with department_number 21239255 already exists'}
ERROR - A group with department_number 21239255 already exists
DEBUG - trace
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/dimcli/__init__.py", line 889, in run
    args.run(self, args)
  File "/usr/lib/python3/dist-packages/dimcli/__init__.py", line 1669, in create_group
    self.client.group_create(args.usergroup, **options)
  File "/usr/lib/python3/dist-packages/dimclient/__init__.py", line 153, in <lambda>
    return lambda *args, **kwargs: self.call(name, *args, **kwargs)
  File "/usr/lib/python3/dist-packages/dimclient/__init__.py", line 150, in call
    return self.raw_call(function, *passed_args)
  File "/usr/lib/python3/dist-packages/dimclient/__init__.py", line 131, in raw_call
    raise DimError(message=rpc_response['error']['message'],
dimclient.DimError: A group with department_number 21239255 already exists

dim-web does not compile with recent node version

WARN notsup Unsupported engine for [email protected]: wanted: {"node":"0.10 || 0.12 || 4 || 5 || 6 || 7 || 8"} (current: {"node":"14.17.0","npm":"6.14.13"})

[L3D] list pools by layer3domain

During specification I seem to have missed something like

ndcli list pools [VLANID|CIDR|POOL] [layer3domain l3d]

Something like this:

ndcli list pools layer3domain default

ndcli list pools de-kae-bs* layer3domain some-layer-3-domain

ndcli list pools de-kae-bs* layer3domain all

There should be auto-complete for layer3domain.

upgrade to flask 2.x

update to flask2

remove test for zone file import with comment

create test for list pools -a

[ndcli][deb] man page missing

rework packaging to include man page in .deb

NameError: name 'basestring' is not defined

ndcli create rr test.mieslinger.de. txt fuh view internal public

fails with

Sep 24 13:59:28 ds11.bs.wzt.de.k2.ai manage_dim[1484649]: INFO  959b9c226c56e777 - miesi called rr_create(type='TXT', views=['internal', 'public'], name='test.mieslinger.d>
Sep 24 13:59:28 ds11.bs.wzt.de.k2.ai manage_dim[1484649]: INFO  959b9c226c56e777 - 0.022 for rr_create(type='TXT', views=['internal', 'public'], name='test.mieslinger.de.'>
Sep 24 13:59:28 ds11.bs.wzt.de.k2.ai manage_dim[1484649]: ERROR 959b9c226c56e777 - name 'basestring' is not defined
Sep 24 13:59:28 ds11.bs.wzt.de.k2.ai manage_dim[1484649]: Traceback (most recent call last):
Sep 24 13:59:28 ds11.bs.wzt.de.k2.ai manage_dim[1484649]:   File "/srv/http/dim/dim/jsonrpc.py", line 188, in jsonrpc_handler
Sep 24 13:59:28 ds11.bs.wzt.de.k2.ai manage_dim[1484649]:     return json.dumps(dict(result=method(*args, **kwargs), **json_response),
Sep 24 13:59:28 ds11.bs.wzt.de.k2.ai manage_dim[1484649]:   File "/srv/http/dim/dim/transaction.py", line 25, in wrapper
Sep 24 13:59:28 ds11.bs.wzt.de.k2.ai manage_dim[1484649]:     return f(*args, **kwargs)
Sep 24 13:59:28 ds11.bs.wzt.de.k2.ai manage_dim[1484649]:   File "/srv/http/dim/dim/transaction.py", line 42, in wrapper
Sep 24 13:59:28 ds11.bs.wzt.de.k2.ai manage_dim[1484649]:     return f(*args, **kwargs)
Sep 24 13:59:28 ds11.bs.wzt.de.k2.ai manage_dim[1484649]:   File "/srv/http/dim/dim/transaction.py", line 83, in wrapped
Sep 24 13:59:28 ds11.bs.wzt.de.k2.ai manage_dim[1484649]:     return f(*args, **kwargs)
Sep 24 13:59:28 ds11.bs.wzt.de.k2.ai manage_dim[1484649]:   File "/srv/http/dim/dim/transaction.py", line 98, in wrapper
Sep 24 13:59:28 ds11.bs.wzt.de.k2.ai manage_dim[1484649]:     result = f(*args, **kwargs)
Sep 24 13:59:28 ds11.bs.wzt.de.k2.ai manage_dim[1484649]:   File "/srv/http/dim/dim/rpc.py", line 1996, in rr_create
Sep 24 13:59:28 ds11.bs.wzt.de.k2.ai manage_dim[1484649]:     **kwargs)
Sep 24 13:59:28 ds11.bs.wzt.de.k2.ai manage_dim[1484649]:   File "/srv/http/dim/dim/rpc.py", line 3225, in _rr_create
Sep 24 13:59:28 ds11.bs.wzt.de.k2.ai manage_dim[1484649]:     self._rr_create_single_pair(zone=zone, view=view, profile=profile, ttl=ttl, comment=comment, **kwargs)
Sep 24 13:59:28 ds11.bs.wzt.de.k2.ai manage_dim[1484649]:   File "/srv/http/dim/dim/rpc.py", line 3108, in _rr_create_single_pair
Sep 24 13:59:28 ds11.bs.wzt.de.k2.ai manage_dim[1484649]:     dim.dns.create_single_rr(rr_type=type, zone=zone_obj, user=self.user, **kwargs)
Sep 24 13:59:28 ds11.bs.wzt.de.k2.ai manage_dim[1484649]:   File "/srv/http/dim/dim/dns.py", line 213, in create_single_rr
Sep 24 13:59:28 ds11.bs.wzt.de.k2.ai manage_dim[1484649]:     new_rr = RR.create(name=name, type=rr_type, view=view, **kwargs)
Sep 24 13:59:28 ds11.bs.wzt.de.k2.ai manage_dim[1484649]:   File "/srv/http/dim/dim/models/dns.py", line 381, in create
Sep 24 13:59:28 ds11.bs.wzt.de.k2.ai manage_dim[1484649]:     kwargs = RR.validate_args(type, **kwargs)
Sep 24 13:59:28 ds11.bs.wzt.de.k2.ai manage_dim[1484649]:   File "/srv/http/dim/dim/models/dns.py", line 370, in validate_args
Sep 24 13:59:28 ds11.bs.wzt.de.k2.ai manage_dim[1484649]:     kwargs[field] = validate(None, field, kwargs[field])
Sep 24 13:59:28 ds11.bs.wzt.de.k2.ai manage_dim[1484649]:   File "/srv/http/dim/dim/rrtype.py", line 205, in validate_strings
Sep 24 13:59:28 ds11.bs.wzt.de.k2.ai manage_dim[1484649]:     if isinstance(value, basestring):
Sep 24 13:59:28 ds11.bs.wzt.de.k2.ai manage_dim[1484649]: NameError: name 'basestring' is not defined

Add Fedora 34 packages

With the next release Fedora 34 packages should be added to the automated build.

The packages that need to be built are ndcli and dimclient.

running dim testsuite has side effects

ndcli does use the users .ndclirc, so it can happen that ont the local dim service is used to apply tests

the code has tons of hardcoded 127.0.0.1:3307 mysql_uris

Solve legacy JDK 8 binding to JDK-internal classes

The statically included (and altered) dnsjava source that is located in pdns-output/jdnssec-dnsjava is based on dnsjava 2.1.7. Dnsjava 2.1.7 has hard dependencies on JDK 8 internal classes that were not meant to be used by application code and that were removed in later JDKs.

When building with JDK 11:

...
> Task :jdnssec-dnsjava:compileJava FAILED
...dim/pdns-output/jdnssec-dnsjava/src/main/java/org/xbill/DNS/spi/DNSJavaNameServiceDescriptor.java:16: error: cannot find symbol
public class DNSJavaNameServiceDescriptor implements NameServiceDescriptor {
                                                     ^
  symbol: class NameServiceDescriptor
...dim/pdns-output/jdnssec-dnsjava/src/main/java/org/xbill/DNS/spi/DNSJavaNameServiceDescriptor.java:18: error: cannot find symbol
private static NameService nameService;
               ^
  symbol:   class NameService
  location: class DNSJavaNameServiceDescriptor
...dim/pdns-output/jdnssec-dnsjava/src/main/java/org/xbill/DNS/spi/DNSJavaNameServiceDescriptor.java:30: error: cannot find symbol
public NameService
       ^
  symbol:   class NameService
  location: class DNSJavaNameServiceDescriptor
...dim/pdns-output/jdnssec-dnsjava/src/main/java/org/xbill/DNS/spi/DNSJavaNameServiceDescriptor.java:7: error: package sun.net.spi.nameservice does not exist
import sun.net.spi.nameservice.*;
^
...dim/pdns-output/jdnssec-dnsjava/src/main/java/org/xbill/DNS/utils/HMAC.java:87: warning: [dep-ann] deprecated item is not annotated with @Deprecated
HMAC(MessageDigest digest, byte [] key) {
^
...dim/pdns-output/jdnssec-dnsjava/src/main/java/org/xbill/DNS/utils/HMAC.java:101: warning: [dep-ann] deprecated item is not annotated with @Deprecated
HMAC(String digestName, byte [] key) {
^
...dim/pdns-output/jdnssec-dnsjava/src/main/java/org/xbill/DNS/spi/DNSJavaNameServiceDescriptor.java:21: error: cannot find symbol
        ClassLoader loader = NameService.class.getClassLoader();
                             ^
  symbol:   class NameService
  location: class DNSJavaNameServiceDescriptor
...dim/pdns-output/jdnssec-dnsjava/src/main/java/org/xbill/DNS/spi/DNSJavaNameServiceDescriptor.java:22: error: cannot find symbol
        nameService = (NameService) Proxy.newProxyInstance(loader,
                       ^
  symbol:   class NameService
  location: class DNSJavaNameServiceDescriptor
...dim/pdns-output/jdnssec-dnsjava/src/main/java/org/xbill/DNS/spi/DNSJavaNameServiceDescriptor.java:23: error: cannot find symbol
                        new Class[] { NameService.class },
                                      ^
  symbol:   class NameService
  location: class DNSJavaNameServiceDescriptor
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
7 errors
2 warnings

Adopt OpenJDK has JDK 8 support until "At Least May 2026".

ndcli login returns 401 unauthorized error but cookie data bypasses error

$ ndcli -d login
DEBUG - Dim server URL: https://server/dim
DEBUG - Username: myuser
Password: 
ERROR - Login failed: HTTP Error 401: UNAUTHORIZED
ERROR - Could not log in
DEBUG - trace
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/dimcli/__init__.py", line 889, in run
    args.run(self, args)
  File "/usr/lib/python3/dist-packages/dimcli/__init__.py", line 966, in login
    self.client.get_username()
  File "/usr/lib/python3/dist-packages/dimcli/__init__.py", line 840, in client
    raise Exception('Could not log in')
Exception: Could not log in

But if I access the DIM website and copy the session cookie data into the .ndcli.cookie like this:

#LWP-Cookies-2.0
Set-Cookie3: session="COOKIE_SESSION_CONTENT_HERE"; path="/"; domain="server"; path_spec; expires="2031-06-09 11:34:22Z"; HttpOnly=None; version=0

Then all the other operations work fine (ndcli show user, ndcli show server-info, etc) and there are no authentication errors anymore.

If this was a permissions issue, shouldn't the error continue even with the cookie data from the website?

OS: Linux Mint 20.2
ndcli: 4.0.1

[L3D] move pool to a different layer3domain

Networking departments will use layer3domain separation to enhance security.

After the DIM 3.0 migration all containers will be in the default layer3domain.

There needs to be an easy way to move a pool from one layer3domain to another.

$ ndcli modify pool P set layer3domain <D>

check for all subnets of pool P that a containers exists in the new layer3domain.
check for all subnets of pool P that no existing subnet in new layer3domain overlaps (see --allow-overlaps)
create rev-zones views if necessary
delete rrs from old rev-zone view and create in new rev-zone view. Put necessary events in outputupdate table.
update subnet, reserved and static entries belonging to pool in ipblocks table to new layer3domain
update layer3domain property of pool

I guess I missed something. Please comment.

[dim] sql_mode fixes for models/ip.py

miesi@ds11:~$ ndcli list ips 192.168.178.0/24
ERROR - Can't determine which FROM clause to join from, there are multiple FROMS which can join to this entity. Please use the .select_from() method to establish an explicit left side, as well as providing an explcit ON clause if not present already to help resolve the ambiguity.
miesi@ds11:~$ ndcli list ips 192.168.178.0/24 status used
ERROR - Can't determine which FROM clause to join from, there are multiple FROMS which can join to this entity. Please use the .select_from() method to establish an explicit left side, as well as providing an explcit ON clause if not present already to help resolve the ambiguity.

    def _used(self, only_static):
        ret = 0
        q = db.session.query(Ipblock.prefix, func.count()).filter(Ipblock.parent == self)
        if only_static:
            q = q.join(IpblockStatus).filter(IpblockStatus.name == 'Static')
        for prefix, count in q.group_by(Ipblock.prefix).all():
            ret += count * 2 ** (self.ip.bits - prefix)
        return ret

unicode in TXT records

There is a testcase rr-create-txt-2.t, which tries to put unicode codepoints into TXT records. From the RFC this should work.

The testcase doesn't work properly because python doesn't like the escaping in that string. Any try to resolve the issue resulted in the wrong data in the table.

In this case either this is fixed so that pdns ends up with the correct data or this is blocked from being entered in the first place.

create pdns-output.jar builder

Currently there is no build process for pdns-output.jar. But this is needed to be able to build a release and get the following 3 tests running:

rr-create-naptr-1.t
zone-dnssec-subzone-3.t
zone-nsec3-attrs.t

[L3D] list pools default output needs to contain layer3domain

The output format of list pools needs to contain the layer3domain

ndcli list pools 562
INFO - Result for list pools 562
layer3domain   name                            vlan subnets
default        de-kae-bs-it-infra-562           562 172.20.36.0/24
default        de-kae-bs-it-infra-562-infra     562 10.77.202.0/23
default        de-kae-bs-it-infra-itodsi01-562  562 10.76.44.192/26
non-default    es-lgr-pl-sistemas-internal-562  562 10.31.122.128/26

user without rights can delete ip from pool

Internally tracked as ITOUDP-2966 and GPHDIM-529.

This was implemented by design to ease clean up of DNS.

Now it has become a burden.

Change testcase

change code

run testsuite

[dim][ndcli] get testsuite running

Missing Dockerfile

I love to test this app via docker but it seems to me Dockerfile was missing.

Further improve LDAP sync

Follow-up to #76
The LDAP sync should use the dryrun feature provided by dim/transaction instead of the if we've added in #76

While the dry-run helps when running the sync manually,
there should be some sort of deletion threshold for periodic/unattended sync to catch misconfiguration or issues with LDAP;
either absolute numbers, or a percentage of entries in the DB, optimally per type: User, Group, Department

update README of dim

Currently the README of the project isn't helpful to get started with the project. There is some documentation in dim-testsuite, but that's it.

What is needed is:

How to Install each component?
How are Pools working?
How are Zones configured?
What needs to be done to get Output of some sort?

ndcli modify pool <P> rmove attrs a:v does not remove attribute

ndcli modify pool de-kae-bs-access-cloud-be-318-qa remove attrs dhcp-pxe-arch-default:ipxe-install/bin-bios/prelive.0 does not delete the attribute, there is also no warning about incorrect usage.

The working usage is ndcli modify pool de-kae-bs-access-cloud-be-318-qa remove attrs dhcp-pxe-arch-default

create testcase (x)
fix behaviour (x)

also autodelegate rev-zones

When a parent zone in DIM exists (e.g. 10.in-addr.apra) and a sub zone is created (e.g. 1.0.10.in-addr.arpa) then add delegating records to the parent zone.

can not change IP of a record

If I try to change an IP i get an errror:

ndcli modify rr  foo.example.com.  a ip 1.3.4.5 
ERROR - Unparsed tokens: 1.3.4.5

When adding a ttl a different error is triggers:

 ndcli modify rr  foo.example.com.  -t 300  a 2.3.4.4
ERROR - Invalid IP 2.3.4.4: IP block '2.3.4.4' does not exist in layer3domain default

ndcli -V
ndcli version 4.0.0

rpm -qi dim
Name        : dim
Version     : 4.0.9
Release     : 1.el8
Architecture: x86_64
Install Date: Thu 14 Oct 2021 01:32:44 PM CEST
Group       : application/system
Size        : 41385440
License     : MIT
Signature   : (none)
Source RPM  : dim-4.0.9-1.el8.src.rpm
Build Date  : Tue 15 Jun 2021 11:30:11 AM CEST
Build Host  : 04d7d22b2f26
Relocations : (not relocatable)
Summary     : DNS and IP management
Description :
DNS and IP management