Objects within layer3domains are missing in the `ndcli list containers` command

[TheRealBecks]

Objects within layer3domains are missing in the `ndcli list containers` command

[Gibheer]

Okay, something weird is going on. I have looked through the code and it seems okay.
There are even testcases that show, that it must have worked at least in the past: https://github.com/1and1/dim/blob/master/dim-testsuite/t/container-layer3domain.t

I will try to reproduce the issue, so that I can come up with a fix. May take some time though.

[zeromind]

Seems like DIM currently does this:
https://github.com/1and1/dim/blob/dim-5.0.2/dim/dim/rpc.py#L857-L867
When one does not provide a container/prefix, it looks for an Ipblock that does not have a parent (parent_id=None), and is of type Container.
It's trying to find the top-most container to return the "whole" tree for the given layer3domain.

However, in the layer3domain from this issue, there are only Ipblocks with a parent.

>> l3d.name
'6724-swh-pwcrypt-vrf'
>>> dim.models.Ipblock.query.filter_by(parent_id=None, layer3domain=l3d).all()
[]

parent_ids of Ipblocks in the layer3domain:

>>> [ipb.parent_id for ipb in dim.models.Ipblock.query.filter_by(layer3domain=l3d).all()]
[7991830, 7991831, 7991831, 7727504, 7935530, 7935533, 7935533, 7935530, 7968862, 7968862, 7935530, 7968865, 7968865, 7935530, 7968868, 7968868, 7935530, 7968871, 7968871, 7935530, 7968874, 7968874, 7935530, 7968877, 7968877, 7935530, 7968880, 7968880, 7968967, 7968883, 7968883, 7968968, 7968886, 7968886, 7968968, 7968889, 7968889, 7968968, 7968892, 7968892, 7968968, 7968895, 7968895, 7968969, 7968898, 7968898, 7968969, 7968901, 7968901, 7968969, 7968904, 7968904, 7968969, 7968907, 7968907, 7968969, 7968910, 7968910, 7968969, 7968913, 7968913, 7968969, 7968916, 7968916, 7968969, 7968919, 7968919, 7727506, 7968922, 7968923, 7968924, 7968924, 7968923, 7968927, 7968927, 7968923, 7968930, 7968930, 7968922, 7968933, 7968934, 7968934, 7968933, 7968937, 7968937, 7968933, 7968940, 7968940, 7727506, 7968943, 7968944, 7968945, 7968945, 7968944, 7968948, 7968948, 7968944, 7968951, 7968951, 7968943, 7968954, 7968955, 7968955, 7968954, 7968958, 7968958, 7968954, 7968961, 7968961, 7727505, 7968967, 7968967]

Not sure what went wrong there, I think this should not happen.
Will have to dig deeper.

[Gibheer]

Can you run ndcli list containers layer3domain 6724-swh-pwcrypt-vrf -d and ndcli list containers -d?
That will show some debug output containing the parameters sent to the dim middleware.

For the first command, that should return all containers properly, if the layer3domain is sent.
For the second command it is possible that you have a default layer3domain set in your .ndclirc.

[TheRealBecks]

dzgaaj@vm-jumphost:~$ ndcli list containers layer3domain 6724-swh-pwcrypt-vrf -d
DEBUG - Dim server URL: https://<server_name>/dim
DEBUG - Username: <my_username>
DEBUG - dim call: get_username()
DEBUG - time taken: 0.026
DEBUG - dim result: '<my_username>'
DEBUG - dim call: container_list({'include_messages': True, 'layer3domain': '6724-swh-pwcrypt-vrf'})
DEBUG - time taken: 0.021
DEBUG - dim result: {'containers': [], 'messages': []}

dzgaaj@vm-jumphost:~$ ndcli list containers -d
DEBUG - Dim server URL: https://<server_name>/dim
DEBUG - Username: <my_username>
DEBUG - dim call: get_username()
DEBUG - time taken: 0.022
DEBUG - dim result: '<my_username>'
DEBUG - dim call: layer3domain_list()
DEBUG - time taken: 0.016
DEBUG - dim result: [{'comment': 'the correct name would be 8560-global',
  'name': 'default',
  'properties': {'rd': '8560:1'},
  'type': 'vrf'},
 {'comment': None, 'name': 'DONOTUSE', 'type': 'DONOTUSE'},
 {'comment': 'Strato global table',
  'name': '6724-global',
  'type': 'global-table'},
 {'comment': 'arsys legacy rfc1918 space',
  'name': '8560-arsys-rfc1918',
  'properties': {'rd': '8560:80'},
  'type': 'vrf'},
 {'comment': 'mgmt/management/oobms (vendor dependent) rt 6724:111',
  'name': '6724-mgmt-vrf',
  'properties': {'rd': '6724:111'},
  'type': 'vrf'},
 {'comment': 'HiDrive internal networks rt 6724:112',
  'name': '6724-hidrive-vrf',
  'properties': {'rd': '6724:112'},
  'type': 'vrf'},
 {'comment': 'Shared Webhosting internal networks rt 6724:113',
  'name': '6724-swh-internal-vrf',
  'properties': {'rd': '6724:113'},
  'type': 'vrf'},
 {'comment': 'Shared Webhosting storage internal networks rt 6724:114',
  'name': '6724-swh-store-vrf',
  'properties': {'rd': '6724:114'},
  'type': 'vrf'},
 {'comment': 'Shared Webhosting pwcrypt internal networks rt 6724:115',
  'name': '6724-swh-pwcrypt-vrf',
  'properties': {'rd': '6724:115'},
  'type': 'vrf'},
 {'comment': 'STRATO to IONOS networks rt 6724:116',
  'name': '6724-ionos-private-vrf',
  'properties': {'rd': '6724:116'},
  'type': 'vrf'},
 {'comment': 'mssd backup networks rt 6724:117',
  'name': '6724-mssd-backup-vrf',
  'properties': {'rd': '6724:117'},
  'type': 'vrf'},
 {'comment': 'mssd sn2g networks rt 6724:118',
  'name': '6724-mssd-sn2g-vrf',
  'properties': {'rd': '6724:118'},
  'type': 'vrf'},
 {'comment': None, 'name': 'nw-services-prelive', 'type': 'rd:8560:999'},
 {'comment': 'test-ionos-cloud',
  'name': 'ICNetwork',
  'properties': {'rd': '54548:123'},
  'type': 'vrf'}]
DEBUG - dim call: container_list({'include_messages': True, 'layer3domain': 'default'})
DEBUG - time taken: 7.500
DEBUG - dim result: {'containers': [{'attributes': {'RIR': 'iana',
[...]
    fd77:294e:66d3:6000::/51 (Available)
  fd77:294e:66d3:8000::/49 (Available)

Additionally I searched for layer3domain in my output-file:

zgaaj@vm-jumphost:~$ grep -A 5 -B 5 layer3domain temp.txt
DEBUG - Dim server URL: https://<server_name>/dim
DEBUG - Username: <my_username>
DEBUG - dim call: get_username()
DEBUG - time taken: 0.022
DEBUG - dim result: '<my_username>'
DEBUG - dim call: layer3domain_list()
DEBUG - time taken: 0.016
DEBUG - dim result: [{'comment': 'the correct name would be 8560-global',
  'name': 'default',
  'properties': {'rd': '8560:1'},
  'type': 'vrf'},
--
 {'comment': None, 'name': 'nw-services-prelive', 'type': 'rd:8560:999'},
 {'comment': 'test-ionos-cloud',
  'name': 'ICNetwork',
  'properties': {'rd': '54548:123'},
  'type': 'vrf'}]
DEBUG - dim call: container_list({'include_messages': True, 'layer3domain': 'default'})
DEBUG - time taken: 7.500
DEBUG - dim result: {'containers': [{'attributes': {'RIR': 'iana',
                                'comment': 'ITOUDP-5051 - IPv4 Internet Part 1',
                                'reverse_dns_profile': 'public-rev'},
                 'children': [{'attributes': {'comment': 'reserved',
--
                               'ip': 'fd00::/8',
                               'status': 'Container'}],
                 'ip': 'fc00::/7',
                 'status': 'Container'}],
 'messages': []}
DEBUG - dim call: container_list({'include_messages': True, 'layer3domain': 'DONOTUSE'})
DEBUG - time taken: 0.027
DEBUG - dim result: {'containers': [], 'messages': []}
DEBUG - dim call: container_list({'include_messages': True, 'layer3domain': '6724-global'})
DEBUG - time taken: 0.273
DEBUG - dim result: {'containers': [{'attributes': {},
                 'children': [{'ip': '10.0.0.0/20', 'status': 'Available'},
                              {'ip': '10.0.16.0/23', 'status': 'Available'},
                              {'attributes': {'comment': "'sne ldp for mpls'"},
--
                              {'ip': 'fd77:294e:66d3:8000::/49',
                               'status': 'Available'}],
                 'ip': 'fd77:294e:66d3::/48',
                 'status': 'Container'}],
 'messages': []}
DEBUG - dim call: container_list({'include_messages': True, 'layer3domain': '8560-arsys-rfc1918'})
DEBUG - time taken: 0.022
DEBUG - dim result: {'containers': [], 'messages': []}
DEBUG - dim call: container_list({'include_messages': True, 'layer3domain': '6724-mgmt-vrf'})
DEBUG - time taken: 0.018
DEBUG - dim result: {'containers': [], 'messages': []}
DEBUG - dim call: container_list({'include_messages': True, 'layer3domain': '6724-hidrive-vrf'})
DEBUG - time taken: 0.020
DEBUG - dim result: {'containers': [], 'messages': []}
DEBUG - dim call: container_list({'include_messages': True, 'layer3domain': '6724-swh-internal-vrf'})
DEBUG - time taken: 0.020
DEBUG - dim result: {'containers': [], 'messages': []}
DEBUG - dim call: container_list({'include_messages': True, 'layer3domain': '6724-swh-store-vrf'})
DEBUG - time taken: 0.021
DEBUG - dim result: {'containers': [], 'messages': []}
DEBUG - dim call: container_list({'include_messages': True, 'layer3domain': '6724-swh-pwcrypt-vrf'})
DEBUG - time taken: 0.021
DEBUG - dim result: {'containers': [], 'messages': []}
DEBUG - dim call: container_list({'include_messages': True, 'layer3domain': '6724-ionos-private-vrf'})
DEBUG - time taken: 0.021
DEBUG - dim result: {'containers': [], 'messages': []}
DEBUG - dim call: container_list({'include_messages': True, 'layer3domain': '6724-mssd-backup-vrf'})
DEBUG - time taken: 0.019
DEBUG - dim result: {'containers': [], 'messages': []}
DEBUG - dim call: container_list({'include_messages': True, 'layer3domain': '6724-mssd-sn2g-vrf'})
DEBUG - time taken: 0.020
DEBUG - dim result: {'containers': [], 'messages': []}
DEBUG - dim call: container_list({'include_messages': True, 'layer3domain': 'nw-services-prelive'})
DEBUG - time taken: 0.020
DEBUG - dim result: {'containers': [], 'messages': []}
DEBUG - dim call: container_list({'include_messages': True, 'layer3domain': 'ICNetwork'})
DEBUG - time taken: 0.021
DEBUG - dim result: {'containers': [], 'messages': []}
layer3domain: default
0.0.0.0/1 (Container) RIR:iana comment:ITOUDP-5051 - IPv4 Internet Part 1 reverse_dns_profile:public-rev
  0.0.0.0/8 (Container) comment:reserved reverse_dns_profile:internal source:RFC5735
    0.0.0.0/8 (Available)
  1.0.0.0/8 (Available)
  2.0.0.0/12 (Available)
--
    fdc8::/13 (Available)
    fdd0::/12 (Available)
    fde0::/11 (Available)


layer3domain: 6724-global
10.0.0.0/8 (Container)
  10.0.0.0/20 (Available)
  10.0.16.0/23 (Available)
  10.0.18.0/23 (Container) comment:'sne ldp for mpls'
    10.0.18.0/23 (Available)

All non-default-layer3domains have an empty result.

That's my config:

dzgaaj@vm-jumphost:~$ cat .ndclirc
server=https://<server_name>/dim
username=<my_username>