Duplicate vlangroup import

[nautobot]

Duplicate vlangroup import

[glennmatthews]

I see that you opened this against nautobot-netbox-importer 1.0.0. Would you mind retesting this against version 1.0.1? I'm unable to reproduce this issue with 1.0.1 and the JSON files you provided.

[pke11y]

Oh sorry Glenn. Thanks a typo. Issue updated.
nautobot-netbox-importer version: 1.0.1

I've added an edited version of the dump. Can you try import this file db_dump_minimal_indented_edited.json and see if it reproduces the error please? I used this file to manually remove the custom field requirements and also avoid the invalid default value error.

[glennmatthews]

Thanks for the update! With the edited JSON file you provided, I still don't see any duplicate-object warnings on VLAN Groups, though I do see some for Racks and for VLANs, as well as a single potential duplicate IP address; I'll investigate these more closely as time permits.

[pke11y]

OK, I've added another data dump db_dump_minimal_indented_edited_2.json to see if that makes a difference.

For ref, I'm getting duplicates for the following models with the db_dump_minimal_indented_edited_2.json dataset:

• rack
• vlan
• ipaddress
• vlangroup

[glennmatthews]

Sorry for the delay! I can confirm that with the latest dump you provided that I can see the same duplicate warnings. This looks to be another instance of the more general issue I'm tracking in #21 - basically the problem is that the database considers two records with a NULL value in one of their unique identifier fields (in your data, in the case of Racks, the group field; in the case of VLANs, the group field as well, etc.) to be distinct (because NULL != NULL in database terms), but Python considers them to be non-unique (because it is representing NULL as None, and None == None in Python). Some design work will be required to figure out how to address this, which I'm working on now.

[pke11y]

Ok, thanks for the update Glenn.

On Thu 1 Apr 2021 at 14:44, Glenn Matthews ***@***.***> wrote: Sorry for the delay! I can confirm that with the latest dump you provided that I can see the same duplicate warnings. This looks to be another instance of the more general issue I'm tracking in #21 <#21> - basically the problem is that the database considers two records with a NULL value in one of their unique identifier fields (in your data, in the case of Racks, the group field; in the case of VLANs, the group field as well, etc.) to be distinct (because NULL != NULL in database terms), but Python considers them to be non-unique (because it is representing NULL as None, and None == None in Python). Some design work will be required to figure out how to address this, which I'm working on now. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#11 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AFQBDUPYGAJPOVPZ7BPDIYDTGR2DBANCNFSM4ZCR6JRA> .

-- Regards, Paddy Sent from Gmail Mobile

[glennmatthews]

Nautobot 1.0.0b3 has been released, and correspondingly, so has version 1.1.0 of this importer plugin, which includes fix #28. I expect that that fix should resolve this issue as well. If you have the chance to re-test with Nautobot 1.0.0b3 + nautobot-netbox-importer 1.1.0 and let me know how it goes, I'd appreciate it!

[pke11y]

Ok thanks Glenn. Will upgrade and try.

…

-- Paddy

On Wed, Apr 7, 2021 at 8:24 PM Glenn Matthews ***@***.***> wrote: Nautobot 1.0.0b3 has been released, and correspondingly, so has version 1.1.0 of this importer plugin, which includes fix #28 <#28>. I expect that that fix should resolve this issue as well. If you have the chance to re-test with Nautobot 1.0.0b3 + nautobot-netbox-importer 1.1.0 and let me know how it goes, I'd appreciate it! — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#11 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AFQBDULVMKCAL7VAJU4GTA3THSWOHANCNFSM4ZCR6JRA> .

[chadell]

@pke11y did you try it?

[pke11y]

Difficult to recall, but I think it's say to assume the issue was fixed. We did a lot of testing around that time.