irrdnet / irrd Goto Github PK

View Code? Open in Web Editor NEW

150.0 18.0 50.0 4.14 MB

IRRd version 4

Home Page: http://irrd.readthedocs.io/en/stable/

License: Other

Python 97.77% Mako 0.03% HTML 2.20%

irrd's Introduction

Internet Routing Registry Daemon (IRRd) Version 4

https://circleci.com/gh/irrdnet/irrd.svg?style=svg

https://readthedocs.org/projects/irrd/badge/?version=stable

Internet Routing Registry daemon version 4 is an IRR database server, processing IRR objects in the RPSL format. Its main features are:

Validating, cleaning and storing IRR data, and extracting information for indexing.
Providing several query interfaces to query the IRR data.
Handling authoritative IRR data, and allowing users with the appropriate authorisation to submit requests to change objects.
Mirroring other IRR databases using file imports and NRTM.
Offering NRTM mirroring and full export services to other databases.

This IRRd version 4 project was originally commissioned in 2018 by NTT and designed and developed by Reliably Coded (known as DashCare until 2021). Since then, Reliably Coded has been maintaining and extending IRRd significantly, for, or with support of, NTT, ARIN, Merit, RIPE NCC Community Projects Fund, LACNIC, Netnod and Internetstiftelsen. This has taken place in the form of development contracts, support contracts, or grants.

Older versions of IRRd are or were in use by various IRR operators. Difficulties with continued maintenance and extension of these older versions led to the IRRd v4 project.

Please see the extensive documentation for more on how to deploy and use IRRd.

irrd's People

Contributors

Stargazers

Watchers

Forkers

itnord netravnen bitcynth xtxerr haussli paulyc snuggles tangledhelix kkirsche troy2914 athna signshop airgapper mirceaulinic icing morrowc meghlakhan36 pwo 1w1521375 traderose sumkincpp tristanbruns reliablycoded adudek benmaddison sjm-steffann briandfoy global-localhost global19 global19-atlassian-net global19-atlassian-net fortplus alopintsev ericparton arineng nicko170 schelcj sichanghe sitedata saulooliveirati wobcom mattkobayashi justin-apnic brunoflores job yu-re-ka jwbensley vidister volodymyr20221 kiraum

irrd's Issues

Decide on handling of RFC variations

Based on the current state of our parser, there are a few areas where we may need to adjust validation.

A full overview of all errors in non-strict mode for all databases is available on irrd01.dashcare.nl:parser_results/nonstrict-others.errors, and a run on the NTTCOM db with strict mode in nttcom.errorsum with the full objects for context listed in nttcom.errors. For the difference between strict and non-strict, see the docs.

The errors about being unable to read public PGP keys should be assumed to be parser bugs at this time, as currently all key-certs generate an error. Other than that, all these errors require a business decision.

In strict mode on NTTCOM:

(6x) MAIL-FROM attributes are used in NTTCOM but will not be supported in IRRDv4.
(40x) as-name is missing on some aut-nums in NTTCOM, but this is mandatory in RFC 2622
(a few hundred times) In many cases, the tech-c/admin-c of objects contains a person or department name, which is not a reference to a nic-hdl of a person/role object. For some or all, there is even no person/role object under that same name. In RFC 2622, these must be nic-handles.

In other databases, on non-strict mode:

(1x) ARIN includes a route object with an origin of 20013, which is not valid as this should be AS20013. As this is a primary key of the route object, validation is enabled for this field even in non-strict.
(1x) RIPE contains one object with an attribute named *mb. Although we generally allow unknown attributes to be added, this is an invalid attribute name syntax - not just one we don't know.
(63x) Objects in ARIN (IPv6) and SAVVIS (IPv4) contain network prefixes in primary key fields which have host bits enabled, e.g. 2605:4d00::1/32 or 23.15.142.1/23.
(8x) Similar to NTTCOM, some tech-c/admin-c contain people's names in RADB - some of which are in this case also a nic-hdl on a person/role.
(6x) Level3 still uses ASDOT notation in a few objects.

These checks verify whether IRRD isn't being too strict, but they don't validate whether it's being too flexible, so we also need to check whether the current list of permitted attributes isn't too permissive.

Lastly, RFC 2622 strictly defines ASCII as the allowed character set, but RIPE, TC, GT, BBOI and ARIN use ISO-8859-1 characters. AFRINIC and APNIC use UTF-8. Many others are just ASCII. What should the permitted character set be, and assuming we don't want to stick to pure 7-bit ASCII, what encoding should be used for queries, NRTM and data export?
(Internally, Python code almost always uses UTF-8, as will the database.)

Configuration for IRRd

Configuration will most likely be stored either in a PostgreSQL database (in which case a very small config is needed to point to the database) or a YAML file. Parameters include:

Parsing issue: ASDOT syntax

as-set:         AS-TEST
descr:          TEST
members:        AS5.1
tech-c:         AAP1-AFRINIC
admin-c:        AAP1-AFRINIC
mnt-by:         AAP-MNT
changed:        ***@afrinic.net 20080922
source:         AFRINIC
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
ERROR: Invalid AS number AS5.1: number part is not numeric
ERROR: Invalid set AS5.1: component AS5.1 is not a valid AS number nor a valid set name

=======================================

as-set:        AS-13407
descr:         One Communications and customer AS
members:       AS-CONV, AS13407, AS13609, AS14751, AS13640,
               AS7956, AS32689, AS32294, AS30205, AS32802,
               AS27249, AS19374, AS25964, AS36451, AS8038,
               AS36068, AS36171, AS22519, AS19339, AS36109,
               AS14480, AS34245, AS11784, AS46155, AS14501,
               AS32281, AS27342, AS12168, AS33592, AS31874,
               AS32830, AS16684, AS46766, AS40690, AS46150,
               AS22130, AS30719, AS18482, AS7153, AS29903,
               AS36544, AS32833, AS32171, AS36314, AS36233,
               AS30596, AS19933, AS40951, AS26836, AS12167,
               AS46254, AS26410, AS11987, AS32610, AS22776,
               AS47004, AS14800, AS25688, AS33486, AS26905,
               AS29869, AS32088, AS32180, AS32180, AS53485,
               AS1204, AS12104, AS22405, AS32043, AS15206,
               AS40737, AS46559, AS25632, AS20342, AS1783,
               AS30473, AS36544, AS31998, AS19961, AS21801,
               AS53551, AS2763, AS13572, AS22421, AS53478,
               AS53537, AS12060, AS23187, AS26287, AS53461,
               AS11370, AS32891, AS46579, AS33564, AS53549,
               AS46386, AS12167, AS6.23, AS6062, AS1773,
               AS10844, AS40295, AS53614, AS53855, AS26029,
               AS30196, AS15099, AS53971, AS14242, AS15231,
               AS10727, AS14242, AS15231, AS10727, AS19079,
               AS25750, AS2738, AS54028, AS33141, AS24008,
               AS36338, AS39982, AS54242, AS54377, AS22421,
               AS26715, AS54625, AS18935, AS11627, AS32102,
               AS23039, AS36437, AS40617, AS35919, AS25639,
               AS55088, AS55059, AS393302, AS20458, AS26366,
               AS32634
tech-c:        BTH1-LEVEL3
mnt-by:        ONECOMM-MNT
changed:       [email protected] 20140129
source:        LEVEL3
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
ERROR: Invalid AS number AS6.23: number part is not numeric
ERROR: Invalid set AS6.23: component AS6.23 is not a valid AS number nor a valid set name

=======================================

as-set:        AS-COLNET
descr:         AS-Set for Columbus Networks
members:       AS8014, AS19429, AS18678, AS6458, AS14754, AS22833,
members:       AS27696, AS28506, AS10269, AS14638, AS27781, AS19447,
members:       AS23550, AS7137, AS10396, AS18840, AS27761, AS30689,
members:       AS15146, AS7910, AS6140, AS12066, AS25607, AS28506,
members:       AS6140, AS27665, AS6503, AS10586, AS11367, AS23520,
members:       AS26505, AS27745, AS27748, AS27805, AS23383, AS17086,
members:       AS26597, AS22698, AS11237, AS27685, AS27742, AS27708,
members:       AS27734, AS15066, AS27650, AS14420, AS22724, AS14522,
members:       AS14187, AS15246, AS14080, AS26603, AS23216, AS14202,
members:       AS27731, AS19114, AS19169, AS26619, AS10620, AS27738,
members:       AS13489, AS7997, AS8065, AS10299, AS18869, AS27758,
members:       AS18768, AS18895, AS26132, AS26611, AS6619, AS22999,
members:       AS16649, AS27737, AS20434, AS22917, AS27027, AS27801,
members:       AS27785, AS14920, AS21510, AS32659, AS33576, AS3586,
members:       AS27810, AS27821, AS27729, AS27654, AS30526, AS8054,
members:       AS27798, AS27695, AS22306, AS19582, AS27784, AS26473,
members:       AS11581, AS7315, AS13878, AS19731, AS26112, AS26596,
members:       AS27698, AS18747, AS26605, AS27753, AS27763, AS27831,
members:       AS7984, AS21578, AS11053, AS27829, AS22539, AS27864,
members:       AS27663, AS23487, AS27820, AS3549, AS40323, AS27886,
members:       AS27902, AS27837, AS27845, AS27789, AS27844, AS27868,
members:       AS27668, AS33392, AS27811, AS27933, AS3816, AS27814,
members:       AS27916, AS10278, AS27762, AS27905, AS39963, AS18881,
members:       AS25887, AS27919, AS27921, AS27759, AS27920, AS20372,
members:       AS27947, AS27887, AS15068, AS27937, AS30440, AS11556,
members:       AS27974, AS27767, AS27757, AS27968, AS26613, AS26124,
members:       AS20299, AS27716, AS27649, AS27842, AS36423, AS26061,
members:       AS27999, AS22888, AS26608, AS27763, AS15246, AS14187,
members:       AS27774, AS27941, AS27998, AS27755, AS27956, AS26426,
members:       AS23414, AS23414, AS27716, AS26608, AS27914, AS27850,
members:       AS10391, AS46650, AS5639, AS28010, AS28014, AS27982,
members:       AS27860, AS27958, AS28027, AS21599, AS27796, AS27990,
members:       AS27873, AS6193, AS27936, AS3551, AS27832, AS27948,
members:       AS11830, AS27740, AS28011, AS22808, AS23541, AS4.0,
members:       AS28043, AS36716, AS32085, AS22581, AS22453, AS28512,
members:       AS28549, AS28018, AS26105, AS23360, AS40336, AS4626,
members:       AS28006, AS47856, AS28058, AS17079, AS27884, AS22869,
members:       AS28039, AS47014, AS28053, AS11081, AS10292, AS26434,
members:       AS26173, AS26608, AS27765, AS27923, AS8163, AS22368,
members:       AS22415, AS27841, AS40168, AS5722, AS27991, AS18747,
members:       AS27773, AS27972, AS20299, AS28042, AS28545, AS7087,
members:       AS27903, AS28062, AS28063, AS53251, AS28064, AS28069,
members:       AS28070, AS18809, AS28025, AS28074, AS27929, AS28072,
members:       AS14974, AS27653, AS27989, AS28021, AS14709, AS27687,
members:       AS27922, AS28005, AS47856, AS28030, AS27649, AS27726,
members:       AS28316, AS6193, AS23360, AS28061, AS22382, AS22010,
members:       AS22227, AS27760, AS27786, AS27990, AS27783, AS27679,
members:       AS27802, AS27812, AS27819, AS27842, AS27860, AS28018,
members:       AS27915, AS27944, AS27938, AS27959, AS28083, AS49933,
members:       AS2134, AS7438, AS14178, AS15236, AS28479, AS28557,
members:       AS28558, AS7864, AS28478, AS27932, AS28088, AS13856,
members:       AS29766, AS27749, AS28085, AS28104
admin-c:       PAA1-LEVEL3
tech-c:        PAA1-LEVEL3
mnt-by:        CN-MNT
changed:       [email protected] 20100113
source:        LEVEL3
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
ERROR: Invalid AS number AS4.0: number part is not numeric
ERROR: Invalid set AS4.0: component AS4.0 is not a valid AS number nor a valid set name

=======================================

as-set:        AS-DIVEO-TRANSIT
descr:         Provide transit to other ASes
members:       AS5745, AS8066, AS10318, AS10417, AS10630, AS11063,
members:       AS12150, AS13316, AS13878, AS14522, AS15034, AS15180,
members:       AS15274, AS16397, AS16712, AS16732, AS16735, AS17255,
members:       AS17401, AS18547, AS18579, AS19037, AS19180, AS19422,
members:       AS19519, AS19960, AS20002, AS12264, AS22129, AS23106,
members:       AS25933, AS11431, AS26090, AS11338, AS17108, AS26607,
members:       AS14187, AS26112, AS25998, AS23201, AS19741, AS11315,
members:       AS26616, AS26104, AS27693, AS14026, AS19182, AS19731,
members:       AS22698, AS14346, AS22453, AS5722, AS28583, AS21574,
members:       AS28588, AS28594, AS28601, AS28608, AS22085, AS28572,
members:       AS28626, AS28660, AS28665, AS28662, AS14457, AS28671,
members:       AS28315, AS26596, AS7195, AS28189, AS28214, AS14080,
members:       AS28228, AS28227, AS27855, AS27992, AS28254, AS28013,
members:       AS3573, AS26599, AS28166, AS53101, AS28358, AS53064,
members:       AS28057, AS4.603
admin-c:       FV298-ARIN
tech-c:        FV298-ARIN
notify:        [email protected]
mnt-by:        DIVEO-MNT
changed:       [email protected] 20110118
source:        LEVEL3
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
ERROR: Invalid AS number AS4.603: number part is not numeric
ERROR: Invalid set AS4.603: component AS4.603 is not a valid AS number nor a valid set name

Suggestion: look into supporting ASDOT syntax, and replacing it with ASPLAIN before accepting the object.

Parsing issue: plain number in origin attribute

=======================================

route:          199.119.212.0/23
descr:          CyrusOne
descr:          Austin IDC
descr:          Austin, TX
origin:         20013
mnt-by:         MNT-CYRS
changed:        [email protected] 20110608
source:         ARIN
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
ERROR: Invalid AS number 20013: must start with 'AS'

Queries: open questions about queries

A few questions for @job, which I haven't been able to clear up using the IRRd source code:

What exactly is searched for in RIPE-style free text queries? E.g. if the only query input is "NTT" or "192.0.2.0/24"?
What are the rules for retrieving related objects? Retrieve all associated tech-c/admin-c/zone-c objects and include them in the result? And for which queries should related objects be included? (provided -r hasn't been included)
What is the -V query flag for? I can't seem to get any results out of it, nor find any documentation on it.

Parsing issue: unclear encodings

The different databases seem to use inconsistent encodings. The dumps received from rr.ntt.net were guessed as:

afrinic.db: utf-8 with confidence 0.99
altdb.db: Windows-1252 with confidence 0.73
apnic.db: Windows-1254 with confidence 0.4888600622828866
arin-whois.db: ascii with confidence 1.0
arin.db: Windows-1252 with confidence 0.7299344525030538
bboi.db: ISO-8859-1 with confidence 0.73
bell.db: ascii with confidence 1.0
gt.db: ISO-8859-1 with confidence 0.73
internal.db: ascii with confidence 1.0
jpirr.db: ascii with confidence 1.0
level3.db: Windows-1252 with confidence 0.7299986263799446
nttcom.db: ISO-2022-JP with confidence 0.99
radb.db: Windows-1254 with confidence 0.5017295329798788
rgnet.db: ascii with confidence 1.0
ripe.db: Windows-1254 with confidence 0.45201648599143607
savvis.db: ISO-8859-1 with confidence 0.73
tc.db: Windows-1254 with confidence 0.4542790826372332

RFC 2622 strictly defines ASCII as the allowed character set, but we can clearly see that a wide variation is used.

Questions:

What should the permitted character set be for objects submitted IRRD?
What encoding should IRRD use for queries, NRTM and data export?

For importing / NRTM receiving there should be a configuration option for the expected encoding of the mirrored database. Although that only partially helps, because I'm not sure on all the encodings above - I'd expect RIPE to be more likely as ISO-8859-1 than Windows-1254.

(Internally, IRRD will use UTF-8 exclusively, but we can convert to/from different encodings easily.)

Parsing issue: invalid attribute *mb in RIPE

mntner:         mine
admin-c:        DUMY-RIPE
upd-to:         [email protected]
auth:           MD5-PW $1$SaltSalt$DummifiedMD5HashValue.   # Real value hidden for security
mb:             Mine
*mb: Mine
mnt-by:         mine
created:        2010-11-02T13:17:09Z
last-modified:  2017-08-21T09:21:05Z
source:         RIPE
remarks:        ****************************
remarks:        * THIS OBJECT IS MODIFIED
remarks:        * Please note that all data that is generally regarded as personal
remarks:        * data has been removed from this object.
remarks:        * To view the original object, please query the RIPE Database at:
remarks:        * http://www.ripe.net/whois
remarks:        ****************************
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
ERROR: Line 6: encountered malformed attribute name: [*mb]

(As previously discussed)

Generally we accept unknown attributes, but *mb is an invalid name (mb is fine, but we ignore it). RIPE ticket #23121.

Parsing issue: dashes in AS numbers in ARIN

as-set:         AS-23016
descr:          Heavyswitch LLC
descr:          1 NE 1st St.
descr:          Miami FL 33132
descr:          US
mnt-by:         MNT-HEAVY6
members:        AS-23016:AS-BACKBONE AS-23016:AS-CUSTOMERS
admin-c:        CRESC-ARIN
tech-c:         NETWO3882-ARIN
changed:        [email protected] 20100830
source:         ARIN
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
ERROR: Invalid AS number AS-23016:AS-BACKBONE AS-23016:AS-CUSTOMERS: number part is not numeric
ERROR: Invalid set AS-23016:AS-BACKBONE AS-23016:AS-CUSTOMERS: component AS-BACKBONE AS-23016 is not a valid AS number nor a valid set name

=======================================

as-set:         AS-8100
descr:          IPTelligent LLC
descr:          36 NE 2nd St.
descr:          Miami FL 33132
descr:          US
mnt-by:         MNT-IPTEL-1
members:        AS-8100:AS-BACKBONE AS-8100:AS-CUSTOMERS
admin-c:        RGC5-ARIN
tech-c:         NOC3572-ARIN
changed:        [email protected] 20101104
source:         ARIN
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
ERROR: Invalid AS number AS-8100:AS-BACKBONE AS-8100:AS-CUSTOMERS: number part is not numeric
ERROR: Invalid set AS-8100:AS-BACKBONE AS-8100:AS-CUSTOMERS: component AS-BACKBONE AS-8100 is not a valid AS number nor a valid set name

I suppose we could allow the dash and just remove it, if the AS otherwise seems valid?

Parsing issue: space-separated AS list

as-set:         AS-MQCUSTOMERS
descr:          MarquisNet LLC
descr:          7185 Pollock Drive
descr:          Las Vegas, NV 89119
descr:          US
tech-c:         ENGIN16-ARIN
admin-c:        ENGIN16-ARIN
members:        AS10270 AS36286 AS20104 AS23521 AS22837 AS13446 AS27193
mnt-by:         MNT-MARQU-3
changed:        [email protected] 20100721
source:         ARIN
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
ERROR: Invalid AS number AS10270 AS36286 AS20104 AS23521 AS22837 AS13446 AS27193: number part is not numeric
ERROR: Invalid set AS10270 AS36286 AS20104 AS23521 AS22837 AS13446 AS27193: component AS10270 AS36286 AS20104 AS23521 AS22837 AS13446 AS27193 is not a valid AS number nor a valid set name

This object is invalid, and we parse members, being a lookup attribute. It is also required for as-set resolving.

Parsing issue: host bits enabled in many databases

In about 400 cases, host bits are enabled in prefixes. This affects AFRINIC, ARIN, BELL (only one object affected), LEVEL3 (only once), RIPE and SAVVIS. In many cases, this concerns member attributes, but also the primary key of a route object.

Some examples:

route:        216.89.79.34/27
descr:        Interglobe cidr block
origin:       AS17248
mnt-by:       MAINT-AS17248
changed:      [email protected] 20070123
source:       SAVVIS
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
ERROR: Invalid address prefix: 216.89.79.34/27: IP('216.89.79.34/27') has invalid prefix length (27)

=======================================

route:        23.15.208.1/20
descr:        Peer1 Route Object
origin:       AS13768
mnt-by:       maint-as13768
changed:      [email protected] 20110627
source:       SAVVIS
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
ERROR: Invalid address prefix: 23.15.208.1/20: IP('23.15.208.1/20') has invalid prefix length (20)

=======================================

route6:         2a07:1c44:711::/44
descr:          IONSWITCH-SEA-IP6-ext
origin:         AS395970
mnt-by:         MNT-IL-446
changed:        [email protected] 20170420
source:         ARIN
remarks:        ****************************
remarks:        * THIS OBJECT CONTAINS PLACEHOLDER DATA
remarks:        * Please note that all data that is generally regarded as personal
remarks:        * data has been removed from this object.
remarks:        * To view the original object, please query the ARIN Database at:
remarks:        * http://www.arin.net/whois
remarks:        ****************************
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
ERROR: Invalid address prefix: 2a07:1c44:711::/44: IP('2a07:1c44:711::/44') has invalid prefix length (44)


=======================================

route-set:      RS-ROKE-FORIS
descr:          FORIS prefixes advertised through ROKE Telkom
members:        AS37048
members:        41.190.192.0/19
members:        41.190.192.0/20
members:        41.190.208.0/20
members:        41.190.192.0/21
members:        41.190.200.8/21
members:        41.190.208.16/21
members:        41.190.216.24/21
members:        41.190.192.0/22
members:        41.190.196.4/22
members:        41.190.200.8/22
members:        41.190.204.12/22
members:        41.190.208.16/22
members:        41.190.212.20/22
members:        41.190.216.24/22
members:        41.190.220.28/22
members:        41.190.192.0/23
members:        41.190.194.0/23
members:        41.190.196.0/23
members:        41.190.198.0/23
members:        41.190.200.0/23
members:        41.190.202.0/23
members:        41.190.204.0/23
members:        41.190.206.0/23
members:        41.190.208.0/23
members:        41.190.210.0/23
members:        41.190.212.0/23
members:        41.190.214.0/23
members:        41.190.216.0/23
members:        41.190.218.0/23
members:        41.190.220.0/23
members:        41.190.222.0/23
members:        41.190.192.0/24
members:        41.190.193.0/24
members:        41.190.194.0/24
members:        41.190.195.0/24
members:        41.190.196.0/24
members:        41.190.197.0/24
members:        41.190.198.0/24
members:        41.190.199.0/24
members:        41.190.200.0/24
members:        41.190.201.0/24
members:        41.190.202.0/24
members:        41.190.203.0/24
members:        41.190.204.0/24
members:        41.190.205.0/24
members:        41.190.206.0/24
members:        41.190.207.0/24
members:        41.190.208.0/24
members:        41.190.209.0/24
members:        41.190.210.0/24
members:        41.190.211.0/24
members:        41.190.212.0/24
members:        41.190.213.0/24
members:        41.190.214.0/24
members:        41.190.215.0/24
members:        41.190.216.0/24
members:        41.190.217.0/24
members:        41.190.218.0/24
members:        41.190.219.0/24
members:        41.190.220.0/24
members:        41.190.221.0/24
members:        41.190.222.0/24
members:        41.190.223.0/24
tech-c:         DUMY-RIPE
admin-c:        DUMY-RIPE
mnt-by:         ROKE-MNT
created:        2016-03-24T06:09:19Z
last-modified:  2016-09-28T07:50:30Z
source:         RIPE
remarks:        ****************************
remarks:        * THIS OBJECT IS MODIFIED
remarks:        * Please note that all data that is generally regarded as personal
remarks:        * data has been removed from this object.
remarks:        * To view the original object, please query the RIPE Database at:
remarks:        * http://www.ripe.net/whois
remarks:        ****************************
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
ERROR: Value is neither a valid set name nor a valid prefix: 41.190.200.8/21: IP('41.190.200.8/21') has invalid prefix length (21)
ERROR: Value is neither a valid set name nor a valid prefix: 41.190.208.16/21: IP('41.190.208.16/21') has invalid prefix length (21)
ERROR: Value is neither a valid set name nor a valid prefix: 41.190.216.24/21: IP('41.190.216.24/21') has invalid prefix length (21)
ERROR: Value is neither a valid set name nor a valid prefix: 41.190.196.4/22: IP('41.190.196.4/22') has invalid prefix length (22)
ERROR: Value is neither a valid set name nor a valid prefix: 41.190.200.8/22: IP('41.190.200.8/22') has invalid prefix length (22)
ERROR: Value is neither a valid set name nor a valid prefix: 41.190.204.12/22: IP('41.190.204.12/22') has invalid prefix length (22)
ERROR: Value is neither a valid set name nor a valid prefix: 41.190.208.16/22: IP('41.190.208.16/22') has invalid prefix length (22)
ERROR: Value is neither a valid set name nor a valid prefix: 41.190.212.20/22: IP('41.190.212.20/22') has invalid prefix length (22)
ERROR: Value is neither a valid set name nor a valid prefix: 41.190.216.24/22: IP('41.190.216.24/22') has invalid prefix length (22)
ERROR: Value is neither a valid set name nor a valid prefix: 41.190.220.28/22: IP('41.190.220.28/22') has invalid prefix length (22)

Full log of all unique errors (#61 was already made to clean these a little bit):

errors.afrinic.db:ERROR: Value is neither a valid set name nor a valid prefix: 105.60.0.0/13: IP('105.60.0.0/13') has invalid prefix length (13)
errors.afrinic.db:ERROR: Value is neither a valid set name nor a valid prefix: 196.223.253.0/23: IP('196.223.253.0/23') has invalid prefix length (23)
errors.afrinic.db:ERROR: Value is neither a valid set name nor a valid prefix: 196.43.208.5/24: IP('196.43.208.5/24') has invalid prefix length (24)
errors.arin.db:ERROR: Invalid address prefix: 2001:420:3400::/34: IP('2001:420:3400::/34') has invalid prefix length (34)
errors.arin.db:ERROR: Invalid address prefix: 2001:470:33::1/48: IP('2001:470:33::1/48') has invalid prefix length (48)
errors.arin.db:ERROR: Invalid address prefix: 2600:C04:1002:C::2/64: IP('2600:c04:1002:c::2/64') has invalid prefix length (64)
errors.arin.db:ERROR: Invalid address prefix: 2605:4d00::1/32: IP('2605:4d00::1/32') has invalid prefix length (32)
errors.arin.db:ERROR: Invalid address prefix: 2620:10A:E0::/40: IP('2620:10a:e0::/40') has invalid prefix length (40)
errors.arin.db:ERROR: Invalid address prefix: 2a07:1c44:711::/44: IP('2a07:1c44:711::/44') has invalid prefix length (44)
errors.arin.db:ERROR: Value is neither a valid set name nor a valid prefix: 192.225.29.0/22: IP('192.225.29.0/22') has invalid prefix length (22)
errors.arin.db:ERROR: Value is neither a valid set name nor a valid prefix: 198.187.213.0/23: IP('198.187.213.0/23') has invalid prefix length (23)
errors.arin.db:ERROR: Value is neither a valid set name nor a valid prefix: 200.13.200.0/20: IP('200.13.200.0/20') has invalid prefix length (20)
errors.arin.db:ERROR: Value is neither a valid set name nor a valid prefix: 208.42.199.1/24: IP('208.42.199.1/24') has invalid prefix length (24)
errors.arin.db:ERROR: Value is neither a valid set name nor a valid prefix: 209.67.253.0/23: IP('209.67.253.0/23') has invalid prefix length (23)
errors.bell.db:ERROR: Value is neither a valid set name nor a valid prefix: 192.197.180.0/16: IP('192.197.180.0/16') has invalid prefix length (16)
errors.bell.db:ERROR: Value is neither a valid set name nor a valid prefix: 192.197.181.0/16: IP('192.197.181.0/16') has invalid prefix length (16)
errors.bell.db:ERROR: Value is neither a valid set name nor a valid prefix: 192.197.182.0/16: IP('192.197.182.0/16') has invalid prefix length (16)
errors.bell.db:ERROR: Value is neither a valid set name nor a valid prefix: 192.197.183.0/16: IP('192.197.183.0/16') has invalid prefix length (16)
errors.bell.db:ERROR: Value is neither a valid set name nor a valid prefix: 192.197.184.0/16: IP('192.197.184.0/16') has invalid prefix length (16)
errors.bell.db:ERROR: Value is neither a valid set name nor a valid prefix: 207.34.32.0/16: IP('207.34.32.0/16') has invalid prefix length (16)
errors.bell.db:ERROR: Value is neither a valid set name nor a valid prefix: 207.34.33.0/16: IP('207.34.33.0/16') has invalid prefix length (16)
errors.bell.db:ERROR: Value is neither a valid set name nor a valid prefix: 207.34.34.0/16: IP('207.34.34.0/16') has invalid prefix length (16)
errors.bell.db:ERROR: Value is neither a valid set name nor a valid prefix: 207.34.35.0/16: IP('207.34.35.0/16') has invalid prefix length (16)
errors.bell.db:ERROR: Value is neither a valid set name nor a valid prefix: 207.34.36.0/16: IP('207.34.36.0/16') has invalid prefix length (16)
errors.bell.db:ERROR: Value is neither a valid set name nor a valid prefix: 207.34.37.0/16: IP('207.34.37.0/16') has invalid prefix length (16)
errors.bell.db:ERROR: Value is neither a valid set name nor a valid prefix: 207.34.38.0/16: IP('207.34.38.0/16') has invalid prefix length (16)
errors.bell.db:ERROR: Value is neither a valid set name nor a valid prefix: 207.34.39.0/16: IP('207.34.39.0/16') has invalid prefix length (16)
errors.bell.db:ERROR: Value is neither a valid set name nor a valid prefix: 207.34.40.0/16: IP('207.34.40.0/16') has invalid prefix length (16)
errors.bell.db:ERROR: Value is neither a valid set name nor a valid prefix: 207.34.41.0/16: IP('207.34.41.0/16') has invalid prefix length (16)
errors.bell.db:ERROR: Value is neither a valid set name nor a valid prefix: 207.34.42.0/16: IP('207.34.42.0/16') has invalid prefix length (16)
errors.bell.db:ERROR: Value is neither a valid set name nor a valid prefix: 207.34.43.0/16: IP('207.34.43.0/16') has invalid prefix length (16)
errors.bell.db:ERROR: Value is neither a valid set name nor a valid prefix: 207.34.44.0/16: IP('207.34.44.0/16') has invalid prefix length (16)
errors.bell.db:ERROR: Value is neither a valid set name nor a valid prefix: 207.34.45.0/16: IP('207.34.45.0/16') has invalid prefix length (16)
errors.bell.db:ERROR: Value is neither a valid set name nor a valid prefix: 207.34.46.0/16: IP('207.34.46.0/16') has invalid prefix length (16)
errors.bell.db:ERROR: Value is neither a valid set name nor a valid prefix: 207.34.47.0/16: IP('207.34.47.0/16') has invalid prefix length (16)
errors.level3.db:ERROR: Value is neither a valid set name nor a valid prefix: 63.245.72.0/20: IP('63.245.72.0/20') has invalid prefix length (20)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 103.1.45.0/23: IP('103.1.45.0/23') has invalid prefix length (23)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 103.11.37.0/23: IP('103.11.37.0/23') has invalid prefix length (23)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 154.117.239.0/20: IP('154.117.239.0/20') has invalid prefix length (20)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 154.117.254.0/22: IP('154.117.254.0/22') has invalid prefix length (22)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 169.254.0.0/0: IP('169.254.0.0/0') has invalid prefix length (0)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 170.252.85.0/23: IP('170.252.85.0/23') has invalid prefix length (23)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 178.20.87.0/21: IP('178.20.87.0/21') has invalid prefix length (21)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 192.176.121.0/23: IP('192.176.121.0/23') has invalid prefix length (23)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 192.194.136.0/20: IP('192.194.136.0/20') has invalid prefix length (20)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 193.104.188.0/2: IP('193.104.188.0/2') has invalid prefix length (2)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 193.149.1.106/25: IP('193.149.1.106/25') has invalid prefix length (25)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 193.188.137.162/24: IP('193.188.137.162/24') has invalid prefix length (24)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 193.218.0.66/24: IP('193.218.0.66/24') has invalid prefix length (24)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 193.238.188.0/21: IP('193.238.188.0/21') has invalid prefix length (21)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 193.42.155.39/24: IP('193.42.155.39/24') has invalid prefix length (24)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 194.146.118.77/24: IP('194.146.118.77/24') has invalid prefix length (24)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 194.53.172.111/25: IP('194.53.172.111/25') has invalid prefix length (25)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 194.68.123.199/24: IP('194.68.123.199/24') has invalid prefix length (24)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 194.68.128.199/24: IP('194.68.128.199/24') has invalid prefix length (24)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 194.9.117.60/24: IP('194.9.117.60/24') has invalid prefix length (24)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 195.182.218.151/23: IP('195.182.218.151/23') has invalid prefix length (23)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 195.245.240.199/24: IP('195.245.240.199/24') has invalid prefix length (24)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 195.66.225.111/22: IP('195.66.225.111/22') has invalid prefix length (22)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 195.66.237.111/22: IP('195.66.237.111/22') has invalid prefix length (22)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 195.69.119.199/24: IP('195.69.119.199/24') has invalid prefix length (24)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 195.69.144.229/22: IP('195.69.144.229/22') has invalid prefix length (22)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 195.69.145.229/22: IP('195.69.145.229/22') has invalid prefix length (22)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 196.202.222.0/22: IP('196.202.222.0/22') has invalid prefix length (22)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 197.232.17.0/23: IP('197.232.17.0/23') has invalid prefix length (23)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 198.32.118.114/24: IP('198.32.118.114/24') has invalid prefix length (24)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 198.32.125.70/23: IP('198.32.125.70/23') has invalid prefix length (23)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 198.32.132.118/24: IP('198.32.132.118/24') has invalid prefix length (24)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 198.32.176.206/24: IP('198.32.176.206/24') has invalid prefix length (24)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 2001:478:124::70/48: IP('2001:478:124::70/48') has invalid prefix length (48)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 2001:478:132::118/64: IP('2001:478:132::118/64') has invalid prefix length (64)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 2001:504:0:2::5580:1/64: IP('2001:504:0:2::5580:1/64') has invalid prefix length (64)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 2001:504:0:4::5580:1/64: IP('2001:504:0:4::5580:1/64') has invalid prefix length (64)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 2001:504:0:5::5580:1/64: IP('2001:504:0:5::5580:1/64') has invalid prefix length (64)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 2001:504:16::15cc/64: IP('2001:504:16::15cc/64') has invalid prefix length (64)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 2001:504:17:115::25/64: IP('2001:504:17:115::25/64') has invalid prefix length (64)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 2001:504:d::5580:1/64: IP('2001:504:d::5580:1/64') has invalid prefix length (64)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 2001:504:f::72/64: IP('2001:504:f::72/64') has invalid prefix length (64)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 2001:7f8:14::3a:1/64: IP('2001:7f8:14::3a:1/64') has invalid prefix length (64)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 2001:7f8:1::a500:5580:1/64: IP('2001:7f8:1::a500:5580:1/64') has invalid prefix length (64)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 2001:7f8:1::a500:5580:2/64: IP('2001:7f8:1::a500:5580:2/64') has invalid prefix length (64)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 2001:7f8:24::ad/64: IP('2001:7f8:24::ad/64') has invalid prefix length (64)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 2001:7f8:26::a500:5580:1/64: IP('2001:7f8:26::a500:5580:1/64') has invalid prefix length (64)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 2001:7f8:35::5580:1/64: IP('2001:7f8:35::5580:1/64') has invalid prefix length (64)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 2001:7f8:42::a500:5580:1/64: IP('2001:7f8:42::a500:5580:1/64') has invalid prefix length (64)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 2001:7f8:4:1::15cc:1/64: IP('2001:7f8:4:1::15cc:1/64') has invalid prefix length (64)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 2001:7f8:4::15cc:1/64: IP('2001:7f8:4::15cc:1/64') has invalid prefix length (64)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 2001:7f8:54::9/64: IP('2001:7f8:54::9/64') has invalid prefix length (64)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 2001:7f8:8:15:0:15cc:0:1/64: IP('2001:7f8:8:15:0:15cc:0:1/64') has invalid prefix length (64)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 2001:7f8:8:20:0:15cc:0:1/64: IP('2001:7f8:8:20:0:15cc:0:1/64') has invalid prefix length (64)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 2001:7f8:8::15cc:0:1/64: IP('2001:7f8:8::15cc:0:1/64') has invalid prefix length (64)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 2001:7f8::15cc:0:1/64: IP('2001:7f8::15cc:0:1/64') has invalid prefix length (64)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 2001:7f8::15cc:0:2/64: IP('2001:7f8::15cc:0:2/64') has invalid prefix length (64)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 2001:7f8:b:100:1d1:a5d0:5580:107/64: IP('2001:7f8:b:100:1d1:a5d0:5580:107/64') has invalid prefix length (64)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 2001:7f8:d:fb::199/64: IP('2001:7f8:d:fb::199/64') has invalid prefix length (64)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 2001:7f8:d:fc::199/64: IP('2001:7f8:d:fc::199/64') has invalid prefix length (64)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 2001:7f8:d:fe::199/64: IP('2001:7f8:d:fe::199/64') has invalid prefix length (64)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 2001:7f8:d:ff::199/64: IP('2001:7f8:d:ff::199/64') has invalid prefix length (64)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 2001:7f8:f::51/64: IP('2001:7f8:f::51/64') has invalid prefix length (64)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 206.126.115.25/24: IP('206.126.115.25/24') has invalid prefix length (24)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 206.126.236.204/22: IP('206.126.236.204/22') has invalid prefix length (22)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 206.223.118.117/24: IP('206.223.118.117/24') has invalid prefix length (24)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 206.223.119.45/24: IP('206.223.119.45/24') has invalid prefix length (24)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 206.223.143.212/24: IP('206.223.143.212/24') has invalid prefix length (24)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 206.81.81.17/23: IP('206.81.81.17/23') has invalid prefix length (23)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 217.29.66.107/23: IP('217.29.66.107/23') has invalid prefix length (23)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 2a02:e982::/29: IP('2a02:e982::/29') has invalid prefix length (29)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 37.49.236.9/23: IP('37.49.236.9/23') has invalid prefix length (23)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 41.138.223.0/23: IP('41.138.223.0/23') has invalid prefix length (23)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 41.190.196.4/22: IP('41.190.196.4/22') has invalid prefix length (22)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 41.190.200.8/21: IP('41.190.200.8/21') has invalid prefix length (21)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 41.190.200.8/22: IP('41.190.200.8/22') has invalid prefix length (22)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 41.190.204.12/22: IP('41.190.204.12/22') has invalid prefix length (22)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 41.190.208.16/21: IP('41.190.208.16/21') has invalid prefix length (21)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 41.190.208.16/22: IP('41.190.208.16/22') has invalid prefix length (22)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 41.190.212.20/22: IP('41.190.212.20/22') has invalid prefix length (22)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 41.190.216.24/21: IP('41.190.216.24/21') has invalid prefix length (21)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 41.190.216.24/22: IP('41.190.216.24/22') has invalid prefix length (22)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 41.190.220.28/22: IP('41.190.220.28/22') has invalid prefix length (22)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 41.221.95.0/2: IP('41.221.95.0/2') has invalid prefix length (2)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 41.75.191.0/22: IP('41.75.191.0/22') has invalid prefix length (22)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 41.84.223.0/22: IP('41.84.223.0/22') has invalid prefix length (22)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 41.86.122.0/20: IP('41.86.122.0/20') has invalid prefix length (20)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 43.230.103.0/23: IP('43.230.103.0/23') has invalid prefix length (23)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 46.43.64.66/23: IP('46.43.64.66/23') has invalid prefix length (23)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 46.43.64.68/22: IP('46.43.64.68/22') has invalid prefix length (22)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 46.43.64.72/22: IP('46.43.64.72/22') has invalid prefix length (22)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 46.43.64.76/24: IP('46.43.64.76/24') has invalid prefix length (24)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 46.43.64.77/24: IP('46.43.64.77/24') has invalid prefix length (24)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 46.43.64.78/24: IP('46.43.64.78/24') has invalid prefix length (24)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 46.43.64.79/24: IP('46.43.64.79/24') has invalid prefix length (24)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 62.69.144.1/23: IP('62.69.144.1/23') has invalid prefix length (23)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 62.69.146.1/23: IP('62.69.146.1/23') has invalid prefix length (23)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 64.18.215.0/23: IP('64.18.215.0/23') has invalid prefix length (23)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 80.81.194.66/22: IP('80.81.194.66/22') has invalid prefix length (22)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 80.81.195.66/22: IP('80.81.195.66/22') has invalid prefix length (22)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 81.178.224.0/17: IP('81.178.224.0/17') has invalid prefix length (17)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 81.179.64.0/17: IP('81.179.64.0/17') has invalid prefix length (17)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 85.210.192.0/17: IP('85.210.192.0/17') has invalid prefix length (17)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 91.194.218.16/24: IP('91.194.218.16/24') has invalid prefix length (24)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 91.206.52.173/23: IP('91.206.52.173/23') has invalid prefix length (23)
errors.ripe.db:ERROR: Value is neither a valid set name nor a valid prefix: 91.210.16.223/22: IP('91.210.16.223/22') has invalid prefix length (22)
errors.savvis.db:ERROR: Invalid address prefix: 107.191.186.1/24: IP('107.191.186.1/24') has invalid prefix length (24)
errors.savvis.db:ERROR: Invalid address prefix: 107.191.187.1/24: IP('107.191.187.1/24') has invalid prefix length (24)
errors.savvis.db:ERROR: Invalid address prefix: 107.191.188.1/24: IP('107.191.188.1/24') has invalid prefix length (24)
errors.savvis.db:ERROR: Invalid address prefix: 107.191.189.1/24: IP('107.191.189.1/24') has invalid prefix length (24)
errors.savvis.db:ERROR: Invalid address prefix: 107.191.190.1/24: IP('107.191.190.1/24') has invalid prefix length (24)
errors.savvis.db:ERROR: Invalid address prefix: 107.191.191.1/24: IP('107.191.191.1/24') has invalid prefix length (24)
errors.savvis.db:ERROR: Invalid address prefix: 109.205.119.0/21: IP('109.205.119.0/21') has invalid prefix length (21)
errors.savvis.db:ERROR: Invalid address prefix: 118.91.8.0/20: IP('118.91.8.0/20') has invalid prefix length (20)
errors.savvis.db:ERROR: Invalid address prefix: 12.45.97.80/21: IP('12.45.97.80/21') has invalid prefix length (21)
errors.savvis.db:ERROR: Invalid address prefix: 151.185.64.0/16: IP('151.185.64.0/16') has invalid prefix length (16)
errors.savvis.db:ERROR: Invalid address prefix: 162.251.164.253/22: IP('162.251.164.253/22') has invalid prefix length (22)
errors.savvis.db:ERROR: Invalid address prefix: 186.148.222.0/22: IP('186.148.222.0/22') has invalid prefix length (22)
errors.savvis.db:ERROR: Invalid address prefix: 188.65.31.0/23: IP('188.65.31.0/23') has invalid prefix length (23)
errors.savvis.db:ERROR: Invalid address prefix: 192.225.213.0/23: IP('192.225.213.0/23') has invalid prefix length (23)
errors.savvis.db:ERROR: Invalid address prefix: 195.229.27.221/30: IP('195.229.27.221/30') has invalid prefix length (30)
errors.savvis.db:ERROR: Invalid address prefix: 196.10.139.0/23: IP('196.10.139.0/23') has invalid prefix length (23)
errors.savvis.db:ERROR: Invalid address prefix: 198.6.16.0/19: IP('198.6.16.0/19') has invalid prefix length (19)
errors.savvis.db:ERROR: Invalid address prefix: 199.36.69.0/23: IP('199.36.69.0/23') has invalid prefix length (23)
errors.savvis.db:ERROR: Invalid address prefix: 201.229.110.0/22: IP('201.229.110.0/22') has invalid prefix length (22)
errors.savvis.db:ERROR: Invalid address prefix: 203.228.216.0/20: IP('203.228.216.0/20') has invalid prefix length (20)
errors.savvis.db:ERROR: Invalid address prefix: 204.10.107.0/23: IP('204.10.107.0/23') has invalid prefix length (23)
errors.savvis.db:ERROR: Invalid address prefix: 205.157.151.224/26: IP('205.157.151.224/26') has invalid prefix length (26)
errors.savvis.db:ERROR: Invalid address prefix: 206.98.113.201/29: IP('206.98.113.201/29') has invalid prefix length (29)
errors.savvis.db:ERROR: Invalid address prefix: 208.122.23.0/22: IP('208.122.23.0/22') has invalid prefix length (22)
errors.savvis.db:ERROR: Invalid address prefix: 208.77.134.0/22: IP('208.77.134.0/22') has invalid prefix length (22)
errors.savvis.db:ERROR: Invalid address prefix: 208.94.225.0/21: IP('208.94.225.0/21') has invalid prefix length (21)
errors.savvis.db:ERROR: Invalid address prefix: 216.135.96.101/24: IP('216.135.96.101/24') has invalid prefix length (24)
errors.savvis.db:ERROR: Invalid address prefix: 216.52.0.1/21: IP('216.52.0.1/21') has invalid prefix length (21)
errors.savvis.db:ERROR: Invalid address prefix: 216.59.57.0/21: IP('216.59.57.0/21') has invalid prefix length (21)
errors.savvis.db:ERROR: Invalid address prefix: 216.89.78.162/29: IP('216.89.78.162/29') has invalid prefix length (29)
errors.savvis.db:ERROR: Invalid address prefix: 216.89.79.34/27: IP('216.89.79.34/27') has invalid prefix length (27)
errors.savvis.db:ERROR: Invalid address prefix: 23.15.142.1/23: IP('23.15.142.1/23') has invalid prefix length (23)
errors.savvis.db:ERROR: Invalid address prefix: 23.15.204.1/22: IP('23.15.204.1/22') has invalid prefix length (22)
errors.savvis.db:ERROR: Invalid address prefix: 23.15.208.1/20: IP('23.15.208.1/20') has invalid prefix length (20)
errors.savvis.db:ERROR: Invalid address prefix: 24.137.34.0/22: IP('24.137.34.0/22') has invalid prefix length (22)
errors.savvis.db:ERROR: Invalid address prefix: 37.34.192.0/17: IP('37.34.192.0/17') has invalid prefix length (17)
errors.savvis.db:ERROR: Invalid address prefix: 41.202.174.0/22: IP('41.202.174.0/22') has invalid prefix length (22)
errors.savvis.db:ERROR: Invalid address prefix: 41.84.223.0/22: IP('41.84.223.0/22') has invalid prefix length (22)
errors.savvis.db:ERROR: Invalid address prefix: 46.32.96.120/21: IP('46.32.96.120/21') has invalid prefix length (21)
errors.savvis.db:ERROR: Invalid address prefix: 61.5.190.0/20: IP('61.5.190.0/20') has invalid prefix length (20)
errors.savvis.db:ERROR: Invalid address prefix: 63.141.38.0/21: IP('63.141.38.0/21') has invalid prefix length (21)
errors.savvis.db:ERROR: Invalid address prefix: 64.152.237.0/22: IP('64.152.237.0/22') has invalid prefix length (22)
errors.savvis.db:ERROR: Invalid address prefix: 64.202.167.0/21: IP('64.202.167.0/21') has invalid prefix length (21)
errors.savvis.db:ERROR: Invalid address prefix: 64.243.224.194/27: IP('64.243.224.194/27') has invalid prefix length (27)
errors.savvis.db:ERROR: Invalid address prefix: 64.243.224.225/27: IP('64.243.224.225/27') has invalid prefix length (27)
errors.savvis.db:ERROR: Invalid address prefix: 64.74.100.253/24: IP('64.74.100.253/24') has invalid prefix length (24)
errors.savvis.db:ERROR: Invalid address prefix: 64.75.81.0/22: IP('64.75.81.0/22') has invalid prefix length (22)
errors.savvis.db:ERROR: Invalid address prefix: 65.114.21.0/16: IP('65.114.21.0/16') has invalid prefix length (16)
errors.savvis.db:ERROR: Invalid address prefix: 66.151.248.128/19: IP('66.151.248.128/19') has invalid prefix length (19)
errors.savvis.db:ERROR: Invalid address prefix: 69.25.196.1/24: IP('69.25.196.1/24') has invalid prefix length (24)
errors.savvis.db:ERROR: Invalid address prefix: 69.90.117.1/24: IP('69.90.117.1/24') has invalid prefix length (24)
errors.savvis.db:ERROR: Invalid address prefix: 71.19.230.0/21: IP('71.19.230.0/21') has invalid prefix length (21)
errors.savvis.db:ERROR: Invalid address prefix: 72.162.128.128/20: IP('72.162.128.128/20') has invalid prefix length (20)
errors.savvis.db:ERROR: Invalid address prefix: 72.162.192.80/20: IP('72.162.192.80/20') has invalid prefix length (20)
errors.savvis.db:ERROR: Invalid address prefix: 76.10.159.0/18: IP('76.10.159.0/18') has invalid prefix length (18)
errors.savvis.db:ERROR: Invalid address prefix: 91.147.191.0/21: IP('91.147.191.0/21') has invalid prefix length (21)
errors.savvis.db:ERROR: Invalid address prefix: 94.128.16.0/17: IP('94.128.16.0/17') has invalid prefix length (17)

Query: complex query parsing and resolving

This expands upon #17 by implementing parsing and resolving for more complex queries.

Hide mirrored & reject authoritative objects based on selection criteria

We'll need the ability to hide objects coming in over NRTM, and reject objects slated for insertion in the authoritative database based on selection criteria such as:

Is the prefix a covered by a list of bogon prefixes? (aka ability to ignore rfc1918 space)
Is the origin ASN a member of the list of bogon ASNs? (ignore route objects originated by private or special ASNs such as 23456)

Database design and interface

This task involves building a basic database design, and implementing it, probably with an ORM, probably SQLAlchemy. (The interface between the database layer and other parts of IRRD should obviously be ORM-agnostic). In design, we need to account for issues like the references between objects, and the various methods of searching by resource or lookup keys. We only need to consider PostgreSQL.

Fix timezone issues regarding updated/created in DB

The internal created/updated timestamps in the DB appear to have inconsistent timezones. This should be checked and fixed.

Prevent expansion of 'unreasonably' large members of an as-set

I'd like to automatically prevent expansion of the top 100 largest AS-SETs when they are included in other AS-SETs.

In other words, if anyone includes AS2914:AS-GLOBAL in their AS-SET - we should have a capability to ignore that AS2914:AS-GLOBAL is a member of that AS-SET.

Updates: authentication, conflicts and notifications

Expanding on #20, this task involves checking the authentication for all requested updates, any conflicts with other objects, and finding out which notifications need to be sent. As these responsibilities all likely require retrieving related objects, they're grouped together as one task. This also includes deletion of objects that are still referenced.

Currently open:

Ensure mntner checks are correct
Add notification resolving and sending for updates
Extract PGP keys
Accept configurable override password
Limit changes to authoritative databases
Send INFO message when replacing auth hashes
Refactoring

Python package distribution

This involves easily creating a nice Python package from the repo, and documenting how others can do releases.

Updates: basic processing

This tasks involves receiving an incoming update with one or more objects, extracting any PGP signatures, validating the objects, saving the changed state.

Document supported queries

Mirroring: NRTM stream client

Building upon #14, once an initial import is successful, follow an NRTMv3 stream and process updates. Streams should be monitored for inactivity, and restarted on suspected staleness.

Parsing issue: email encoding artifacts in ARIN

as-set:         AS11976:AS-CUSTOMERS
descr:          Fidelity Communications
members:        AS11976
members:        AS31919
members:        AS46611
members:        AS17286=20
members:        AS5638
mnt-by:         MNT-FIDN
changed:        [email protected] 20100806
source:         ARIN
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
ERROR: Invalid AS number AS17286=20: number part is not numeric
ERROR: Invalid set AS17286=20: component AS17286=20 is not a valid AS number nor a valid set name

=======================================

route-set:      RS-IODC-15299-ROUTES
descr:          --------------------------------------------------
descr:          -
descr:          - IO DATA CENTERS, LLC.
descr:          -
descr:          - i/o PHOENIX ONE
descr:          - 615 N. 48th St.
descr:          - Phoenix, AZ 85008 US
descr:          -
descr:          - i/o SCOTTSDALE ONE
descr:          - 8521 E. Princess Dr.
descr:          - Scottsdale, AZ 85255 US
descr:          -
descr:          - http://www.iodatacenters.com
descr:          -
descr:          --------------------------------------------------
members:        =20
remarks:        --------------------------------------------------
remarks:        - For abuse complaints, please contact
remarks:        - [email protected]
remarks:        -
remarks:        - For general network inquiries, please contact
remarks:        - [email protected]
remarks:        --------------------------------------------------
tech-c:         NETWO1766-ARIN
admin-c:        NETWO1766-ARIN
mnt-by:         MNT-ICPL-3
changed:        [email protected] 20100714
source:         ARIN
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
ERROR: Value is neither a valid set name nor a valid prefix: =20: IP Address format was invalid: =20

I suppose we could automatically strip these values, but I don't immediately know if that's risk-free.

Mirroring: providing exports and NRTM streams to other registries

Including an access list for NRTM access.

Updates: submission via TCP socket

Provide a TCP interface to IRRd for submitting updates, with further processing as built in #20. This has some overlap with #16.

Prioritise same-database objects for set resolving

When resolving AS sets or route sets, we need to prefer an object from the same database, before trying a different one.

(See also #39)

Parsing issue: MAIL-FROM auth in NTTCOM

errors.nttcom-strict.db:ERROR: Invalid auth attribute: MAIL-FROM .*@.*blackrose.org: supported options are CRYPT-PW, MD5-PW and PGPKEY-xxxxxxxx
errors.nttcom-strict.db:ERROR: Invalid auth attribute: MAIL-FROM [email protected]: supported options are CRYPT-PW, MD5-PW and PGPKEY-xxxxxxxx
errors.nttcom-strict.db:ERROR: Invalid auth attribute: MAIL-FROM [email protected]: supported options are CRYPT-PW, MD5-PW and PGPKEY-xxxxxxxx
errors.nttcom-strict.db:ERROR: Invalid auth attribute: MAIL-FROM [email protected]: supported options are CRYPT-PW, MD5-PW and PGPKEY-xxxxxxxx
errors.nttcom-strict.db:ERROR: Invalid auth attribute: MAIL-FROM [email protected]: supported options are CRYPT-PW, MD5-PW and PGPKEY-xxxxxxxx
errors.nttcom-strict.db:ERROR: Invalid auth attribute: MAIL-FROM [email protected]: supported options are CRYPT-PW, MD5-PW and PGPKEY-xxxxxxxx

I suppose we can just discard these, but how? Refuse to import the entire mntner?

(Note that only for NTTCOM, we do strict validation, so we validate all attributes as opposed to the lighter validation on mirrored databases.)

Ability to submit objects via a HTTPS based API

Set resolving queries should take member-of into account

The mbrs-by-ref attribute is a list of maintainer names or the
   keyword ANY.  If this attribute is used, the route set also includes
   address prefixes whose route objects are registered by one of these
   maintainers and whose member-of attribute refers to the name of this
   route set.  If the value of a mbrs-by-ref attribute is ANY, any route
   object referring to the route set name is a member.  If the mbrs-by-
   ref attribute is missing, only the address prefixes listed in the
   members attribute are members of the set.

Parsing issue: IPv6 addresses in IPv4 members in ARIN

route-set:      rs-asgaard
descr:          Asgaard routes
tech-c:         ZA74-ARIN
admin-c:        ZA74-ARIN
mnt-by:         MNT-ASGAAR
members:        204.29.149.0/24, 204.29.150.0/23, 204.29.152.0/23, 2620:0:3f0::/48
notify:         [email protected]
notify:         [email protected]
changed:        [email protected] 20091105
source:         ARIN
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
ERROR: Value is neither a valid set name nor a valid prefix: 2620:0:3f0::/48: IPv4 Address can't be larger than ffffffff: 2620000003f000000000000000000000

=======================================

route-set:      RS-CAPEQUILOG6
descr:          CapEquiLog networks and customer routes (IPv6)
members:        2607:ff18::/32
tech-c:         CNO41-ARIN
admin-c:        JW44-ARIN
mnt-by:         MNT-CELL-1
changed:        [email protected] 20100108
source:         ARIN
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
ERROR: Value is neither a valid set name nor a valid prefix: 2607:ff18::/32: IPv4 Address can't be larger than ffffffff: 2607ff18000000000000000000000000

Queries: query parsing and resolving for basic queries.

This task covers handling an individual query in string form, parsing it, retrieving the necessary objects, and returning the answer. This task only covers basic primary key queries.

Allow one data source to 'drown out' other data sources

Allow one data source to 'drown out' other data source. For instance RPKI data supersedes IRR data, by using RPKI data we can combat stale IRR proxy route registrations.

Consider the following:

route: 192.0.2.0/24
origin: AS1
source: NTTCOM

route: 192.0.2.0/24
origin: AS2
source: RPKI

It would be nice if !gAS1 would not return 192.0.2.0/24 because of the existence of the RPKI entry covering 192.0.2.0/24. In operational terms the owner of prefix 192.0.2.0/24 should create a second RPKI ROA if they'd want to allow AS1 to originate the prefix.

Similarly not all IRR sources are equal, the APNIC database contains better data than ALTDB or NTTCOM.

Clean up errors for invalid IP addresses

This is a bit redundant:

ERROR: Value is neither a valid set name nor a valid prefix: 192.197.180.0/16: IP('192.197.180.0/16') has invalid prefix length (16)

Shouldn't be too hard to clean up a bit.

Mirroring: initial database import

For NRTM mirroring, load the initial copy of the database over http(s)/ftp, and load it into the database (replacing existing objects from the same source). The serial should also be retrieved and recorded.

This should be based on a basic configuration which includes the ability to filter for certain object types.

Explore whether GraphQL is sensible for querying routing data

Multiple people suggested that perhaps https://graphql.org/ is a good fit for this problem space. We should research whether this is a good fit to replace the IRRd WHOIS protocol

Make inverse attribute searches case insensitive

Currently, -i and !o queries are case sensitive - they should not be.

Updates: submission via e-mail

Provide a nice interface to deliver incoming e-mail to IRRd for further processing as built in #20.

Auto-build IRRd docker images

Router set members validator refuses other router set

rtr-set:        AS28816:rtrs-arbinet-rs
descr:          Arbinet Routeservers
members:        AS28816:rtrs-arbinet-customer-rs
members:        AS28816:rtrs-arbinet-transit-rs
mbrs-by-ref:    AS12885-MNT
admin-c:        DUMY-RIPE
tech-c:         DUMY-RIPE
mnt-by:         ABEL-MNT
changed:        [email protected] 20000101
source:         RIPE
remarks:        ****************************
remarks:        * THIS OBJECT IS MODIFIED
remarks:        * Please note that all data that is generally regarded as personal
remarks:        * data has been removed from this object.
remarks:        * To view the original object, please query the RIPE Database at:
remarks:        * http://www.ripe.net/whois
remarks:        ****************************
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
ERROR: Invalid DNS name: AS28816:rtrs-arbinet-customer-rs
ERROR: Invalid DNS name: AS28816:rtrs-arbinet-transit-rs

This looks like a valid router set name, so it should be accepted.

Parsing issue: Invalid route-set members in RIPE

rtr-set:        rtrs-stanbicbank-uganda
members:        196.8.207.0/24
members:        196.8.208.0/24
descr:          STANBIC BANK UGANDA
tech-c:         DUMY-RIPE
admin-c:        DUMY-RIPE
mnt-by:         RIPE-NCC-LOCKED-MNT
created:        2014-05-12T14:46:40Z
last-modified:  2016-04-25T13:15:17Z
source:         RIPE
remarks:        ****************************
remarks:        * THIS OBJECT IS MODIFIED
remarks:        * Please note that all data that is generally regarded as personal
remarks:        * data has been removed from this object.
remarks:        * To view the original object, please query the RIPE Database at:
remarks:        * http://www.ripe.net/whois
remarks:        ****************************
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
ERROR: Invalid DNS name: 196.8.207.0/24
ERROR: Invalid DNS name: 196.8.208.0/24

(Members should be other rtr-set names, an inet-rtr name or an IPv4/IPv6 address.

Finish RPSL parser

Fix a number of open TODO's
Add generation of templates
Raise test coverage
Double check that non-strict mode does check for primary key presence
Validate for: "All the set name components
of an hierarchical name has to be of the same type. For example, the
following names are valid: AS1:AS-CUSTOMERS, AS1:RS-EXPORT:AS2, RS-
EXCEPTIONS:RS-BOGUS."
Fix suspected PGP key reading bug in NTTCOM

Queries: TCP server handling

This task covers the listening to and handling incoming connections on port 43, splitting them into individual query strings to be handled elsewhere, and sending the replies back. All handling of multiple command mode is part of this task.

Mirroring: monitoring interface for mirror status

This includes the !j query.

Parsing issue: invalid PGP keys in NTTCOM

key-cert:           PGPKEY-32D3E619
method:             PGP
owner:              Mark Otoupal <[email protected]>
fingerpr:           DC 1B 9D 76 22 B0 97 21  57 35 90 78 9D E9 E1 72
certif:             
 -----BEGIN PGP PUBLIC KEY BLOCK-----
 Version: 2.6.2
+
 mQCNAzn0jCMAAAEEAL9g91cxcECfVrmtgoYKkCETfdVz50wHZspsslbCegIRhUpL
 /qA2Br+MJRaG2UdYhGMNIYmYHqLt9f+SA6V6ZmFBloiD05+7oJfgm18lwt5wRAW3
 I8q5jSi9uf+2+glV8jG2vdZVu7fmDb5m++Uj3mavj1ttlfJcPqeviPEy0+YZAAUR
 tCRNYXJrIE90b3VwYWwgPG1vdG91cGFsQGJlc3RsaW5lLm5ldD4=
 =/3La
 -----END PGP PUBLIC KEY BLOCK-----
mnt-by:             MAINT-BESTLINE
changed:            [email protected] 20001023
source:        NTTCOM
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
ERROR: Unable to read public PGP key: key corrupt or multiple keys provided

=======================================

key-cert:           PGPKEY-CD2C3BC9
method:             PGP
owner:              NOC <[email protected]>
fingerpr:           C4 A3 02 70 3E CC BA 29  A3 0A 5E 2B DE F8 9D 38
certif:             
 -----BEGIN PGP PUBLIC KEY BLOCK-----
 Version: 2.6.3ia
+
 mQCNAzxW7VsAAAEEAOoqRMnitOgrc9zmkdxOLVMtVHgtI2LwDVHKo9pIroeuezjr
 XVbhG00IjcYsGqII+m3yIfo5N5/Eaff3s+nZ+LQANW35PdFEBOjtzcBLauGYGZmT
 juca+JRQhhtd6j29Si5/1L5q0swXIsI+zREb+jyZm5tzUIo3qxVUvPnNLDvJAAUR
 tBNOT0MgPG5vY0BhcmdpYS5uZXQ+iQCVAwUQPFbuavjAhoFcoG4NAQGXxwQAgEpZ
 UQ1NinPzthCES9i+Snrcgq1XM0NsIyYPpnQxFgq3KKuIe57ehQbK2VuTRuIsWaqB
 CiFYzoCHSfPlKtCO/yUgT1BM3JsqzOBZFwHg+/WwLARyDln2/q0EfEsABjdVH6W/
 /ZLIkeOXujk9c28Qdmlv2r9KwDJexOlf/dX1JyCJAJUDBRA8Vu1bFVS8+c0sO8kB
 AZGEBADbRy1IF9JZKlbbwFLOa5XH3HXmx3RVUWI9bB3B/iTfV9zLGQTCrDsGCY5f
 CI2aStGV9B6nhAcPo2/M1RAftKryzU7j+yZUc1LlEJEKtZ+kLqd1f7VqBexDE1gq
 Rl/Pj3y1ZR8lShFhhDcJt49ZcjW2oe9DwydA1Rnp+ZxwB4qjEQ==
 =jnTU
 -----END PGP PUBLIC KEY BLOCK-----
notify:             [email protected]
mnt-by:             MAINT-ARGIABGP
changed:            [email protected] 20020129
source:        NTTCOM
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
ERROR: Unable to read public PGP key: key corrupt or multiple keys provided

This affects a total of five objects.

Same question as #57: we should ignore these, but how?

(Note that only for NTTCOM, we do strict validation, so we validate all attributes as opposed to the lighter validation on mirrored databases.)

Fix parsing/storage of route-set members

Route-set members are currently an RPSLTextField, which means they are not validated and not split correctly into individual member names/ranges. This breaks set resolving queries.

Route-set members can refer to other route sets or to prefixes, but may also include a range operator - so this isn't so simple as changing the field type. Probably it needs a custom variant of RPSLReferenceField with a slightly easier validation.

Once this change is done, !i queries that use route sets should work - we should make sure they are fully tested in TestWhoisQueryParserIRRD.

Also note that we need to look at members and mp-members.

Quality assurance / v3 consistency checks

We would like to have a process to ensure the new IRRd provides the same answers on the same datasets as the IRRDv3. Complexity is that they obviously need to have the same data at the same point in time, therefore both receive any updates made.

Previously we discussed a proxy that runs on live queries and sends them to both. This can work but has some operational challenges. A more practical alternative discussed would be taking fixed database dumps, loading them into IRRDv3 and v4, and running a pcap of actual queries against them. That is probably a preferred solution, as it's easier to reproduce and retest. At some point, we need to make final decisions about this.

(In such tests, we do need to pay special attention to ensuring no actual notification mails are sent out, but ideally do want to capture those.)

Parsing issue: invalid unicode characters in mnt-by in RIPE

route:          192.170.96.0/19
descr:          Hewlett-Packard Company
origin:         AS7430
mnt-by:         Â  AS1889-MNT
mnt-routes:     COLT-UK
changed:        [email protected] 20000101
created:        2009-05-28T14:19:14Z
last-modified:  2016-01-14T07:07:02Z
source:         RIPE
remarks:        ****************************
remarks:        * THIS OBJECT IS MODIFIED
remarks:        * Please note that all data that is generally regarded as personal
remarks:        * data has been removed from this object.
remarks:        * To view the original object, please query the RIPE Database at:
remarks:        * http://www.ripe.net/whois
remarks:        ****************************
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
ERROR: Invalid name: Â  AS1889-MNT: contains invalid characters, does not start with a letter, or does not end in a letter/digit

=======================================

route:          176.52.224.0/20
descr:          Â Â Â Â Â Â Â Â Â Â  armorconnectic
origin:         AS49112
mnt-by:         Â Â Â Â Â Â Â  ARMORCONNECTIC-MNT
created:        2016-06-09T09:39:51Z
last-modified:  2016-06-09T09:39:51Z
source:         RIPE
remarks:        ****************************
remarks:        * THIS OBJECT IS MODIFIED
remarks:        * Please note that all data that is generally regarded as personal
remarks:        * data has been removed from this object.
remarks:        * To view the original object, please query the RIPE Database at:
remarks:        * http://www.ripe.net/whois
remarks:        ****************************
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
ERROR: Invalid name: Â Â Â Â Â Â Â  ARMORCONNECTIC-MNT: contains invalid characters, does not start with a letter, or does not end in a letter/digit

=======================================

route:          185.147.232.0/22
descr:          Â Â Â Â Â Â Â Â Â Â  armorconnectic
origin:         AS49112
mnt-by:         Â Â Â Â Â Â Â  ARMORCONNECTIC-MNT
created:        2016-06-09T09:39:52Z
last-modified:  2016-06-09T09:39:52Z
source:         RIPE
remarks:        ****************************
remarks:        * THIS OBJECT IS MODIFIED
remarks:        * Please note that all data that is generally regarded as personal
remarks:        * data has been removed from this object.
remarks:        * To view the original object, please query the RIPE Database at:
remarks:        * http://www.ripe.net/whois
remarks:        ****************************
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
ERROR: Invalid name: Â Â Â Â Â Â Â  ARMORCONNECTIC-MNT: contains invalid characters, does not start with a letter, or does not end in a letter/digit

=======================================

route:          185.147.228.0/22
descr:          Â Â Â Â Â Â Â Â Â Â  capaix-connectic
origin:         AS49540
mnt-by:         Â Â Â Â Â Â Â  CAPAIX-CONNECTIC-MNT
created:        2016-06-09T09:39:52Z
last-modified:  2016-06-09T09:39:52Z
source:         RIPE
remarks:        ****************************
remarks:        * THIS OBJECT IS MODIFIED
remarks:        * Please note that all data that is generally regarded as personal
remarks:        * data has been removed from this object.
remarks:        * To view the original object, please query the RIPE Database at:
remarks:        * http://www.ripe.net/whois
remarks:        ****************************
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
ERROR: Invalid name: Â Â Â Â Â Â Â  CAPAIX-CONNECTIC-MNT: contains invalid characters, does not start with a letter, or does not end in a letter/digit

=======================================

route:          185.147.148.0/22
descr:          Â Â Â Â Â Â Â Â Â Â  memonet
origin:         AS49083
mnt-by:         Â Â Â Â Â Â Â FR-MEMONET-MNT
created:        2016-06-09T09:39:52Z
last-modified:  2016-06-09T09:39:52Z
source:         RIPE
remarks:        ****************************
remarks:        * THIS OBJECT IS MODIFIED
remarks:        * Please note that all data that is generally regarded as personal
remarks:        * data has been removed from this object.
remarks:        * To view the original object, please query the RIPE Database at:
remarks:        * http://www.ripe.net/whois
remarks:        ****************************
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
ERROR: Invalid name: Â Â Â Â Â Â Â FR-MEMONET-MNT: contains invalid characters, does not start with a letter, or does not end in a letter/digit

=======================================

mntner:         Neutrinet-MNT
admin-c:        DUMY-RIPE
upd-to:         [email protected]
auth:           MD5-PW $1$SaltSalt$DummifiedMD5HashValue.   # Real value hidden for security
mnt-by:         Neutrinet-MNT
mnt-by:         Â  Â Gitoyen-NCC
created:        2014-04-15T13:48:34Z
last-modified:  2017-04-02T19:51:55Z
source:         RIPE
remarks:        ****************************
remarks:        * THIS OBJECT IS MODIFIED
remarks:        * Please note that all data that is generally regarded as personal
remarks:        * data has been removed from this object.
remarks:        * To view the original object, please query the RIPE Database at:
remarks:        * http://www.ripe.net/whois
remarks:        ****************************
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
ERROR: Invalid name: Â  Â Gitoyen-NCC: contains invalid characters, does not start with a letter, or does not end in a letter/digit

Parsing issue: as-name missing on aut-nums in NTTCOM

aut-num:     AS7334
descr:       Wall Street Autonomous System Number
admin-c:     JG825-ORG
tech-c:      JG825-ORG
import:       from AS2914 action pref=10; accept ANY
import:       from AS701 action pref=10; accept ANY
export:      to AS2914 announce AS7334
export:      to AS701 announce AS7334
mnt-by:      MAINT-WALLST
changed:     [email protected] 20000127
source:        NTTCOM
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
ERROR: Mandatory attribute 'as-name' on object aut-num is missing

(Note that only for NTTCOM, we do strict validation, so we validate all attributes as opposed to the lighter validation on mirrored databases.)

Review twisted configuration

We're currently using a very naive twisted protocol implementation, which probably requires further tweaking.

Validate IRR object creation against the authoritative RIR database

APNIC expressed interest to jointly develop an authentication mechanism to facilitate route-object creation and still validate against the authoritative RIR database.

Parsing issue: invalid tech-c, admin-c on many objects in NTTCOM

The tech-c and admin-c is invalid in 828 cases in NTTCOM

It's a bit hard to figure out exactly what the syntax requirements are. RFC 2280 says:

   <nic-handle> is a uniquely assigned identifier word used by routing,
      address allocation, and other registries to unambiguously refer to
      contact information.  Person and role classes map NIC handles to
      actual person names, and contact information.

However, being the nic-hdl of the person/role object, it also seems like it is a name:

   <object-name>
      Many objects in RPSL have a name.  An <object-name> is made up of
      letters, digits, the character underscore "_", and the character
      hyphen "-"; the first character of a name must be a letter, and
      the last character of a name must be a letter or a digit.  The
      following words are reserved by RPSL, and they can not be used as
      names:

That is the current validation, and it seems widely used too.

Some examples of invalid objects:

aut-num:    AS18070
as-name:    NDAC
descr:      Global Network Core Co.,Ltd.
import:     from AS2497   accept ANY
import:     from AS4713   accept ANY
import:     from AS2518   accept ANY
export:     to AS2497   announce AS-18070
export:     to AS4713   announce AS-18070
export:     to AS2518   announce AS-18070
admin-c:    Yasuyuki Kaneko
tech-c:     Yasuyuki Kaneko
notify:     [email protected]
mnt-by:     MAINT-AS18070
changed:    [email protected] 20161011
source:     NTTCOM
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
ERROR: Invalid name: Yasuyuki Kaneko: contains invalid characters, does not start with a letter, or does not end in a letter/digit
ERROR: Invalid name: Yasuyuki Kaneko: contains invalid characters, does not start with a letter, or does not end in a letter/digit
ERROR: Invalid name: Yasuyuki Kaneko: contains invalid characters, does not start with a letter, or does not end in a letter/digit
ERROR: Invalid name: Yasuyuki Kaneko: contains invalid characters, does not start with a letter, or does not end in a letter/digit

=======================================

route:      159.211.224.0/24
descr:      DIC Yokohama DC
            Yokohama Dia Bld. Kohoku Annex. No.1 3F
            1-1,Sakura-namiki,Tsuzuki-ku,Yokohama,Kanagawa
tech-c:     DIC Corp. Information System dept. Masahiko Goto
origin:     AS4713
mnt-by:     MAINT-AS4713
changed:    [email protected] 20160810
source:     NTTCOM
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
ERROR: Invalid name: DIC Corp. Information System dept. Masahiko Goto: contains invalid characters, does not start with a letter, or does not end in a letter/digit
ERROR: Invalid name: DIC Corp. Information System dept. Masahiko Goto: contains invalid characters, does not start with a letter, or does not end in a letter/digit

=======================================

mntner:     MAINT-AS6057
descr:      ANTEL
admin-c:    ANTEL NOC
tech-c:     ANTEL NOC
upd-to:     [email protected]
mnt-nfy:    [email protected]
auth:       CRYPT-PW xxxxx
notify:     [email protected]
mnt-by:     MAINT-AS6057
changed:    [email protected] 20150316
source:     NTTCOM
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
ERROR: Invalid name: ANTEL NOC: contains invalid characters, does not start with a letter, or does not end in a letter/digit
ERROR: Invalid name: ANTEL NOC: contains invalid characters, does not start with a letter, or does not end in a letter/digit
ERROR: Invalid name: ANTEL NOC: contains invalid characters, does not start with a letter, or does not end in a letter/digit
ERROR: Invalid name: ANTEL NOC: contains invalid characters, does not start with a letter, or does not end in a letter/digit

Architecture design

We should make an (initial) design of the various components and their interactions. This should, as much as reasonable, also make it simple to bolt on ideas suggested for phase 2.

Both of these designs should be part of the documentation and updated as the project progresses, as new insights will lead to changes.

irrdnet / irrd Goto Github PK

irrd's Introduction

Internet Routing Registry Daemon (IRRd) Version 4

irrd's People

Contributors

Stargazers

Watchers

Forkers

irrd's Issues

Recommend Projects

Recommend Topics

Recommend Org