ammario/ipisp

Query IP address network information in Go

ipisp's Issues

"NA" ASNs cause panic

There are times when Cymru incorrectly returns "NA" as the ASN and the ipisp package returns the error. I've only seen these types of errors when submitting a large number of IPs. I ran into the error below when querying some 10k ips.

For example:
NA | 193.32.71.189 | NA | UA | ripencc | 2018-04-26 | NA

And the resulting error (backtrace provided by the errors package on the client side):

➜  example ./main
2018/10/08 11:26:14 There was an error: strconv.Atoi: parsing "NA": invalid syntax
failed to conv into to string
github.com/ammario/ipisp.ParseASN
	/Users/dcam/golang/src/github.com/ammario/ipisp/asn.go:23
github.com/ammario/ipisp.parseASNs
	/Users/dcam/golang/src/github.com/ammario/ipisp/client.go:32
github.com/ammario/ipisp.(*whoisClient).LookupIPs
	/Users/dcam/golang/src/github.com/ammario/ipisp/whois_client.go:133
main.main
	/Users/dcam/example/main.go:49
runtime.main
	/usr/local/Cellar/go/1.11.1/libexec/src/runtime/proc.go:201
runtime.goexit
	/usr/local/Cellar/go/1.11.1/libexec/src/runtime/asm_amd64.s:1333
failed to parse asn
github.com/ammario/ipisp.parseASNs
	/Users/dcam/golang/src/github.com/ammario/ipisp/client.go:34
github.com/ammario/ipisp.(*whoisClient).LookupIPs
	/Users/dcam/golang/src/github.com/ammario/ipisp/whois_client.go:133
main.main
	/Users/dcam/example/main.go:49
runtime.main
	/usr/local/Cellar/go/1.11.1/libexec/src/runtime/proc.go:201
runtime.goexit
	/usr/local/Cellar/go/1.11.1/libexec/src/runtime/asm_amd64.s:1333
failed to parse asn list NA
github.com/ammario/ipisp.(*whoisClient).LookupIPs
	/Users/dcam/golang/src/github.com/ammario/ipisp/whois_client.go:135
main.main
	/Users/dcam/example/main.go:49
runtime.main
	/usr/local/Cellar/go/1.11.1/libexec/src/runtime/proc.go:201
runtime.goexit
	/usr/local/Cellar/go/1.11.1/libexec/src/runtime/asm_amd64.s:1333

There are times when Cymru correct responds with the right data (that IP has ASN AS205127) and nothing panics. The ideas I've come up with to handle the case would be to
A. Drop the incorrect request completely, returning no information about the IP to the user or returning a non-fatal error.
B. Retry the IP a limited number of times. On success, return information to the user or on failure return a non-fatal error.
C. Do nothing to the package and expect users to manage the error client side.

I'm willing to help with the patch, I'm just curious on your intended outcome first.

Let me know if I can clarify anything and thank you for the package, it's really useful!

v2

This issue documents my plans to refactor this package (and break API) for v2:

1. Return per IP errors from LookupIPs
2. Use xerrors instead of pkg/errors
3. Use table driven tests, try to remove use of testify/assert
4. Use idiomatic Go file names
5. Remove tests that are prone to failure (or hide them under build tag)
6. Remove concept of Long and Short ISP name
7. Fix #17

Allow providing a custom DNS resolver

A new resolver is created at the start of the LookupIP and LookupASN functions.

ipisp/dnsclient.go

Line 18 in a1960f6

var r net.Resolver

Would it be possible to provide a custom resolver in the following way?

client := ipisp.NewClient(&net.Resolver{})
client.LookupIP(...)
client.LookupASN(...)

Error parsing multiple ASNs from reverse lookup

Ran into this issue using GOSINT, originally opened an issue there (ciscocsirt/GOSINT#19), but moving it here as it seems to be a bug in this library.

When the lookup returns multiple ASNs for a given IP, it tries and fails to parse as a number:

2018/02/22 09:43:48 Error doing reverse lookup on %v: %v 103.235.224.237 Could not parse ASN (4808 23724 56282): %!s(MISSING): strconv.Atoi: parsing "4808 23724 56282": invalid syntax
2018/02/22 09:43:52 Error doing reverse lookup on %v: %v 103.75.156.237 Could not parse ASN (134877 135216): %!s(MISSING): strconv.Atoi: parsing "134877 135216": invalid syntax
2018/02/22 09:43:59 Error doing reverse lookup on %v: %v 106.75.101.69 Could not parse ASN (4808 23724): %!s(MISSING): strconv.Atoi: parsing "4808 23724": invalid syntax
2018/02/22 09:43:59 Error doing reverse lookup on %v: %v 106.75.18.132 Could not parse ASN (4808 23724): %!s(MISSING): strconv.Atoi: parsing "4808 23724": invalid syntax

(Note the above output is from GOSINT.)

These IPs are from the CI Army blacklist, http://cinsscore.com/list/ci-badguys.txt, there's about 160 IPs out of the 15k on the list that trigger the error.

Release v2.0.1

The last released version (v2.0.0) was released on Dec 25, 2021 and does not contain the following fix: #23
See v2.0.0...master

This currently requires the use of the master branch.

Can you release a new version?

Thanks!

parse asn error when there are multiple asn for a host

When a host with multiple asn value is present, the application returns parse asn error.

Error message:

parse ASN "397213 397215 397218 397220 397224 397233 397235 397238": parse "397213 397215 397218 397220 397224 397233 397235 397238": strconv.Atoi: parsing "397213 397215 397218 397220 397224 397233 397235 397238": invalid syntax

Steps to reproduce:
ip: 204.74.99.100
Host url: japan.teslamotors.com

Bug in IPv4 Parsing

In dns_client.go, line 144, the following code causes an array out of bounds condition when a valid IPv4 address in 4 byte representation is passed in:

if p4 := ip.To4(); len(p4) == net.IPv4len {
		return fmt.Sprintf("%d.%d.%d.%d.origin.asn.cymru.com", ip[15], ip[14], ip[13], ip[12]), nil
}

Inconsistent error between whois and dns for non existent record

DNS errors, whois resolver returns NA. Particularly difficult to avoid when doing bulk requests in DNS.

I just ended up using whois for my small project.

Add connection persistence/timeouts

IPs returning multiple ASNs cuts off legitimate responses

When using the whoisClient and the LookupIPs function there are times when Cymru returns multiple ASNs (I haven't looked into why).

Because of the finished boolean checking the length of the response vs the capacity of the response some legitimate IPs are getting dropped from the response:

if len(resp) == cap(resp) {
        finished = true
}

I added in some debugging

Raw Value:  <AS> | <IP>  | <BGP Prefix> | <CC> | <Registry> | <Allocated>  | <AS Name> # Edited
isisp: whoisClient.LookupIPs: bad ASN count: 2 # related to issue 10
isisp: whoisClient.LookupIPs: Scan count: 2500
========================================================

isisp: whoisClient.LookupIPs: ATTENTION: saw <IP> 2 number of times # Edited
isisp: whoisClient.LookupIPs: ATTENTION: saw <IP> 2 number of times # Edited
isisp: whoisClient.LookupIPs: ATTENTION: saw <IP> 2 number of times # Edited
isisp: whoisClient.LookupIPs: ATTENTION: saw <IP> 2 number of times # Edited
isisp: whoisClient.LookupIPs: ATTENTION: saw <IP> 2 number of times # Edited
isisp: whoisClient.LookupIPs: ATTENTION: saw <IP> 2 number of times # Edited
isisp: whoisClient.LookupIPs: ATTENTION: saw <IP> 2 number of times # Edited
isisp: whoisClient.LookupIPs: ATTENTION: saw <IP> 2 number of times # Edited
========================================================

isisp: whoisClient.LookupIPs: Scanner Open? true
isisp: whoisClient.LookupIPs: Scanner Unprocessed Value: <AS> | <IP>  | <BGP Prefix> | <CC> | <Registry> | <Allocated>  | <AS Name> # Edited
isisp: whoisClient.LookupIPs: Scanner Open? true
isisp: whoisClient.LookupIPs: Scanner Unprocessed Value: <AS> | <IP>  | <BGP Prefix> | <CC> | <Registry> | <Allocated>  | <AS Name> # Edited
isisp: whoisClient.LookupIPs: Scanner Open? true
isisp: whoisClient.LookupIPs: Scanner Unprocessed Value: <AS> | <IP>  | <BGP Prefix> | <CC> | <Registry> | <Allocated>  | <AS Name> # Edited
isisp: whoisClient.LookupIPs: Scanner Open? true
isisp: whoisClient.LookupIPs: Scanner Unprocessed Value: <AS> | <IP>  | <BGP Prefix> | <CC> | <Registry> | <Allocated>  | <AS Name> # Edited
isisp: whoisClient.LookupIPs: Scanner Open? true
isisp: whoisClient.LookupIPs: Scanner Unprocessed Value: <AS> | <IP>  | <BGP Prefix> | <CC> | <Registry> | <Allocated>  | <AS Name> # Edited
isisp: whoisClient.LookupIPs: Scanner Open? true
isisp: whoisClient.LookupIPs: Scanner Unprocessed Value: <AS> | <IP>  | <BGP Prefix> | <CC> | <Registry> | <Allocated>  | <AS Name> # Edited
isisp: whoisClient.LookupIPs: Scanner Open? false
isisp: whoisClient.LookupIPs: Scan missed results count: 6
isisp: whoisClient.LookupIPs: Scanner Error: <nil>

`go get -v github.com/ammario/ipisp/v2` fails while `go get -v github.com/ammario/ipisp` succeeds

publish official releases?

Hi!

First, thank you for this software, it looks quite useful and is used in a tool I am packaging for debian (dt).

One problem with this dependency is that it doesn't have official releases. It makes it hard to find out when changes happen in the code base. It also makes tracking module dependencies with the new golang module system harder.

Would you consider using official, tagged releases for your software?

Again thank you for your time.

"NA" BGP Prefix causes error

Sorry I didn't catch this in issue #10, but the BGP Prefix can also be "NA", which causes an error on the client side.

The "AS Name" field is also NA, so that may cause an issue too.

Raw Value: NA      | 193.32.71.189    | NA                  | UA | ripencc  | 2018-04-26 | NA
2018/10/08 14:17:41 There was an error: invalid CIDR address: NA

type ASN uint32

Should change type ASN int to type ASN uint32

https://www.arin.net/resources/guide/asn/

A 4-byte ASN is a 32-bit number. This format provides for 2^32 or 4,294,967,296 ASNs (0 to 4294967295).

https://datatracker.ietf.org/doc/html/rfc6793#section-8

... the InetAutonomousSystemNumber textual convention [RFC4001] is defined as Unsigned32.

int could be either 32 or 64 bits depending on the system. If 32 bits then its range (-2147483648 to 2147483647) would be insufficient to cover the full 4-byte ASN range.

Lookup via DNS

Team-Cymru asks, "PLEASE use the DNS based service since it is faster and more efficient, particularly for larger deployments of individual IP based queries"

Why didn't you choose to do it this way? Go has what you need in the standard library already:
https://golang.org/pkg/net/#LookupTXT

ASN Name Returns Bad Value

It appears that ParseName() has an issue with ASN names using -. The method tokenizes using - as a separator, which causes part of the data to be truncated. Using a slice and then concatenating elements [1+] might be a workable solution...