dnsdb/dnsdb-query

DNSDB query scripts

dnsdb-query's Issues

CSV output format for python DNSDB_query.py

Feature request:

The C DNSDB query client offers CSV output (with the "-p csv" output format option)

Please consider adding a --csv output format option to the Python dnsdb_query.py client, too, for those who may want CSV output for easy input to Excel, etc., while using the Python dnsdb_query.py client.

Thank you!

Need to add CURLOPT_PATH_AS_IS option to curl invocation

Trying to query for name . without curl's --path-as-is flag or CURLOPT_PATH_AS_IS option, curl rewrites the URL and changes the end from name/. to just name/

TypeError: raise: arg 3 must be a traceback or None

Traceback (most recent call last):
  File "/usr/local/bin/dnsdb_query.py", line 263, in <module>
    main()
  File "/usr/local/bin/dnsdb_query.py", line 217, in main
    cfg = parse_config(options.config)
  File "/usr/local/bin/dnsdb_query.py", line 124, in parse_config
    raise IOError, 2, 'dnsdb_query: No config files found'
TypeError: raise: arg 3 must be a traceback or None

--rrtype fails when using --rdataname

Working:

$ dnsdb_query.py --rdataname dns1.iu.edu/NS
uiuc.biz. IN NS dns1.iu.edu.
...

Failure:

$ dnsdb_query.py --rdataname dns1.iu.edu --rrtype NS
Traceback (most recent call last):
  File "/usr/local/bin/dnsdb_query.py", line 286, in <module>
    main()
  File "/usr/local/bin/dnsdb_query.py", line 257, in main
    results = client.query_rdata_name(*qargs, before=options.before, after=options.after)
TypeError: query_rdata_name() got multiple values for keyword argument 'before'

python3 compatibility

PR #30 added python3 compatibility, however after being merged @hstern said

I had to revert this commit. It broke some python 2.7 stuff.

Can you expand on what it broke? A quick test with both pythons doesn't cause issues.
I'm trying to use the dnsdb library to add an expansion plugin in python3 software, like MISP-modules.

If needed I can spend some time to try to fix things and make it python3 compatible in the few missing places. Not being python3 compatible is becoming a real pain. And cherry picking the #30 commits is not very practical.

Thanks
Christophe

DNSDB clients make ambiguous URLs if parameters contain slashes

Running dnsdb_query.py -r "226.0/24.75.70.64.in-addr.arpa" calls out to https://api.dnsdb.info/lookup/rrset/name/226.0/24.75.70.64.in-addr.arpa which causes DNSTLI to report an error because it splits parameters on slashes. This requires both a server-side and a client-side fix. This issue is for the client-side fix.

Add option for ISO8601 Timestamps?

(this replaces a stale PR from 2014 "New option when emitting JSON to add an ISO 8601 timestamp" that featured a frankenstein's monster of a patch)

• Should we add an option to the JSON output to include a human readable ISO8601 UTC timestamp? Discuss.

Sorting by time fields results in an error

Working query to verify setup

dnsdb-query$ ./dnsdb_query.py -l 4 -r *.bac.com/ns
;; bailiwick: com.
;; count: 1,275
;; first seen in zone file: 2013-01-12 17:16:08 -0000
;; last seen in zone file: 2016-07-20 16:15:34 -0000
bac.com. IN NS ns6.bac.com.
bac.com. IN NS ns7.bac.com.
bac.com. IN NS ns8.bac.com.
bac.com. IN NS ns10.bac.com.
bac.com. IN NS ns11.bac.com.
bac.com. IN NS ns12.bac.com.

;; bailiwick: com.
;; count: 364
;; first seen in zone file: 2012-01-14 17:13:33 -0000
;; last seen in zone file: 2013-01-11 17:16:41 -0000
bac.com. IN NS ns10.bac.com.
bac.com. IN NS ns11.bac.com.
bac.com. IN NS ns12.bac.com.
bac.com. IN NS ns1.bankofamerica.com.
bac.com. IN NS ns3.bankofamerica.com.
bac.com. IN NS ns4.bankofamerica.com.

;; bailiwick: com.
;; count: 630
;; first seen in zone file: 2010-04-24 16:12:21 -0000
;; last seen in zone file: 2012-01-13 17:14:40 -0000
bac.com. IN NS ns1.bankofamerica.com.
bac.com. IN NS ns3.bankofamerica.com.
bac.com. IN NS ns4.bankofamerica.com.

;; bailiwick: com.
;; count: 2
;; first seen: 2015-10-16 08:45:12 -0000
;; last seen: 2015-10-16 08:45:12 -0000
bac.com. IN NS ns6.bac.com.

Failure-mode A

dnsdb-query$ ./dnsdb_query.py -r *.bac.com/ns --sort time_last
dnsdb_query: invalid sort key "time_last". valid sort keys are bailiwick, count, rdata, rrname, rrtype, zone_time_first, zone_time_last

Failure-mode B

dnsdb-query$ ./dnsdb_query.py -r *.bac.com/ns --sort zone_time_last
Traceback (most recent call last):
File "./dnsdb_query.py", line 317, in
main()
File "./dnsdb_query.py", line 201, in f
return func(_args, *_kwargs)
File "./dnsdb_query.py", line 309, in main
results.sort(key=lambda r: r[options.sort], reverse=options.reverse)
File "./dnsdb_query.py", line 309, in
results.sort(key=lambda r: r[options.sort], reverse=options.reverse)
KeyError: 'zone_time_last'

Option for proxy config

Some users wish to use dnsdb-query from behind a restrictive firewall and need to direct HTTPS queries to a proxy. It would be helpful to have a way to specify a proxy.

--before and --after options do not work with --json

Hello. Your code breaks when the --json output specifier is used with the timestamp filtering options --before and/or --after as per the following:

[14:41]mike(ttys003):dnsdb-query[1]$ ./dnsdb_query.py -r google.com/A -j --before=1382894906 --after=1385573346
Traceback (most recent call last):
  File "./dnsdb_query.py", line 259, in <module>
    main()
  File "./dnsdb_query.py", line 249, in main
    res_list = filter_before(res_list, options.before)
  File "./dnsdb_query.py", line 160, in filter_before
    if res['time_first'] < before_time:
TypeError: string indices must be integers, not str

The problem is in the filter_before() and filter_after() functions. When not using --json they index res as a dictionary, which is fine. When using --json, res is now a string being indexed with a string which throws an exception.

dnsdb_query.py needs pydocs

dnsdb_query.py exits silently when no config files are present

On a system with no dnsdb-query.conf, the program raises IOError but silently exits without printing the intended error message.

$ ./dnsdb_query.py
$

I suspect the IOError class already sets errno. The fix is to simply change line 125 to:

raise IOError('dnsdb_query: No config files found\n')

(and remove the now unneeded import errno statement. The program now works as expected:

$ ./dnsdb_query.py
dnsdb_query: No config files found
$