joepie91/python-whois

A python module for retrieving and parsing WHOIS data

python-whois's Issues

Contact information for .co.uk domains is not parsed

Hi,

Domain "co.uk" are not parsed correctly. Domain: tattitude.co.uk

Example:

updated_date :  [datetime.datetime(2013, 6, 14, 0, 0)]

status :    [u'Registered until expiry date.']

contacts :  {'admin': None, 'tech': None, 'registrant': None, 'billing': None}

nameservers :   [u'ns1.pipedns.com', u'ns2.pipedns.com', u'ns3.pipedns.com']

expiration_date :   [datetime.datetime(2017, 7, 14, 0, 0)]

creation_date : [datetime.datetime(2007, 7, 14, 0, 0), datetime.datetime(2007, 7, 14, 0, 0), datetime.datetime(2007, 7, 14, 0, 0)]

raw :   [u"\n    Domain name:\n        tattitude.co.uk\n\n    Registrant:\n        Paul Hartley\n\n    Trading as: \n        Private Registration US\n\n    Registrant type:\n        UK Individual\n\n    Registrant's address:\n        The registrant is a non-trading individual who has opted to have their\n        address omitted from the WHOIS service.\n\n    Data validation:\n        Registrant contact details validated by Nominet on 10-Dec-2012\n\n    Registrar:\n        Melbourne IT t/a Internet Names Worldwide [Tag = MELBOURNE-IT]\n        URL: http://www.melbourneit.com.au/contacts\n\n    Relevant dates:\n        Registered on: 14-Jul-2007\n        Expiry date:  14-Jul-2017\n        Last updated:  14-Jun-2013\n\n    Registration status:\n        Registered until expiry date.\n\n    Name servers:\n        ns1.pipedns.com\n        ns2.pipedns.com\n        ns3.pipedns.com\n\n    WHOIS lookup made at 04:53:56 23-Jul-2014\n\n-- \nThis WHOIS information is provided for free by Nominet UK the central registry\nfor .uk domain names. This information and the .uk WHOIS are:\n\n    Copyright Nominet UK 1996 - 2014.\n\nYou may not access the .uk WHOIS or use any data from it except as permitted\nby the terms of use available in full at http://www.nominet.org.uk/whoisterms,\nwhich includes restrictions on: (A) use of the data for advertising, or its\nrepackaging, recompilation, redistribution or reuse (B) obscuring, removing\nor hiding any or all of this notice and (C) exceeding query rate or volume\nlimits. The data is provided on an 'as-is' basis and may lag behind the\nregister. Access may be withdrawn or restricted at any time. \n"]

registrar : [u'Melbourne IT t/a Internet Names Worldwide [Tag = MELBOURNE-IT]']

Original Record from Domaintools:

Domain name:
        attitude.co.uk

    Registrant:
        Attitude Media Ltd

    Registrant type:
        UK Limited Company, (Company number: 5841474)

    Registrant's address:
        207 Old Street, 3rd Floor
        London
        EC1V 9NR
        United Kingdom

    Data validation:
        Registrant contact details validated by Nominet on 10-Dec-2012

    Registrar:
        1 & 1 Internet AG [Tag = SCHLUND]
        URL: https://www.1and1.co.uk

    Relevant dates:
        Registered on: before Aug-1996
        Expiry date:  22-Nov-2015
        Last updated:  21-Nov-2013

    Registration status:
        Registered until expiry date.

    Name servers:
        ns1.bdm.microsoftonline.com
        ns2.bdm.microsoftonline.com

Disclaimer
WHOIS lookup made at 04:59:49 23-Jul-2014

-- 
This WHOIS information is provided for free by Nominet UK the central registry
for .uk domain names. This information and the .uk WHOIS are:

    Copyright Nominet UK 1996 - 2014.

You may not access the .uk WHOIS or use any data from it except as permitted
by the terms of use available in full at http://www.nominet.org.uk/whoisterms,
which includes restrictions on: (A) use of the data for advertising, or its
repackaging, recompilation, redistribution or reuse (B) obscuring, removing
or hiding any or all of this notice and (C) exceeding query rate or volume
limits. The data is provided on an 'as-is' basis and may lag behind the
register. Access may be withdrawn or restricted at any time.

incomplete whois data for .fi

Incomplete data for .fi domains.

example domains:
why.fi
wi.fi

example output:

pwhois why.fi
Status : Granted
Registration date : 2007-01-22 00:00:00
Expiration date : 2018-01-22 00:00:00
Last update : 2014-10-03 00:00:00
Name server : dns1.louhi.net
Name server : dns2.louhi.net
Name server : dns3.louhi.fi

raw output:

pwhois why.fi --raw

domain: why.fi
descr: OHJAAMO Group Oy
descr: 19101477
address: --snip--
address: --snip--
address: --snip--
address: --snip--
phone: --snip--
status: Granted
created: 22.1.2007
modified: 3.10.2014
expires: 22.1.2018
nserver: dns1.louhi.net [Ok]
nserver: dns2.louhi.net [Ok]
nserver: dns3.louhi.fi [Ok]
dnssec: no

More information is available at https://domain.fi/
Copyright (c) Finnish Communications Regulatory Authority

.it domains not parsed

Hi there,

pythonwhois doesn't parse properly google.it and others (e.g., yahoo.it). I suspect it applies to all .it domains.

Here is the test I've done so far:

it = pythonwhois.get_whois("google.it")
pprint(it)

{'contacts': {'admin': None,
'billing': None,
'registrant': None,
'tech': None},
'creation_date': [datetime.datetime(1999, 12, 10, 0, 0),
datetime.datetime(2013, 4, 21, 1, 5, 35)],
'expiration_date': [datetime.datetime(2015, 4, 21, 0, 0)],
'raw': ['\n********************************************************************\n Please note that the following result could be a subgroup of \n the data contained in the database. \n \n Additional information can be visualized at: \n http://www.nic.it/cgi-bin/Whois/whois.cgi \n********************************************************************\n\nDomain: google.it\nStatus: ok\nCreated: 1999-12-10 00:00:00\nLast Update: 2014-05-07 00:52:45\nExpire Date: 2015-04-21\n\nRegistrant\n Name: Google Ireland Holdings\n Organization: Google Ireland Holdings\n ContactID: DUP430692088\n Address: 70 Sir John Rogersons Quay\n Dublin\n 2\n IE\n IE\n Created: 2013-04-21 01:05:35\n Last Update: 2013-04-21 01:05:35\n\nAdmin Contact\n Name: Tsao Tu\n Organization: Tu Tsao\n ContactID: DUP142437129\n Address: 70 Sir John Rogersons Quay\n Dublin\n 2\n IE\n IE\n Created: 2013-04-21 01:05:35\n Last Update: 2013-04-21 01:05:35\n\nTechnical Contacts\n Name: Google Ireland Holdings\n Organization: Google Ireland Holdings\n ContactID: DUP430692088\n Address: 70 Sir John Rogersons Quay\n Dublin\n 2\n IE\n IE\n Created: 2013-04-21 01:05:35\n Last Update: 2013-04-21 01:05:35\n\nRegistrar\n Organization: MarkMonitor International Limited\n Name: MARKMONITOR-REG\n Web: https://www.markmonitor.com/\n\nNameservers\n ns1.google.com\n ns4.google.com\n ns2.google.com\n ns3.google.com\n\n'],
'status': ['ok'],
'updated_date': [datetime.datetime(2014, 5, 7, 0, 52, 45),
datetime.datetime(2013, 4, 21, 1, 5, 35)]}

nameservers not parsed for .li

name servers aren't parsed for .li domain names.

example domains:
presseportal.li
ragu.li

other information seems to be parsed correctly.

Unable to get whois record for .ac.uk domains

Attempted to pull whois record for UK HE institutions as follows (for example):

pythonwhois.get_whois("imperial.ac.uk")

This responds with the whois record for "ac.uk" not "imperial.ac.uk". From the response data:

Domain name:\r\n        ac.uk

I guess you require explicit lists of supported extensions?

Directly Query mandatory whois services ICANN requires new gTLDs to run

If you look up the content of the ICANN agreements for all the new TLD operators. Whois tool writers get a giant big helping hand. They mandate some whois standards!

The easiest way to do it would be to query whois.nic.tld first, then fall back to the current mechanism.

The current mechanism is widely inconsistent with the new gTLDs. A quick run of the 'status' field results looking for 'nic.tld' shows a lot of empty values where they return valid ICANN required whois data for instance:

• nic.webcam
• nic.wang
• nic.vote
• nic.vi

A significant portion of the domains in ICANNs new gTLD batches, return results like this pythonwhois.shared.WhoisException: No root WHOIS server found for domain.

---

SPECIFICATION 4

REGISTRATION DATA PUBLICATION SERVICES

1.          Registration Data Directory Services.  Until ICANN requires a different protocol, Registry Operator will operate a WHOIS service available via port 43 in accordance with RFC 3912, and a web-based Directory Service at <whois.nic.TLD> providing free public query-based access to at least the following elements in the following format.  ICANN reserves the right to specify alternative formats and protocols, and upon such specification, the Registry Operator will implement such alternative specification as soon as reasonably practicable.
   

Registry Operator shall implement a new standard supporting access to domain name registration data (SAC 051) no later than one hundred thirty-five (135) days after it is requested by ICANN if: 1) the IETF produces a standard (i.e., it is published, at least, as a Proposed Standard RFC as specified in RFC 2026); and 2) its implementation is commercially reasonable in the context of the overall operation of the registry.

1.1. The format of responses shall follow a semi-free text format outline below, followed by a blank line and a legal disclaimer specifying the rights of Registry Operator, and of the user querying the database.

1.2. Each data object shall be represented as a set of key/value pairs, with lines beginning with keys, followed by a colon and a space as delimiters, followed by the value.

1.3. For fields where more than one value exists, multiple key/value pairs with the same key shall be allowed (for example to list multiple name servers). The first key/value pair after a blank line should be considered the start of a new record, and should be considered as identifying that record, and is used to group data, such as hostnames and IP addresses, or a domain name and registrant information, together.

1.4. The fields specified below set forth the minimum output requirements. Registry Operator may output data fields in addition to those specified below, subject to approval by ICANN, which approval shall not be unreasonably withheld.

1.5. Domain Name Data:

1.5.1 Query format: whois EXAMPLE.TLD

1.5.2 Response format:

Domain Name: EXAMPLE.TLD
Domain ID: D1234567-TLD
WHOIS Server: whois.example.tld
Referral URL: http://www.example.tld
Updated Date: 2009-05-29T20:13:00Z
Creation Date: 2000-10-08T00:45:00Z
Registry Expiry Date: 2010-10-08T00:44:59Z
Sponsoring Registrar: EXAMPLE REGISTRAR LLC
Sponsoring Registrar IANA ID: 5555555
Domain Status: clientDeleteProhibited
Domain Status: clientRenewProhibited
Domain Status: clientTransferProhibited
Domain Status: serverUpdateProhibited
Registrant ID: 5372808-ERL
Registrant Name: EXAMPLE REGISTRANT
Registrant Organization: EXAMPLE ORGANIZATION
Registrant Street: 123 EXAMPLE STREET
Registrant City: ANYTOWN
Registrant State/Province: AP
Registrant Postal Code: A1A1A1
Registrant Country: EX
Registrant Phone: [phone]
Registrant Phone Ext: 1234
Registrant Fax: [phone]
Registrant Fax Ext: 4321
Registrant Email: [email protected]
Admin ID: 5372809-ERL
Admin Name: EXAMPLE REGISTRANT ADMINISTRATIVE
Admin Organization: EXAMPLE REGISTRANT ORGANIZATION
Admin Street: 123 EXAMPLE STREET
Admin City: ANYTOWN
Admin State/Province: AP
Admin Postal Code: A1A1A1
Admin Country: EX
Admin Phone: [phone]
Admin Phone Ext: 1234
Admin Fax: [phone]
Admin Fax Ext:
Admin Email: [email protected]
Tech ID: 5372811-ERL
Tech Name: EXAMPLE REGISTRAR TECHNICAL
Tech Organization: EXAMPLE REGISTRAR LLC
Tech Street: 123 EXAMPLE STREET
Tech City: ANYTOWN
Tech State/Province: AP
Tech Postal Code: A1A1A1
Tech Country: EX
Tech Phone: [phone]
Tech Phone Ext: 1234
Tech Fax: [phone]
Tech Fax Ext: 93
Tech Email: [email protected]
Name Server: NS01.EXAMPLEREGISTRAR.TLD
Name Server: NS02.EXAMPLEREGISTRAR.TLD
DNSSEC: signedDelegation
DNSSEC: unsigned

Last update of WHOIS database: 2009-05-29T20:15:00Z <<<

1.6. Registrar Data:

1.6.1 Query format: whois “registrar Example Registrar, Inc.”

1.6.2 Response format:

Registrar Name: Example Registrar, Inc.
Street: 1234 Admiralty Way
City: Marina del Rey
State/Province: CA
Postal Code: 90292
Country: US
Phone Number: +1.3105551212
Fax Number: +1.3105551213
Email: [email protected]
WHOIS Server: whois.example-registrar.tld
Referral URL: http://www.example-registrar.tld
Admin Contact: Joe Registrar
Phone Number: +1.3105551213
Fax Number: +1.3105551213
Email: [email protected]
Admin Contact: Jane Registrar
Phone Number: +1.3105551214
Fax Number: +1.3105551213
Email: [email protected]
Technical Contact: John Geek
Phone Number: +1.3105551215
Fax Number: +1.3105551216
Email: [email protected]

Last update of WHOIS database: 2009-05-29T20:15:00Z <<<

1.7. Nameserver Data:

1.7.1 Query format: whois “NS1.EXAMPLE.TLD”, whois “nameserver (nameserver name)”, or whois “nameserver (IP Address)”

1.7.2 Response format:

Server Name: NS1.EXAMPLE.TLD
IP Address: 192.0.2.123
IP Address: 2001:0DB8::1
Registrar: Example Registrar, Inc.
WHOIS Server: whois.example-registrar.tld
Referral URL: http://www.example-registrar.tld

Last update of WHOIS database: 2009-05-29T20:15:00Z <<<

1.8. The format of the following data fields: domain status, individual and organizational names, address, street, city, state/province, postal code, country, telephone and fax numbers (the extension will be provided as a separate field as shown above), email addresses, date and times should conform to the mappings specified in EPP RFCs 5730-5734 so that the display of this information (or values return in WHOIS responses) can be uniformly processed and understood.

1.9. In order to be compatible with ICANN’s common interface for WHOIS (InterNIC), WHOIS output shall be in the format outline above.

VeriSign responses that only contain partial matches are not handled correctly

When a query is made for which no exact match exists, but one or more partial matches exist - such as www.google.com, which is not the same as google.com - pythonwhois will currently return the first registrar name it finds, but not include any other data.

The correct behaviour would be to not return any parsed information at all, as there was no exact match, and partial matches should be ignored.

Example WHOIS response:

sven@linux-rfa7:~/projects> pwhois --raw www.google.com
Domain not found.

--

Whois Server Version 2.0

Domain names in the .com and .net domains can now be registered
with many different competing registrars. Go to http://www.internic.net
for detailed information.

   Server Name: WWW.GOOGLE.COM.AR
   Registrar: ENOM, INC.
   Whois Server: whois.enom.com
   Referral URL: http://www.enom.com

   Server Name: WWW.GOOGLE.COM.AU
   Registrar: MELBOURNE IT, LTD. D/B/A INTERNET NAMES WORLDWIDE
   Whois Server: whois.melbourneit.com
   Referral URL: http://www.melbourneit.com

   Server Name: WWW.GOOGLE.COM.BR
   Registrar: ENOM, INC.
   Whois Server: whois.enom.com
   Referral URL: http://www.enom.com

   Server Name: WWW.GOOGLE.COM.CO
   Registrar: ENOM, INC.
   Whois Server: whois.enom.com
   Referral URL: http://www.enom.com

   Server Name: WWW.GOOGLE.COM.DO
   Registrar: ENOM, INC.
   Whois Server: whois.enom.com
   Referral URL: http://www.enom.com

   Server Name: WWW.GOOGLE.COM.HK
   Registrar: GKG.NET, INC.
   Whois Server: whois.gkg.net
   Referral URL: http://www.gkg.net

   Server Name: WWW.GOOGLE.COM.MX
   Registrar: ENOM, INC.
   Whois Server: whois.enom.com
   Referral URL: http://www.enom.com

   Server Name: WWW.GOOGLE.COM.PE
   Registrar: HOSTOPIA.COM INC. D/B/A APLUS.NET
   Whois Server: whois.names4ever.com
   Referral URL: http://www.aplus.net

   Server Name: WWW.GOOGLE.COM.PK
   Registrar: INTERNET.BS CORP.
   Whois Server: whois.internet.bs
   Referral URL: http://www.internet.bs

   Server Name: WWW.GOOGLE.COM.SA
   Registrar: OMNIS NETWORK, LLC
   Whois Server: whois.omnis.com
   Referral URL: http://domains.omnis.com

   Server Name: WWW.GOOGLE.COM.TR
   Registrar: TUCOWS DOMAINS INC.
   Whois Server: whois.tucows.com
   Referral URL: http://domainhelp.opensrs.net

   Server Name: WWW.GOOGLE.COM.TW
   Registrar: ENOM, INC.
   Whois Server: whois.enom.com
   Referral URL: http://www.enom.com

   Server Name: WWW.GOOGLE.COM.VN
   Registrar: ENOM, INC.
   Whois Server: whois.enom.com
   Referral URL: http://www.enom.com

>>> Last update of whois database: Sat, 07 Jun 2014 11:00:40 UTC <<<

NOTICE: The expiration date displayed in this record is the date the 
registrar's sponsorship of the domain name registration in the registry is 
currently set to expire. This date does not necessarily reflect the expiration 
date of the domain name registrant's agreement with the sponsoring 
registrar.  Users may consult the sponsoring registrar's Whois database to 
view the registrar's reported date of expiration for this registration.

TERMS OF USE: You are not authorized to access or query our Whois 
database through the use of electronic processes that are high-volume and 
automated except as reasonably necessary to register domain names or 
modify existing registrations; the Data in VeriSign Global Registry 
Services' ("VeriSign") Whois database is provided by VeriSign for 
information purposes only, and to assist persons in obtaining information 
about or related to a domain name registration record. VeriSign does not 
guarantee its accuracy. By submitting a Whois query, you agree to abide 
by the following terms of use: You agree that you may use this Data only 
for lawful purposes and that under no circumstances will you use this Data 
to: (1) allow, enable, or otherwise support the transmission of mass 
unsolicited, commercial advertising or solicitations via e-mail, telephone, 
or facsimile; or (2) enable high volume, automated, electronic processes 
that apply to VeriSign (or its computer systems). The compilation, 
repackaging, dissemination or other use of this Data is expressly 
prohibited without the prior written consent of VeriSign. You agree not to 
use electronic processes that are automated and high-volume to access or 
query the Whois database except as reasonably necessary to register 
domain names or modify existing registrations. VeriSign reserves the right 
to restrict your access to the Whois database in its sole discretion to ensure 
operational stability.  VeriSign may restrict or terminate your access to the 
Whois database for failure to abide by these terms of use. VeriSign 
reserves the right to modify these terms at any time. 

The Registry database contains ONLY .COM, .NET, .EDU domains and
Registrars.

incomplete whois data for domains registered at onamae/gmo

Incomplete data for domains registered at onamae.com/gmo.jp - example domains:

tempescope.com
buzzjp.com
lovemerci.com

example output:

pwhois tempescope.com
Domain ID : 1758738035_DOMAIN_COM-VRSN
Status : Active
Registrar : GMO Internet, INC.
Registration date : 2012-11-12 06:53:17
Expiration date : 2017-11-12 06:53:17
Last update : 2014-10-24 00:50:28
Name server : ns1.jimdo.com
Name server : ns2.jimdo.com
E-mail address : [email protected]
E-mail address : [email protected]

full output:

pwhois --raw tempescope.com
Domain Name: tempescope.com
Registry Domain ID: 1758738035_DOMAIN_COM-VRSN
Registrar WHOIS Server: whois.discount-domain.com
Registrar URL: http://www.onamae.com
Updated Date: 2014-10-24 00:50:28
Creation Date: 2012-11-12 06:53:17.0
Registrar Registration Expiration Date: 2017-11-12 06:53:17.0
Registrar: GMO INTERNET, INC.
Registrar IANA ID: 49
Registrar Abuse Contact Email: [email protected]
Registrar Abuse Contact Phone:
Domain Status: ACTIVE
Registry Registrant ID:
Registrant Name: Whois Privacy Protection Service by onamae.com
Registrant Organization: Whois Privacy Protection Service by onamae.com
Registrant Street1: 26-1 Sakuragaoka-cho
Registrant Street2: Cerulean Tower 11F
Registrant City: Shibuya-ku
Registrant State/Province: Tokyo
Registrant Postal Code: 150-8512
Registrant Country: JP
Registrant Phone: [phone]
Registrant Phone Ext:
Registrant Fax:
Registrant Fax Ext:
Registrant Email: [email protected]
Registry Admin ID:
Admin Name: Whois Privacy Protection Service by onamae.com
Admin Organization: Whois Privacy Protection Service by onamae.com
Admin Street1: 26-1 Sakuragaoka-cho
Admin Street2: Cerulean Tower 11F
Admin City: Shibuya-ku
Admin State/Province: Tokyo
Admin Postal Code: 150-8512
Admin Country: JP
Admin Phone: [phone]
Admin Phone Ext:
Admin Fax:
Admin Fax Ext:
Admin Email: [email protected]
Registry Tech ID:
Tech Name: Whois Privacy Protection Service by onamae.com
Tech Organization: Whois Privacy Protection Service by onamae.com
Tech Street1: 26-1 Sakuragaoka-cho
Tech Street2: Cerulean Tower 11F
Tech City: Shibuya-ku
Tech State/Province: Tokyo
Tech Postal Code: 150-8512
Tech Country: JP
Tech Phone: [phone]
Tech Phone Ext:
Tech Fax:
Tech Fax Ext:
Tech Email: [email protected]
Name Server: ns1.jimdo.com
Name Server: ns2.jimdo.com
DNSSEC: Unsigned
URL of the ICANN WHOIS Data Problem Reporting System: http://wdprs.internic.net/

Last update of WHOIS database: 2014-10-24 00:50:28

Domain Name: TEMPESCOPE.COM
Registrar: GMO INTERNET, INC. DBA ONAMAE.COM
Whois Server: whois.discount-domain.com
Referral URL: http://www.onamae.com
Name Server: NS1.JIMDO.COM
Name Server: NS2.JIMDO.COM
Status: ok
Updated Date: 24-oct-2014
Creation Date: 12-nov-2012
Expiration Date: 12-nov-2017

they have a fair few domains: http://www.webhosting.info/registrars/reports/total_domains/onamae.com

python-whois can only retrieve a little information for .hk domain

Run pwhois on this domain and compare the result with this example:

http://whois.domaintools.com/unwire.hk

domain .eu not parsed

Having problemas with domains .eu:
Example: mapfre-es.eu
Exception in thread Thread-6:
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/pythonwhois/init.py", line 8, in get_whois
return parse.parse_raw_whois(raw_data, normalized=normalized, never_query_handles=False, handle_server=server_list[-1])
File "/usr/local/lib/python2.7/dist-packages/pythonwhois/parse.py", line 516, in parse_raw_whois
data["registrar"].insert(0, match.group(1).strip())
KeyError: 'registrar'

pythonwhois version:
pip search pythonwhois
pythonwhois - Module for retrieving and parsing the WHOIS data for a domain. Supports most domains. No dependencies.
INSTALLED: 2.4.3 (latest)

Cannot parse whois data of IP address, e.g. google IP 173.194.70.106

Not sure different RIR (e.g. ARIN, APNIC, RIPE) will have different format of whois data

All the country name are not in standard format

Hi,

I was retrieving the whois data for multiple ccTLD and gTLD and I found that some Countries name are in abbreviation and while some has full name.

While parsing the data, can we parse those name into one standard format. In my script I have used following country code mapping data:

http://www.worldatlas.com/aatlas/ctycodes.htm

No WHOIS server for .ps TLD

echo whois; whois v.ps; echo pwhois;./pwhois v.ps;echo pls fix
whois
Domain Name: v.ps
Domain Status: Available
Notes: This name is not allowed by the policy of this registry, and can not be registered

TERMS OF USE: You are not authorized to access or query our Whois
database through the use of electronic processes that are high-volume and
automated. Whois database is provided by PNINA as a service to the internet
community on behalf of PNINA and Its Certified Registrars (CR). (http://www.pnina.ps/registrars/registrars-list/)

The data is for information purposes only. PNINA does not
guarantee its accuracy. By submitting a Whois query, you agree to abide
by the following terms of use: You agree that you may use this Data only
for lawful purposes and that under no circumstances will you use this Data
to: (1) allow, enable, or otherwise support the transmission of mass
unsolicited, commercial advertising or solicitations via e-mail, telephone,
or facsimile; or (2) enable high volume, automated, electronic processes
that apply to PNINA and it's Certified Registrar's (or PNINA or CR computer systems). The
compilation, repackaging, dissemination or other use of this Data is
expressly prohibited.

Last update of WHOIS database: 2014-05-22T15:18:54.252Z <<<
pwhois
Traceback (most recent call last):
File "./pwhois", line 23, in
data = pythonwhois.net.get_whois_raw(args.domain[0])
File "/tmp/python-whois/pythonwhois/net.py", line 12, in get_whois_raw
target_server = get_root_server(domain)
File "/tmp/python-whois/pythonwhois/net.py", line 48, in get_root_server
raise shared.WhoisException("No root WHOIS server found for TLD.")
pythonwhois.shared.WhoisException: No root WHOIS server found for TLD.
pls fix

Error with new TLD

Hello, i'm want get whois for new TLD, you can get the liste here : http://www.opensrs.com/site/services/new-tlds/price-list-new-tlds

But i'am have this error :

root@LHA:~# pwhois ikoula.support
Traceback (most recent call last):
File "/usr/local/bin/pwhois", line 23, in
data = pythonwhois.net.get_whois_raw(args.domain[0])
File "/usr/local/lib/python2.7/dist-packages/pythonwhois/net.py", line 61, in get_whois_raw
return get_whois_raw(domain, referal_server, new_list)
File "/usr/local/lib/python2.7/dist-packages/pythonwhois/net.py", line 37, in get_whois_raw
response = whois_request(request_domain, target_server)
File "/usr/local/lib/python2.7/dist-packages/pythonwhois/net.py", line 75, in whois_request
sock.connect((server, port))
File "/usr/lib/python2.7/socket.py", line 224, in meth
return getattr(self._sock,name)(*args)
socket.gaierror: [Errno -2] Name or service not known

Can you solve this error ?
Thank you

Limitations

Is there any limitations on amount of URLs per day sent? I can't find this info in documentation, but I saw somewhere 100 per day?
Also maybe I reached the daily limit, but method
pythonwhois.get_whois(url here)
just stopped responding, hanging up my python script.
Any ideas?

.cn, .co.jp, .com.mx, .com.tw, and other domains not parsed

Hi there again :)

First, thanks for the quick reply for the .it problems. Then, I did some further test and I have found several TLDs that are not parsed correctly. The .cn, .co.jp, .com.mx, .com.tw are few of them that I have checked manually, but there are others.

I created a tarball that you can take from here http://trouge.net/misc/slds_pwhois.tar.bz2 which contains two bash scripts and the folder test/.

The script pwhois_test.sh runs pwhois -j on a number of domains and store the stdout and stderr in test/. It creates 2 files for each domain. The script parse_pwhois_json.sh shows the field "contacts" by using the tool jq to parse json files.

Please, consider that in test/ there are combinations of SLDs and TLDs that makes no sense, e.g., google.ca.gov and I haven't filtered them out. Sorry for that :(

Hope this will help improving the tool, I am planning to use it in few experiments and it would be great if the TLDs above could be supported as well.

If you like, I could help you in testing the domains again to confirm the fix.

Error while parsing some domains.

I am having issues while parsing some domains:

whois_data = get_whois(domain)

File "/usr/local/lib/python2.7/dist-packages/pythonwhois/init.py", line 4, in get_whois
raw_data, server_list = net.get_whois_raw(domain, with_server_list=True)
File "/usr/local/lib/python2.7/dist-packages/pythonwhois/net.py", line 69, in get_whois_raw
return get_whois_raw(domain, referal_server, new_list, server_list=server_list, with_server_list=with_server_list)
File "/usr/local/lib/python2.7/dist-packages/pythonwhois/net.py", line 44, in get_whois_raw
response = whois_request(request_domain, target_server)
File "/usr/local/lib/python2.7/dist-packages/pythonwhois/net.py", line 94, in whois_request
return buff.decode("utf-8")
File "/usr/lib/python2.7/encodings/utf_8.py", line 16, in decode
return codecs.utf_8_decode(input, errors, True)
UnicodeDecodeError: 'utf8' codec can't decode byte 0xf3 in position 1345: invalid continuation byte

Looks like library is not parsing well.
pythonwhois version:
pip search pythonwhois
pythonwhois - Module for retrieving and parsing the WHOIS data for a domain. Supports most domains. No dependencies.
INSTALLED: 2.4.3 (latest)

domain tested:
sesise.com
gcbcapp.com

Trying with linux whois you can see especial characters on some text or contacts, respectively.
Example for gcbapp.com:
$ whois gcbcapp.com
[...]
Technical Contact:
technical_contact_organization: Uni▒ Empresaris Hostaleria i Turisme Costa Brava
technical_contact_name: Teresa Marqu▒s
technical_contact_address1: Ctra. Palam▒s, km. 328

.AX only email address is parsed

Only the email address is parsed from the .AX whois output.

Python 3.x support?

It does not appear python-whois has Python 3.x support. As 3.x grows more commonplace, this will become a serious obstacle to adoption.

Some problems I ran into:

~/python-whois$ python3 pwhois
  File "pwhois", line 6
    except ImportError, e:
                      ^
SyntaxError: invalid syntax


~/python-whois$ python3
Python 3.4.0 (default, Apr 11 2014, 13:05:11) 
[GCC 4.8.2] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from pythonwhois import get_whois
>>> get_whois('google.com')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/fsufitch/python-whois/pythonwhois/__init__.py", line 4, in get_whois
    raw_data = net.get_whois_raw(domain)
  File "/home/fsufitch/python-whois/pythonwhois/net.py", line 15, in get_whois_raw
    domain = encode( domain if type(domain) is unicode else decode(domain, "utf8"),    "idna" )
NameError: name 'unicode' is not defined

There may be other problems too, but I do not have the time to conduct a code review of it right now.

Thanks for looking into this! I really wanted to use pythonwhois but my project is unfortunately python3-only.

IP range WHOIS

Proper WHOIS lookups for IP ranges should eventually be implemented.

TODO:

• Investigate whether root server will answer correctly for IP lookups
• Add patterns for recognizing IP WHOIS data

.ai isn't parsed correctly

.ai isn't parsed correctly, tested on the domain wit.ai. pwhois is the latest version.

pwhois output:

Name server : primary
Name server : secondary
Name server : third
Name server : fourth
E-mail address : [email protected]

whois output:

DOMAIN INFORMATION

Complete Domain Name........: wit.ai
Organization Using Domain Name
Organization Name..........:
Street Address.............:
City.......................:
etc.

KR

How to handle "Operation Timed Out" exception.

Hi,

For some domains, process is hanging as it is getting struck in ".get_whois()" call. It is taking too long before throwing exception,for example: domain "clavosdecristo.es"

---> 11 w = pythonwhois.get_whois('clavosdecristo.es')
     12 
     13 for k in w.keys():

/Users/rakumar/anaconda/lib/python2.7/site-packages/pythonwhois-2.4.1-py2.7.egg/pythonwhois/__init__.pyc in get_whois(domain, normalized)
      2 
      3 def get_whois(domain, normalized=[]):
----> 4         raw_data, server_list = net.get_whois_raw(domain, with_server_list=True)
      5         # Unlisted handles will be looked up on the last WHOIS server that was queried. This may be changed to also query
      6         # other servers in the future, if it turns out that there are cases where the last WHOIS server in the chain doesn't

/Users/rakumar/anaconda/lib/python2.7/site-packages/pythonwhois-2.4.1-py2.7.egg/pythonwhois/net.pyc in get_whois_raw(domain, server, previous, rfc3490, never_cut, with_server_list, server_list)
     40         else:
     41                 request_domain = domain
---> 42         response = whois_request(request_domain, target_server)
     43         if never_cut:
     44                 # If the caller has requested to 'never cut' responses, he will get the original response from the server (this is

/Users/rakumar/anaconda/lib/python2.7/site-packages/pythonwhois-2.4.1-py2.7.egg/pythonwhois/net.pyc in whois_request(domain, server, port)
     86         buff = b""
     87         while True:
---> 88                 data = sock.recv(1024)
     89                 if len(data) == 0:
     90                         break

error: [Errno 60] Operation timed out

Web-based WHOIS

Need to investigate the possibilities for (dependency-less) parsing of web-based WHOIS data, for registries such as SIDN and EURid. CAPTCHAs might be a problem here.

Detect non-existent domain

There should be some way to determine that a specified domain name does not exist. Not sure whether to make it throw an exception, or return a special 'status' value.

Detection could be done through checking for the existence of common fields, such as creation date or expiration date. I'll have to look into the specifics of this, to make sure that no false positives occur.

IndexError: list index out of range

Hello!
I've just installed this module and created simple file with:

import sys, pythonwhois
testlist = open("domains.txt").readlines()
raw, result = pythonwhois.whois(sys.argv[1])
print raw
print result

Python interpreter say me:

"IndexError: list index out of range" in the 3-th line.
How to fix it?

With regards,
Ivan.

Decoding issues for bidtheatre.com

#44 includes a fix for dealing with Unicode decoding issues, but in the case of bidtheatre.com (WHOIS served by "Active 24 AS", which is not correctly parsed yet), some characters still appear to be lost. This needs some looking into.

A specific source encoding may have to be used for particular WHOIS servers.

cannot parse OVH dot eu domains

$ whois hl3.eu;pwhois hl3.eu;echo pls fix
% The WHOIS service offered by EURid and the access to the records
% in the EURid WHOIS database are provided for information purposes
% only. It allows persons to check whether a specific domain name
% is still available or not and to obtain information related to
% the registration records of existing domain names.
%
% EURid cannot, under any circumstances, be held liable in case the
% stored information would prove to be wrong, incomplete or not
% accurate in any sense.
%
% By submitting a query you agree not to use the information made
% available to:
%
% - allow, enable or otherwise support the transmission of unsolicited,
% commercial advertising or other solicitations whether via email or
% otherwise;
% - target advertising in any possible way;
%
% - to cause nuisance in any possible way to the registrants by sending
% (whether by automated, electronic processes capable of enabling
% high volumes or other possible means) messages to them.
%
% Without prejudice to the above, it is explicitly forbidden to extract,
% copy and/or use or re-utilise in any form and by any means
% (electronically or not) the whole or a quantitatively or qualitatively
% substantial part of the contents of the WHOIS database without prior
% and explicit permission by EURid, nor in any attempt hereof, to apply
% automated, electronic processes to EURid (or its systems).
%
% You agree that any reproduction and/or transmission of data for
% commercial purposes will always be considered as the extraction of a
% substantial part of the content of the WHOIS database.
%
% By submitting the query you agree to abide by this policy and accept
% that EURid can take measures to limit the use of its WHOIS services
% in order to protect the privacy of its registrants or the integrity
% of the database.
%
% The EURid WHOIS service on port 43 (textual whois) never
% discloses any information concerning the registrant.
% Registrant and onsite contact information can be obtained through use of the
% webbased whois service available from the EURid website www.eurid.eu
%
% WHOIS hl3
Domain: hl3

Registrant:
NOT DISCLOSED!
Visit www.eurid.eu for webbased whois.

Reseller:

Technical:
Name: Klaba Octave
Organisation: OVH
Language: fr
Phone: [phone]
Fax: [phone]
Email: [email protected]

Registrar:
Name: OVH SAS
Website: www.ovh.com/

Name servers:
noah.ns.cloudflare.com
lady.ns.cloudflare.com

Keys:

Please visit www.eurid.eu for more info.
Registrar : Name: OVH SAS
E-mail address : [email protected]
pls fix

No whois server for .moe

Hi,

pwhois gives the below error when looking up .moe domains:

"No root WHOIS server found for domain."

looking at the IANA webpage for .moe, there isn't a whois server listed:

http://www.iana.org/domains/root/db/moe.html

but querying manually whois.nic.me works:

whois get.moe -h whois.nic.moe

I'm not sure if you'd add this as an exception in net.py? Not sure when/if IANA would be updated to include the correct whois server.

KR

eurodns.com parsing

Hi.
Love the software.
there seems to be a problems with subject, following is the data from the Whois command:

whois vulnweb.com
[Querying whois.verisign-grs.com]
[Redirected to whois.eurodns.com]
[Querying whois.eurodns.com]
[whois.eurodns.com]

The Data in EuroDNS WHOIS database is provided for information purposes only.
The fact that EuroDNS display such information does not provide any guarantee
expressed or implied on the purpose for which the database may be used, its
accuracy or usefulness. By submitting a WHOIS query, you agree that you will
use this Data only for lawful purposes and that, under no circumstances will
you use this Data to:

(1) allow, enable, or otherwise support the transmission of mass unsolicited,
commercial advertising or solicitations via e-mail (spam); or
(2) enable high volume, automated, electronic processes that apply to EuroDNS
(or its systems). EuroDNS reserves the right to modify these terms at any time.

By submitting this query, you agree to abide by the above policy.

Domain Name: vulnweb.com
Registry Domain ID: D4569573-COM
Registrar WHOIS Server: whois.eurodns.com
Registrar URL: http://www.eurodns.com
Updated Date: 2014-04-08T10-03-21Z
Creation Date: 2010-06-14T00-00-00Z
Registrar Registration Expiration Date: 2017-06-13T00-00-00Z
Registrar: Eurodns S.A.
Registrar IANA ID: 1052
Registrar Abuse Contact Email: legalservices[at]eurodns.com
Registrar Abuse Contact Phone: [phone]
Domain Status: clientTransferProhibited
Registry Registrant ID:
Registrant Name: Abela Robert
Registrant Organization: Acunetix Ltd
Registrant Street: Block B, Office 303, Engomi Business Center, 1, 28th October Street
Registrant City: Nicosia
Registrant State/Province:
Registrant Postal Code: 2414
Registrant Country: CYPRUS
Registrant Phone: [phone]
Registrant Fax:
Registrant Email: [email protected]
Registry Admin ID:
Admin Name: Abela Robert
Admin Organization: Acunetix Ltd
Admin Street: Block B, Office 303, Engomi Business Center, 1, 28th October Street
Admin City: Nicosia
Admin State/Province:
Admin Postal Code: 2414
Admin Country: CYPRUS
Admin Phone: [phone]
Admin Fax:
Admin Email: [email protected]
Registry Tech ID:
Tech Name: Abela Robert
Tech Organization: Acunetix Ltd
Tech Street: Block B, Office 303, Engomi Business Center, 1, 28th October Street
Tech City: Nicosia
Tech State/Province:
Tech Postal Code: 2414
Tech Country: CYPRUS
Tech Phone: [phone]
Tech Fax:
Tech Email: [email protected]
Name Server: ns1.eurodns.com
Name Server: ns2.eurodns.com
Name Server: ns3.eurodns.com
Name Server: ns4.eurodns.com
DNSSEC:
ICANN WHOIS Data Problem Reporting System: http://www.icann.org/en/resources/compliance/complaints/whois/inaccuracy-form

Whois History: 164 records have been archived since 2010-06-15
http://www.domaintools.com/research/whois-history/?page=results&Affiliate_ID=1001861&q=vulnweb.com

And following is the content of the python whois dict:

{
'status': ['clientTransferProhibited'], 
'updated_date': [datetime.datetime(2014, 4, 8, 0, 0)], 
'contacts': {'admin': None, 'tech': None, 'registrant': None, 'billing': None}, 
'nameservers': ['ns1.eurodns.com', 'ns2.eurodns.com', 'ns3.eurodns.com', 'ns4.eurodns.com'], 
'expiration_date': [datetime.datetime(2017, 6, 13, 0, 0)], 
'emails': ['[email protected]'], 
'raw': ['\nThe Data in EuroDNS WHOIS database is provided for information purposes only.\n
The fact that EuroDNS display such information does not provide any guarantee\n
expressed or implied on the purpose for which the database may be used, its\n
accuracy or usefulness. By submitting a WHOIS query, you agree that you will\n
use this Data only for lawful purposes and that, under no circumstances will\n
you use this Data to:\n\n
(1) allow, enable, or otherwise support the transmission of mass unsolicited,\n
commercial advertising or solicitations via e-mail (spam); or\n
(2) enable high volume, automated, electronic processes that apply to EuroDNS\n
(or its systems). EuroDNS reserves the right to modify these terms at any time.\n\n

By submitting this query, you agree to abide by the above policy.\n\n

Domain Name: vulnweb.com\n
Registry Domain ID: D4569573-COM\n
Registrar WHOIS Server: whois.eurodns.com\n
Registrar URL: http://www.eurodns.com\n
Updated Date: 2014-04-08T10-03-21Z\n
Creation Date: 2010-06-14T00-00-00Z\n
Registrar Registration Expiration Date: 2017-06-13T00-00-00Z\n
Registrar: Eurodns S.A.\n
Registrar IANA ID: 1052\n
Registrar Abuse Contact Email: legalservices[at]eurodns.com\n
Registrar Abuse Contact Phone: [phone]\n
Domain Status: clientTransferProhibited\n
Registry Registrant ID: \n
Registrant Name: Abela Robert\n
Registrant Organization: Acunetix Ltd\n
Registrant Street: Block B, Office 303, Engomi Business Center, 1, 28th October Street\n
Registrant City: Nicosia\n
Registrant State/Province:\n
Registrant Postal Code: 2414\n
Registrant Country: CYPRUS\n
Registrant Phone: [phone]\n
Registrant Fax: \n
Registrant Email: [email protected]\n
Registry Admin ID: \n
Admin Name: Abela Robert\n
Admin Organization: Acunetix Ltd\n
Admin Street: Block B, Office 303, Engomi Business Center, 1, 28th October Street\n
Admin City: Nicosia\n
Admin State/Province:\n
Admin Postal Code: 2414\n
Admin Country: CYPRUS\n
Admin Phone: [phone]\n
Admin Fax: \n
Admin Email: [email protected]\n
Registry Tech ID: \n
Tech Name: Abela Robert\n
Tech Organization: Acunetix Ltd\n
Tech Street: Block B, Office 303, Engomi Business Center, 1, 28th October Street\n
Tech City: Nicosia\n
Tech State/Province:\n
Tech Postal Code: 2414\n
Tech Country: CYPRUS\n
Tech Phone: [phone]\n
Tech Fax: \n
Tech Email: [email protected]\n
Name Server: ns1.eurodns.com\n
Name Server: ns2.eurodns.com\n
Name Server: ns3.eurodns.com\n
Name Server: ns4.eurodns.com\n
DNSSEC: \n
ICANN WHOIS Data Problem Reporting System: http://www.icann.org/en/resources/compliance/complaints/whois/inaccuracy-form\n\n

Whois History: 164 records have been archived since 2010-06-15\n
http://www.domaintools.com/research/whois-history/?page=results&Affiliate_ID=1001861&q=vulnweb.com\n', '\n

Whois Server Version 2.0\n\n

Domain names in the .com and .net domains can now be registered\n
with many different competing registrars. Go to http://www.internic.net\n
for detailed information.\n\n   

Domain Name: VULNWEB.COM\n   
Registrar: EURODNS S.A\n   
Whois Server: whois.eurodns.com\n   
Referral URL: http://www.eurodns.com\n   
Name Server: NS1.EURODNS.COM\n   
Name Server: NS2.EURODNS.COM\n   
Name Server: NS3.EURODNS.COM\n   
Name Server: NS4.EURODNS.COM\n   
Status: clientTransferProhibited\n   
Updated Date: 08-apr-2014\n   
Creation Date: 14-jun-2010\n   
Expiration Date: 14-jun-2017\n\n

>>> Last update of whois database: Sat, 26 Apr 2014 11:46:30 UTC <<<\n\n

NOTICE: The expiration date displayed in this record is the date the \n
registrar\'s sponsorship of the domain name registration in the registry is \n
currently set to expire. This date does not necessarily reflect the expiration \n
date of the domain name registrant\'s agreement with the sponsoring \n
registrar.  Users may consult the sponsoring registrar\'s Whois database to \n
view the registrar\'s reported date of expiration for this registration.\n\n

TERMS OF USE: You are not authorized to access or query our Whois \n
database through the use of electronic processes that are high-volume and \n
automated except as reasonably necessary to register domain names or \n
modify existing registrations; the Data in VeriSign Global Registry \n
Services\' ("VeriSign") Whois database is provided by VeriSign for \n
information purposes only, and to assist persons in obtaining information \n
about or related to a domain name registration record. VeriSign does not \n
guarantee its accuracy. By submitting a Whois query, you agree to abide \n
by the following terms of use: You agree that you may use this Data only \n
for lawful purposes and that under no circumstances will you use this Data \n
to: (1) allow, enable, or otherwise support the transmission of mass \n
unsolicited, commercial advertising or solicitations via e-mail, telephone, \n
or facsimile; or (2) enable high volume, automated, electronic processes \n
that apply to VeriSign (or its computer systems). The compilation, \n
repackaging, dissemination or other use of this Data is expressly \n
prohibited without the prior written consent of VeriSign. You agree not to \n
use electronic processes that are automated and high-volume to access or \n
query the Whois database except as reasonably necessary to register \n
domain names or modify existing registrations. VeriSign reserves the right \n
to restrict your access to the Whois database in its sole discretion to ensure \n
operational stability.  VeriSign may restrict or terminate your access to the \n
Whois database for failure to abide by these terms of use. VeriSign \n
reserves the right to modify these terms at any time. \n\n

The Registry database contains ONLY .COM, .NET, .EDU domains and\n
Registrars.'], 
'whois_server': ['whois.eurodns.com'], 
'registrar': ['Eurodns S.A.'], 
'creation_date': [datetime.datetime(2010, 6, 14, 0, 0)], 
'id': ['D4569573-COM']
}

And if I run the whois command against the IP Address I get the following whois result:

whois 87.230.29.167
[Querying whois.ripe.net]
[whois.ripe.net]
% This is the RIPE Database query service.
% The objects are in RPSL format.
%
% The RIPE Database is subject to Terms and Conditions.
% See http://www.ripe.net/db/support/db-terms-conditions.pdf

% Note: this output has been filtered.
%       To receive output for a database update, use the "-B" flag.

% Information related to '87.230.29.0 - 87.230.29.255'

% Abuse contact for '87.230.29.0 - 87.230.29.255' is '[email protected]'

inetnum:        87.230.29.0 - 87.230.29.255
remarks:        INFRA-AW
netname:        HE-WVPS-29-CGN3-NET
descr:          Hosteurope GmbH
descr:          [email protected]
country:        DE
admin-c:        HER
tech-c:         HER
status:         ASSIGNED PA
mnt-by:         HOSTEUROPE-MNT
source:         RIPE # Filtered

role:           Host Europe Ripehandle
address:        Welserstrasse 14
address:        51149 Koeln
phone:          [phone]
abuse-mailbox:  [email protected]
admin-c:        BURN
admin-c:        HONK
admin-c:        JUPP
admin-c:        MATE
admin-c:        METT
admin-c:        MOMO
admin-c:        OUZO
admin-c:        SEPP
admin-c:        WIRR
tech-c:         BURN
tech-c:         HONK
tech-c:         JUPP
tech-c:         MATE
tech-c:         METT
tech-c:         MOMO
tech-c:         OUZO
tech-c:         SEPP
tech-c:         WIRR
nic-hdl:        HER
mnt-by:         HOSTEUROPE-MNT
source:         RIPE # Filtered

% Information related to '87.230.0.0/17AS20773'

route:          87.230.0.0/17
descr:          DE-HER-87-230-SLASH-17
origin:         AS20773
member-of:      AS20773:RS-HOSTEUROPE
mnt-by:         HOSTEUROPE-MNT
source:         RIPE # Filtered

% This query was served by the RIPE Database Query Service version 1.72 (DBC-WHOIS4)

And once more the content of the python whois dict is:

{
'status': ['ASSIGNED PA'], 
'raw': ['% This is the RIPE Database query service.\n
% The objects are in RPSL format.\n
%\n
% The RIPE Database is subject to Terms and Conditions.\n
% See http://www.ripe.net/db/support/db-terms-conditions.pdf\n\n

% Note: this output has been filtered.\n
%       To receive output for a database update, use the "-B" flag.\n\n

% Information related to \'87.230.29.0 - 87.230.29.255\'\n\n

% Abuse contact for \'87.230.29.0 - 87.230.29.255\' is \'[email protected]\'\n\n

inetnum:        87.230.29.0 - 87.230.29.255\n
remarks:        INFRA-AW\n
netname:        HE-WVPS-29-CGN3-NET\n
descr:          Hosteurope GmbH\n
descr:          [email protected]\n
country:        DE\nadmin-c:        HER\n
tech-c:         HER\n
status:         ASSIGNED PA\n
mnt-by:         HOSTEUROPE-MNT\n
source:         RIPE # Filtered\n\n

role:           Host Europe Ripehandle\n
address:        Welserstrasse 14\n
address:        51149 Koeln\n
phone:          [phone]\n
abuse-mailbox:  [email protected]\n
admin-c:        BURN\n
admin-c:        HONK\n
admin-c:        JUPP\n
admin-c:        MATE\n
admin-c:        METT\n
admin-c:        MOMO\n
admin-c:        OUZO\n
admin-c:        SEPP\n
admin-c:        WIRR\n
tech-c:         BURN\n
tech-c:         HONK\n
tech-c:         JUPP\n
tech-c:         MATE\n
tech-c:         METT\n
tech-c:         MOMO\n
tech-c:         OUZO\n
tech-c:         SEPP\n
tech-c:         WIRR\n
nic-hdl:        HER\n
mnt-by:         HOSTEUROPE-MNT\n
source:         RIPE # Filtered\n\n

% Information related to \'87.230.0.0/17AS20773\'\n\n

route:          87.230.0.0/17\n
descr:          DE-HER-87-230-SLASH-17\n
origin:         AS20773\n
member-of:      AS20773:RS-HOSTEUROPE\n
mnt-by:         HOSTEUROPE-MNT\n
source:         RIPE # Filtered\n\n

% This query was served by the RIPE Database Query Service version 1.72 (DBC-WHOIS3)\n\n\n'], 
'emails': ['[email protected]', '[email protected]'], 
'contacts': {'admin': {'handle': 'HER'}, 'tech': {'handle': 'HER'}, 
'registrant': None, 'billing': None}
}

Change behaviour for update date for non-existent domains

When searching for a non-existent domain with a .com TLD, pythonwhois returns an empty parsed record, with the exception of updated_date, which is set from the returned raw string "Last update of whois database: [date]". I see the library falls back to this at parse.py:131.

I'd propose that this string is not matched, as it relates to the whois database as a whole, not the record in question. Removing this match will make it easier to identify non-existent records.

registrant & nameservers not parsed for .rs

registrant & name servers aren't parsed for .rs domain names.

example domains:
tvbest.rs - whois privacy
lobste.rs - no whois privacy

hidden registrant:

pwhois tvbest.rs
Status : Active
Registrar : BeotelNet-ISP d.o.o.
Registration date : 2012-08-30 00:00:00
Expiration date : 2015-08-30 00:00:00

raw:

pwhois tvbest.rs --raw
--snip--

Domain name: tvbest.rs.
Domain status: Active
Registration date: 30.08.2012 13:42:24
Modification date: 08.08.2014 09:09:20
Expiration date: 30.08.2015 13:42:24
Registrar: BeotelNet-ISP d.o.o.

Whois privacy has been activated for domain.
DNS: ns1.itplatforma.com. -
DNS: ns2.itplatforma.com. -
DNS: ns3.itplatforma.com. -

domains that don't have whois privacy activated have a "Registrant:" line that isn't parsed (pwhois lobste.rs --raw).

KeyError: 'registrar'

Using version 2.4.3

import pythonwhois

pythonwhois.get_whois('crushmarketing.ca')

Resulting traceback:

KeyError                                  Traceback (most recent call last)
<ipython-input-2-1bf6b8459b38> in <module>()
----> 1 pythonwhois.get_whois('crushmarketing.ca')

~/.virtualenvs/dev/local/lib/python2.7/site-packages/pythonwhois/__init__.pyc in get_whois(domain, normalized)
      6         # other servers in the future, if it turns out that there are cases where the last WHOIS server in the chain doesn't
      7         # actually hold the handle contact details, but another WHOIS server in the chain does.
----> 8         return parse.parse_raw_whois(raw_data, normalized=normalized, never_query_handles=False, handle_server=server_list[-1])
      9
     10 def whois(*args, **kwargs):

~/.virtualenvs/dev/local/lib/python2.7/site-packages/pythonwhois/parse.pyc in parse_raw_whois(raw_data, normalized, never_query_handles, handle_server)
    514                 match = re.search("Registrar:\n\s+(?:Name:\s*)?(\S.*)", segment)
    515                 if match is not None:
--> 516                         data["registrar"].insert(0, match.group(1).strip())
    517                 match = re.search("(?:Domain nameservers|Name servers):([\s\S]*?\n)\n", segment)
    518                 if match is not None:

KeyError: 'registrar'

Sina.com.cn does not parses registration date

It cannot parse registration date for this domain I suspect other domains ending with .cn have same fate , although a cli WHOIS , shows registration date and in the below output you can see it in the 'raw' value but parser cannot find and only shows expiry date !

input:
w = pythonwhois.get_whois("sina.com.cn")
pprint.pprint(w)

output:

{'contacts': {'admin': None,
'billing': None,
'registrant': None,
'tech': None},
'emails': ['[email protected]'],
'expiration_date': [datetime.datetime(2019, 12, 4, 9, 32, 35)],
'nameservers': ['ns3.sina.com.cn',
'ns2.sina.com.cn',
'ns4.sina.com.cn',
'ns1.sina.com.cn'],
'raw': ['Domain Name: sina.com.cn\nROID: 20021209s10011s00082127-cn\nDomain Status: clientDeleteProhibited\nDomain Status: serverDeleteProhibited\nDomain Status: clientUpdateProhibited\nDomain Status: serverUpdateProhibited\nDomain Status: clientTransferProhibited\nDomain Status: serverTransferProhibited\nRegistrant ID: sinacomcn2\nRegistrant: \xe5\x8c\x97\xe4\xba\xac\xe6\x96\xb0\xe6\xb5\xaa\xe4\xba\x92\xe8\x81\x94\xe4\xbf\xa1\xe6\x81\xaf\xe6\x9c\x8d\xe5\x8a\xa1\xe6\x9c\x89\xe9\x99\x90\xe5\x85\xac\xe5\x8f\xb8\nRegistrant Contact Email: [email protected]\nSponsoring Registrar: \xe5\x8c\x97\xe4\xba\xac\xe6\x96\xb0\xe7\xbd\x91\xe6\x95\xb0\xe7\xa0\x81\xe4\xbf\xa1\xe6\x81\xaf\xe6\x8a\x80\xe6\x9c\xaf\xe6\x9c\x89\xe9\x99\x90\xe5\x85\xac\xe5\x8f\xb8\nName Server: ns3.sina.com.cn\nName Server: ns2.sina.com.cn\nName Server: ns4.sina.com.cn\nName Server: ns1.sina.com.cn\nRegistration Date: 1998-11-20 00:00:00\nExpiration Date: 2019-12-04 09:32:35\nDNSSEC: unsigned\n'],
'registrar': ['\xe5\x8c\x97\xe4\xba\xac\xe6\x96\xb0\xe7\xbd\x91\xe6\x95\xb0\xe7\xa0\x81\xe4\xbf\xa1\xe6\x81\xaf\xe6\x8a\x80\xe6\x9c\xaf\xe6\x9c\x89\xe9\x99\x90\xe5\x85\xac\xe5\x8f\xb8'],
'status': ['clientDeleteProhibited',
'serverDeleteProhibited',
'clientUpdateProhibited',
'serverUpdateProhibited',
'clientTransferProhibited',
'serverTransferProhibited']}

Issue with new TLDs

I just installed your library yesterday and it saved me quite a bit or headache in extracting individual items out of whois records. Many Thanks!!!

I did have an issue looking up some of the new TLDs and found the following: article:http://superuser.com/questions/758647/how-to-whois-new-tlds

I added these as described to my Linux server but pythonwhois still didn't work.

I then dug into pythonwhois and found net.py does the lookups internally, and added the whois servers from the article into the "exceptions" structure. This seems to have fixed the issue.

Just thought I would let you know

V/R

Unicode parsing problems

Looking up some IP addresses seems to lead to an UnicodeDecodeError:

>>> pythonwhois.get_whois('179.175.242.131')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/fsufitch/python-whois/pythonwhois/__init__.py", line 4, in get_whois
    raw_data, server_list = net.get_whois_raw(domain, with_server_list=True)
  File "/home/fsufitch/python-whois/pythonwhois/net.py", line 42, in get_whois_raw
    response = whois_request(request_domain, target_server)
  File "/home/fsufitch/python-whois/pythonwhois/net.py", line 92, in whois_request
    return buff.decode("utf-8")
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xd4 in position 565: invalid continuation byte

Also:

>>> pythonwhois.get_whois('80.148.135.28')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/fsufitch/python-whois/pythonwhois/__init__.py", line 8, in get_whois
    return parse.parse_raw_whois(raw_data, normalized=normalized, never_query_handles=False, handle_server=server_list[-1])
  File "/home/fsufitch/python-whois/pythonwhois/parse.py", line 553, in parse_raw_whois
    data["contacts"] = parse_registrants(raw_data, never_query_handles, handle_server)
  File "/home/fsufitch/python-whois/pythonwhois/parse.py", line 894, in parse_registrants
    contact = fetch_nic_contact(data_reference["handle"], handle_server)
  File "/home/fsufitch/python-whois/pythonwhois/parse.py", line 981, in fetch_nic_contact
    response = net.get_whois_raw(handle, lookup_server)
  File "/home/fsufitch/python-whois/pythonwhois/net.py", line 42, in get_whois_raw
    response = whois_request(request_domain, target_server)
  File "/home/fsufitch/python-whois/pythonwhois/net.py", line 92, in whois_request
    return buff.decode("utf-8")
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xdf in position 706: invalid continuation byte

More IPs that break in this way:

• 177.169.129.223
• 80.148.135.28
• 181.1.55.141
• 91.6.97.224
• 179.246.105.144
• 94.92.58.77

I ran into this while testing a separate program on completely random IPs. To reproduce:

import ipaddress, random, traceback
import pythonwhois

randip = lambda: ipaddress.ip_address(random.randint(0,256**4-1)).compressed

for i in range(50):
   ip = randip()
   try:
       pythonwhois.get_whois(ip)
   except UnicodeDecodeError as e:
       print('========', ip, '========')
       traceback.print_exc()
   except Exception as e:
       continue

Thanks!

another regex explosion

This is similar to issue #2. The call takes ~400s to complete; if interrupted, the traceback is somewhere is re module.

>>> import pythonwhois
>>> pythonwhois.get_whois('drpciv.biz')
^CTraceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "[...]/lib/python2.7/site-packages/pythonwhois/__init__.py", line 5, in get_whois
    return parse.parse_raw_whois(raw_data, normalized=normalized)
  File "[...]/lib/python2.7/site-packages/pythonwhois/parse.py", line 255, in parse_raw_whois
    data["contacts"] = parse_registrants(raw_data)
  File "[...]/lib/python2.7/site-packages/pythonwhois/parse.py", line 639, in parse_registrants
    match = re.search(regex, segment)
  File "[...]/lib/python2.7/re.py", line 148, in search
    return _compile(pattern, flags).search(string)
KeyboardInterrupt

Different behavior between pwhois and pwhois -f for .ch TLDs

pwhois swisscom.ch does not parse the name servers:

Registrant
    Name              : Karin Hug
                        Domain Registration
    Organization      : Swisscom AG
    Street address    : alte Tiefenaustrasse 6
    Postal code       : CH-3050
    City              : Bern
    Country           : Switzerland

Technical Contact
    Name              : Andreas Disteli
    Organization      : Swisscom IT Services AG
    Street address    : Ostermundigenstrasse 99 6
    Postal code       : CH-3050
    City              : Bern
    Country           : Switzerland

But saving it to file and parsing it from there works (pwhois --raw swisscom.ch > swisscom.ch && pwhois -f swisscom.ch .):

Name server       : dns1.swisscom.com
Name server       : dns2.swisscom.com
Name server       : dns3.swisscom.com
Name server       : dns6.swisscom.ch
Name server       : dns7.swisscom.ch

Registrant
    Name              : Karin Hug
                        Domain Registration
    Organization      : Swisscom AG
    Street address    : alte Tiefenaustrasse 6
    Postal code       : CH-3050
    City              : Bern
    Country           : Switzerland

Technical Contact
    Name              : Andreas Disteli
    Organization      : Swisscom IT Services AG
    Street address    : Ostermundigenstrasse 99 6
    Postal code       : CH-3050
    City              : Bern
    Country           : Switzerland

ISO 8601 date format

I'm using python-whois (a much needed library) with Celery. Celery commonly uses the standard json library, which doesn't easily handle datetime objects.

Would it be possible to add a new argument to get_whois allowing for ISO 8601 output? My naive test was basically replacing parse.py:413 and 417 with:

parsed_dates.append(datetime.datetime(year, month, day, hour, minute, second).isoformat()

Is that something you think would be possible / advisable? It would always be possible to convert any dates externally, but it would certainly be very convenient.

no whois server for .CD

Hi,

pwhois gives the below error when looking up .cd domains:

"No root WHOIS server found for domain."

looking at the IANA webpage for .cd, there isn't a whois server listed:

http://www.iana.org/domains/root/db/cd.html

but querying manually whois.nic.cd works:

whois domain.cd -h whois.nic.cd

no real information contained in the whois results though..

KR

incomplete whois data for .cz

Incomplete data for .cz domains.

example domains:
dmail.cz
jakubarnold.cz

example output:

pwhois dmail.cz
Registrar : Reg-active24
Registration date : 2007-10-31 00:00:00
Expiration date : 2015-10-31 00:00:00
Last update : 2008-03-12 00:00:00
Name server : ns2.register.it
Name server : ns1.register.it
E-mail address : [email protected]
E-mail address : [email protected]

Registrant
NIC handle : DMAIL

Technical Contact
NIC handle : SB:ACTIVE24-SREG030420

raw output:

pwhois dmail.cz --raw
-- snip --
domain: dmail.cz
registrant: DMAIL
nsset: NSS:ACTIVE24-SREG030420:3
registrar: REG-ACTIVE24
registered: 31.10.2007 08:49:56
changed: 12.03.2008 15:15:46
expire: 31.10.2015

contact: DMAIL
org: D-Mail Srl
name: David Mozzetti
address: Via Aretina 25
address: Pontassieve
address: 50069
address: FI
address: IT
phone: [phone]
fax-no: +390.558363057
e-mail: [email protected]
registrar: REG-ACTIVE24
created: 11.03.2008 14:45:02

nsset: NSS:ACTIVE24-SREG030420:3
nserver: ns2.register.it
nserver: ns1.register.it
tech-c: SB:ACTIVE24-SREG030420
registrar: REG-ACTIVE24
created: 01.10.2007 02:00:00
changed: 08.03.2011 15:26:00

contact: SB:ACTIVE24-SREG030420
org: Register.it spa
name: Register.it spa
address: Via Zanchi 22
address: Bergamo
address: 24126
address: IT
e-mail: [email protected]
registrar: REG-ACTIVE24
created: 14.02.2005 15:05:00
changed: 08.03.2011 15:22:48

another domain causing regex issue

pythonwhois 2.4.3, Python 2.7.5 on Mac OS X
get_whois('kawaihayao.jp') loads the CPU and runs forever; after Ctrl-C here is the traceback:

>>> import pythonwhois
>>> pythonwhois.get_whois('kawaihayao.jp')
.....^C
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/lib/python2.7/site-packages/pythonwhois/__init__.py", line 8, in get_whois
    return parse.parse_raw_whois(raw_data, normalized=normalized, never_query_handles=False, handle_server=server_list[-1])
  File "/lib/python2.7/site-packages/pythonwhois/parse.py", line 558, in parse_raw_whois
    data["contacts"] = parse_registrants(raw_data, never_query_handles, handle_server)
  File "/lib/python2.7/site-packages/pythonwhois/parse.py", line 848, in parse_registrants
    match = re.search(regex, segment)
  File "/lib/python2.7/re.py", line 148, in search
    return _compile(pattern, flags).search(string)
KeyboardInterrupt
>>>

ValueError: month must be in 1..12

>>> pythonwhois.get_whois("ALBARELLI.NET")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python2.7/dist-packages/pythonwhois/__init__.py", line 8, in get_whois
    return parse.parse_raw_whois(raw_data, normalized=normalized, never_query_handles=False, handle_server=server_list[-1])
  File "/usr/local/lib/python2.7/dist-packages/pythonwhois/parse.py", line 563, in parse_raw_whois
    data['expiration_date'] = parse_dates(data['expiration_date'])
  File "/usr/local/lib/python2.7/dist-packages/pythonwhois/parse.py", line 812, in parse_dates
    parsed_dates.append(datetime.datetime(year, day, month, hour, minute, second))
ValueError: month must be in 1..12

appeared on: ALBARELLI.NET
normal whois gives the following data:

Domain name: ALBARELLI.NET
Registrar: Premium Registrations
[...]
Record created: 2013-06-26
Record expires: 0000-00-00
Name Server: p1.internetbolaget.se
Name Server: p2.internetbolaget.se

however this only appears on some whois server, others have updated information where the expire data is in a correct format and valid date

it would be nice if pythonwhois would not fail with an exception, but rather silently just ignore this wrong date

Nominet parsing

Hi,

When running against .co.uk domains this seems to be excluding contact information. (when searching 'google.co.uk' address information is included in raw output but not parsed by the code). I've also noticed it seems to be missing the "Registration status:" field too.

Thanks

"pwhois wa.us" hangs forever

"pwhois wa.us" hangs forever
same result with the api.

Feature Request: XML output

Hi
Would it be possible to get an XML output of the query run, in the same format as the python-whois dict?

Cheers
Tom

Add flag field for non-existent domains

Suggestion: it would be useful to add rules identifying if a domain exists / is registered. Like everything, this is non-standard between TLDs:

.com.au: 'No Data Found\n'
.com: '...No match for "NOSUCHDOMAINEXAMPLE.COM"
.io: 'Domain nosuchdomainexample.io is available for purchase'

If we had a set of regexes to detect this state, and set a boolean "domain_registered" field, it would be most useful.

.je Registrant Name not parsed

Registrant Name field of .je whois isn't parsed.

whois jerseylovesfood.je contains:
Registrant Name: Guernsey Press Company Ltd

pwhois jerseylovesfood.je misses this out.

example.com - No root WHOIS server found for TLD

v2.1.4

>>> pythonwhois.get_whois('example.com')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "pythonwhois/__init__.py", line 4, in get_whois
    raw_data = net.get_whois_raw(domain)
  File "pythonwhois/net.py", line 24, in get_whois_raw
    target_server = get_root_server(domain)
  File "pythonwhois/net.py", line 60, in get_root_server
    raise shared.WhoisException("No root WHOIS server found for TLD.")
pythonwhois.shared.WhoisException: No root WHOIS server found for TLD.

Strange performance characteristics

After implementing a much-needed optimization (pre-compiling regexes), I found that not only does pythonwhois now parse much faster in Python 2.x, it is now also significantly faster than in Python 3.x. This should not be the case. Timing results below:

Prior to optimization

2.6
Timing in default mode: 333ms avg, 152ms min, 476ms max
Timing in normalized mode: 336ms avg, 130ms min, 482ms max

2.7
Timing in default mode: 283ms avg, 101ms min, 444ms max
Timing in normalized mode: 287ms avg, 109ms min, 453ms max

3.3
Timing in default mode: 78ms avg, 4ms min, 180ms max
Timing in normalized mode: 80ms avg, 4ms min, 338ms max

After optimization

2.6
Timing in default mode: 30ms avg, 1ms min, 135ms max
Timing in normalized mode: 30ms avg, 1ms min, 132ms max

2.7
Timing in default mode: 18ms avg, 0ms min, 82ms max
Timing in normalized mode: 18ms avg, 0ms min, 84ms max

3.3
Timing in default mode: 79ms avg, 3ms min, 178ms max
Timing in normalized mode: 79ms avg, 4ms min, 178ms max

several domains not parsable

The parser cannot parse the following whois domains:

• .co.th like starbucks.co.th
• .ir like nic.ir
• .com like servequake.com
• .com.mx like expopack.com.mx (already referred to issue #22 )

Edit: I'll add non-working tlds with example domains to this post.