Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make WHOIS lookup fail gracefully #231

Open
WilliamDEdwards opened this issue Sep 28, 2024 · 5 comments
Open

Make WHOIS lookup fail gracefully #231

WilliamDEdwards opened this issue Sep 28, 2024 · 5 comments

Comments

@WilliamDEdwards
Copy link

WilliamDEdwards commented Sep 28, 2024

When --whois is enabled, and one lookup fails, dnstwist dies with an exception such as:

Traceback (most recent call last):
  File "/usr/local/bin/dnstwist", line 8, in <module>
    sys.exit(run())
             ^^^^^
  File "/usr/local/lib/python3.11/dist-packages/dnstwist.py", line 1182, in wrapper
    result = func(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/dnstwist.py", line 1540, in run
    wreply = whois.whois('.'.join(domain_tld(domain['domain'])[1:]))
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/dnstwist.py", line 258, in whois
    return self._extract(self.query(domain, server))
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/dnstwist.py", line 253, in query
    return self.query(query, refer.group('server'))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/dnstwist.py", line 237, in query
    sock.connect((server, 43))
socket.gaierror: [Errno -2] Name or service not known

As you can see, the WHOIS server 'refers' to another. In this case, it's whois.iana.org, and I don't know why it fails. But that's irrelevant to this issue; I believe such a situation should be handled gracefully, by skipping the domain silently and logging this to the debug log. After all, not all registries provide reliable WHOIS servers.

@WilliamDEdwards WilliamDEdwards changed the title Make DNS lookup fail gracefully Make WHOIS lookup fail gracefully Sep 28, 2024
@WilliamDEdwards
Copy link
Author

Upon further inspection, (at least) .ph apparently returns an empty 'refer':

refer:

For comparison, for .com:

refer:        whois.verisign-grs.com

This -obviously- confuses dnstwist.

@elceef
Copy link
Owner

elceef commented Sep 28, 2024

Could you please provide domain name and DNS resolver (if publicly available) so I can reproduce this issue?

@elceef
Copy link
Owner

elceef commented Sep 28, 2024

Meanwhile I pushed some changes which hopefully should resolve this.

@WilliamDEdwards
Copy link
Author

Could you please provide domain name and DNS resolver (if publicly available) so I can reproduce this issue?

Did you see my follow-up comment (#231 (comment))?

@elceef
Copy link
Owner

elceef commented Sep 29, 2024

Empty refer fields are skipped. Initially domain name for IANA server is resolved which seems to be the cause in your network. Commit 464bb42 should resolve it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants