hrun/sa-haveibeenpwned

Splunk add-on providing a custom search command to query Troy Hunt's haveibeenpwned API (https://haveibeenpwned.com/api/v3/) for known breaches of your domains or mail addresses.

sa-haveibeenpwned's Issues

Proxy connection Error

Hi,
we noticed that inside the python code (haveibeenpwned.py) when the connection is made through the proxy, the call is not executed because it lacks the call to the function r.read() that completes the call of the http_client and allows to execute a new request.

Here is the code:
Line 156

150    if proxy_type == "http":
151              try:
152                        connection = http_client.HTTPSConnection('{0}'.format(proxy_url.split('//')[-1].rstrip('/')), port=proxy_port)
153                        connection.set_tunnel('haveibeenpwned.com', port=443, headers=auth_headers)
154                        connection.request('HEAD', '/api/v3', headers=headers)
155                        r = connection.getresponse()
156                        r.read()

This is one of the solutions we have set up.

Best regards,

Roberto and team

Partial results using haveibeenpwned search command

Hi hRun,

I deployed the add on in my splunk environment but I'm having some issue getting it to work as expected. When using the search command while passing email addresses as results I only get ouput on a few of the results. This only applies when using "mode=mail". When using "mode=domain" I receive breach results for all of the results. Reviewing the search log I see that when using mode=domain the input and output values match and seem to not have much of a limit, I've been able to get results on 1,000 unique email addresses (though I know this applies to the domain rather than the actual email address). Running the same exact search and only changing "mode=mail" I see the inputs is a fraction of the actual results and the outputs seems to max out at 3.

I'm hoping you can tell me if this is by design or maybe if I'm just using the add-on incorrectly. The idea was to run this command to check a large number of results for breach information on a large number of email addresses at one time. The search syntax I'm using is similar to this:

index= sourcetype= FieldName=
| table EventDateTime, OldValue, NewValue
| dedup OldValue
| rename NewValue as email
| table email
| haveibeenpwned mode=mail threshold=90 email

I'm attaching a screenshot of what the job inspection looks like on an unsuccessful run. The number of events or emails that this should have returned values for is 1,181

Thanks in advance for any help provided!

Extend App with additional "modes" (api endpoints)

Hi Harun,

it would be great if you could modify the app to include additional endpoints.
I would be intrested in this one:
GET https://haveibeenpwned.com/api/v3/breacheddomain/{domain}
I understand, that it requires an Api key which might make it less valuable/interesting for a lot of users. But the https://haveibeenpwned.com/api/v3/breaches endpoint does not return any result if a breach was not against your company, but some company empoyees had accounts with their email in the breach.

Regards
Chris

Query result not found

Hi,
I am writing you because sometimes it happens that some email addresses are not analyzed and therefore the query in splunk does return errors.

Let me explain:
when I try to check an email address [email protected] the query returns the result with all Breach and pastes.
Sometimes, instead, with causal addresses like [email protected] nothing comes back.

I also see within the curl that nothing return in the event, there is an error: page not found 404 at the time of response of the site haveibeenpwned.

I can not understand why this occurs only for n mail address and not for all.

HTTPS request failed: Request-sent

Hello!

I am getting this error when running a query with SA-haveibeenpwned

RuntimeWarning at "/opt/splunk/etc/apps/SA-haveibeenpwned/bin/haveibeenpwned.py", line 194 : HTTPS request failed: Request-sent

Splunk 9.0.0.1
Python 3.7.11

I have confirmed the following

• Splunk server can reach https://haveibeenpwned.com/API/v3 through the proxy
• API key does work
• Tested this SA-haveibeenpwned on a virtual server and no issues

I did some research to see if I can fix this but no luck. I have some thoughts thinking it has be an SSL error but I am unsure.

Here are sources I used from Splunk base but I cant seem to fix the issue.

SSL error while trying to connect to splunk web from python in CentOS-7 - https://community.splunk.com/t5/Security/SSL-error-while-trying-to-connect-to-splunk-web-from-python-in/m-p/295077

SSL Certificate issue - https://community.splunk.com/t5/Splunk-Enterprise/SSL-Certificate-issue/m-p/577055

How to get Splunk to run my Python shell script? - https://community.splunk.com/t5/All-Apps-and-Add-ons/How-to-get-Splunk-to-run-my-Python-shell-script/m-p/223138

Also I can curl haveibeenpwned.com from the server CLI with no issues but if I curl with ./splunk cmd I am not getting out to https://haveibeenpwned.com

Curl with ./splunk cmd

Curl -vk with ./splunk cmd (insecure)

Not working for Splunk version 8.2

The configuration page was unable to load and errors found in log
REST Error [500]: Internal Server Error -- Traceback (most recent call last):\n File "/opt/splunk/etc/apps/SA-haveibeenpwned/bin/sa_haveibeenpwned/aob_py3/splunktaucclib/rest_handler/handler.py", line 117,

Error while executing search in Splunk ES

Hi!

Please help me with a problem I have faced.

Splunk ES version: v8.2.4

An error occurred while performing a search query:

<skipped>
| table email
| haveibeenpwned mode=mail pastes=all email

The error message in interface:

Full error message from index _internal:

source = /opt/splunk/search/splunk/var/log/python_upgrade_readiness_app/pura_utils.log
sourcetype = python_upgrade_readiness_app

2022-02-22 14:49:43,890 ERROR 140591149778752 - [Errno 111] Connection refused
Traceback (most recent call last):
  File "/opt/splunk/search/splunk/etc/apps/python_upgrade_readiness_app/bin/libs_py3/pura_libs_utils/pura_utils.py", line 760, in one_shot_str_wrapper
    oneshot_job = service.jobs.oneshot(path)
  File "/opt/splunk/search/splunk/etc/apps/python_upgrade_readiness_app/bin/libs_py3/pura_libs_utils/splunklib/client.py", line 3054, in oneshot
    **params).body
  File "/opt/splunk/search/splunk/etc/apps/python_upgrade_readiness_app/bin/libs_py3/pura_libs_utils/splunklib/client.py", line 821, in post
    return self.service.post(path, owner=owner, app=app, sharing=sharing, **query)
  File "/opt/splunk/search/splunk/etc/apps/python_upgrade_readiness_app/bin/libs_py3/pura_libs_utils/splunklib/binding.py", line 290, in wrapper
    return request_fun(self, *args, **kwargs)
  File "/opt/splunk/search/splunk/etc/apps/python_upgrade_readiness_app/bin/libs_py3/pura_libs_utils/splunklib/binding.py", line 71, in new_f
    val = f(*args, **kwargs)
  File "/opt/splunk/search/splunk/etc/apps/python_upgrade_readiness_app/bin/libs_py3/pura_libs_utils/splunklib/binding.py", line 759, in post
    response = self.http.post(path, all_headers, **query)
  File "/opt/splunk/search/splunk/etc/apps/python_upgrade_readiness_app/bin/libs_py3/pura_libs_utils/splunklib/binding.py", line 1235, in post
    return self.request(url, message)
  File "/opt/splunk/search/splunk/etc/apps/python_upgrade_readiness_app/bin/libs_py3/pura_libs_utils/splunklib/binding.py", line 1252, in request
    response = self.handler(url, message, **kwargs)
  File "/opt/splunk/search/splunk/etc/apps/python_upgrade_readiness_app/bin/libs_py3/pura_libs_utils/splunklib/binding.py", line 1392, in request
    connection.request(method, path, body, head)
  File "/opt/splunk/search/splunk/lib/python3.7/http/client.py", line 1281, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/opt/splunk/search/splunk/lib/python3.7/http/client.py", line 1327, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/opt/splunk/search/splunk/lib/python3.7/http/client.py", line 1276, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/opt/splunk/search/splunk/lib/python3.7/http/client.py", line 1036, in _send_output
    self.send(msg)
  File "/opt/splunk/search/splunk/lib/python3.7/http/client.py", line 976, in send
    self.connect()
  File "/opt/splunk/search/splunk/lib/python3.7/http/client.py", line 1443, in connect
    super().connect()
  File "/opt/splunk/search/splunk/lib/python3.7/http/client.py", line 948, in connect
    (self.host,self.port), self.timeout, self.source_address)
  File "/opt/splunk/search/splunk/lib/python3.7/socket.py", line 729, in create_connection
    raise err
  File "/opt/splunk/search/splunk/lib/python3.7/socket.py", line 717, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused

Pates issues - not displayed

Hello,
I’m having problems with the haveibeenpwned command.
When I run the command on Splunk

| haveibeenpwned mode=mail

This is what I expected from the search:

_time	emai	breach	paste
2022-01-01 00:00:00	[email protected]	No breach reported for given account and time frame.	No paste reported for given account and time frame.

But sometimes the column is not displayed.

Has anyone had this problem before?

--- Thanks --

Setup Question

Setup notes say to install on the SH- I assume for the knowledge objects. No configuration or API calls being done here - correct?

Since this is making API calls - shouldn't the configuration be done on a HF?
The notes don't reference this so I wanted to confirm.