thescriptguy/certificatechecker

certificateChecker Python Script

certificatechecker's Issues

Backwards compatibility for older websites

It seems there are still issues with some websites being stuck behind the times (i.e. not keeping their website up to date with latest security trends).

Creating a argument to allow reduced security protocol checking.

mongoDB upload error

Running version 0.31.

Attempting to upload data to MongoDB.

Get python error:
Traceback (most recent call last):
File "/projects/github/certificateChecker/certCheck.py", line 350, in
processQueryFile()
File "/projects/github/certificateChecker/certCheck.py", line 267, in processQueryFile
uploadResult = sdMDB.uploadDataToMongoDB(myJsonScriptData)
File "/projects/github/certificateChecker/data/sendDataMongoDB.py", line 158, in uploadDataToMongoDB
uploadResult = self.sendResults(__jsonScriptData, collection)
File "/projects/github/certificateChecker/data/sendDataMongoDB.py", line 39, in sendResults
__uploadResult = __destCollection.insert_one(__results)
File "/usr/lib/python3/dist-packages/pymongo/collection.py", line 698, in insert_one
self._insert(document,
File "/usr/lib/python3/dist-packages/pymongo/collection.py", line 613, in _insert
return self._insert_one(
File "/usr/lib/python3/dist-packages/pymongo/collection.py", line 602, in _insert_one
self.__database.client._retryable_write(
File "/usr/lib/python3/dist-packages/pymongo/mongo_client.py", line 1498, in _retryable_write
return self._retry_with_session(retryable, func, s, None)
File "/usr/lib/python3/dist-packages/pymongo/mongo_client.py", line 1384, in _retry_with_session
return self._retry_internal(retryable, func, session, bulk)
File "/usr/lib/python3/dist-packages/pymongo/mongo_client.py", line 1416, in _retry_internal
return func(session, sock_info, retryable)
File "/usr/lib/python3/dist-packages/pymongo/collection.py", line 590, in _insert_command
result = sock_info.command(
File "/usr/lib/python3/dist-packages/pymongo/pool.py", line 699, in command
self._raise_connection_failure(error)
File "/usr/lib/python3/dist-packages/pymongo/pool.py", line 683, in command
return command(self, dbname, spec, slave_ok,
File "/usr/lib/python3/dist-packages/pymongo/network.py", line 120, in command
request_id, msg, size, max_doc_size = message._op_msg(
File "/usr/lib/python3/dist-packages/pymongo/message.py", line 714, in _op_msg
return _op_msg_uncompressed(
bson.errors.InvalidDocument: documents must have only string keys, key was 128

issue with data structure and sending resulting data to Mongo database

Getting KeyError value as a result of json format structure change in version 0.30

Traceback when allowing untrusted issuer

When querying a hostname in a file that has the optional arguments ["local_untrusted_allow"], the following error message appears.

Traceback (most recent call last):
File "/projects/github/certificateChecker/certCheck.py", line 401, in
processQueryFile()
File "/projects/github/certificateChecker/certCheck.py", line 282, in processQueryFile
jsonCertificateInfo = o_myCertificate.convertCertificateObject2Json(myHostname["hostname"], myHostname["port"], o_startTime, o_endTime, myCertificate)
File "/projects/github/certificateChecker/certificate/certificateModule.py", line 357, in convertCertificateObject2Json
myJsonCertificateInfo["certificateInfo"]["certificateIssuer"] = dict(x[0] for x in __certificateObject["certificateMetaData"]["issuer"])
KeyError: 'issuer'

Python traceback when MongoDB experiences operation failure

When the MongoDB has reached capacity, the operation failure exception does not catch due to the datetime.datetime field within the exception (wrong import for datetime)

`Traceback (most recent call last):
File "/certificateChecker/data/sendDataMongoDB.py", line 107, in sendResults
__mongoResult = __destCollection.insert_one(__results)
File "/opt/venv/lib/python3.10/site-packages/pymongo/collection.py", line 628, in insert_one
self._insert_one(
File "/opt/venv/lib/python3.10/site-packages/pymongo/collection.py", line 569, in _insert_one
self.__database.client._retryable_write(acknowledged, _insert_command, session)
File "/opt/venv/lib/python3.10/site-packages/pymongo/mongo_client.py", line 1476, in _retryable_write
return self._retry_with_session(retryable, func, s, None)
File "/opt/venv/lib/python3.10/site-packages/pymongo/mongo_client.py", line 1349, in _retry_with_session
return self._retry_internal(retryable, func, session, bulk)
File "/opt/venv/lib/python3.10/site-packages/pymongo/_csot.py", line 105, in csot_wrapper
return func(self, *args, **kwargs)
File "/opt/venv/lib/python3.10/site-packages/pymongo/mongo_client.py", line 1390, in _retry_internal
return func(session, sock_info, retryable)
File "/opt/venv/lib/python3.10/site-packages/pymongo/collection.py", line 557, in _insert_command
result = sock_info.command(
File "/opt/venv/lib/python3.10/site-packages/pymongo/pool.py", line 767, in command
return command(
File "/opt/venv/lib/python3.10/site-packages/pymongo/network.py", line 166, in command
helpers._check_command_response(
File "/opt/venv/lib/python3.10/site-packages/pymongo/helpers.py", line 181, in _check_command_response
raise OperationFailure(errmsg, code, response, max_wire_version)
pymongo.errors.OperationFailure: you are over your space quota, using 512 MB of 512 MB, full error: {'ok': 0, 'errmsg': 'you are over your space quota, using 512 MB of 512 MB', 'code': 8000, 'codeName': 'AtlasError'}

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/certificateChecker/certCheck.py", line 402, in
processQueryFile()
File "/certificateChecker/certCheck.py", line 304, in processQueryFile
uploadResult = sdMDB.uploadDataToMongoDB(myJsonScriptData)
File "/certificateChecker/data/sendDataMongoDB.py", line 236, in uploadDataToMongoDB
uploadResult = self.sendResults(__jsonScriptData, collection)
File "/certificateChecker/data/sendDataMongoDB.py", line 120, in sendResults
errTime = str(datetime.datetime.utcnow())
AttributeError: type object 'datetime.datetime' has no attribute 'datetime'
`

Environment variables

Create an argument --environmentVariables that allows the certCheck.py script to retrieve the Tenant ID and tag.

Environment variables that certCheck.py script should look for:

• TENANT_ID for the Tenant ID
• TAG for the tag

Add some error handling that makes the script exit if the --environmentVariables argument is passed but the environment variables don't exist, exit.

This feature request is to facilitate a more efficient way of executing the script when run in a Docker container.

Store data locally if Mongo connection doesn't work

If Mongo connection can't be established to upload the results of the certificate polling, then store data locally.
Next time script runs:

1. Poll sites for certificate data
2. Check to see if a local data file exists.
3. If local data exists (essentially from previous failed uploads), write the results of the most recent polling into that file.
4. Attempt to upload the locally stored data to Mongo DB.
5. If successful, then delete locally stored data.

Add timestamp to error message for troubleshooting purposes

If the script is unable to upload data to a MongoDB server, the error message that's generated does not include a timestamp for that particular time the script is executed.

Update the output errors to include the timestamp for better troubleshooting.

Current example:

2023-04-15 16:40:38.600808 - [<pymongo.results.InsertOneResult object at 0x7f1209c5d840>]
2023-04-15 17:40:40.212915 - [<pymongo.results.InsertOneResult object at 0x7f0ebabe9840>]
Server connection timeout error when uploading data.
Saving to certificateData.json

Expected result:

2023-04-15 16:40:38.600808 - [<pymongo.results.InsertOneResult object at 0x7f1209c5d840>]
2023-04-15 17:40:40.212915 - [<pymongo.results.InsertOneResult object at 0x7f0ebabe9840>]
2023-04-15 18:40:40.123413 - Server connection timeout error when uploading data. Saving to certificateData.json

Timeout errors do not indicate the time

If there's an error accessing a hostname, the error message does not include the timestamp when it occurred.

For example:

2023-10-13 00:16:23.772568 - [<pymongo.results.InsertOneResult object at 0x7f58597d3be0>]
www.website.com:443, options: None - Timeout error -  None
www.website.com:443, options: None - Timeout error -  None
www.website.com:443, options: None - Timeout error -  None
2023-10-13 01:16:52.076241 - [<pymongo.results.InsertOneResult object at 0x7fe7725abbb0>]

The request is to have the timeout error (or any other error) include the timestamp as well.

Add socket timeout values

It seems that by default that socket connections will wait indefinitely before it is terminated.

This can inadvertently cause the script to hang when there's a problem with the socket.

As a first version of this feature request, set the default timeout value to 10 seconds.