texadactyl/diyca

Do-It-Yourself Certificate Authority

Overview

*** Python 3 only ***

(Python 2 was deprecated in 2020)

*** It is very important to follow docs/preparation_notes.txt precisely. ***

This git project constitutes a Do-It-Yourself Certificate Authority (diyca), suitable for unit testing (developer testing) just prior to entering integration system testing with a wider audience: more developers, end-users, and/or auditors. A web server approach is used in obtaining an X.509 certificate signed by this unit testing CA, based on a Certificate Signing Request (CSR) provided by the user, employing a web browser (Firefox, Safari, etc.).

The inspiration of this project is the myriad of Internet of Things (IoT) projects that might be falling into one of the following categories of undesirable patterns:

• No data security at all: (1) no authentication of the endpoints, (2) no message integrity checking, and (3) data is transmitted over the network in cleartext.
• Partners are using weak cryptography (E.g. RC4 or Single-DES) and there is no secure methodology of installing or managing the secret keys. E.g. coding the secret keys as program constants.
• Partners are using strong secret key cryptography but there is no secure methodology of installing or managing the secret keys. E.g. The secret keys are never changed.

Even when developers agree to securely use public key infrastructure and strong cryptography, I have seen cases where they stub this aspect out of their project during unit testing. This just puts off the inevitable and may cause project delays when the stubs are later replaced with operational code. Better to design and develop a project from the very beginning as it is intended to be in production.

Therefore, the primary goal of this project is to allow developers to unit test programs which make use of X.509 certificate based authentication and cryptography. Thus, when it is time to migrate to more stringent testing environments (E.g. integrated system testing and user acceptance testing), the only thing new will be the target environment details since the developers will have gained experience with a Certificate Authority operations, X.509 certificates, and managing the user's private key.

This project is already proven running the web server on various Intel/AMD hardware and Raspberry Pi 2, 3, and 4.

Licensing

This is NOT commercial software; instead, usage is covered by the GNU General Public License version 3 (2007). In a nutshell, please feel free to use the project and share it as you will but please don't sell it. Thanks!

See the LICENSE file for the GNU licensing information.

Getting Started

Subfolders:

• app_web - Python 3 source code for the web server running in a Flask infrastructure (see docs/preparation_notes.txt for references to all of the supporting software as well as how to install, test, etc.)
• bin - Bash scripts for setting up diyca and other tools
• calvin - Self-signed Certificate Authority
• certs - Calvin's certificate and the web server's certificate
• docs - project documentation (admittedly, skimpy at the moment)
• example.users - example SSL programs that use certificates signed by Calvin, "alice" and "bob"
• log - Holds all of the log files which cutoff at midnight; aged to keep a maximum of 10 files
• signer - uploaded CSRs (temporarily) and downloaded CRTs (accumulating, for the moment)

The starting point with this project is the docs/preparation_notes.txt file. Follow the instructions therein precisely with possible exceptions related to Raspbian set up changes or bugs that you found (please open an issue).

External Python Package Dependencies

dnspython
flask
pyOpenSSL
werkzeug

Feel free to open an issue for bugs and enhancement requests. I'll respond as soon as I can.

Richard Elkins

Dallas, Texas, USA, 3rd Rock, Sol, ...

diyca's Issues

Duplicate Serial numbers in the certificates

I signed multiple certificates using the CA. All had the serial number of 42. While that is the meaning of life, firefox and chrome detect the same serial number on multiple certificates and error. I used your CA to sign my certificates I generated for the cockpit web based remote administration system.

Relevant section of code from app_web/diyca_web_signer.py
try:
cert = crypto.X509()
cert.set_serial_number(42) <-------------------------------
cert.gmtime_adj_notBefore(0)
cert.gmtime_adj_notAfter(EXPIRY_PERIOD)
cert.set_issuer(CAcertificate.get_subject())
subject = csr.get_subject() # will log the subject later
cert.set_subject(subject)
cert.set_pubkey(csr.get_pubkey())
cert.sign(CAprivatekey, DIGEST)
except Exception as e:
app.logger.error("sign_csr: Cannot sign CSR {%s} for user {%s}, reason: {%s}", arg_csr_path, arg_userid, repr(e))
return False
# Store signed CRT

This should be imported from the file /opt/diyca/calvin/db/serial and then incremented or left to the openssl CA function. The results in the certificates are:
Certificate:
Data:
Version: 1 (0x0)
Serial Number: 42 (0x2a)
Signature Algorithm: sha256WithRSAEncryption
Issuer: C = AU, O = SqXXXs Org, OU = SqXXXs Admin, CN = sXXXXs.id.au
Validity
Not Before: Mar 27 12:42:38 2019 GMT
Not After : Mar 26 12:42:38 2021 GMT
Subject: C = AU, ST = Queensland, L = Samford Valley, O = sXXXs, CN = cosmos.sqXXXs.id.au, emailAddress = nXXXl@XXXXXXXXX
Subject Public Key Info:
Public Key Algorithm: rsaEncryption
RSA Public-Key: (2048 bit)
Modulus:
Certificate:
Data:
Version: 1 (0x0)
Serial Number: 42 (0x2a)
Signature Algorithm: sha256WithRSAEncryption
Issuer: C = AU, O = SqXXXs Org, OU = SqXXXs Admin, CN = sqXXXs.id.au
Validity
Not Before: Mar 27 09:46:15 2019 GMT
Not After : Mar 26 09:46:15 2021 GMT
Subject: C = au, ST = Queensland, L = Samford Valley, O = sXXXs, CN = certis.sqXXXs.id.au, emailAddress = [email protected]
Subject Public Key Info:
Public Key Algorithm: rsaEncryption
RSA Public-Key: (2048 bit)
Modulus:
Based on the config for the openssl
[ca_default]
home = .
database = $home/db/index
serial = $home/db/serial
crlnumber = $home/db/crlnumber

Version 64 of Chrome detects ERR_SSL_SERVER_CERT_BAD_FORMAT

There is a certificate consistency issue when using the version 64 of Chrome. The gross symptom appears as follows:

This site can’t provide a secure connection
doesn't adhere to security standards.
ERR_SSL_SERVER_CERT_BAD_FORMAT

The web server certificate version generated by diyca's use of openssl = 1 but the certificate contain version 3 extensions; this has been reported by other software engineers on the Internet. Firefox, MSIE, Safari, and the older versions of Chrome are forgiving; the newer versions of Chrome are not.

This requires some adjustments in the diyca project that should have been there in the beginning.

It would have been helpful if Chrome had said this was a version conflict explicitly instead of making me guess/google! )=: All part of the fun.

X509v3 Subject Alternative Names

Richard thank you for publishing your cert signer, it has helped me learn quite a bit about CAs.

I wanted to ask if it supports specifying an Alternative Name via IP. I have successfully loaded the cert and it shows as valid when connecting to the device via hostname i.e. https://hostname.local, but if I access via the specified IP the certificate is shown to be invalid.

New Variation of Defect #24 ("Enhanced" Browser Security), now with Firefox 105.0.3

Describe the bug
With sincere apologies, I am afraid that I have to report what looks to be the re-emergence of defect #24, this time with Firefox 105.0.3.

As reported for my testing of the fix for defect #24, at that time I created a new certificate for my domestic WordPress environment. That functioned perfectly. Today I went back and created an all-new certificate, using the brand new DIYCA installation, for my NextCloud environment.

To my surprise, I saw that I was getting the same Bad_Cert_Domain error that I had previously reported.

My default assumption was, "idiot user"... but when I checked my access to my local family "WordPress" site, which had previously been tested and which had worked perfectly with the latest edition of DIYCA, I see that the same error has returned.

However, one thing which has changed is my browser, since Mint Update gave me Firefox 105.0.3 - replacing 105.0.2 - first thing this morning. Given the observed failure modes - previously working site now not working immediately after a Firefox update - my inclination is to suggest that this is a variation of the issue we observed last time [i.e. Firefox raising the security bar for certificates]. But I accept that my evidence is a bit thin on the ground.

As before, very happy to provide support, more evidence, copies of pem/key/csr/crt files, should that help diagnosis.

Very sorry to put you to the trouble.

Need to update cert generations given enhanced browser security (Good job and thank you, Firefox and Google!)

Describe the bug
I am not sure if this is a bug or an idiot user.

Symptoms: Having created and deployed a DIYCA-signed certificate to a local web server (Apache2 running on an RPi4B, to host WordPress and NextCloud for family use) I see an error from Firefox (104.0.2, Linux, 64-bit) when I first access any DIYCA-protected web site. The "error" is a FireFox message, "Warning: Potential Security Risk Ahead", with Error code: "SSL_ERROR_BAD_CERT_DOMAIN".

This is a relatively recent change in behaviour - when I first set up this site, I did not observe this error. Research leads me to believe this might be related to changes in Firefox, but I think that Firefox may be generating an error based on discrepancies it thinks it sees in DIYCA-generated certificates. But I'm not sure of that. See attached "ErrorGenerationSteps.txt" for more details.

Please see attached file, "000 - On-Screen Error as seen in browser.png" for actual render of message...

If I then choose to view the certificate, this is what I see:-

I am able to bypass/ignore this "bug" simply by selecting "Accept the Risk and Continue", but would obviously like to eliminate that if I can...

I suspect/wonder (but do not know) if the issue might be related to DIYCA's root CA?

To Reproduce
Steps to reproduce the behavior:

I want to be detailed in terms of the steps I took to reproduce this error, including details regarding the generation of the CSR using the OpenSSL utility and the way I have this deployed with Apache2, so I put more detailed notes in this TXT file:-
ErrorGenerationSteps.txt

Expected behavior
Ideally, I would like to be able to navigate to a URL protected by a DIYCA server and have the browser accept the certificate without error and take me to the site directly.

Screenshots
One more screen shot for you... here's a view of my Firefox "Certificate Store", showing the DIYCA Root Certificate successfully imported:-

Desktop (please complete the following information):

• OS: Mint Linux 20.1 Ulyssa, Kernel 5.15-0-46-generic-x86_64
• Browser: Firefox
• Version: 104.0.2

Additional context
I have changed my local network to Class-B Address 172.16.. to allow me to use address ranges to identify devices.
I have deployed Pi-Hole DNS Sink to provide local DNS Services and add/manage my own local DNS records for access to local resources (e.g. the certificate in question is "nc2.saurian.net" for NextCloud Server 2...) Don't think this should be impactful, but you never know...

Obviously entirely happy to provide more details if required.

Thank you.

Show software version in web server log

Also, revise the diyca.version file for reading into the launched web server.

In the example.users, "certs" --> "csrs"

Having "certs" at 2 levels is confusing. Also, the folder only holds CSR files.

diyca does not have proper support for SubjectAlternateNames (SAN)

Is your feature request related to a problem? Please describe.
I cannot create an SSL cert via that can be used in chrome as it mandates the inclusion of SubjectAlternateName field.
I have tried to pass in a CSR that includes the SAN fields, but the resulting cert does not contain reference to the SubjectAlternateName in the request

Describe the solution you'd like
diyca should produce a cert that contains the SubjectAlternateName if it is in the request.

Describe alternatives you've considered
Safari still accepts the generated cert without the SubjectAlternateName but most of our resting is done with Chrome.

Additional context
I'm using a Mac here.

Request: Check for SAN extension and add to crt

I am using diyca to quickly sign CSR's for internal testing of web apps and servers. I know that this is not the intended use of this CA but it works really well for me.
The only issue I have is that I have to use the X509Extension "subjectAltName" which is included in the CSR.
Is there a way that the diyca_web_signer.py could be modified to check the CSR for such extentions and include them in the output CRT?

I have tried to add this myself but unfortunately I have failed so far. For OpenSSL I can add extensions using the "-extfile" and "-extensions" -flags, but I don't know how to translate this to python.

New openssl requires non-nil sections in certificate config files

Remove section ca_req and all references to it (1) in file
bin/diyca_calvin_cert.cfg

"No such file" when attempting "2.diyca_calvin_init.sh"

I fear this is almost certainly something I've done/am doing wrong, but I get a warning when running the "1.diyca_cleanup.sh" file, and a failure when running "2.diyca_calvin_init.sh"...

HW: Raspberry Pi 4 / 4Gb RAM
SW: clean (brand new) Raspbian Jessie build with "apt-get update", "apt-get upgrade" and "apt-get dist-upgrade" (latter invoked only after observing held back packages)...
Python Version: 2.7.16

Actual observed failure copied from command line, below:-
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
root@raspberrypi:/opt/diyca/bin# /opt/diyca/bin/1.diyca_cleanup.sh
<13>Nov 10 17:27:37 1.diyca_cleanup.sh: Begin
<13>Nov 10 17:27:37 1.diyca_cleanup.sh: Remove all CRT files
<13>Nov 10 17:27:37 1.diyca_cleanup.sh: Remove all CSR files
<13>Nov 10 17:27:37 1.diyca_cleanup.sh: Remove all KEY files
<13>Nov 10 17:27:37 1.diyca_cleanup.sh: Remove all PYC files
<13>Nov 10 17:27:37 1.diyca_cleanup.sh: Remove all DB files
<13>Nov 10 17:27:37 1.diyca_cleanup.sh: Re-create app_web/private subdirectory
mkdir: cannot create directory ‘app_web/private’: No such file or directory
<13>Nov 10 17:27:37 1.diyca_cleanup.sh: Re-create the calvin subdirectory
<13>Nov 10 17:27:37 1.diyca_cleanup.sh: Re-create the signer subdirectory
<13>Nov 10 17:27:37 1.diyca_cleanup.sh: Re-create the certs subdirectory
<13>Nov 10 17:27:37 1.diyca_cleanup.sh: Re-create the log subdirectory
<13>Nov 10 17:27:37 1.diyca_cleanup.sh: End
root@raspberrypi:/opt/diyca/bin# /opt/diyca/bin/2.diyca_calvin_init.sh
<13>Nov 10 17:28:15 2.diyca_calvin_init.sh: Begin
/opt/diyca/bin/2.diyca_calvin_init.sh: line 14: bin/diyca_common.bash: No such file or directory
<13>Nov 10 17:28:15 2.diyca_calvin_init.sh: *** bin/diyca_common.bash is inaccessible
root@raspberrypi:/opt/diyca/bin# ls -l
total 60
-rwxr-xr-x 1 root root 1113 Nov 10 17:23 1.diyca_cleanup.sh
-rwxr-xr-x 1 root root 2497 Nov 10 17:23 2.diyca_calvin_init.sh
-rwxr-xr-x 1 root root 1932 Nov 10 17:23 3.diyca_web_init.sh
drwxr-xr-x 2 root root 4096 Nov 10 17:27 calvin
drwxr-xr-x 2 root root 4096 Nov 10 17:27 certs
-rwxr-xr-x 1 root root 1040 Nov 10 17:23 diyca_admin.sh
-rw-r--r-- 1 root root 1895 Nov 10 17:23 diyca_calvin_cert.cfg
-rw-r--r-- 1 root root 873 Nov 10 17:23 diyca_common.bash
-rw-r--r-- 1 root root 529 Nov 10 17:26 diyca_web_cert.cfg
drwxr-xr-x 2 root root 4096 Nov 10 17:27 log
-rwxr-xr-x 1 root root 731 Nov 10 17:23 rc.diyca.sh
-rwxr-xr-x 1 root root 404 Nov 10 17:23 rcerts.sh
-rwxr-xr-x 1 root root 245 Nov 10 17:23 rdb.sh
-rwxr-xr-x 1 root root 3102 Nov 10 17:23 rpi_sensors.sh
drwxr-xr-x 2 root root 4096 Nov 10 17:27 signer
root@raspberrypi:/opt/diyca/bin#
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Entirely happy to try again, grab additional environment data, etc., should that be required.

Not sure if this is relevant, but I am running these scripts as actual root, on account of hitting a weirdness when trying to grant my base user sudo privilege...

Update docs/preparation_notes.txt for Buster

Update 3-year-old Raspbian set up sub-procedure from the docs/preparation_notes.txt.

Add certificate instructions for Chrome

Documentation omission in preparation_notes.txt

Update preparation notes for recent changes to Raspbian & Ubuntu

Debian and Ubuntu appear to be evolving to changes in system startup. It was noticed that /etc/rc.local is not always present and that systemd does not necessarily the presence of that file. However, there is an rc-local.service available for compatibility.

example.users run-time configs have the wrong certs directory reference

Both

• bob_run.cfg
• alice_run.cfg

are pointing to the certs directory at the wrong level.

"DeprecationWarning: str for buf is no longer accepted, use bytes"

Describe the bug
Relatively new deprecation warnings in example programs alice and bob for not presenting the socket routines with bytes objects in stead of str objects. Currently, they work anyways. No telling about tomorrow.

To Reproduce
Steps to reproduce the behavior:

1. Run the alice and bob example programs as documented.
2. See: "DeprecationWarning: str for buf is no longer accepted, use bytes"

Expected behavior
No warnings.

Convert to Python 3

2020 is the end of Python 2

New openssl requires pre-existing index.attr (can be nil)

Fix bin/2.diyca_calvin_init.sh to touch db/index.attr just after it touches db/index.

Remove potential project information from Calvin's certificate

Is your feature request related to a problem? Please describe.
People are using Calvin's self-signed CA certificate on the public Internet.

Describe the solution you'd like
People should be using this project for which it was intended: LAN. Barring that, information that might identify this project should be removed from Calvin's certificate configuration file, same for the diy web server, and elsewhere.

Additional context
Guidance in the preparation notes or elsewhere is needed to guide folks who only want to set up a web site like you commonly find on the Internet: one-way authentication of the web site based on the CA who signed the web site's user certificate.

Request How-To Notes for Root CA Refresh

Is your feature request related to a problem? Please describe.
Very happy DIYCA User here... been running flawlessly on my RPi4 since November 2019... but I imported my Calvin root Cert to a new home machine this weekend and noticed that it will expire in November this year [2 year lifespan].

Describe the solution you'd like
Would it be possible to extend the documentation to show how to create a new root certificate please? Also, very conscious that the codebase hasn't been touched since installation time in 2019... it looks from a cursory read of the instructions that the way to upgrade to the latest version might be to wipe and start over? Should I be concerned about migrating to the latest version?

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
N/A

Additional context
Add any other context or screenshots about the feature request here.
Happy to provide any other documentation or information that may be required.