blockchain-certificates/cert-tools

Command line tools for designing certificate templates and instantiating a certificate batch

V3 is available

Blockcerts V3 is now based off W3C Verifiable Credentials specification and does not follow Open Badges V2 nomenclature anymore. To understand the differences between v2 and v3, please refer to the docs. You may also have a look at the JSONLD and JSON-SCHEMA documents

cert-tools

Command line tools for designing certificate templates, instantiating a certificate batch, and import/export tasks

See example of certificate template and batch creation in sample_data.

For more information on working with legacy V2, please see the docs here.

Install

1. Ensure you have an python environment. Recommendations

2. Git clone the repository and change to the directory

git clone https://github.com/blockchain-certificates/cert-tools.git && cd cert-tools

3. Run the setup script

pip install .

Scripts

The cert-tools setup script installs 2 scripts, which are described below:

create_certificate_template.py

Run

create-certificate-template -c conf.ini

Configuration

The conf.ini fields are described below. Optional arguments are in brackets.

At this moment this will give a very simple V3 blockcerts that you will likely need to customize further.

create-certificate-template --help

usage: create_v3_certificate_template.py [-h] 
                                         [-c MY_CONFIG] 
                                         [--data_dir DATA_DIR] 
                                         [--issuer_url ISSUER_URL] --issuer_id ISSUER_ID [--template_dir TEMPLATE_DIR] 
                                         [--template_file_name TEMPLATE_FILE_NAME]
                                         [--additional_global_fields ADDITIONAL_GLOBAL_FIELDS] 
                                         [--additional_per_recipient_fields ADDITIONAL_PER_RECIPIENT_FIELDS]


Args that start with '--' (eg. --data_dir) can also be set in a config file (./cert-tools/conf.ini or specified via -c). Config file syntax allows: key=value, flag=true, stuff=[a,b,c] (for details, see syntax at https://goo.gl/R74nmi). If an arg is specified in more than one place, then commandline values override config file values which override defaults.


Argument details:
  -h, --help            show this help message and exit
  -c MY_CONFIG, --my-config MY_CONFIG
                        config file path (default: None)
  --data_dir DATA_DIR   where data files are located (default: None)
  --issuer_url ISSUER_URL
                        issuer URL (default: None)
  --issuer_id ISSUER_ID
                        issuer profile (default: None)
  --template_dir TEMPLATE_DIR
                        the template output directory (default: None)
  --template_file_name TEMPLATE_FILE_NAME
                        the template file name (default: None)
  --additional_global_fields ADDITIONAL_GLOBAL_FIELDS
                        additional global fields (default: None)
  --additional_per_recipient_fields ADDITIONAL_PER_RECIPIENT_FIELDS
                        additional per-recipient fields (default: None)

About

Creates a certificate template populated with the setting you provide in the conf.ini file. This will not contain recipient-specific data; such fields will be populated with merge tags.

instantiate_certificate_batch.py

Run

instantiate-certificate-batch -c conf.ini

About

Populates the certificate template (created by the previous script) with recipient data from a csv file. It generates a certificate per recipient based on the values in the csv file.

The csv file location is configurable via the conf.ini file.

The csv file must always contain:

• name
• pubkey
• identity

Configuration

The conf.ini fields are described below. Optional arguments are in brackets

instantiate-certificate-batch --help

usage: instantiate_v2_certificate_batch.py [-h] 
                                           [-c MY_CONFIG]
                                           [--data_dir DATA_DIR]
                                           [--issuer_certs_url ISSUER_CERTS_URL]
                                           [--template_dir TEMPLATE_DIR]
                                           [--template_file_name TEMPLATE_FILE_NAME]
                                           [--hash_emails]
                                           [--additional_per_recipient_fields ADDITIONAL_PER_RECIPIENT_FIELDS]
                                           [--unsigned_certificates_dir UNSIGNED_CERTIFICATES_DIR]
                                           [--roster ROSTER]

Args that start with '--' (eg. --data_dir) can also be set in a config file (./cert-tools/conf.ini or specified via -c). Config file syntax allows: key=value, flag=true, stuff=[a,b,c] (for details, see syntax at https://goo.gl/R74nmi). If an arg is specified in more than one place, then commandline values override config file values which override defaults.

Argument details:
  -h, --help            show this help message and exit
  -c MY_CONFIG, --my-config MY_CONFIG
                        config file path (default: None)
  --data_dir DATA_DIR   where data files are located (default: None)
  --issuer_certs_url ISSUER_CERTS_URL
                        issuer certificates URL (default: None)
  --template_dir TEMPLATE_DIR
                        the template output directory (default: None)
  --template_file_name TEMPLATE_FILE_NAME
                        the template file name (default: None)
  --hash_emails         whether to hash emails in the certificate (default:
                        False)
  --additional_per_recipient_fields ADDITIONAL_PER_RECIPIENT_FIELDS
                        additional per-recipient fields (default: None)
  --unsigned_certificates_dir UNSIGNED_CERTIFICATES_DIR
                        output directory for unsigned certificates (default:
                        None)
  --roster ROSTER       roster file name (default: None)

Adding custom fields

You can specify additional global fields (fields that apply for every certificate in the batch) and additional per-recipient fields (fields that you will specify per-recipient).

Important: defining your custom fields in a JSON-LD context

When adding either global or per-recipient custom fields, you must define each of your new terms in a JSON-LD context. You can either point to an existing JSON-LD context, or embed them directly in the context of the certificate. For an example of the latter, see the JSON-LD specification section 3.1. In this case, the @context value would be an array listing the existing context links, and your new definition.

Examples of both options are below:

{
  "@context": [
        "https://www.w3.org/2018/credentials/v1",
        "https://w3id.org/blockcerts/v3",
        "https://your-custom-context/v1",                                <-- option 1: point to custom JSON-LD context
        {                                                                <-- option 2: directly embed in certificate
             "xyz_custom_field": "http://path/to/xyz_custom_field",
              ... // and all other custom fields
        }
    ]
}

Custom global fields

You can specify custom global fields in the conf.ini file with the additional_global_fields entry

For each additional global field, you must indicate:

• the jsonpath to the field
• the global value to use

Example:

conf.ini:

additional_global_fields = {"fields": [{"path": "$.certificate.subtitle","value": "custom subtitle"}]}

or, expanded for readability:

    additional_global_fields = {
        "fields": 
            [
                {
                    "path": "$.certificate.subtitle",
                    "value": "custom subtitle"
                }
            ]
    }

Custom per-recipient fields

See above note on (current) manual step of defining custom JSON-LD context.

Per-recipient fields are specified in a combination of the conf.ini file, with the additional_per_recipient_fields entry, and the .csv file containing per-recipient data. Per-recipient fields are used in both template creation and certificate instantiaion. During the template creation process, we apply placeholder merge tags as values. This helps you preview your template before running instantiate_certificate_batch.py.

For each additional per-recipient field, you must indicate the following in the additional_per_recipient_fields config field:

• the jsonpath to the field
• the merge_tag placeholder to use
• the csv column where the value (per recipient) can be found. This is used by instantiate_certificate_batch

Example:

conf.ini version:

    additional_per_recipient_fields = {"fields": [{"path": "$.xyz_custom_field","value": "*|THIS WILL CONTAIN XYZ CUSTOM VALUES|*","csv_column": "xyz_custom_field"}]}

above expanded for readability:

TODO

create_revocation_addresses.py (currently unused)

Run (optional)

create-revocation-addresses -k tpubD6NzV...H66KUZEBkf

About

Generates Bitcoin addresses using an HD extended public (or private) key to be used as the issuer's revocation addresses for the certificates. This would be useful only if the issuer requires to be able to revoke specific certificates later on. It creates a list of addresses that could then be easily merged with the roster file, e.g. using unix's paste command.

To create 20 revocation address for a testnet extended public key for the first batch of 2016 certificates run:

echo "revkey" > rev_addresses.txt

create-revocation-addresses -n 20 -p "2016/1" -k tpubD6NzV...H66KUZEBkf >> rev_addresses.txt

To merge to roster (in unix) run:

paste -d , roster.txt rev_addresses.txt > roster_with_rev.txt

Example

See sample_data for example configuration and output. conf-mainnet.ini was used to create a batch of 2 unsigned certificates on the Bitcoin blockchain.

The steps were:

• Create the template
  • Update the config file to contain the correct data to populate the certificates
  • Place the needed images in images/ and point to them in the config file
  • Run create_certificate_template.py, which resulted in the certificate template /certificate_templates/test.json
• Instantiate the batch
  • Add the recipient roster (in this case rosters/roster_testnet.csv) with the recipient's Bitcoin addresses.
  • Run 'instantiate_certificate_batch.py', which resulted in the files in unsigned_certificates

Then the unsigned certificates were copied to cert-issuer for signing and issuing on the blockchain.

Contact

Contact us at the Blockcerts community forum.

cert-tools's Issues

Improve config file documentation

Problem with "issuer_logo_file"

Hi, the verifier use the isuser logo from the issuer profile.json BUT we can still use the "issuer_logo_file" parameter during the template creation.
That image is inserted in the certificate.json BUT is never used.

The "issuer_logo_file" parameter is deprecated? If yes, it would not be added to the certificate.json
The "issuer_logo_file" parameter is NOT deprecated? If yes, it would be used by the verifier

Thanks

Add certificate import/export tools

Add certificate template creator

Add CLI for creating a certificate JSON template given the issuer's parameters

CSV field limitation

I have a problem related that I have reached the field additional_fields_per_recipient limit.

It's solves by this code

# maybe necessary add flag for configuring this parametter 
csv.field_size_limit(100000000)

remove familyName and giveName from readme

Looks like we're just using name to cover both first and last name.

Add certificate batch instantiator

Given a certificate template and a roster of recipients (csv format), instantiate a batch of unsigned certificates

Update to OBI v1.1 format and retrofit customizations as OBI/JSON LD extensions

Update these tools when digital-certs v2 schema is finalized. At this time, add detailed usage information and examples.

Note that extensions are currently flagged as TODOs.

ImportError: No module named 'helpers'

running the command
create-certificate-template -c conf.ini
gives the error
ImportError: No module named 'helpers'

I think there was an issue with my virtual machine and how it was handling my site-packages.

responsive html in display_html

Is it possible to add media queries in display_html so that the certificate is responsive?

cert tools should warn or error when missing config info will result in invalid cert

I created a cert with no revocation_list in the conf.ini this resulted in the cert having revocation_list: null which cannot be imported to the iOS wallet. The tool shouldn't output broken certs.

display_html should have a sensible default and support substitutions from cert fields

It would be nice if when display_html isn't specified it would be populated with something reasonable like:

<img /><!-- the certificate image -->
<h1>{{Certificate Title}}</h1>
<p>Issued on: {{issue date}}</p>
<p>Issuer: {{issuer}}</p>
<p>Recipient: {{recipient}}</p>

Additionally being able to reference fields from the conf or resulting json so that the displayed cert can show recipient info would be nice.

Update Readme

I had to spend a couple hours searching through the code to finally get create-issuer to work properly. Adding some documentation about the function and the intro_url config will be helpful for newcomers!

badge Id and certificate Id

What are difference in Badge ID and Certificate Id?

AttributeError: 'str' object has no attribute 'isoformat'

hi, i followed the cert-tools readme guide, but when i try to run this script instantiate-certificate-batch -c conf.ini, it returns me this err

(venv) C:\Users\yeyei\OneDrive\Documentos\cert-tools\certs>instantiate-certificate-batch -c conf.ini
Traceback (most recent call last):
File "c:\users\yeyei\appdata\local\programs\python\python39\lib\runpy.py", line 197, in _run_module_as_main
return _run_code(code, main_globals, None,
File "c:\users\yeyei\appdata\local\programs\python\python39\lib\runpy.py", line 87, in run_code
exec(code, run_globals)
File "C:\Users\yeyei\OneDrive\Documentos\cert-tools\certs\venv\Scripts\instantiate-certificate-batch.exe_main.py", line 7, in
File "c:\users\yeyei\onedrive\documentos\cert-tools\certs\venv\lib\site-packages\cert_tools\instantiate_v2_certificate_batch.py", line 150, in main
instantiate_batch(conf)
File "c:\users\yeyei\onedrive\documentos\cert-tools\certs\venv\lib\site-packages\cert_tools\instantiate_v2_certificate_batch.py", line 114, in instantiate_batch
certs = create_unsigned_certificates_from_roster(template, recipients, use_identities, config.additional_per_recipient_fields, config.hash_emails)
File "c:\users\yeyei\onedrive\documentos\cert-tools\certs\venv\lib\site-packages\cert_tools\instantiate_v2_certificate_batch.py", line 73, in create_unsigned_certificates_from_roster
issued_on = helpers.create_iso8601_tz()
File "c:\users\yeyei\onedrive\documentos\cert-tools\certs\venv\lib\site-packages\cert_tools\helpers.py", line 61, in create_iso8601_tz
return ret.isoformat()
AttributeError: 'str' object has no attribute 'isoformat'

Add documentation about needing JSON-LD definitions with custom attributes

Issue described here:
http://community.blockcerts.org/t/how-to-add-custom-additional-fields-blockcertvalidationerror/276/2?u=kim

Creating an Issuer ID

Hi,
I wanted to learn the best practices for creating an Issuer ID (that needs to be stored in the conf.ini file). Also wanted to learn how do you generate the public key for the issuer for the conf.ini file.
Thanks for the help!

Blockcerts schema version 2.1 compatibility

The cert-schema project is being updated for Blockcerts schema v2.1, and cert-tools needs to be adjusted to accommodate. This may include support for Open Badges' hosted verification, as well as ensuring that both signed and hosted badges can be verified against the IMS Open Badges Validator.

Adding custom fields in conf.ini

Hi.
I've uncommented the line:

# additional_global_fields = {"fields": [{"path": "$.certificate.subtitle","value": "kim custom subtitle"}]}

in conf.ini.

So I've run the script:

create-certificate-template -c conf.ini

the error is:

path is not valid! : certificate.subtitle
Traceback (most recent call last):
File "/Users/leonardol/anaconda3/bin/create-certificate-template", line 11, in
sys.exit(main())
File "/Users/leonardol/anaconda3/lib/python3.6/site-packages/cert_tools/create_v2_certificate_template.py", line 186, in main
template = create_certificate_template(conf)
File "/Users/leonardol/anaconda3/lib/python3.6/site-packages/cert_tools/create_v2_certificate_template.py", line 131, in create_certificate_template
assertion = jsonpath_helpers.set_field(assertion, field['path'], field['value'])
File "/Users/leonardol/anaconda3/lib/python3.6/site-packages/cert_tools/jsonpath_helpers.py", line 85, in set_field
raise (Exception(msg))
Exception: path is not valid! : certificate.subtitle

Changing that line removing .subtitle:

additional_global_fields = {"fields": [{"path": "$.certificate","value": "kim custom subtitle"}]}

all was ok with the following result:

Writing template to /Users/leonardol/cert-tools/sample_data/certificate_templates/test.json
Created template!

Could you help me?

Thanks in advance.

PyPi out of date

The package on pypi hasn't been updated in a year, can the latest changes be published there?