mozilla-services/autograph-canary

An AWS lambda for checking Autograph signing is working correctly

autograph-canary's Issues

Add documentation on test creation

fix CI to fail when integration test fails

Update bin/test_canary.sh to fail when the invocation fails.

test addon installs are blocked or not blocked

see also #43

Currently, we only test that an addon installs in https://github.com/mozilla-services/autograph-canary/blob/main/tests/addon_signature_test.js#L106

but we should update the blocklist then check that an addon install is blocked or not blocked for the following cases:

• a known blocked addon is blocked
• a known unblocked addons is not blocked
• newly signed addons are not blocked (I'm not sure what a sufficiently large set would be)

replace python and tlscanary dependencies or cache Fx tarball

While useful in the initial PoC tlscanary pulls in a lot of unused functionality and support for more platforms than we need. Instead we can:

• bake in a given Firefox binary using env or build args (replacing https://github.com/mozilla/tls-canary/blob/master/tlscanary/tools/firefox_downloader.py and https://github.com/mozilla/tls-canary/blob/master/tlscanary/tools/firefox_extractor.py) https://github.com/stephendonner/socorro/blob/master/e2e-tests/Dockerfile is an option
• provide Fx prefs and profile at a known path
• extract the relevant parts of the tlscanary namely https://github.com/mozilla/tls-canary/blob/master/tlscanary/tools/xpcshell_worker.py ; https://github.com/mozilla/tls-canary/blob/master/tlscanary/tools/firefox_app.py ; https://github.com/mozilla/tls-canary/blob/master/tlscanary/js/worker_common.js and https://github.com/mozilla-services/autograph-canary/blob/main/autograph.py
• run the XPCShell tests directly, print logs, and exit

Add documentation on Lambda configuration

Add top comment on the existing test files to summarize their purpose

• addon_signature_test.js
• content_signature_test.js

Add expanded documentation

Including:

• Information on adding tests
• Lambda configuration

Should the prefs to fetch (on failure) be specified by environment?

Two possible mechanisms here:

1. Have a global set of prefs that are fetched by every worker
2. Have a mechanism where per-test prefs are specified by special name (e.g. FETCH_ADDON_SIGN_PREFS=an.example.pref, another.example.pref )

teach container to time travel

Specifically, we'd like to run it in the future to detect pending expirations.

Something like the following should work:

timedatectl set-ntp no
timedatectl set-time YYYY-MM-DD
timedatectl set-time $(date -u -d '+60 day' '+%F')

https://www.cyberciti.biz/faq/howto-set-date-time-from-linux-command-prompt/

^ results in:

root@85daaeb79bd9:/function# timedatectl set-time $(date -u -d '+60 day' '+%F')
System has not been booted with systemd as init system (PID 1). Can't operate.
Failed to connect to bus: Host is down

so that approach might not be possible in a container.

Lambda fails when accessing environment

from @erkolson:

[ERROR] AttributeError: 'LambdaContext' object has no attribute 'get'
Traceback (most recent call last):
  File "/var/task/autograph.py", line 86, in run_tests
    env = lambda_context.get("env")

Add open source software license

This Mozilla repository has been identified as lacking a license. Consistent with Mozilla's Licensing Policy an open source license should be applied to the code in this repository.

Please add an appropriate LICENSE.md file to the root directory of the project. In general, Mozilla's licensing policies are as follows:

• Client-side products created by Mozilla employees or contributors should use the Mozilla Public License, Version 2.0 (MPL).

• Server-side products or utilities that support Mozilla products may use either the MPL or the Apache License 2.0 (Apache 2.0).

In special cases, another license might be appropriate. If the repository is a fork of another repository it must apply the license of the original. Similarly, another license might be appropriate to match that of a broader project (for example Rust crates that Firefox depends on are published under an Apache 2.0 / MIT dual license, as that is the dual license used by the Rust programming language and projects).

Please ensure that any license added to the LICENSE.md file matches other licensing information in the repository (for example, it should match any license indicated in a setup.py or package.json file).

Mozilla staff can access more information in our Software Licensing Runbook – search for “Licensing Runbook” in Confluence to find it.

If you have any questions you can contact Daniel Nazer who can be reached at dnazer on Mozilla email or Slack.

OPENLIC-2023-01

Ensure the native / non-native package extraction is documented adequately

Create a consistent setup procedure

In order to have a consistent build pipeline, we need a requirements.txt for this project.

Provide configurable options to allow testing of non-production signatures

Currently we assume the production signing root and remote settings infra. Since this is for testing signature issues, it makes sense to provide coverage on stage too.

test scripts should error if there's a timeout

Add environment for adjusting logging verbosity

To help with silent failures: e.g. #17 (comment)

Sort out pip package for pypi

fix failing content signature tests

Seeing this error mozilla/tls-canary#227

One possible solution is pulling in the content signature JS from about remove settings / remote settings dev tools.

Add documentation on lambda packaging

convert to containerized lambda

https://docs.aws.amazon.com/lambda/latest/dg/images-create.html

We can run it in the lambda emulator using docker-compose as in mozilla-services/autograph#652

Create mechanism for the Lambda configuration (environment, etc) to reach worker scripts

add docker compose override example for dev setup

https://docs.docker.com/compose/extends/#adding-and-overriding-configuration

publish canary as docker image

• get an image name on dockerhub e.g. mozilla/autograph-canary (alternatively push directly to ecr)
• add creds to CI config
• add publish step to CI

AFAIK it's fine for the image to be public and to make this repo public.

Grant read-only access to https://github.com/orgs/mozilla-services/teams/everyone

That way more people can view this repo

bundle fixture XPIs in docker image

• add test fixture XPIs to docker image
• update addons test to test fixture XPIs from path or file:/// url

make repo public

refs: #48

I ran a quick scan in #48 (comment) and don't think there's anything sensitive in here.

I spent awhile debugging deploys today and yesterday. mirrorDockerImage pulls build info from CircleCI but doesn't error for 404s (which CircleCI returns for private repos), so making this public will make it easier to deploy (alternatively we could set up a CircleCI API token).

handle shutdown cleanly

We sleep an amount of time scaled by the number of things a test verifies, but it'd be better to verify things complete or fail and shut down gracefully.

One possible route:

• send a tls canary quit command
• wait for an ack response https://github.com/mozilla/tls-canary/blob/5e243f4dd8557d61c358b9d6da641cbd807c957e/tlscanary/js/worker_common.js#L192
• then terminate the XPCShell worker