mozilla-extensions/xpi-template

XPI template repo

Special cases

If a repository contains multiple package.json files, and any of them does not refer to an Add-on, a dontbuild file can be dropped in the directory to make automation skip building and signing.

xpi-template's Issues

update .tc.yml to ignore tags

If we can also update the PR behavior to only run tasks if the PR has changed, that would be good too. Otherwise, let's split that out into its own ticket.

AssertionError: artifact-reference only supports public artifacts, not /path/to/xpi

This issue was raised in Slack. It looks like our TaskCluster configuration does not support private repositories anymore because of the use of artifact-reference (added for the addons-linter task).

[task 2022-08-08T18:16:20.114Z] 2022-08-08 18:16:20,114 - INFO - Loading kinds
[task 2022-08-08T18:16:20.127Z] 2022-08-08 18:16:20,127 - INFO - Generating full task set
[task 2022-08-08T18:16:20.161Z] 2022-08-08 18:16:20,161 - INFO - Generated 1 tasks for kind build
[task 2022-08-08T18:16:20.164Z] 2022-08-08 18:16:20,164 - INFO - Generated 1 tasks for kind dep-signing
[task 2022-08-08T18:16:20.166Z] 2022-08-08 18:16:20,166 - INFO - Generated 1 tasks for kind linter
[task 2022-08-08T18:16:20.186Z] 2022-08-08 18:16:20,186 - INFO - Generated 2 tasks for kind test
[task 2022-08-08T18:16:20.187Z] 2022-08-08 18:16:20,187 - INFO - Generated 1 tasks for kind pr
[task 2022-08-08T18:16:20.187Z] 2022-08-08 18:16:20,187 - INFO - Generating full task graph
[task 2022-08-08T18:16:20.187Z] 2022-08-08 18:16:20,187 - INFO - Full task graph contains 6 tasks and 2 dependencies
[task 2022-08-08T18:16:20.187Z] 2022-08-08 18:16:20,187 - INFO - writing artifact file `full-task-graph.json`
[task 2022-08-08T18:16:20.190Z] 2022-08-08 18:16:20,190 - INFO - writing artifact file `runnable-jobs.json`
[task 2022-08-08T18:16:20.190Z] 2022-08-08 18:16:20,190 - INFO - Generating target task set
[task 2022-08-08T18:16:20.191Z] 2022-08-08 18:16:20,191 - INFO - Filter filter_target_tasks pruned 1 tasks (5 remain)
[task 2022-08-08T18:16:20.191Z] 2022-08-08 18:16:20,191 - INFO - writing artifact file `target-tasks.json`
[task 2022-08-08T18:16:20.192Z] 2022-08-08 18:16:20,191 - INFO - Generating target task graph
[task 2022-08-08T18:16:20.192Z] 2022-08-08 18:16:20,191 - INFO - Adding 0 tasks with `always_target` attribute
[task 2022-08-08T18:16:20.193Z] 2022-08-08 18:16:20,192 - INFO - Generating optimized task graph
[task 2022-08-08T18:16:20.193Z] 2022-08-08 18:16:20,192 - INFO - No tasks removed during optimization
[task 2022-08-08T18:16:20.616Z] 2022-08-08 18:16:20,616 - INFO - No tasks replaced during optimization
[task 2022-08-08T18:16:20.618Z] Traceback (most recent call last):
[task 2022-08-08T18:16:20.618Z]   File "/builds/worker/.local/lib/python3.6/site-packages/taskgraph/main.py", line 742, in main
[task 2022-08-08T18:16:20.619Z]     args.command(vars(args))
[task 2022-08-08T18:16:20.619Z]   File "/builds/worker/.local/lib/python3.6/site-packages/taskgraph/main.py", line 615, in decision
[task 2022-08-08T18:16:20.619Z]     taskgraph_decision(options)
[task 2022-08-08T18:16:20.619Z]   File "/builds/worker/.local/lib/python3.6/site-packages/taskgraph/decision.py", line 118, in taskgraph_decision
[task 2022-08-08T18:16:20.619Z]     write_artifact("task-graph.json", tgg.morphed_task_graph.to_json())
[task 2022-08-08T18:16:20.619Z]   File "/builds/worker/.local/lib/python3.6/site-packages/taskgraph/generator.py", line 220, in morphed_task_graph
[task 2022-08-08T18:16:20.619Z]     return self._run_until("morphed_task_graph")
[task 2022-08-08T18:16:20.620Z]   File "/builds/worker/.local/lib/python3.6/site-packages/taskgraph/generator.py", line 402, in _run_until
[task 2022-08-08T18:16:20.620Z]     k, v = next(self._run)
[task 2022-08-08T18:16:20.620Z]   File "/builds/worker/.local/lib/python3.6/site-packages/taskgraph/generator.py", line 387, in _run
[task 2022-08-08T18:16:20.620Z]     existing_tasks=existing_tasks,
[task 2022-08-08T18:16:20.620Z]   File "/builds/worker/.local/lib/python3.6/site-packages/taskgraph/optimize.py", line 77, in optimize_task_graph
[task 2022-08-08T18:16:20.620Z]     decision_task_id,
[task 2022-08-08T18:16:20.620Z]   File "/builds/worker/.local/lib/python3.6/site-packages/taskgraph/optimize.py", line 358, in get_subgraph
[task 2022-08-08T18:16:20.620Z]     dependencies=named_task_dependencies,
[task 2022-08-08T18:16:20.620Z]   File "/builds/worker/.local/lib/python3.6/site-packages/taskgraph/util/parameterization.py", line 95, in resolve_task_references
[task 2022-08-08T18:16:20.621Z]     "artifact-reference": artifact_reference,
[task 2022-08-08T18:16:20.621Z]   File "/builds/worker/.local/lib/python3.6/site-packages/taskgraph/util/parameterization.py", line 28, in _recurse
[task 2022-08-08T18:16:20.621Z]     return recurse(val)
[task 2022-08-08T18:16:20.621Z]   File "/builds/worker/.local/lib/python3.6/site-packages/taskgraph/util/parameterization.py", line 24, in recurse
[task 2022-08-08T18:16:20.621Z]     return {k: recurse(v) for k, v in val.items()}
[task 2022-08-08T18:16:20.621Z]   File "/builds/worker/.local/lib/python3.6/site-packages/taskgraph/util/parameterization.py", line 24, in <dictcomp>
[task 2022-08-08T18:16:20.621Z]     return {k: recurse(v) for k, v in val.items()}
[task 2022-08-08T18:16:20.621Z]   File "/builds/worker/.local/lib/python3.6/site-packages/taskgraph/util/parameterization.py", line 24, in recurse
[task 2022-08-08T18:16:20.621Z]     return {k: recurse(v) for k, v in val.items()}
[task 2022-08-08T18:16:20.622Z]   File "/builds/worker/.local/lib/python3.6/site-packages/taskgraph/util/parameterization.py", line 24, in <dictcomp>
[task 2022-08-08T18:16:20.622Z]     return {k: recurse(v) for k, v in val.items()}
[task 2022-08-08T18:16:20.622Z]   File "/builds/worker/.local/lib/python3.6/site-packages/taskgraph/util/parameterization.py", line 24, in recurse
[task 2022-08-08T18:16:20.622Z]     return {k: recurse(v) for k, v in val.items()}
[task 2022-08-08T18:16:20.622Z]   File "/builds/worker/.local/lib/python3.6/site-packages/taskgraph/util/parameterization.py", line 24, in <dictcomp>
[task 2022-08-08T18:16:20.622Z]     return {k: recurse(v) for k, v in val.items()}
[task 2022-08-08T18:16:20.622Z]   File "/builds/worker/.local/lib/python3.6/site-packages/taskgraph/util/parameterization.py", line 23, in recurse
[task 2022-08-08T18:16:20.622Z]     return param_fn(val[param_key])
[task 2022-08-08T18:16:20.622Z]   File "/builds/worker/.local/lib/python3.6/site-packages/taskgraph/util/parameterization.py", line 89, in artifact_reference
[task 2022-08-08T18:16:20.622Z]     return ARTIFACT_REFERENCE_PATTERN.sub(repl, val)
[task 2022-08-08T18:16:20.622Z]   File "/builds/worker/.local/lib/python3.6/site-packages/taskgraph/util/parameterization.py", line 85, in repl
[task 2022-08-08T18:16:20.622Z]     artifact_name
[task 2022-08-08T18:16:20.623Z] AssertionError: artifact-reference only supports public artifacts, not `xpi/build/search_engine_usage_study.xpi`
[taskcluster 2022-08-08 18:16:20.895Z] === Task Finished ===
[taskcluster 2022-08-08 18:16:22.134Z] Unsuccessful task run with exit code: 1 completed in 8.732 seconds

private repos are using `public/build` as their artifactPrefix

This used to work, and stopped at some point. Let's figure out what happened, and fix.

Consider preventing signing when the linter emits errors (i.e. when it fails)

Currently, if the linter task fails, it does not prevent signing. I am wondering if this something we'd want because ideally xpi-manifest would prevent signing when the linter emits errors.

investigate alternate solutions for template params

I initially put these in to give us debugging information about which revision of the template repo we used in a given graph. Per this thread, we may want to adjust this.

We probably want to weigh the effort of doing so with the likelihood of wanting to reuse this code. I think mozilla-extensions is a special use case, and we may not need generic support for this.

investigate alternate solutions for the taskcluster directory

Things work the smoothest by nuking and linking the xpi-template taskcluster/ directory into the xpi source dir. There are potentially other options as mentioned in this thread.

I'm avoiding doing so because:

• using --root implies the source is in the xpi-template directory, which breaks
• we could also run in the xpi-template directory (with or without --root), and specify a known location for the xpi source.
  • I believe this will reuse the level from the xpi-template repo instead of the source repo. I'm intentionally keeping xpi-template level 3 and the source repos level 1, to reflect the amount of trust given to each repo. I'd rather not treat the source repos as level 3, or treat the xpi-template repo as level 1, since we want to restrict what changes go into the template repo.

Opening this to track investigating other solutions, or revisit the above reasoning.

Pass `--privileged` flag to addons-linter

A new CLI flag is going to be added to addons-linter in order to enable special checks for privileged extensions (see: mozilla/addons-linter#4324). This flag will eventually enforce the linted XPIs to be privileged extensions, which should be the case for all the extensions that use this template.

The idea is to pass this flag now so that nothing will break when we'll land the actual changes (because omitting this flag will emit errors if the extension is privileged).