mobilefirstllc/extension-cli

Command line tool for building extensions/add-ons for chromium based browsers.

Extension CLI

Extension CLI is a command-line application that facilitates chromium∗-based web extension development by providing a systematic way to build, test and document extension projects. It handles the project setup and builds and lets you focus on the extension you are creating.

---

Features

• 🖥️   Javascript Bundling — Compiles, bundles and minifies javascript files
  

• 🎨   CSS Bundling — Compiles, bundles, and minifies CSS and SASS files
  

• 💄   Linting — lint JavaScript using ESLint
  

• 📦   ZIP Generation — Generates a .zip file for publishing
  

• 📝   Document Source Code — Generates code documentation using JSDoc
  

• ⚗️   Unit Testing — Provides a unit test environment preloaded with mocha, chai and sinon-chrome
  

• ⚔️   Cross-Browser Compatibility - develop extensions for Chrome, Edge, Firefox, Opera and Brave.
  

Getting Started

Note: Using this CLI assumes you have Node.js installed. If you do not, you can install it here.

Create new extension project

npx extension-cli

Add to an existing project

npm install extension-cli

Commands Reference

Command	Description
xt-build	Run builds; env flags: -e prod and -e dev
xt-test	Run unit tests
xt-docs	Generate docs
xt-sync	Update project config files to match the latest defaults supplied by this CLI
xt-clean	Remove automatically generated files

---

Read the Docs

 
   User Guide →

CLI Developer Guide

If you are interested in extending this project or forking see this guide →

---

Motivation

After developing multiple browser extensions, it became clear that there are several steps in the development process that stay the same between every project.

Instead of setting up these tasks individually for each project, it made more sense to combine everything in a utility tool that could be shared between projects.

This approach helps with creating a common, consistent development approach between multiple projects, reduces time to get started, and makes it easier to update build tools and scripts across multiple projects as many npm packages inevitably need to be updated (frequently!).

---

Issues & Feature Requests: Submit on Github

Maker: made by developers behind several popular extensions!

License: MIT

extension-cli's Issues

Show more output when docs command fails

Describe the bug
Show more output when docs fail

To Reproduce
Write some bad JSDoc3 value to make it fail

Expected behavior
Should show reason why the docs build is failing. Saying it failed but not why is not helpful at all.

Loading jQuery fails because of webpack

We need options to customize the webpack engine so we can add the provider plugin to fix webpack messing up the jQuery library. Or if this library can already do this, how do you do it?

xt-docs template changes

xt-docs command builds documentation site for an extension project using JSDoc3, which is quite useful for documentation purposes. The current default configuration uses a template called foodoc. But foodoc source is not being maintained, uses old packages, etc. so it is time to switch to something else.

Possible options (I would say in order of preference):

• find a different template that is being maintained
• fork foodoc template and start maintaining it
• make a new template
• default to default / no template

(I'm not able to find many that are being maintained) but here are some candidates:

• clean-jsdoc-theme
• Braintree JSDoc Template
• JSDoc Template
• tidy-jsdoc

Should also update the user guide and list all these options on there. A project can use any template they want. This issue is about choosing the default template.

EDIT: After trying some of these templates, not sure about using them. I am now thinking: set default to default in this project. Then fork foodocs and modify it (a lot) to update dependencies and the dated bootstrap style at which point is becomes a new-ish template. Add to the user guide a list of templates to use. So this is a mix of the last 3 options.

Dependency Dashboard

This issue provides visibility into Renovate updates and their statuses. Learn more

This repository currently has no open or pending branches.

---

• Check this box to trigger a request for Renovate to run again on this repository

Add JSX loader to webpack configs

Want to write the UI using react but cannot do it currently because there is no loader for it

Eslint

related to: PR 68

It should be .json file extension (quick fix) -or- change the config file to actually use js syntax (slow fix)

Add renovate

Renovate is a good bot to keep your npm dependencies in check. I use it for my projects, and it makes things easier for me.

Here's where you can find it.

Add github actions config yml

Extension CLI allows pulling config files into a project using xt-sync.

It currently allows pulling in CI configs for gitlab and travis. It should also have an option for github actions and (possibly?) circle-ci.

Side note: the command name should probably better describe this functionality, since what is does is "pull in latest config/config starter code", which is slightly different in semantics from the original use case. Maybe rename to xt-config? Besides CI configs, it also has option for eslint config and gitignore matching default project organization.

For added value could also add in user guide, how to configure the CI to upload to store(s) automatically.

Review dependencies

Make a list of project dependencies and what they are used for (in docs so it can be re-reviewed periodically?) and remove any that are no longer used (if any). Some of these dependencies raise security alerts, would like to eliminate those, even though not overly concerned about them, since this is a dev tool.

Developer instructions

Describe the bug

The instructions for how to build and develop the CLI locally are unclear and too vague.

Expected behavior

Write a step by step guide explaining local dev steps, to include the following facts:

• need two projects: (1) CLI (2) extension project to drive the CLI
• better explain the steps to link the two projects (step by step commands)
• clearly explain the config/build.json path issue when debugging locally
• step by step of how to unlink CLI and project once done

Enable running custom commands before generating zip file

Is your feature request related to a problem? Please describe.

Running the basic build with some additional custom build steps; would want to be able to have the build not compile the zip file until all build scripts are done.

Describe the solution you'd like
Run the build, then any custom build commands, then generate the zip file.

Describe alternatives you've considered

Changing the order of build steps but the standard build clears the output directory in the beginning of the build. As a workaround I had custom build run first and output to a specific directory, then copy the outputs from that directory to dist during xt-build.

Alternatively, allow disabling clearing of output directory or create separate command for generating zip.

xt-build generates incompatible output (us-ascii character set instead of utf-8)

Describe the bug
Some part of the code generation side of this tool converts the character set from UTF-8 to us-ascii, resulting in errors like the following:

Running file -I dist/content.js returns:

dist/content.js: text/plain; charset=us-ascii

This has effectively stopped me in my tracks and ended my use of this tool — I'm not remotely pleased at having to re-scaffold my project right now.

To Reproduce
Steps to reproduce the behavior:

1. Scaffold
2. npm start
3. Load unpacked extension in Chrome; navigate to dist/
4. See error

Expected behavior
The in-dev extension loaded in Chrome

Desktop (please complete the following information):

• OS: MacOS 12.6
• Browser Chrome
• Version 106.0.5249.119
• Node.js 16.13.2

Use Tailwind CSS

Hey team,

Pretty straightforward, how can I set up Tailwind CSS within the extension?
Is it possible to suggest the Tailwind setup during the creation of the extension?

Cheers!

xt-clean: check that .gitignore exists

Describe the bug
If there is no .gitignore in the root the command terminates with error.

Expected behavior
allow configuring cleanable files in package.json similar to other commands.
If there is no .gitignore or config, then show appropriate message and terminate without error.

TypeScript support

Is your feature request related to a problem? Please describe.
I just found this, and I wanted to ask if TypeScript support is coming in the future.

Describe the solution you'd like
Support for the TypeScript compiler. I can make a pr for it if needed.

Describe alternatives you've considered
It isn't hard to modify the npm scripts to run npx tsc and then xt-build.

Additional context
Could be a options on initial setup.

Manifest version for new projects

When creating a new project, currently manifest defaults to v2, which is not recommended for chrome, and FF will move to v3 some day (?), but this will be a very gradual migration overall.

• change starter manifest to v3 (extension-cli/config/init/manifest.json)

EDIT: this task is to change current starter MV2 to MV3 since it is widely supported. If project owner wants to downgrade it is fairly simple to change the manifest keys locally per project.

Unit test utility functions

Figure out how to unit test cli commands.

• test utility functions that do not perform I/O
• test utility functions that perform I/O

test CLI commands (see #21, different effort)

Support for using different xtbuild config for firefox and chrome

Is your feature request related to a problem? Please describe.
I would like to be able to define different behavior for a chrome release vs a firefox release. For example, I would like to be able to have a different release name for my chrome and firefox releases.

Describe the solution you'd like
I would like to be able to add a "firefox" and/or "chrome" key to the .xtbuild.json file or xt-build config in package.json with different values for the configuration options. My config would then look like this

{
	"manifest": "./src/extension/manifest.json",
	"js_bundles": [
		{
			"src": "./src/extension/background.js",
			"name": "background",
		},
		{
			"src": [
				"./src/extension/file1",
				"./src/extension/file2",
				"./src/extension/file3"
			],
			"name": "bundle2",
		}
	],
	"firefox": {
		"release_name": "release_firefox"
	}
}

which would change the release name when releasing for firefox

Describe alternatives you've considered
Another way to achieve this would be to allow overriding config options on the command line, e.g. xt-build -e prod -p firefox --release_name=release_firefox. I would imagine that this solution would be harder to implement, and less robust

Thanks for the awesome project!

JS build on watch doesn't rebuild js files correctly

To Reproduce
When you run dev build with watch enabled, it appears to be rebuilding but is actually not. Have to stop it from running and run again manually to work around it. Does not happen with e.g. stylesheets; this is specific to js files.

Expected behavior
When running dev debug mode with watch enabled, should keep rebuilding the js file whenever there are changes (and actually do it)

Watching changes during dev

A recent change implemented more fine-grained watch behavior to watch individual bundles. This generally works, but the problem is that within a script I can import module and unless I list those modules in the build config, changes to those modules are not being watched, which is not ideal. This is only alpha version, probably need to change it back to "watch all js changes" then "build all on some change". I can see how scss could cause the same behavior with imports, so same fix. Use the watch path config key to specify what to watch.

Passing version from package.json to manifest.json

I would like to pass an autofield to the manifest that will be replaced by the package version. Something like "version": "__AUTO_package_version__" in manifest.json which will be replaced by the version from package.json on build.

Utilities.keyReplace does not add keys if they do not exist in original config

Describe the bug

xt-docs allows overriding all defined keys but not adding new ones

To Reproduce
Steps to reproduce the behavior:

Create project and attempt to set config options for properties not defined in the default docs config

Expected behavior

Command should allow extending configs with new keys

Document config files better

Describe the issue

There are multiple config files (excluding init/) with file ending .js meaning they are executable code and can cause failures at runtime.

These need to be better documented and ideally verified.

Goal

Minimize chance of error in these files

Tasks

• update createPrompts.js
• update rootSuite.js
• update texts.js
• add config/README.md w/ short description of each file

Update .travis config

Describe the bug
The configuration should cause build to error if unit tests do not pass. Also update Node version.

Watch mode for docs

Is your feature request related to a problem? Please describe.

Investigate adding watch mode to docs command

Describe the solution you'd like

Be able to build docs in watch mode.

Describe alternatives you've considered

Can build it manually which is not a big deal, but watch mode would be better.

Additional context

Research JsDoc if this is doable; not sure why this was excluded in the first place?

Updating underlying packages

Sorry if I'm missing something obvious here but I'm unable to figure out how to update the underlying packages.

I installed extension-cli last month and created a project. That was 1.2.0 which used webpack-stream 6.1.2.

I want to update webpack-stream to 7.0.0 but am unable to figure out how to do this.

Specifically, webpack-stream uses webpack 5 which supports conditional chaining.

I guess this is quite the conditional chain in itself!

Updating manifest while watching causes looping behavior

To Reproduce
Create new project and run npm run start with watch enabled. Make a change to the manifest and proceeds to rebuild in a loop.

Expected behavior
Should rebuild the dist and not loop

Desktop (please complete the following information):

• OS: macOS
• Version: latest
• Node.js Version: 10.15.3

Unit test commands

Is your feature request related to a problem? Please describe.

Yes -- adding testing to these commands would give more confidence in not breaking them on changes.

Issue is these commands perform IO so they are not straightforward to test. Looking for examples.

ACTION: chalk breaking change

About a month ago, I merged #134 into master, which was a breaking version of chalk. I hadn't properly tested the change, and it seems that chalk has changed directions to use ESM, and not support CommonJS. This can be demonstrated by using the current state of master (at commit 4684574), and running npx extension-cli. You should see this error (unneeded dirs blacked out):

I'll be working on fixing this, but this will remain a tracking issue.

Create examples directory

Describe the solution you'd like

Add examples directory that contains simple starter code examples for different types of extensions. Lookup some common types, e.g. ad blocker starter code.

xt-build with watch flag doesn't rebuild the background bundle on *.js files change

Describe the bug

By running npm run start or xt-build -e dev -w (the default development script) - background bundle gets build once, but any further changes to javascript files located in ./src/**/*.js have no effect and the resulting bundled file in dist/background.js stays the same.

Expected behavior

The output js bundles should be rebuild on any file change occurred in a listed bundle(s).

Additional context

It's similar to #4 and the issue was fixed already, but at some point that fix disappeared. The issue happens in the following task:

extension-cli/config/gulpfile.js

Lines 34 to 55 in 00ab352

	const scripts = done => {
	let bundles = (Array.isArray(paths.js_bundles) ? paths.js_bundles : [paths.js_bundles]);
	const buildScript = () => {
	if (!bundles.length) {
	return typeof done !== 'function' || done();
	}
	const b = bundles.pop();
	return gulp.src(b.src)
	.pipe(webpack({'mode': 'production'}))
	.pipe(plugins.rename(function (path) {
	path.dirname = '';
	path.basename = b.name;
	}))
	.pipe(gulp.dest(paths.dist))
	.on('end', buildScript);
	};
	buildScript();
	};

Specifically here:

extension-cli/config/gulpfile.js

Line 42 in 00ab352

const b = bundles.pop();

^this line pops all the bundles provided by the original config object and mutates the object, thus all the bundles get build during the first run and ignored on watch event since they do not exist anymore in the object.

To solve the issue, the array of bundles should be copied to a temporary list. This line:

extension-cli/config/gulpfile.js

Line 35 in 00ab352

let bundles = (Array.isArray(paths.js_bundles) ? paths.js_bundles : [paths.js_bundles]);

Should be replaced with this one:

let bundles = [].concat(Array.isArray(paths.js_bundles) ? paths.js_bundles : [paths.js_bundles]);

copyAsIs on directory with html file still moves html file

Describe the bug
If copyAsIs is used for a directory with an html file inside it, the complete directory gets copied - as expected. At the same time, the html file inside the directory is put into the root of dist which is the default behavior - this is unexpected.

To Reproduce
Steps to reproduce the behavior:

1. create a directory "test" with an html page inside it "page.html"
2. add this to the build config: copyAsIs: ["test/*"]
3. note that dist/test contains page.html
4. there is also a processed version at dist/page.html

Expected behavior
Ignore *.html files when using copyAsIs command

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

• OS: Mac
• Browser: Chome
• Version: latest
• Node.js: latest

Enable specifying multiple manifests for different target browsers

Is your feature request related to a problem? Please describe.

The use case is this: I have some extension source, and I want to compile it for different browsers. The core src should be the same but perhaps some configs change between builds. This is especially the case with chrome and edge.

Current workaround is have browser specific build config file for each target

Describe the solution you'd like

I would like to be able to do this in the project root directory instead, and configure different builds, e.g. npm run build:chrome, npm run build:edge and then specify which files need to go in each bundle.

Building shell commands

Need to address there security alerts (all similar):

• xt-build
• xt-docs
• xt-test

The example fix shows use of array as args -> try it for the listed commands and see if it works. If it does not then wontfix.

• regarding all alerts: suggested fix is not applicable
• second/third issue: the indicated path is a constant with the exception of package name, but package name cannot contain whitespaces -> described issue is not applicable.

package.json name formating

Describe the bug
when adding name property to package.json, make it all lowercase and replace all spaces with hyphens

Custom webpack configuration

I'm trying to use ejs in my content script but getting an error related to using webpack 5. The solution it says is to add a polyfill to webpack.config.js. But I can't figure out how to do this using extension-cli as the config file is not generated and looking at the gulpfile it seems there's no simple way to add one.

The error output is

BREAKING CHANGE: webpack < 5 used to include polyfills for node.js core modules by default.
This is no longer the case. Verify if you need this module and configure a polyfill for it.

If you want to include a polyfill, you need to:
	- add a fallback 'resolve.fallback: { "path": require.resolve("path-browserify") }'
	- install 'path-browserify'
If you don't want to include a polyfill, you can use an empty module like this:
	resolve.fallback: { "path": false }

So it looks like I just need to have a config file with the following...

module.exports = {
   ...
   resolve: {
    fallback: {
      "fs": false,
      "path": require.resolve("path-browserify")
    }
  },
};

Is there currently a way to do this?

Debug vs prod configuration support

Describe the solution you'd like
Enable way to setup project config files that can be easily changed between dev and prod builds, for example, to switch out API endpoints.

Copy directories during build

Describe the bug
Copy as-is does copy directories.

Expected behavior
Make copy support copying of whole directories

No images on create

Describe the bug

When creating new extension the assets/img path is empty.

To Reproduce

run npx extension-cli and observe the result

Expected behavior

assets/img directory should have default icons, 16x16.png 24x24.png, 32x32.png and 128x128.png. There is currently assets/icon.svg so I can repair this manually but create command should create these for me.

Webpack Stream does not load babel configuration

Describe the bug
Webpack Stream does not take into account Babel configuration @babel/preset-env in package.json

To Reproduce
Use ES2021 syntax result in failed build by webpack-stream

Expected behavior
Build files transpiled successfully using babel configuration

Additional context
Relates to following issues:

• shama/webpack-stream#71
• babel/babel-loader#463

Add publish action

Add action to publish on tagged commit example

Complexity and tech debt considerations

After moving all code files to cli the maintainability decreased. The task is to address these complexity issues or mark them as wontfix.

Tasks

Code smells

Fixing this one function would take care of most smell issues:

• mouseEvent has 35 lines of code
• mouseEvent has a Cognitive Complexity of 8
• mouseEvent has 5 arguments
• keyReplace has a Cognitive Complexity of 10
• iterateConfigs has a Cognitive Complexity of 10
• ignore: Consider simplifying this complex logical expression → likely fix by #25

Duplication

• Similar blocks of code...
• ignore: Similar blocks of code... → likely fix by #25

Production build is really slow.

Describe the bug
Production build of scripts is really slow without any significant reason.

To Reproduce
npm run build

Expected behavior
I think than production build should go faster than current speed. There is not generation of source map or anything just same compilation as development with some minification. is the minifaction the issue?
When I just used webpack for compiling, the speeds were significantly faster than current time even when source maps where generated with minifcation.

clean inside app where extension is a subdirectory

Describe the bug

Create an extension inside another project where extension is part of a bigger solution e.g. CRA -> run xt-clean on .gitignore with additional files and first it hangs on file locks, second it clears the root dir.

Expected behavior

The command should not hang
The command should only clear ignored files

Desktop (please complete the following information):

• OS: MacOS
• Node.js Version 12+

Clearing files

Describe the bug
This happens to me frequently when I clear something, there are local files I do not actually want cleared. Another related issue is the command will not clear without .gitignore.

To Reproduce
remove .gitignore and run clear

Expected behavior
Add alternative way to specify what should be cleared. Perhaps in this order:

1. custom config for clearing
2. if not specified, then .gitignore
3. if none found then do nothing

Additional context
n/a

Make sync prompt

Describe the bug
Change the sync to prompt for options, similar to xt-create. This is a one-off command, should not be set to use a fixed flag. Also make appropriate changes when creating new extension.

Expected behavior
Make it easier to change flags

How do you import something different in tests vs release?

Is your feature request related to a problem? Please describe.

In tests, we'd like to use:

import * as ort from 'onnxruntime-node';

But in the actual browser extension:

import * as ort from 'onnxruntime-web';

Is there a way to do this?

Describe the solution you'd like

Point me to a JavaScript / extension-cli example on how to do this?

Additional context

I have mostly a C#/.NET/Java background, so I really have no idea about JavaScript. Any suggestions will help a lot! Thanks!

Failed unit tests not terminating on travis

Describe the bug
Running failing unit tests on travis does not terminate the build

Expected behavior
Test command should return non-zero code when tests complete to know it failed

Screenshots
see build example

research web-ext

Came across this today. Adding this note to research it further, specifically these questions:

1. Is web-ext same format as chromium extension or some other format? ^(*
2. How old is this project and who makes it?
3. What kind of options they offer through the CLI?
4. What is the docs and support like?
5. What are the marketing channels (besides direct link from mozilla)?

* There used to be a legacy way of building extensions. Also project seems very active.

npm install on create

Describe the bug
Sometimes creating a new extension project finished without running npm install. Maybe due to upgrades of dependencies.
Can work around it by running npm install in the project directory but not ideal.

To Reproduce
Steps to reproduce the behavior:

1. npx extension-cli - create extension project
2. Open project in IDE
3. node_modules does not exist

Expected behavior
npm install should run on create and process should wait for install to complete

Desktop (please complete the following information):

• OS: MacOS
• Node.js Version v12.18.2