brave/token-lists

Manages custom token lists for Brave Wallet

token-lists

Manages custom token lists for Brave Wallet

Automated Publishing

We have the following setup in place to automatically build and publish token-lists to wallet data files.

1. A daily cron job on GitHub Actions that performs the following tasks:
   1. use @solflare-wallet/utl-aggregator to fetch the top SPL tokens on CoinGecko.
   2. commit the above list to data/solana/tokenlist.json.
   3. download and post-process token logos, and build a new NPM package.
   4. publish the above NPM package.
2. A weekly cron job on Jenkins that publishes the latest NPM package (in 1.iv) to wallet data files.

The purpose of 2 independent cron jobs is to factor in flakiness of 1., and avoid performing PNG post-processing in 2.

Development

To test out the output of this package you have to use a docker. It has been tested on Linux and macOS:

• Install docker
• Create a docker image docker build -t token-lists .
• Launch the docker image docker run -u "$(id -u):$(id -g)" -v "$PWD/build:/token-lists/build" -ti token-lists
• You will see an output in the build folder

Publishing token list to npm

brave/brave-core-crx-packager uses the npm package published here brave-wallet-lists.

It will be automatically published when your PR is merged and release is created.

This outputs a no dependency package with output images and token lists.

Testing a deployment

To test wallet data files use the development component updater with a fresh profile.

To do this you can use the command line argument --use-dev-goupdater-url.

You can use a clean profile without clearing with this as well: --user-data-dir=<tmp-dir>.

If you're using a development build, you can set the dev server via this npmrc environment in ~/.npmrc:

updater_dev_endpoint=https://go-updater-dev.bravesoftware.com/extensions

You can test a deployment by running the Jenkins job named: brave-core-ext-wallet-data-files-update-publish-dev Please check to make sure it succeeds.

Wait 5-10 minutes as the server will purge its cache during that timeframe and start serving the new component.

Then startup Brave using: open -a Brave\ Browser\ Beta.app --args --use-dev-goupdater-url --user-data-dir=$(mktemp -d)

After things are tested you can run the Jenkins job: brave-core-ext-wallet-data-files-update-publish and then after success, test on your normal Brave profile. The change will be live within 5-10 minutes. Please also test on production.

After testing on production, gives sign off in Slack on #releases and #prod-changes.

Troubleshooting deployment

You can see a list of the components that the component updater serves by going to these URLs:

• Prod: https://go-updater.brave.com/extensions/test
• Dev: https://go-updater-dev.bravesoftware.com/extensions/test

token-lists's Issues

Move the package to brave

We should move https://www.npmjs.com/package/brave-wallet-lists to https://www.npmjs.com/org/brave for a better management. Here is the concern for a context https://github.com/brave/security/issues/1012#issuecomment-1233392213

Dependency Dashboard

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

• Update Node.js to v16.19.1
• Update dependency qyu to ^0.7.0
• Update dependency sharp to ^0.31.0
• Update Node.js to v19
• Update dependency node-fetch to v3
• Update dependency qyu to v1
• Click on this checkbox to rebase all open PRs at once

Detected dependencies

dockerfile

Dockerfile

• node 16.13.0

github-actions

.github/workflows/audit.yml

• actions/checkout v3
• actions/setup-node v3

.github/workflows/build.yml

• actions/checkout v3
• actions/setup-node v3

.github/workflows/codeql-analysis.yml

• actions/checkout v3
• github/codeql-action v2
• github/codeql-action v2
• github/codeql-action v2

.github/workflows/npm-publish.yml

• actions/checkout v3
• actions/setup-node v3

.github/workflows/solana-tokenlist.yml

• actions/checkout v3
• actions/setup-node v3
• stefanzweifel/git-auto-commit-action v4

npm

package.json

• @solflare-wallet/utl-aggregator ^0.0.11
• node-fetch 2
• nodejs-file-downloader ^4.9.3
• qyu ^0.4.7
• sharp ^0.30.5
• imagemin ^8.0.1
• imagemin-pngquant fd51901
• semver-regex >=4.0.5
• uuid >=8.3.2
• got >=11.8.5
• bin-build da4209b
• bin-wrapper 4bc5c9e
• download c9c2589

---

• Check this box to trigger a request for Renovate to run again on this repository

Create npm audit action

dependabot alert

see https://github.com/brave/token-lists/security/dependabot/8

Sandboxed environment

@diracdeltas expresses concern about running scripts which contain vulnerabilities on real machines. It would be good to find a way to run them always on isolated environments. We have it running in a docker on CI, but not on dev machines.

can we make sure these scripts only get run in a sandboxed environment (like a docker container that gets destroyed after outputting the compressed PNG icons)? that way if there are malicious images or vulns in the dependencies, it shouldn't compromise any machines.