paularmstrong/build-tracker

A set of tools to track the size of your build artifacts over time.

I am a Principal Software Engineer with deep experience leading teams and building amazing experiences using many different “front-end” web technologies. The list is deep and topics can range from JavaScript, Typescript, Node.js, React, CSS, Progressive Web Apps, developer experience, testing, continuous integration, and so much more.

You may be familiar with some of my open source work, like Normalizer, Build Tracker, React Component Benchmark, and more on Github.

Or possibly you’ve seen me speak, like at React Europe 2019 about “Moving fast with confidence”.

Sometimes I write things down. Hopefully I do that in my blog and I’m not too long-winded about it. When I don’t write enough about something, it usually appears on Mastodon @[email protected].

What am I working on?

build-tracker's Issues

Add validation to insert build routing

Problem

When inserting a build, the content is not validated to be well-formed.

Solution

The server's API should validate that the build contains meta and artifacts in an appropriate format with all required pieces (revision, timestamp, parentRevision, etc)

@build-tracker/cli

Problem

There's no unified way to read artifacts and report builds from repositories.

Solution

Create a CLI that can be configured to read glob patterns, calculate stat, gzip, brotli (configurable?) sizes, commit information, and upload to a configured API.

Docs: issues inferring branch (e.g. in Travis build)

I just noticed that all of my builds are uploading with their branch set to HEAD. IIUC, this is because of the way Travis builds work—they clone the repository then checkout a specific commit (e.g. abc) rather than a branch name (e.g. master):

$ git clone  --branch=master https://github.com/unsplash/unsplash-web.git unsplash/unsplash-web
Cloning into 'unsplash/unsplash-web'...
$ cd unsplash/unsplash-web
$ git checkout -qf 67babfc8143aab900f9660c2c933388e0b9e7125

This means that when the build tracker CLI tries to infer the branch name via git rev-parse --abbrev-ref HEAD, we will get HEAD instead of the actual branch name.

You can see this for yourself by running these commands locally:

$ git checkout master
$ git rev-parse --abbrev-ref HEAD
master
$ git checkout `git rev-parse master`
$ git rev-parse --abbrev-ref HEAD
HEAD

I understand this is why the --branch flag is useful. Perhaps we could have some docs on the site to outline the problem and the recommended solution, as otherwise it might not be obvious why this happens nor how to fix it?

For future reference, I managed to specify the branch name using the environment variables provided by Travis: TRAVIS_PULL_REQUEST_BRANCH and TRAVIS_BRANCH (https://docs.travis-ci.com/user/environment-variables/#default-environment-variables).

Add tooltips to graph

Problem

There's no information while hovering over the graph in the app that can help verify what build/revision you're about to interact with

Solution

Add tooltips while hovering with a mouse that includes more information

Nice to haves:

• Build revision
• Build date
• Build summary
• Budget warnings & errors

Remove react-native-testing-library

Problem

react-native-testing-library is somewhat unnecessary. The newest version has breaking changes and it's confusing to switch between @testing-library/react and react-native-testing-library, sometimes mixing the two in the same test files

Solution

I think it would be best to remove react-native-testing-library from the @build-tracker/app tests in favor of @testing-library/react

Allow artifact stats to be configurable

Use case

Not all projects are concerned with gzip sizes of artifacts. For example: native iOS and Android applications.

This is a bit 2.0.0 thinking, but I would still like others to weigh in on how this type of thing might be achieved and/or if it's necessary.

Solution

Artifact stats could be configurable via the server application and passed to the frontend application. The application would then give toggles for each type defined.

It could still default to gzip and stat declaring themselves as bytes and work the same as the application does now:

type ArtifactStats = {
  [key: string]: 'bytes' | 'duration'
};

createServer({
  artifactStats: ({ gzip: 'bytes', stat: 'bytes' }: ArtifactStats)
});

Error when uploading/creating build inside Travis job

Thanks again for this fantastic library. I hope my issues are helpful…

I'm filing this issue to help others who might run into the same thing when using Travis. (Perhaps we could add something to the docs?) I've found and noted a fix, so I'll close this immediately.

When I ran bt-cli upload-build inside of a Travis job, I got an error:

$ bt-cli upload-build
Usage: bt-cli upload-build
Create a build
  --branch, -b        Set the branch name and do not attempt to read from git  [string]
  --config, -c        Override path to the build-tracker CLI config file  [string]
  --skip-dirty-check  Skip the git work tree state check  [boolean] [default: false]
Global:
  --help, -h  Show this help screen  [boolean]
  --verbose
Options:
  --version  Show version number  [boolean]
  -v         Set the logging verbosity. Use -vv for more verbose  [count] [default: 0]
{
  code: 128,
  stderr: 'fatal: Not a valid object name origin/master\n',
  errors: { spawn: null, stdout: null, stderr: null }
}

This might have something to do with defaultBranch:

build-tracker/src/cli/src/commands/create-build.ts

Line 82 in f66274b

const defaultBranch = await Git.getDefaultBranch(config.cwd);

build-tracker/src/cli/src/modules/git.ts

Line 14 in f66274b

export function getDefaultBranch(cwd: string = process.cwd()): Promise<string> {

I suspect origin/master doesn't exist because of the way Travis clones the Git repo: it only creates a shallow clone.

I was able to fix this issue by disabling the shallow clone. In .travis.yml:

git:
  depth: false

Github action for posting artifact diff details to PRs

Problem

This repo doesn't have an example of how to get comparison output into its own PRs

Solution

Build a Github action to provide as a comment to each PR, maybe even commit?

Revision details incorrect

Problem

A short explanation of your problem or use-case is helpful!

Steps to Reproduce:

1. Go to https://build-tracker-demo.herokuapp.com/
2. Select any build on the graph
3. Focus the build via the "More info" link in the build header dropdown

Expected Result

All meta fields should display the right data in a human-readable format.

Actual Result

Note bad date. Author shows a number?? Subject is the Author. Revision and ParentRevision are the same value.

Update CLI config to be able to add links for revisions and commits

Problem

One of the neat features of Build Tracker is that each build can link to its revision and/or commit. The demo application at https://build-tracker-demo.herokuapp.com does not link any builds.

Solution

This feature was missed in the CLI configuration. We should be able to add a key like buildUrlFormat?: string that uses path-to-regexp to replace the revision key in the string with the build revision.

Example:

module.exports = {
  buildUrlFormat: 'https://github.com/paularmstrong/build-tracker/commit/:revision`
};

The CLI will then need to pass this information per the Build meta interface

The application/UI is already feature-complete for this 🎉, so no other changes should be necessary.

Add link generator

Should generate links for specific revisions, comparison, toggles, etc.

Show budgets on graph

Problem

When viewing an artifact or group with budget, it would be good to see that on the graph as well.

Solution

This is a difficult problem for UI/UX.

Each individual artifact may have a budget. How should this be visualized? Should it be on every view when that artifact is visible? Should there be an icon at each point for warning/error?

Since groups of artifacts can also have budgets, would it be more useful to provide a warning or error icon for each build that has one or both of those across the entire graph (and not per artifact)?

Could this be done instead of adding tooltips to the graph? It would reduce the overhead, allowing only presenting tooltips for only the builds that are necessary to look at (#12)

Getting Started and the Demo folder

I have postgres running.
I tried to follow Getting Started but cannot get my app to upload a build because it cannot find bt-cli.

I then gave up on Getting Started and switched to Demo folder in your repo.
I cloned your repo.
cd demo
sh build.sh (looks good)
yarn bt-cli upload-build cannot find bt-cli
sh run.sh. (starts server, but I already have that going based on Getting Started
I hit localhost:3000 and I get your cool GUI.

Cannot figure out how to upload one of my builds or your demo build.

[blog]

Problem

The Blog section of the site is blank.

Solution

Ideas:

• Invite a guest blogger to get things moving?
• Update existing first post to get some minimal meat. If the blog is RSS-fed anywhere, better to make new posts instead.
• Add a repost to point out the React Conf talk where this project is showcased.
• Add a post showin some good twitter thread about this project. This might make the project showcase where its community is.

Blog: Add case study

Problem

There are no success stories to help others.

Solution

Add a case-study style blog post that helps others get ideas on how they might set up their own performance budgets.

Admin panel for artifact, group, & budget configuration

Problem

Changing any of the configurations for artifacts, groups, and budgets requires re-deploy of the application. This is mostly fine for some use-cases, but when tracking large applications, modules, bundling, and requirements can change without warning. It would be nice to be able to change this configuration without needing to redeploy.

Solution

• Some sort of admin panel that allows setting various pieces of the application cofiguration
• Considerations:
  • users, permissions, security, etc
  • validation of input data (regexes, etc)

Add build generation

Sample CLI:

build-tracker create-build --dir=dist/static/js --replaceName="\.[a-f0-9]\.js$" --meta.foo="bar" --ext="js" --ext="css"

Expectations:

• Automatically pulls git revision and commit message (and can be turned off)
• Has options to manually provide revision and commit message
• Has --meta.<key> options to provide any extra data
• Accepts a regular expression to replace parts of filenames
• Accepts multiple ext flags to specify the file extensions to read
• Automatically gets the stat and gzip sizes for each artifact
• Automatically calculates a hash for each artifact

Buildtracker support for multiple applications

Problem

In my organization, we have several microservices, so if build tracker could be configured in a way that at one place we can see the graphs for all applications, it would be helpful.

Add ability to link to short revisions

Problem

Git SHAs are 40-characters long. This make URLs annoying to look at when they get generated for CI or shared from the UI.

Solution

• Add a DB column shortRevision, populated if revision matches a SHA1. e.g /^[0-9a-f]{40}$/. The shortRevision should be 7 characters (as is git default for --short option).
• Add migrations to the plugins postgres, mariadb, and mysql to migrate any existing data
• Allow API lookups by shortRevision if the revision is 7 characters long
• Set the default in the application to use shortRevision in the URL any time it is updated

@build-tracker/plugin-with-mongo

Problem

It would be useful to have a plugin that enables you to quickly get set up with a recommended MongoDB database.

Solution

Create a plugin, @build-tracker/plugin-with-mongo that can be used as a compositional wrapper around a server config to easily connect to a Mongo database.

Help Wanted 👍

I haven't had any experience with MongoDB in the past 6 years, so if anyone is interested in using Mongo as their DB backend and has experience, please help!

@build-tracker/plugin-with-postgres

Problem

It would be useful to have a plugin that enables you to quickly get set up with a recommended PostgreSQL database.

Solution

Create a plugin, @build-tracker/plugin-with-postgres that can be used as a compositional wrapper around a server config to easily connect to a PostgreSQL database.

[docs] Add optional best-practice starter?

Problem

To convert visitors to users (especially those coming here from React Europe 2019) they could be offered a sample project to clone and get all the goodies: husky doing eslint, pretty, lint-staged, etc. greenkeeper. snyk? yarn! tsconfig. README, maybe with a link to RDD.

Solution

It looks like this could basically be a copy of your existing build-tracker repo, trimmed down.

Maybe a script could do more or less:

DIR=sample-$(date "+%Y-%m-%d_%H%M")
mkdir $DIR
cd $DIR
git init
git remote add sample-parent https://github.com/paularmstrong/build-tracker.git
git fetch --depth 1 sample-parent next
git merge sample-parent/next
git rm -r src/{comparator,fixtures,formatting,server}
# git rm ....... OK I should've picked some other repo as parent :)

So this would be the basic idea. Probably still keep lerna, to also let people get comfortable with monorepos and @package/separations without causing much developer-unhappiness

Do you have some favorite git repo sample that would be better to start from and add in build-tracker on top? Ideally a repo with all the user-specific fields written in commits on top of a git --depth 1 base, so the user can easily see and edit right in the first commits their project name etc. But stepwise, basic now is better than perfect ...someday.

Update Jest

Jest 22.2.x has support for multi-project config. Would reduce the necessity of using lerna run test

Add message when there are no builds to show

Problem

The default view when first deploying the application results in a blank screen. It's unclear what is happening.

Solution

• A message that the data set is empty
• A link to documentation for help

docs/docs/installation.md for dummies

Is the installation page WIP? It probably needs more words, or hints (links?) to where things are explained.

Specifically, reading the page, I still don't understand these three:

module.exports = {
  // TODO
};

applicationUrl: 'https://your-build-tracker-app',

  // TODO
};

Silent error when server config is missing `artifacts`

Hi, thanks for the great tool!

I am in the process of setting this up for Unsplash.

I've been following the docs but got stuck: the app was not showing any data, despite the fact I had uploaded builds and these were being returned by the API.

After some debugging I discovered the issue: there was an exception happening in the reducer, when processing the BUILDS_SET action. (For some reason the exception is not shown in the console?)

The exception was from this line:

build-tracker/src/app/src/store/reducer.ts

Line 13 in f66274b

artifactBudgets: artifactConfig.budgets,

artifactConfig is undefined at runtime.

To fix this I had to add an empty artifacts object to my server config.

Does this requirement need to be documented, or should the app handle a missing artifacts?

Docs: Update homepage to include more screenshots/features

Problem

The homepage for docs is not clear with the app does.

Solution

Add "features" and list out various stuff.

Indicate unexpected hash changes in Markdown table

Problem

In you inspect this build comparison, you will see that several artifacts have unexpected hash changes. This is clearly indicated in the UI:

Here is the same comparison in the form of a Markdown table (after using an artifact filter to remove rows that didn't change). As you can see, many rows are included but the table doesn't make it clear why they are included. We can see from the UI that they are included because of unexpected hash changes, or because there was a delta but it was too insignificant for any difference to be visible when viewed in KiBs.

	30607b0	0a7b976	Δ1
All	935.24 KiB	936.17 KiB	0.93 KiB (0.1%)
vendors~main	229.32 KiB	230.2 KiB	0.88 KiB (0.4%)
main	89.34 KiB	89.33 KiB	-0 KiB (-0.0%)
explore-old-route	59.95 KiB	59.95 KiB	0 KiB (0.0%)
user-stats-sub-route	19.31 KiB	19.31 KiB	0 KiB (0.0%)
uploader	15.55 KiB	15.6 KiB	0.05 KiB (0.3%)
photos-route	15.54 KiB	15.54 KiB	-0 KiB (-0.0%)
collection-routeeditorial-routeexplore-routefollowing-routephotos-routesearch-photos-routetopi~50ad0392	12.23 KiB	12.23 KiB	0 KiB (0.0%)
user-route	11.36 KiB	11.36 KiB	0 KiB (0.0%)
collection-route	11.31 KiB	11.31 KiB	0 KiB (0.0%)
edit-modal	9.01 KiB	9.01 KiB	-0 KiB (-0.0%)
editorial-route	8.47 KiB	8.47 KiB	0 KiB (0.0%)
campaign-proposal-route	7.85 KiB	7.85 KiB	0 KiB (0.0%)
explore-route	7.34 KiB	7.35 KiB	0 KiB (0.0%)
following-route	6.11 KiB	6.11 KiB	0 KiB (0.0%)
explore-route~topics-route	5.57 KiB	5.57 KiB	0 KiB (0.0%)
collections-feed-route	5.27 KiB	5.27 KiB	0 KiB (0.0%)
search-photos-route	5.01 KiB	5.01 KiB	0 KiB (0.0%)
collection-routephotos-routeuser-feed-sub-route	4.32 KiB	4.32 KiB	0 KiB (0.0%)
search-collections-route	3.88 KiB	3.89 KiB	0 KiB (0.0%)
feed	1.34 KiB	1.34 KiB	0 KiB (0.0%)

Solution

Add some sort of indication to rows in the Markdown table which are included because of unexpected hash changes.

Add proper response output to insert build routing

Something like...

{
  budgets: {
    groups: { [key: string]: BudgetResponse },
    artifacts: { [key: string]: BudgetResponse }
  },
  comparator: comparator.toJSON()
}

Add ability to visualize build artifact deltas on the graph

Allow changing size key view from drawer

npx bt-cli create-build gives TypeError: cannot read property 'forEach' of undefined

Problem

Trying to upload a build.

Steps to Reproduce:

0. npm run build (builds my webpack bundle in /dist/

1. In one cmd prompt: npx bt-server run

{"level":30,"time":1574359053032,"pid":1239,"hostname":"johns-mbp.lan","msg":"Build Tracker server running on port 3000","v":1}

2. npx bt-cli create-build

Expected Result

I'm not sure...

It's not clear why the server and the client need a url to my app. Should I be running an http-server so that my production build is served up? The docs make it sound like the server is doing that..."Your application will now be accessible at the provided url."

Actual Result

Here's what I actually see when I run the above:

TypeError: Cannot read property 'forEach' of undefined
    at Object.<anonymous> (/Users/johnmairs/Documents/gear/node_modules/@build-tracker/cli/dist/commands/stat-artifacts.js:75:31)
    at step (/Users/johnmairs/Documents/gear/node_modules/@build-tracker/cli/dist/commands/stat-artifacts.js:32:23)
    at Object.next (/Users/johnmairs/Documents/gear/node_modules/@build-tracker/cli/dist/commands/stat-artifacts.js:13:53)
    at fulfilled (/Users/johnmairs/Documents/gear/node_modules/@build-tracker/cli/dist/commands/stat-artifacts.js:4:58)

Here's the contents of my webpack dist folder

johnmairs@johns-mbp gear (213-addBuildTracker) $ ls dist/
1c8b268ec020a3aa5cd211cdda05fe2b.png	bundle.js.br				main.9847c6831b516c2e5a76.css.br
200b3fc810ff360a023c4cc309f77e5a.png	bundle.js.map				main.9847c6831b516c2e5a76.css.map
7a564d5604efdab6ca2325dcced48457.png	favicon.ico				mapTiles
995a1d96b1fef026c61ac03f4c6ff841.png	index.html				report.html
bundle.js				main.9847c6831b516c2e5a76.css

Here's my server config (build-tracker.config.js):

const withPostgres = require('@build-tracker/plugin-with-postgres').default;

module.exports = withPostgres({
  url: 'https://localhost:443',
  pg: omitted
});

Here's my client config (build-tracker-cli.config.js):

module.exports = {
  applicationUrl: 'https://localhost:443', // The same as your server config `url`
  artifacts: ['dist/**/*.js'], // an Array of glob-style file paths
};

Use declaration files for TypeScript types (instead of source code)

The types field in the package.json of @build-tracker/plugin-with-postgres currently refers to the TS source code:

build-tracker/plugins/with-postgres/package.json

Line 10 in f66274b

"types": "index.ts",

This means that TS users who import from this module will get lots of errors relating to the source code.

It would be better if the types field instead just referred to some declaration files (.d.ts). These could be generated using the declaration compiler option. This way, users will only import the types—they won't be forced to type check the actual source code itself.

I'm happy to raise a PR if that sounds good to you?

Add custom meta information with bt-cli

Problem

I want to add some custom Meta information to Builds. There is no option for this.

Solution

• add --meta option to bt-cli create-build and upload-build commands. The value should be either a string-value or a JSON-encoded BuildMetaItem

Make "Deploy to Heroku" button

Problem

Deploying and getting started requires reading a decent amount of the docs.

Solution

Create "deploy to heroku" button and add to the readme, so folks can quickly spin up a server. If more config is required, maybe a little SPA to generate the config based on some options might be required.

install docs show cli using build-tracker.config.js when it should be build-tracker-cli.config.js

Problem

Getting Started and CLI sections both say create a config file and name it build-tracker.config.js which is the server one not the cli one.

Steps to Reproduce:

2. ...

Expected Result

Here's what I expect to see when I run the above:

Actual Result

Here's what I actually see when I run the above:

Add ability to get deltas for groups of artifacts

Add bar chart option

Problem

When using an isolated comparison of just 2 or 3 builds, area charts aren't very effective. They're difficult to view and follow.

Solution

Build Tracker pre-1.0 had an option to switch to a bar chart for this very reason. It also auto-selected based on the number of builds you were viewing.

Comparator generated by API insert: builds are in wrong order

Problem

A short explanation of your problem or use-case is helpful!

Steps to Reproduce:

If I have two commits, A and B, the generated Markdown looks like this:

Expected Result

	A	B	Δ1
All	1,108.14 KiB	1,108.2 KiB	0.06 KiB (-0.0%)
main	126.08 KiB	126.13 KiB	0.06 KiB (-0.0%)

Actual Result

	B	A	Δ1
All	1,108.2 KiB	1,108.14 KiB	-0.06 KiB (-0.0%)
main	126.13 KiB	126.08 KiB	-0.06 KiB (-0.0%)

featuring build-tracker in Docusaurus

Hello, we would like to feature your project to the Docusaurus users

If you are okay with it, are you able to provide a logo we can use? Additional information can be found here

@build-tracker/plugin-with-mysql

Problem

It would be useful to have a plugin that enables you to quickly get set up with a recommended MySQL database.

Solution

Create a plugin, @build-tracker/plugin-with-msyql that can be used as a compositional wrapper around a server config to easily connect to a MySQL database.

Cannot find module '@build-tracker/formatting'

Problem

I am trying to install the Build Tracker server following the steps in the documentation but when trying to run bt-server i am getting:

Error: Cannot find module '@build-tracker/formatting'

Steps to Reproduce:

1. npm install --save @build-tracker/server@latest
2. create build-tracker.config.js on project root with:

module.exports = {
  url: 'https://my-url',
};

3. add "start": "bt-server run" to the scripts section of package.json
4. npm start

Safari refuses to load scripts

Problem

In Safari (I'm using Tech Preview Release 81 (Safari 12.2, WebKit 14608.1.19.5)) it seems like CSP is enforced more thoroughly, and some scripts are not allowed to be loaded

Steps to Reproduce:

1. open demo in safari

2. click on a place on the graph

Expected Result

Here's what I expect to see when I run the above:

a new item being added to the comparison list

Actual Result

Here's what I actually see when I run the above:

`getParentRevision` returns invalid revision when parent is a merge commit

All of our commits on our master branch are merge commits, which means they have two parents. Because of this, getParentRevision returns two commits when it's expected to only return one:

$ git log --pretty=%P -n 1 master
143abcfc1c4416d2f1d045f309aadbd7baf2f7b2 ece9964638c730e9be046c6d23187b9b57b243bd

This in turn means we get an error when trying to upload the build, because the parent revision is too long to fit into the database (max characters is 64).

What should happen in this case? I guess we have to pick one of the merge commit's parents to act as the "main parent".

Add URL/routing to app

Likely a bit of a significant refactor. This will require moving local state from <Main /> to a higher level that passes props for active & focused revisions, enabled/disabled artifacts, etc.

Problem

There's no way to link to a specific view of builds

Solution

Step 1: Make the application stateful with routing

Don't show date range unless manually set

accept/use artifact filters and thresholds in ascii/csv output

POST /api/builds responds with 404

Problem

A short explanation of your problem or use-case is helpful!

Steps to Reproduce:

1. With a fresh database, run bt-cli upload-build

Expected Result

200 response

Actual Result

404 response

This seems to happen as a result of queries.byRevision(build.getMetaValue('parentRevision')). In this case, it seems that there is no build for the parent revision, because this is the first time a build has been uploaded.

When there is no parent for the "parent revision", perhaps we should still respond with a 200? WDYT?

Show ActivityIndicator when loading data

Budgets silently fail when there is no "parent build"

Problem

When using budgets, if there is no parent build and the artifacts from the current build fail the defined budgets, comparator will not complain.

I suspect this is because of

build-tracker/src/comparator/src/index.ts

Line 237 in 8f1df8c

return this.builds.slice(0, i).map(prevBuild => {

Most of the time this won't be an issue, as there will be a parent build, but it could confuse people (like me) who are just getting started with this tool.

I guess it doesn't make sense to run delta budgets when there is no parent build, but it does make sense to run standard size budgets?

Expected Result

Here's what I expect to see when I run the above:

Actual Result

Here's what I actually see when I run the above: