datadesk/baker

A build tool by and for the Los Angeles Times

baker's Issues

Add i18n support to datefilter

Example is here

Add a "date" template filter that allows for reformatting a datetime string

We need this to be able to auto populate machine readable fields that require different date formats. We probably would want it anyway because it's just a handy thing to have when building pages.

There are some off-the-shelf ones on npm that ship with moment. We could also roll our own. I'm a bear, I don't care. Either way sounds good to me.

Q: Is there a way to pass `true` or `false` into ArchieML meta files and have them converted into booleans

I suspect there is some kindergarten way to do this I am simply ignorant of.

Failed builds should always throw the proper error code

If it's not possible for other tools to determine that baker failed to build, that's no bueno.

Move to server-based dev environment

The "render everything in dev" approach is outdated and has become untenable in major projects such as the coronavirus tracker and elections. There are a lot of little details that'd go into getting this right, but it's time to get serious about having pages and assets build/render on the fly based on the page that's currently being viewed.

New features like the {% asset %} tag are already a move more toward doing work as needed instead of ahead of time. CSS/SCSS processing and JS processing will also have to be reworked to be more compatible with this, but overall this is the direction we should go anyway for UX reasons.

The biggest question IMO is whether this attempts to land independent of the major changes happening in the next branch or not.

Baker should be smart enough to find a new port if 3000 is taken

And it should yell very loudly in the console that it had to be done.

The {% static %} tag should ignore and pass through absolute URLs

Actually the real solution is to completely migrate away from the static tag, but this probably would make sense for now. Maybe it warns you? I dunno. But it'd be nice (as @andyroberson raised) to be able to pass a variable in and not have to care whether it was local or not.

Template tags should be able to "emit" files in a way Baker can process

I'm thinking of something very, very similar to Rollup's this.emitFile method. Right now all a tag can do is take in a string and return a string, which means if it'd want to, for example, output more than one file (say, an {% image %} tag that creates multiple images for srcset) it would be impossible.

Swap in @babel/preset-modules for modern build

Will make our modern build even smaller!

https://github.com/babel/preset-modules

The dev server should warn when static files aren't found

Currently the only way you learn about a broken {% static %} link is... you don't. Until a build attempts to happen at least, or you are waiting for a successful deploy and it can't do it.

We should make the dev server warn when {% static %} tags it attempted to resolve didn't have a valid file. (It currently just allows it to happen quietly.) The idea was you shouldn't have the server crash and burn while in development just because you made a typo, but it should be louder about it.

The requested module 'totalist' is expected to be of type CommonJS

Our AWS buildbot threw this error when trying to release harris-polls

2021-08-01T23:39:25.032-07:00 | > page-template@ build /usr/src/app
  | 2021-08-01T23:39:25.032-07:00 | > NODE_ENV=production bake build --config
  | 2021-08-01T23:39:26.131-07:00 | Build failed. Here's what possibly went wrong:
  | 2021-08-01T23:39:26.132-07:00 | The requested module 'totalist' is expected to be of type CommonJS, which does not support named exports. CommonJS modules can be imported by importing the default export.
  | 2021-08-01T23:39:26.132-07:00 | For example:
  | 2021-08-01T23:39:26.132-07:00 | import pkg from 'totalist';
  | 2021-08-01T23:39:26.132-07:00 | const { totalist } = pkg;
  | 2021-08-01T23:39:26.145-07:00 | npm ERR! code ELIFECYCLE
  | 2021-08-01T23:39:26.145-07:00 | npm ERR! errno 1
  | 2021-08-01T23:39:26.147-07:00 | npm ERR! page-template@ build: `NODE_ENV=production bake build --config`
  | 2021-08-01T23:39:26.147-07:00 | npm ERR! Exit status 1
  | 2021-08-01T23:39:26.147-07:00 | npm ERR!
  | 2021-08-01T23:39:26.147-07:00 | npm ERR! Failed at the page-template@ build script.
  | 2021-08-01T23:39:26.147-07:00 | npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
  | 2021-08-01T23:39:26.159-07:00 | npm ERR! A complete log of this run can be found in:
  | 2021-08-01T23:39:26.159-07:00 | npm ERR! /home/node/.npm/_logs/2021-08-02T06_39_26_148Z-debug.log
  |  

Add ability to provide generic context to templates

Right now the only way for context to be provided to a template is if it gets consumed via the _data directory. We'll probably want a less convoluted way to get extra variables into templates (and process.env.* variables in JavaScript) without having to roll the dice in that directory.

When someone tries to use a JS entrypoint that's not configured, it should tell you

Right now you'll just get a cryptic error about the JS file the template tries to load being missing, but we know the truth and should tell the user — they may have forgot to pass in the entrypoint!

Integrate Google Docs into _data folder system

Is there a way to catch template undefined errors and give more context?

It's incredibly ambiguous what's missing when a Nunjucks template crashes and burns.

Directory index for the dev server

Right now it's kind of a mystery what pages could be found, especially if you don't have a base index.html. This wouldn't solve the problem of finding other pages if you are on the default one, but it'd be helpful to have.

Support for adding new global variables via baker.config.js

Rewrite in TypeScript

It's gonna be a beast to pull this off, but I think it'd be net positive if we do.

Images should be read in as streams not buffers

It's fairly easy for an image to get too big and overflow the max buffer limit. (Of course no one should have images that big in the first place. But it'll happen.)

Engines need to be able to build manifests even if they aren't active

The assets engine doesn't run during a serve, but when it doesn't run this means blocks that expect to tap into its manifest cannot do it. This currently breaks the inject block when you attempt to use it with any file that'd be in the assets directory.

Is there a way to use the static tag for hashed assets in JavaScript?

We got it working in HTML, we got it working in Sass — this is the final boss.

A flag for a "modern only" build

It should be possible to say "don't worry about transpiling/outputting any JavaScript for old browsers."

It might be nice to be able to pass a port to the start command

Something like

npm start 8001

This would allow me to easily start multiple baker test servers in multiple tabs at once.

A config setting for the "src" directory where the templates and pages are rooted

This would allow nesting the pages below the root when you have a lot of templates.

Explore using filename driven dynamic page generation

Next.js has this cool method for rendering dynamic pages by adding extra info in the file. Sapper has a version of it too. Eleventy does it in its front matter but I find this incredibly confusing.

These all expect you to write some additional JS to react to that situation, but I wonder if we could either 1) come up with a solution that doesn't require that, and can determine everything directly in the file name, or 2) have our own version of front matter/pre-load JS that fills out the necessary fields.

I think the most ideal scenario would be to have the template and the instructions for the template as close as possible to each other if not in the same file.

_data file validation should throw more explicit errors

Find way to remove the polyfills

This makes some big assumptions about what polyfills you need in a given project, and while that'd normally be good, this will make it harder for other folks to consider using this — what if their use case is different from ours?

create-react-app has a separate polyfill install they use to do something similar to this. Our situation is trickier however because we need two separate outputs (which means two separate polyfills).

How can we support Svelte's use of CSS in Baker?

Svelte has native support for writing CSS which we currently have disabled. (You can write it but it won't appear anywhere.) Ideally folks could use it though. I think it would require having nothing to do with the Sass we use, but for one off scripts that'd probably be fine.

It should be possible to reference static assets in CSS

h/t @caseymm

Currently the only way to get a path to a hashed asset file is with the {% static %} tag in Nunjucks. If you want to use them in CSS (or JavaScript, for that matter), it's currently not possible.

A few options:

1. The most straightforward path would be allow (S)CSS and JS files to be additionally passed through the Nunjucks renderer. It'd work, but oof is it nasty. We don't want to encourage people to start writing loops and other crazy things in files that aren't HTML. If I knew I could disable that stuff, maybe it'd work.

2. Use a more simple find and replace approach that attempts to re-path any mentions of assets in CSS and ensure they point at the correct place. I used this trick in data-visuals-create and it went okay. The biggest challenge is ensuring the paths correctly match. Being off by even one character is enough to make it fail.

Some method to generate full URLs for meta tags and static files

Some meta tags insist on a full URL to share images and other resources. Others want to know your page's canonical URL and your site's bare domain. Somehow we need to get these into our templates.

An experimental downstream pull request in bakery-demo solved this problem by introducing a set of nunjuck macros. It works, but it would be cool if the problem could be solved here instead.

I could very well be wrong, but I believe at least the following stuff will need to be in the mix here for it to happen.

• The site domain
• Any base path prefixes, like /projects/
• The page slug

It should be okay to not have a JavaScript entrypoint

Rollup currently gets mad if you don't.

You must supply options.input to rollup

Add TypeScript support to Svelte

It's super easy to do now thanks to svelte-preprocess.

Sitemap-like functionality

Baker projects should be aware of all its possible HTML (and other?) outputs and be able to surface that information for you. This would help with creating relative link helpers and other sorts of navigation building tools.

When your build breaks because the `static` tag can't find an image, it would be better if it told you which image was missing

This is a build breaker I've seen a couple times, including tonight, when we ended up rooting around for a half hour just to figure out the social_image tag in the meta file two levels up was failing to resolve via the static tag.

Add support for dynamic pages

It should be possible to create pages dynamically via a data file. The best example I've seen of this is with Middleman. You should be able to provide a data set that "fills in the blanks" of a template and generates multiple outputs.

pathPrefix should always have a leading slash

And baker should make sure it's there. If we don't, if you try to pass it to path.resolve (like we do in some of the custom blocks) it'll get resolved against the current working directory, which is no bueno.

Make jsonScript native to baker

This is a little tricky because it requires a client-side script too to be used properly. While we can obviously put it directly into bakery-template it probably makes a little more sense to somehow be packaged here. Or maybe it lives in cookbook?

Add option to preserve whitespace and prevent nunjucks engine from collapsing

There are circumstances, like 1920s modernist poetry, for example, where preserving the whitespace of the HTML is necessary.

I propose some kind of baker.config setting or other environment variable that can globally switch off the hardcoded configuration found here.

I doubt this would be difficult, but I can imagine several different ways to implement it. I'm happy to take a stab at the patch, but I wanted to ask beforehand in case you had opinions on the proper route to take.

Support for Django-style "context processors" passed into baker.config.js

Upgrade to support pure ESM packages like imagemin

My reading of the recent imagemin release notes suggests we'd need to refactor along these lines to support future versions.

As far as I can tell, this is not an immediate need, but probably something we should considering at some point down the road. I suspect @rdmurphy is miles ahead of me on the thinking here.

If for nothing else, I'm filing this ticket to record what I've learned looking at some dependabot errors and release notes this morning.

What's the best way to choose what HTML pages get built/deployed?

Say you're working on a multi-page project that's going to release in stages. How do you work on a future page while parts of the project are already live? It seems like there needs to be a way to tell Baker to not deploy something while allowing it to exist in the project.

A few ideas:

Deploy/no-deploy globs

There could be another config option available where you explicitly say which pages should be going live. The issue with using globs is that, well, everyone needs to understand globs. Could possibly be opt-in or opt-out — in other words you could be required to denote which pages go live, and the default template already flags the default index.html.

A new file naming structure

Another option would be to overload the file name in a new way similar to how we currently use underscores. Maybe you do something like !page.html or #page.html and it knows to skip that. I have no clue if operating systems would allow file names like that, however. This also isn't very compatible with the aspirations of dynamic page generation via #142.

Any other ideas?

Finish up the file contents "inject" block

It's technically "here" but barely.

Need this so it's possible to use SCSS to generate CSS that can be injected into an AMP page. Wooooooo.

The dev server lies about when its ready

If it's taking longer than normal to prep a dev version of a project, it shouldn't tell you the URL to go visit before everything is golden.

Race condition may cause CURRENT_URL to be incorrect in multi-builds

Due to the async method used to render Nunjucks templates, it's possible for the CURRENT_URL to have already changed to another file's URL before a template can render.

starting build /Users/ryan/Projects/datagraphics/elections-2020-frontend/embed.html
starting build /Users/ryan/Projects/datagraphics/elections-2020-frontend/index.html
starting build /Users/ryan/Projects/datagraphics/elections-2020-frontend/precincts.html
^ this is the last one they all use
file ready /Users/ryan/Projects/datagraphics/elections-2020-frontend/embed.html
file ready /Users/ryan/Projects/datagraphics/elections-2020-frontend/precincts.html
file ready /Users/ryan/Projects/datagraphics/elections-2020-frontend/index.html

Is it possible to inject error logs into the browser?

I've seen it done before with a few tools.

create-react-app has this error-overlay library which is terribly convoluted but gets the job done. Wonder if we could raid it for parts to pull this off?

We need the ability to pass parameters to the CLI via environment variables

This came up when attempting to automate deployed via the CI. Right now the only way to provide a custom base path is with a CLI parameter. But sometimes we may want to be able to do this entirely with an environment variable so we can trust it gets seen when a bake build gets ran.

New files should get added to watcher without server restart

I notice this the most with Sass files. You'll add a new dependency file and add it to the app.scss entrypoint (which is getting watched), and slowly drive yourself mad because changes to that added file are not being watched.

Template tag errors are getting eaten

Template tag errors are quietly not raising and just stopping builds. My gut it's an issue with the tag mechanism itself, but always a chance I broke it elsewhere.

"staticRoot" shouldn't be a required field

Right now when you setup a new Baker instance it will crash and burn if you don't pass in staticRoot, even when you want the default of "use the current directory."

Investigate replacing image processing with jimp

We hit a number of issues getting baker working in less flexible environments — like random Alpine Linux Docker containers we have no control over. If we could remove the image plugins with jimp it'd clear it up.