datadesk / baker Goto Github PK
View Code? Open in Web Editor NEWA build tool by and for the Los Angeles Times
License: MIT License
A build tool by and for the Los Angeles Times
License: MIT License
Example is here
We need this to be able to auto populate machine readable fields that require different date formats. We probably would want it anyway because it's just a handy thing to have when building pages.
There are some off-the-shelf ones on npm that ship with moment. We could also roll our own. I'm a bear, I don't care. Either way sounds good to me.
I suspect there is some kindergarten way to do this I am simply ignorant of.
If it's not possible for other tools to determine that baker failed to build, that's no bueno.
The "render everything in dev" approach is outdated and has become untenable in major projects such as the coronavirus tracker and elections. There are a lot of little details that'd go into getting this right, but it's time to get serious about having pages and assets build/render on the fly based on the page that's currently being viewed.
New features like the {% asset %}
tag are already a move more toward doing work as needed instead of ahead of time. CSS/SCSS processing and JS processing will also have to be reworked to be more compatible with this, but overall this is the direction we should go anyway for UX reasons.
The biggest question IMO is whether this attempts to land independent of the major changes happening in the next
branch or not.
And it should yell very loudly in the console that it had to be done.
Actually the real solution is to completely migrate away from the static
tag, but this probably would make sense for now. Maybe it warns you? I dunno. But it'd be nice (as @andyroberson raised) to be able to pass a variable in and not have to care whether it was local or not.
I'm thinking of something very, very similar to Rollup's this.emitFile
method. Right now all a tag can do is take in a string and return a string, which means if it'd want to, for example, output more than one file (say, an {% image %}
tag that creates multiple images for srcset
) it would be impossible.
Will make our modern build even smaller!
Currently the only way you learn about a broken {% static %}
link is... you don't. Until a build attempts to happen at least, or you are waiting for a successful deploy and it can't do it.
We should make the dev server warn when {% static %}
tags it attempted to resolve didn't have a valid file. (It currently just allows it to happen quietly.) The idea was you shouldn't have the server crash and burn while in development just because you made a typo, but it should be louder about it.
Our AWS buildbot threw this error when trying to release harris-polls
2021-08-01T23:39:25.032-07:00 | > page-template@ build /usr/src/app
| 2021-08-01T23:39:25.032-07:00 | > NODE_ENV=production bake build --config
| 2021-08-01T23:39:26.131-07:00 | Build failed. Here's what possibly went wrong:
| 2021-08-01T23:39:26.132-07:00 | The requested module 'totalist' is expected to be of type CommonJS, which does not support named exports. CommonJS modules can be imported by importing the default export.
| 2021-08-01T23:39:26.132-07:00 | For example:
| 2021-08-01T23:39:26.132-07:00 | import pkg from 'totalist';
| 2021-08-01T23:39:26.132-07:00 | const { totalist } = pkg;
| 2021-08-01T23:39:26.145-07:00 | npm ERR! code ELIFECYCLE
| 2021-08-01T23:39:26.145-07:00 | npm ERR! errno 1
| 2021-08-01T23:39:26.147-07:00 | npm ERR! page-template@ build: `NODE_ENV=production bake build --config`
| 2021-08-01T23:39:26.147-07:00 | npm ERR! Exit status 1
| 2021-08-01T23:39:26.147-07:00 | npm ERR!
| 2021-08-01T23:39:26.147-07:00 | npm ERR! Failed at the page-template@ build script.
| 2021-08-01T23:39:26.147-07:00 | npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
| 2021-08-01T23:39:26.159-07:00 | npm ERR! A complete log of this run can be found in:
| 2021-08-01T23:39:26.159-07:00 | npm ERR! /home/node/.npm/_logs/2021-08-02T06_39_26_148Z-debug.log
|
Right now the only way for context to be provided to a template is if it gets consumed via the _data
directory. We'll probably want a less convoluted way to get extra variables into templates (and process.env.*
variables in JavaScript) without having to roll the dice in that directory.
Right now you'll just get a cryptic error about the JS file the template tries to load being missing, but we know the truth and should tell the user — they may have forgot to pass in the entrypoint!
It's incredibly ambiguous what's missing when a Nunjucks template crashes and burns.
Right now it's kind of a mystery what pages could be found, especially if you don't have a base index.html
. This wouldn't solve the problem of finding other pages if you are on the default one, but it'd be helpful to have.
It's gonna be a beast to pull this off, but I think it'd be net positive if we do.
It's fairly easy for an image to get too big and overflow the max buffer limit. (Of course no one should have images that big in the first place. But it'll happen.)
The assets engine doesn't run during a serve, but when it doesn't run this means blocks that expect to tap into its manifest cannot do it. This currently breaks the inject
block when you attempt to use it with any file that'd be in the assets
directory.
We got it working in HTML, we got it working in Sass — this is the final boss.
It should be possible to say "don't worry about transpiling/outputting any JavaScript for old browsers."
Something like
npm start 8001
This would allow me to easily start multiple baker test servers in multiple tabs at once.
This would allow nesting the pages below the root when you have a lot of templates.
Next.js has this cool method for rendering dynamic pages by adding extra info in the file. Sapper has a version of it too. Eleventy does it in its front matter but I find this incredibly confusing.
These all expect you to write some additional JS to react to that situation, but I wonder if we could either 1) come up with a solution that doesn't require that, and can determine everything directly in the file name, or 2) have our own version of front matter/pre-load JS that fills out the necessary fields.
I think the most ideal scenario would be to have the template and the instructions for the template as close as possible to each other if not in the same file.
This makes some big assumptions about what polyfills you need in a given project, and while that'd normally be good, this will make it harder for other folks to consider using this — what if their use case is different from ours?
create-react-app
has a separate polyfill install they use to do something similar to this. Our situation is trickier however because we need two separate outputs (which means two separate polyfills).
Svelte has native support for writing CSS which we currently have disabled. (You can write it but it won't appear anywhere.) Ideally folks could use it though. I think it would require having nothing to do with the Sass we use, but for one off scripts that'd probably be fine.
h/t @caseymm
Currently the only way to get a path to a hashed asset file is with the {% static %}
tag in Nunjucks. If you want to use them in CSS (or JavaScript, for that matter), it's currently not possible.
A few options:
The most straightforward path would be allow (S)CSS and JS files to be additionally passed through the Nunjucks renderer. It'd work, but oof is it nasty. We don't want to encourage people to start writing loops and other crazy things in files that aren't HTML. If I knew I could disable that stuff, maybe it'd work.
Use a more simple find and replace approach that attempts to re-path any mentions of assets in CSS and ensure they point at the correct place. I used this trick in data-visuals-create
and it went okay. The biggest challenge is ensuring the paths correctly match. Being off by even one character is enough to make it fail.
Some meta tags insist on a full URL to share images and other resources. Others want to know your page's canonical URL and your site's bare domain. Somehow we need to get these into our templates.
An experimental downstream pull request in bakery-demo solved this problem by introducing a set of nunjuck macros. It works, but it would be cool if the problem could be solved here instead.
I could very well be wrong, but I believe at least the following stuff will need to be in the mix here for it to happen.
/projects/
Rollup currently gets mad if you don't.
You must supply options.input to rollup
It's super easy to do now thanks to svelte-preprocess
.
Baker projects should be aware of all its possible HTML (and other?) outputs and be able to surface that information for you. This would help with creating relative link helpers and other sorts of navigation building tools.
This is a build breaker I've seen a couple times, including tonight, when we ended up rooting around for a half hour just to figure out the social_image
tag in the meta file two levels up was failing to resolve via the static tag.
It should be possible to create pages dynamically via a data file. The best example I've seen of this is with Middleman. You should be able to provide a data set that "fills in the blanks" of a template and generates multiple outputs.
And baker
should make sure it's there. If we don't, if you try to pass it to path.resolve
(like we do in some of the custom blocks) it'll get resolved against the current working directory, which is no bueno.
This is a little tricky because it requires a client-side script too to be used properly. While we can obviously put it directly into bakery-template
it probably makes a little more sense to somehow be packaged here. Or maybe it lives in cookbook?
There are circumstances, like 1920s modernist poetry, for example, where preserving the whitespace of the HTML is necessary.
I propose some kind of baker.config setting or other environment variable that can globally switch off the hardcoded configuration found here.
I doubt this would be difficult, but I can imagine several different ways to implement it. I'm happy to take a stab at the patch, but I wanted to ask beforehand in case you had opinions on the proper route to take.
My reading of the recent imagemin release notes suggests we'd need to refactor along these lines to support future versions.
As far as I can tell, this is not an immediate need, but probably something we should considering at some point down the road. I suspect @rdmurphy is miles ahead of me on the thinking here.
If for nothing else, I'm filing this ticket to record what I've learned looking at some dependabot errors and release notes this morning.
Say you're working on a multi-page project that's going to release in stages. How do you work on a future page while parts of the project are already live? It seems like there needs to be a way to tell Baker
to not deploy something while allowing it to exist in the project.
A few ideas:
There could be another config option available where you explicitly say which pages should be going live. The issue with using globs is that, well, everyone needs to understand globs. Could possibly be opt-in or opt-out — in other words you could be required to denote which pages go live, and the default template already flags the default index.html
.
Another option would be to overload the file name in a new way similar to how we currently use underscores. Maybe you do something like !page.html
or #page.html
and it knows to skip that. I have no clue if operating systems would allow file names like that, however. This also isn't very compatible with the aspirations of dynamic page generation via #142.
Any other ideas?
It's technically "here" but barely.
Need this so it's possible to use SCSS to generate CSS that can be injected into an AMP page. Wooooooo.
If it's taking longer than normal to prep a dev version of a project, it shouldn't tell you the URL to go visit before everything is golden.
Due to the async method used to render Nunjucks templates, it's possible for the CURRENT_URL
to have already changed to another file's URL before a template can render.
starting build /Users/ryan/Projects/datagraphics/elections-2020-frontend/embed.html
starting build /Users/ryan/Projects/datagraphics/elections-2020-frontend/index.html
starting build /Users/ryan/Projects/datagraphics/elections-2020-frontend/precincts.html
^ this is the last one they all use
file ready /Users/ryan/Projects/datagraphics/elections-2020-frontend/embed.html
file ready /Users/ryan/Projects/datagraphics/elections-2020-frontend/precincts.html
file ready /Users/ryan/Projects/datagraphics/elections-2020-frontend/index.html
I've seen it done before with a few tools.
create-react-app
has this error-overlay
library which is terribly convoluted but gets the job done. Wonder if we could raid it for parts to pull this off?
This came up when attempting to automate deployed via the CI. Right now the only way to provide a custom base path is with a CLI parameter. But sometimes we may want to be able to do this entirely with an environment variable so we can trust it gets seen when a bake build
gets ran.
I notice this the most with Sass files. You'll add a new dependency file and add it to the app.scss
entrypoint (which is getting watched), and slowly drive yourself mad because changes to that added file are not being watched.
Template tag errors are quietly not raising and just stopping builds. My gut it's an issue with the tag mechanism itself, but always a chance I broke it elsewhere.
Right now when you setup a new Baker instance it will crash and burn if you don't pass in staticRoot
, even when you want the default of "use the current directory."
We hit a number of issues getting baker
working in less flexible environments — like random Alpine Linux Docker containers we have no control over. If we could remove the image plugins with jimp it'd clear it up.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.