deathbeds/jupyak

A tool for building JupyterLite preview sites from Jupyter PRs

Deathbeds is an open digital folk community sharing the bleeding edges of open source and open science.

• Sign up for the deathbeds newsletter.
• Donate to deathbeds.
• Deathbeds/Future Warning Calendar
• Deathbeds on opencollective

Deathbeds friends and family night.

Supported Projects

Bleeding Edge Projects

Upcoming presentations

Past presentations

jupyak's Issues

Add badges from CI

Follow some of the patterns from JupyterLite, etc. to add an RTD badge to PRs.

Investigate single yarn workspace

elevator pitch

See if it's possible to only need to do a single yarn install.

motivation

While the build time is dominated by webpack (and wrappers thereof), running yarn install 7 times is quite expensive.

design ideas

During env, discover all of the package.json files and build a work/package.json, and run a single yarn.

Sadly, a lot of things (in .js, .ts, .json, .toml and .py files) depend on files being deployed in a specific place in {cwd}/node_modules, so may require a lot of weird patching magic. Further, some packages run their own yarn install inside other scripts or elsewhere.

Add jupyter(lab)-lsp to stack

Elevator Pitch

Add jupyter-lsp python and js packages to the stack of built packages.

Motivation

Evaluating LSP in JupyterLite will likely require some upstream changes.

Design Ideas

Aside from a few interrelated steps, such as generating js types from schema, this should be relatively straightforward.

Implement some kind of form-based PR building solution

Elevator Pitch

Restore the accessible form-based solution to the GitHub Pages build in the docs, as mentioned in the docs.

Motivation

Part of the original idea of this repo was to reduce the friction from

I have two PR URLs, and want to see them working together

to

Here is a URL for the most recent state of those PRs

Design Ideas

Originally, this repo started as a JSON schema for jupyak_config.json, but moved to a traitlets-based model as the number of derived values became more complicated. A static JSON schema for the rough form of the system is likely a good thing to get back to, even if it's not wholly accurate.

Using an existing tool like rjsf ended up being pretty heavyweight to build, while a jinja-based HTML form had similar complexity.

Another issue (ha) was that the GitHub Issue Forms are... not really JSON Schema, and the resulting UI was not pleasing... scoping down to a smaller number of fields might be useful.

As things are ... easier if the requester owns the PR, using the "New File" URL params is probably easiest:

https://github.com/deathbeds/jupyak/new/main?filename=jupyak_config.toml&value=URL%20ENCODED%20TOML

Include previous lite builds in case of recoverable failure

elevator pitch

If a full shave build passes, and then fails on the next build (but sphinx still runs), try to include the last-known-good build on all future failing builds.

motivation

On both GitHub Pages and ReadTheDocs, once a build is overwritten, it's gone. As we collect and re-report shave failures as a best-effort human-readable site, this means a previously-working, and still inspect-able product might be overwritten with boring error messages.

design ideas

At the end of a full shave, jupyter lite archive makes an npm-compatible tarball, currently always with the name work/dist/lite/lite-jupyterlite.tgz, which also gets uploaded to the host, e.g.

• https://deathbeds.github.io/jupyak/_static/work/lite/lite-jupyterlite.tgz
• https://jupyak--{:pr-number}.org.readthedocs.build/en/16/_static/work/lite/lite-jupyterlite.tgz

It may also be desirable to include the "version" (e.g. PR number), e.g.

• _static/work/lite/jupyak-{:pr-number}.tgz

Any further information (e.g. git hash, info about the repos) is likely not a good idea, though more information (such as build logs) could easily be brought over in a similar manner without changing the lite archive, a useful artifact in its own right.

Based on state of the work folder, and well-known environment variables, a failing sweep-up job can:

• derive the URL of a previous success (or previous pulled-along failure)
• check if it exists
• download it
• extract it into e.g. work/dist/last-known-good/
• copy itself into it
• point to it in e.g. the build info sidebar in the HTML site