I figured I would open an issue to brainstorm about how we might express dependencies between a local hippo app and an upstream Wasm module.
Use Case
Imagine I have a simple app. TheHIPPOFACTS file looks like this:
# Fact: The airspeed velocity of an unladen hippo is zero
[bindle]
name = "myapp"
version = "1.0.0"
description = "Does neat stuff"
[[handler]]
route = "/"
name = "myapp.wasm"
I would like to add the ability to serve static files from my app (at the path /static/...). And rather than write that code, I would like to use an existing fileserver. The HIPPOFACTS file for that project looks like this:
# Fact: Tawaret was the ancient Egyptian hippo goddess
[bindle]
name = "fileserver"
version = "0.2.0"
description = "Provides static file serving for Wagi"
[[handler]]
route = "/static/..."
name = "fileserver.gr.wasm"
files = ["README.md", "LICENSE.txt"]
(While the actual artifact we care about is the invoice.toml, the HIPPOFACTS above gives us all the information we could reasonably expect a user to know about a Bindle).
So how might I, as a hippofactory user, express my desire to use the fileserver inside of my own app.
Option 1: Out-of-band Handling
It is perfectly reasonable to say that the solution to this is that the user figures out how to get a copy of fileserver.gr.wasm on their own, download it locally, and include it directly:
# Fact: The airspeed velocity of an unladen hippo is zero
[bindle]
name = "myapp"
version = "1.0.0"
description = "Does neat stuff"
[[handler]]
route = "/"
name = "myapp.wasm"
[[handler]]
route = "/static/..."
name = "fileserver.gr.wasm"
files = ["index.html", "style.css"]
In this case, the user merely adds the downloaded Wasm module to their HIPPOFACTS, and the user takes on all of the responsibilities of managing that module.
Option 2: Add Dependencies in HIPPOFACTS
In this option, hippofactory is extended to declare additional dependencies more like Cargo.toml or package.json. Because bindles are immutable, we can punt here on the entire topic of lockfiles and such and focus just on the DevEx for now.
In this case, we allow a user to declare, in HIPPOFACTS that the user intends to use parcels located in an existing bindle. One possible syntax for this is:
# Fact: The airspeed velocity of an unladen hippo is zero
[bindle]
name = "myapp"
version = "1.0.0"
description = "Does neat stuff"
[[handler]]
route = "/"
name = "myapp.wasm"
[[dependency]]
[dependency.bindle]
name = "fileserver/0.2.0" # Or whatever the actual bindle name is
[dependency.handler]
route = "/static/..."
While the exact structure of the [[dependency]] object is certainly a wide open area for conversation, the example above illustrates two features that I think are necessary:
- It needs an unambiguous way to address the bindle and its parcels
- It needs an unambiguous way to bind one or more parcels to a handler clause.
Let's treat each one separately:
Addressing a Bindle and Parcels
A bindle is composed of one or more parcels organized into groups. When pulling a bindle into an app, the user may have to make some decisions about how that bindle is to be pulled in.
For example, the bindle for our fileserver application has just one Wasm parcel. But a bindle could have several Wasm parcels, each doing a different thing. Bindle's design philosophy makes it possible for one parcel to declare dependencies on other parcels. And it also makes it possible to switch parcels on or off based on group membership or features.
So a key ability when importing a bindle is to be able to specify which parcels you want. And the traditional means of doing so are through specifying groups and features.
dependency
bindle: Object. The top-level description of a bindle
name: String (REQUIRED). The full name of a bindle, e.g. example.com/foo/1.2.3-alpha.99groups: Array. Zero or more group names. Any group listed here is included in full (all parcels) unless a feature flag turns off the parcel.features: Map<String, String>. Feature name and feature value to enable: (feature.wagi.file, true)parcel: String. The SHA (or we could do the name, which is probably better) for an exact parcel to pull (maybe not a good idea) If this is specified, groups and features are ignoredexcludeGlobalGroup: boolean. If set to true, the global group will not be imported from the parcel.
Given a dependency.bindle definition, the runtime should be able to determine what bindle to load, and which parcels to fetch for that bindle.
Binding parcels to features
The previous definition gave us a bindle and associated parcels, but it provided no instructions on how those parcels are to be included in the application. My suggestion is that we include a handler definition in a dependency, and that this definition matches the handler definition for a local object:
[[dependency]]
[dependency.bindle]
name = "fileserver/0.2.0" # Pull the fileserver bindle and use its defaults (global group, no special features)
[dependency.handler]
name = "fielserver.gr.wasm" # name of the parcel
route = "/resources/..." # Override the `route` feature on the `fileserver.gr.wasm` parcel
The fields are the same as those on an existing HIPPOFACTS file, but the following clarications apply:
name: This refers to the parcel name within the Bindle. As a design constraint on this system, parcel names should be unique.- Features: When a feature is specified (e.g.
route = or host = ), it will override the feature on the imported module. We might need a reserved way of unsetting a value. (e.g. route = "-" effectively sets route to its empty value)
An open question: File parcels
Right now files are attached to a handler using the files array:
[[handler]]
route = "/static/..."
name = "fileserver.gr.wasm"
files = ["index.html", "style.css"]
When pulling in a bindle and its parcels, it is desirable that we pull in the file parcels attached to it.
But, as the present fileserver case illustrates, it may also be desirable to supply files from the local project to be loaded into the external parcel. E.g. if I load a fileserver parcel, it is very likely that I will want to tell that external parcel which of my local files I want it to serve.
It seems there are two sets of goals, then:
- I want to manage which file parcels I load from upstream, with default being "all of them"
- I want to manage which local files I want to attach to the upstream parcel, with the default being "none of them"
Here are some example use-cases:
| Upstream Parcel |
Local Files |
Desired Outcome |
| index.html |
index.html |
local index.html |
| - |
my.js |
local my.js |
| style.css |
- |
upstream style.css |
| README.md |
- |
I don't want the upstream, but I don't want to override |
The last case illustrates an intent to "unset" a file that appears in the upstream parcel without replacing it with a local file. E.g. I just don't want a README.md at all.
Because a handler can easily have hundreds of files attached, it does not seem like manually building a list would be a good approach.
Possible solutions:
- Default to local files only, and require explicit inclusions for the parcel:
files = ['local.txt', 'parcel:README.md'].
- Default to parcel files, but whenever
files is specified, use only local files (e.g. all parcels or all locals, no mixing)
- Local files are additive. The parcel files are all turned on by default, and any
files = [] appends to the list. When duplicates occur, the local overrides the parcel file.
- Provide a
parcelFiles directive in addition to files: files = ['mylocal.txt'] and parcelFiles = ['README.md'], with the result being the union of the two (with naming conflicts favoring local)
- Provide a
omitParcelFiles directive that removes parcel files, and default to all parcel files. Then use the same strategy as #4 to resolve
The issue I have is that we don't want this process to be burdensome to the user. All of these feel either burdensome or too limiting.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent