motion-canvas/motion-canvas

Visualize Your Ideas With Code

Motion Canvas

Motion Canvas is two things:

• A TypeScript library that uses generators to program animations.
• An editor providing a real-time preview of said animations.

It's a specialized tool designed to create informative vector animations and synchronize them with voice-overs.

Aside from providing the preview, the editor allows you to edit certain aspects of the animation which could otherwise be tedious.

Using Motion Canvas

Check out our getting started guide to learn how to use Motion Canvas.

Developing Motion Canvas locally

The project is maintained as one monorepo containing the following packages:

Name	Description
2d	The default renderer for 2D motion graphics
core	All logic related to running and rendering animations.
create	A package for bootstrapping new projects.
docs	Our documentation website.
e2e	End-to-end tests.
examples	Animation examples used in documentation.
internal	Internal helpers used for building the packages.
player	A custom element for displaying animations in a browser.
template	A template project included for developer's convenience.
ui	The user interface used for editing.
vite-plugin	A plugin for Vite used for developing and bundling animations.

After cloning the repo, run npm install in the root of the project to install all necessary dependencies. Then run npx lerna run build to build all the packages.

Developing Editor

When developing the editor, run the following command:

npm run template:dev

It will start a vite server that watches the core, 2d, ui, and vite-plugin packages. The template package itself contains a simple Motion Canvas project that can be used during development.

Developing Player

To develop the player, first build the template: npm run template:build. Then, start npm run player:dev.

Installing a local version of Motion Canvas in a project

It can be useful to install a local version of Motion Canvas in a standalone project. For example, when you want to use your own fork with some custom-made features to create your animations.

Let's assume the following project structure:

projects/
├── motion-canvas/ <- your local monorepo
└── my-project/ <- a bootstrapped project
    └── package.json

You can link the local packages from the monorepo by updating the package.json of your project. Simply replace the version with a file: followed by a relative path to the package you want to link:

  "dependencies": {
-   "@motion-canvas/core": "^3.11.0",
+   "@motion-canvas/core": "file:../motion-canvas/packages/core",
    // ...
  },

If you're linking the ui package, you'll also need to modify vite.config.ts to allow vite to load external files:

import {defineConfig} from 'vite';
import motionCanvas from '@motion-canvas/vite-plugin';

export default defineConfig({
  server: {
    fs: {
      // let it load external files
      strict: false,
    },
  },
  plugins: [motionCanvas()],
});

This is necessary because the editor styles are loaded using the /@fs/ prefix and since the linked ui package is outside the project, vite needs permission to access it.

Then run npm install in to apply the changes and that's it.

You can use the same technique to test out any custom package you're working on.

Contributing

Read through our Contribution Guide to learn how you can help make Motion Canvas better.

motion-canvas's Issues

Allow "Scene's" to be Components.

Warning, I'm shooting for the moon on this one. I'll let you decide what's sane.

Allow Thread Generators (Scene's) to be used as JSX style components.

import {MyScene} from './MyScene.scene'
export default makeKonvaScene(function* () {
  view.add(<MyScene />);
})

This would work analogously to how React lets you use functions that return elements as elements. So in React, if I write

function MyComp() { return <button>Text</button> }

I can just use <MyComp />, and React will replace it with the button. Similarly, a thread generator called MyGen could be used as <MyGen />. The MyGen view would receive a width, height, and translated origin based on the layout engine, and MC would just replace that "component" with whatever elements MyGen adds to its view.

Consolidation of Ideas

My suggestion is a bit more conceptual though. I think that "Projects" and "Scenes" should be consolidated, both in terms of the design of MC and its documentation. I also think that "Scenes," in most contexts, should be referred to as "Generator Components," but this is more of a conceptual change. "Generator Component" would be the technical term, and they would be used to create "Projects" and "Scenes."

Project becomes a Scene

If Scenes can be nested inside other scenes, and if the recalculation and hot module replacement still worked for arbitrarily nested Scenes, the "Project" could just be a root Scene with nested Scenes. This leads to the biggest feature of the ticket.

If the Project is just a Scene, people can work off an edited video, just as they can work off edited audio.

Motion-Canvas is built around the use case of recording audio, then rending the animations, then ingesting the animations into a video editor for final edits, but After Effects (which motion-canvas is intended to replace in some use cases) is built around editing together a video first, and then adding animations. This is the workflow that MC can't do, but if the Project was just a Scene, there's no reason it couldn't. All you would have to do is add a <Video /> component in the Project Scene and layer the individual Scenes on top or behind it. Voila. This use case is the number one reason to consolidate Projects and Scenes. It's a huge boost in the utility of Motion-Canvas, and is entirely feasible.

Scenes become Generator Components

In many cases, it wouldn't make sense for nested Scenes to be referred to as a "Scene"; it would just be an animated component, referred to as a Generator Component (like "Function Component" from React). For instance, a Generator Component could be used to add a sparkle effect to a portion of a scene with <Sparkle />. You wouldn't exactly want to think of the Sparkle component as Scene.

The terms "Project," "Scene," and "Generator Component" would then be conventions, rather than being separate APIs, just like how in React you tend to have a top-level <App /> component with an arbitrarily deep structure. The documentation would present the idea of a "Project" Generator with nested "Scenes" generators that can use "Generator Components," but they would all be the same thing.

Recalculation and Hot Module Replacement

Unlike normal threads, a nested Scene cannot interact with closured variables. Since the interaction between parent and child scenes are limited, this would allow Motion-Canvas to skip recalculation of nested scenes that haven't been effected by a code or input change, just as things work now. Except, if Scenes can be nested, then you can use Thread Components to package expensive component animations in a way that allows for hot replacement of the animation. The <Sparkle /> component from earlier could be skipped entirely during recomputation if neither it nor its properties have changed. Instead, the parent scene would just receive "finished" events based on cached timing, which you know is still valid. In the tree of scenes, only the updated scene and all its ancestors would need to be recomputed.

Updating the UI

This change would require an update to the UI scene view, but it could probably display second-level Generator Components just as easily as it displays Scenes now.

Documentation for logging

Description
Since the Question arose a couple of times in the Discord and asked myself the same thing recently, it would be good
to have a short introduction / guide on how to use the build in logger, so the messages get displayed in the UI.

Doesn't have to be long since there is not much to write about, but I think not a lot of people know that you need to use the useLogger function to get a reference.

---

Proposed Content
One method of debugging your code or animation flow is using logging messages. For this, motion-canvas has its own build-in way to log messages.

To get a reference to the Logger in motion-canvas you can use the useLogger function:

import { makeScene2D } from '@motion-canvas/2d/lib/scenes';
import { useLogger } from '@motion-canvas/core/lib/utils';

export default makeScene2D(function* (view) {
  const logger = useLogger();
});

On this reference, you are then able to call different functions corresponding to the log level you want to log at.

logger.debug('Just here to debug some code.');
logger.info('All fine just a little info.');
logger.warn('Be careful somethis has gone wrong.');
logger.error('Ups. An error occured.');

These messages get then displayed in the UI under the Console tab on the left side.

Code Component: Starting Line Number

Description
The Code component currently only allows you to turn codenumbers on or off, but sometimes you might want to try to show a piece of code from different lines, for example as different views from the same bit of code. Therefore, it would be useful to have the ability to offset the codenumbers

Proposed solution
Take the existing numbers prop and change it to boolean | number, allowing the user to use it like before and have it's default behaviour or provide a number. Codewise this should also be fairly straight forward as the current code is

        for (let i = 0; i < lines.length; i++) {
            context.fillText(i.toString(), -20, (i + 0.5) * lineHeight);
        }

Considered alternatives
It's possible for the user to override this method but considering that almost all the code would have to be duplicated, this doesn't seem like the most feasible thing.

Switch to Vitest.

Description
There's a test runner and library called Vitest which is almost a one-for-one replacement for jest, except it uses Vite to build your code before running the tests. It's supposedly very fast, and uses your Vite config, reducing configuration. They also have a UI for tests, which is something I've been missing in front-end testing for a while. At some point I think MC should migrate from Jest to Vitest, since the configs are already Vite based.

Add usage examples in documentation

Description

While working with the Motion Canvas API, it can be difficult to know how to use a function or component. While some examples are available through in the examples repo or tests, they are not extensive and it can be difficult for beginners to discover and find the relevant usage.

To improve on this, the proposal is to include usage examples within the doc blocks of public APIs. This allows both discoverability through IDEs intellisense/autocomplete as well as the generated docs.

The idea comes from Rust's documentation, which does a good job of providing such examples.

This is not an effortless task but would be worth the effort to make the API more accessible for everyone, and make it easier to use.

Limitations

Evaluation of code blocks

AFAIK, while Docusaurus does support code blocks, it does not provide a way to verify that they work and could include broken examples.

To remedy this, one solution to this could be to write a TypeScript/Babel AST transformer, which visits doc blocks and specifically code blocks tagged with a JSDoc @example block tag to type-check and possibly evaluate the code to verify that it works. Additionally if the code includes assertions, these could also function as tests.

Docs for third-party libraries.

Third-party libraries like Konva can be difficult to use with Motion Canvas as they don't provide documentation and usage examples.

However, this could be made possible through declaration merging. For example:

// @motion-canvas/core/third-party/konva.d.ts
import { Rect } from "konva/lib/shapes/Rect";

declare module "konva/lib/shapes/Rect" {
  /**
   * Description of `Rect` component.
   * 
   * @example
   * ```ts
   * const square = useRef<Rect>()
   * view.add(<Rect ref={square} x={10} y={10} width={100} height={100} />);
   * square.x = 50;
   * ```
   */
  interface Rect {}
}

These can be referenced directly from @motion-canvas/core/project, or a separate @motion-canvas/core/third-party.

This would also allow adding documentation specifically for usage with Motion Canvas.

API stability

As previously mentioned, this is a lot of work, and would require a lot of effort. Especially given that the API may change at any time. So this should be done when the API is stable enough. Tools around it could still be developed for future support.

Delete prior output frames when rendering.

Description
If the video becomes smaller between renders the old last frames will remain in the output folder, which might be difficult to realize at first.

Proposed solution
The output folder could be purged before a render.

Considered alternatives
Alternatively, the output folder could be timestamped so that it's new each time.

Docs typo

Text says 320, code says 240

Switch to a monorepo

Description
Currently, Motion Canvas is split into two packages: core and ui.
motion-canvas/core#31 will most likely introduce yet another package for rendering.
Having multiple packages in separated repositories is problematic when making changes that affect more than one of them.

Proposed solution
A monorepo seems to be the industry standard that aims to reduce all these problems.
Even right now, developing Motion Canvas locally is easiest done by setting up a local monorepo.

Talking points
It seems that there's no official way of setting up semantic-release in a monorepo.
I have no idea how difficult it would be to give up semantic-release and replace it with something else.
Currently, SR is used for:

• generating the changelog
• deriving the next package version
• publishing to GitHub Packages
• creating a GitHub release
• committing changes to GitHub (package.json and CHANGELOG.md)
• commenting on GitHub issues that were published in the current release.

Maybe a better idea would be to use one of the unofficial forks of semantic-release, such as the one by Atlassian.

Or maybe SR is a bit of an overkill. Some repos - for example React - seem to redact their changelogs manually.
If we get rid of changelog-related tasks, we're left with:

• deriving the next package version
• publishing to GitHub Packages
• committing changes to GitHub (package.json)

Which I believe could be done with Lerna alone.

Add caching

Description
The current implementation recalculates the entire animation each time it seeks backward.
This may result in poor performance and low responsiveness for more complex scenes.

Proposed solution
A caching mechanism that would store the already rendered frames and reuse them when possible.

Add Husky for git hooks

Description
Husky is a package for git hooks, which allows to run commands when git commands are run.

Proposed solution
Some use cases I've seen in previous pull requests were

• wrong commit message format
• forgetting to run prettier

Husky could eliminate these (and other) issues by running a validator on the commit message and prettier-fix pre-commit.

Other use cases, like testing, could be done pre-push, so that unit tests have to pass locally before being able to push to GitHub.

Considered alternatives
-

Additional context
-

Add E2E tests

Description
There's no easy way to verify if the next versions of packages work correctly in a real environment.

Proposed solution
Create a repository/workflow for end-to-end testing, that will:

• use the create package to scaffold a new project.
• install the next versions of motion-canvas packages.
• serve the project.
• render a simple animation and compare it with a snapshot.
• build the project.

Considered alternatives
Testing it manually - it's really cumbersome.

Additional context

install the next versions of motion-canvas packages

This may be tricky since we want to run the tests before the packages are published to GitHub/npm.
I believe that npm pack can be used to simulate a real installation while using the code from the repo.

Code node

Description
Whenever you need to add some syntax-highlighted code you have to do something like this:

view.add(
    <Layout>
        <Text fill={...}>...</Text>
        <Text fill={...}>...</Text>
        <Text fill={...}>...</Text>
        {/* And so on, for each token... */}
    </Layout>
);

Proposed solution
A Code node, which takes care of all this. You pass a code snippet to it, the language is written on, and a color palette. It then automatically renders correctly, based on the syntax of that language.

Considered alternatives
A more concise way of formatting text, something like:

<Text><red>var</red> num = <green>3</green></Text>

Additional context
It may be tricky to specify the palette to use, but it's totally doable

npm init @motion-canvas doesn't work

Description
When I run npm init @motion-canvas it prints out the following. It's confusing for developers that are not web developers and are unfamiliar with the npm system and how to use it with packages that are not on npm.

npm ERR! code E404
npm ERR! 404 Not Found - GET https://npm.pkg.github.com/@motion-canvas%2fcreate - npm package "create" does not exist under owner "motion-canvas"
npm ERR! 404 
npm ERR! 404  '@motion-canvas/create@latest' is not in this registry.
npm ERR! 404 
npm ERR! 404 Note that you can also install from a
npm ERR! 404 tarball, folder, http url, or git url.

npm ERR! A complete log of this run can be found in:
npm ERR!     /Users/danemackier/.npm/_logs/2023-02-05T13_14_09_505Z-debug-0.log

I think there needs to be clearer instructions on how to get started from scratch. If you give me clear instructions I'll make a PR to update the docs as well.

Automatically convert to video

Description
This is really just a quality of life. Presumably most people want a video and not merely a bunch of frames. You can run ffmpeg manually, but it would be nice if it were simply done for you.

ffmpeg -framerate 60 -i 'frame%06d.png' -c:v h264 -pix_fmt yuv420p

Proposed solution
Do a fetch after all frames are completed rendering. Node would then try to convert to ffmpeg.

Considered alternatives
Not sure if there are any—every single result for "convert images to video node" refer to ffmpeg.

Set the next frame time at the project level.

Description
Currently, each thread controller computes the time of the next frame individually given the project time and framerate. I believe it would be advantageous to set the "target time" at the project level to be used by the flow functions. There are two main advantages to this.

The first is performance. If the target time was set at the project, the target time could change dynamically whenever the play speed was adjusted. At the moment, (as far as I recall), switching to a x8 slowdown causes a recomputation that requires eight times as many frames to be computed. For most statements this isn't noticeably slower, but tweens will cause far more computations.

There is another effect on performance, however. If the target time can change without a recomputation, then the performance of seeking can be boosted by using a lower frame rate during a seek and switching to the correct frame rate once the requested frame arrives. In fact, the seek could run at no frame rate. I'll explain the two parts of this.

A seek would set the target time as the time selected by the user. If that time is 10 seconds in, the project would set the current time to 0, as usual when seeking backwards, and set the target time to 10, as if the frame-rate was 1/10th frames per second.

Tweens that take less than a frame run twice, once at value=0, and once at value=1. This cleanly extends to a ten-second frame. A tween that starts at t=1 and ends at t=3 will see that the "next frame" is beyond it, and will run twice. A tween that starts at t=9 and ends at t=11 will see that the next frame is inside it, and will also run twice, at 0 and 0.5. All tweens would only run twice during a seek, rather than running all their frames as usual.

The time to seek would then be a multiple of the number of tweens and other statements rather than the number of frames. For heavily animated videos this would be a lot faster.

The second advantage of this change is better synchronization with the audio. Currently, if the thread runner gets out of sync with the audio, the audio time is adjusted, but that just turns stuttery video into stuttery audio. With this change the project could adjust the frame rate in real-time to correct mismatch, such that the audio is perfect and the video actually corrects for errors in real-time. My guess is that the browser works very hard to keep audio from stuttering in most cases, so the video speed might become more stable, and if the adjustments are always small, the animations should only become smoother.

Disadvantage
If any imperative code uses a value that is adjusted by a tween then that code will break while seeking, as the tweens will not run as usual. Granted, this is already bad practice, as it causes the frame rate to effect the outcome, but it could be a gotcha for some people. There could be an option to turn this adjustment off, but users won't understand it well enough to know when to use it.

A note on performance (for Jacob)
You've mentioned in the past that the seek performance is acceptable in your use case, and that is a valid reason not to make this change, however, it is possible that some users will attempt to use Motion-Canvas as more of an animation library rather than more of an instructional library. There is massive overlap (the Astortion devlog uses animation, of course), but if many users find themselves using 20x as many tweens in a scene, it's possible this change could become necessary. This ticket can serve as a ripcord to pull if the performance needs a boost.

Support for running the server in Docker

Description

Some people don't want to install and run node things manually. Docker would allow us to execute a single docker run command to get the server running and ready to use.

A user should be able to either:

1. Follow the guide, run npm init @motion-canvas, and follow the prompts like normal to create a new project
2. Use a docker run command to specify project config with environment variables to bypass the CLI prompts.

Proposed solution
Do everything in the Getting Started guide with a Dockerfile. Here is an example of what it might look like.

FROM node:19

WORKDIR /app

ENV PROJECT_NAME my-animation
ENV PROJECT_PATH my-anmiation
ENV PROJECT_LANGUAGE ts

# Fails because there is currently no way around the CLI prompts for project name, project path, and language
# These prompts should be skippable if you provide the values as environment variables
RUN npm init -y @motion-canvas && cd "$PROJECT_PATH" && npm install

CMD ["npm", "run", "serve"]

# TODO Use a volume to mount the project directory so it can be edited on the host machine

Then,

docker build -t "motion-canvas" .
docker run --name "motion-canvas" \
  -p "9000:9000" \
  -e "PROJECT_NAME=demo-animation" \
  -e "PROJECT_PATH=." \
  -e "PROJECT_LANGUAGE=ts" \
  -v "$(pwd)/app:/app" \
  motion-canvas

Considered alternatives
I don't think a docker-compose.yml makes sense for such a simple server, but correct me if I'm wrong here. Whatever makes it easiest for the end user to get something up and running is the goal.

Additional context
Would also be nice if this were integrated into GitHub Actions to publish the image to Docker Hub.

Pseudorandom number generator

Description
There's no easy way to use randomly generated numbers inside of animations.

Proposed solution
A dedicated random() function that would use the current thread time and/or invocation count as the seed.
This way, when no changes to the animation are made, the generated sequence of numbers would always be the same.
The animation would look the same every time on any machine.

Considered alternatives
Using Math.random() is good enough for certain uses. But it won't work when the generated numbers influence the timing/duration of a scene. There's also no way to provide a custom seed for Math.random().

Try out Vite as a replacement for Webpack.

Description
Vite, supposedly, has faster hot module replacement. It does this by not fully bundling development code in the dev server and instead relying on the browser's native es module support for replacement. It also appears to have a nicer hot module replacement API than Webpack, which could allow for a cleaner implementation.

This could be combined with an update the hot module replacement code, which has some TODO-style comments pertaining to scene detection. One possibility would be to add hot module replacement for components as well, which transfers properties but reruns the layout and render. This could allow for changes to appear nearly instantly when paused on a scene, though the ~100ms delay is not currently much of a problem in that regard.

Considered alternatives
If the hot module replacement isn't faster, the change should likely be abandoned, though the API might still come in handy.

Bring in eslint-plugin-tsdoc and fix all errors.

Description
eslint-plugin-tsdoc is made by the tsdoc team. I tried bringing it in to hunt down my erroneous use of @arg tags and found a lot more errors. If vscode still works it isn't a big deal, but this might hurt the documentation generation. Eventually it should be fixed, just to ensure that the different tsdoc plugins in different editors don't choke on whatever random syntax.

Proposed solution

• Introduce eslint-plugin-tsdoc
• Fix all errors that appear
• Repo is then protected from any future errors

Order of import is relevant.

Describe the bug
This breaks

import {CodeBlock} from '@motion-canvas/2d/lib/components';
import {make2DScene} from '@motion-canvas/2d/lib/scenes';
import {waitFor, waitUntil} from '@motion-canvas/core/lib/flow';

This does not break

import {make2DScene} from '@motion-canvas/2d/lib/scenes';
import {waitFor, waitUntil} from '@motion-canvas/core/lib/flow';
import {CodeBlock} from '@motion-canvas/2d/lib/components';

To Reproduce
I presume that importing any component before the scene (or possibly flow) will break MC. If it's directly related to my new Code component I'll look into it further myself, but this seems like an odd result.

Console errors

Add support for other image formats

Description
The Canvas API supports a variety of image formats that can be exported.
Currently it defaults to image/png and there's no way to customize it.

Proposed solution
Add a format control to the export settings as well as the quality setting used by some of the formats.

Can't use motion canvas in GitHub Codespaces

Describe the bug
When using GitHub Codespaces, the browser does not show motion canvas at all. Instead it just shows what I assume is the source of the page.

<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="UTF-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
    <link
      rel="icon"
      type="image/png"
      sizes="16x16"
      href="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAAACXBIWXMAAAOwAAADsAEnxA+tAAABKklEQVQ4jcWTvUoDURCFv/y0A2tnZ95AW7FZwQdQSO9aRLAJbp0mPkGCTSCBmPQW8Q1SuKTU4AvkEeQO2K6MuSFmN+pCCg9cLhdmzpwzh1tK05RdUN6p+yeCftKK+kmrW4SguvGaNc3PGGRUVEE18x4Dl9dlheP76OZxMQUCIOzVa++/W2jENYaV7oqEWTMETMmhv7fi+w4i4IVhxaaeAq+9es0aL4CJqoaqOsmSrGNsxCZ16ideMeiMVPVrJyISqerC7IhIsN3CoGMeTfYceKARn6/sqOozcADkktmMcU3y5KeZrQQ4ARxwpqpts5O3kIEvGvnJ1vwB7PuquYgc5RVs4tZHeAe8WbOIlIA9r3IJU/DXcc61nXOpcy7I1hb9C5aOLTeHf/6NwCdua48fJxuYPgAAAABJRU5ErkJggg=="
    />
    <link rel="stylesheet" href="/@fs//workspaces/motion-canvas-test/my-animation/node_modules/@motion-canvas/ui/dist/style.css" />
    <link
      rel="stylesheet"
      href="https://unpkg.com/[email protected]/styles/atom-one-dark.css"
    />
    <title>Motion Canvas</title>
  </head>
  <body>
    <script type="module" src="/@id/__x00__virtual:editor"></script>
  </body>
</html>

To Reproduce

1. Create a new repository
2. Create a codespace
3. Run npm init @motion-canvas and npm install as usual
4. Run npm run serve
5. If the codespace doesn't notify you about the app running on port 9000, add it to the ports
6. Open it in the browser

Console errors
npm run serve shows this warning. I don't know if it means anything.

(!) Could not auto-determine entry point from rollupOptions or html files and there are no explicit optimizeDeps.include patterns. Skipping dependency pre-bundling.

Package versions (please complete the following information):

• core: 2.0.0
• ui: 2.0.0

Use time events for duration

Description
Time events allow for editing when something should happen, but they could also define how long it should last.

Proposed solution
From a technical perspective, this is already possible.
The following example will make the tween last as long as the event event:

useScene().timeEvents.register('event')
const duration = useScene().timeEvents.get('event').offset;
yield* circle().scale(2, duration);

But it would be nice to have a compact helper function for this:

yield* circle().scale(2, useDuration('event'));

Displaying available shortcuts

Description
There's no way to view the existing keyboard shortcuts.

Proposed solution
A setting window for viewing the available shortcuts.
In the future, this window could also allow for editing said shortcuts.

Considered alternatives

• Listing the shortcuts in README - requires updating the docs manually whenever shortcuts are added/changed/removed.

Make bundling an animation possible

Description
Currently, Motion Canvas can only operate in conjunction with the webpack dev server.
There's no way to bundle an animation and include it on a standalone website.

Proposed solution
Extract all logic related to development into the UI package.
This would turn it into a dev-only dependency used for editing and bundling.
The core itself could then be bundled together with the animation and a player.

SmartLinearLayout with auto-sizing children

Description
The LinearLayout component currently calculates its size based on the size of its children. However, it would be useful to have a SmartLinearLayout component that automatically sizes its children based on its own size while maintaining the aspect ratio of the children. This would especially be useful for laying out images of different sizes.

Proposed solution
As I know there are talks of redoing the Image API, these images are loaded using a theoretical API with minimal user input.

// Note: Image API is hypothetical
import pic1 from './img/pic1.png'
import pic2 from './img/pic2.png'
import pic3 from './img/pic3.png'
import pic4 from './img/pic4.png'

Vertical tiling with explicit height and spacing

scene.add(
    <SmartLinearLayout direction={Center.Vertical} height={875} spacing={25}>
        <Image image={pic1}/>
        <Image image={pic2}/>
        <Image image={pic3}/>
    </SmartLinearLayout>
)

The current proposal includes the spacing value in the height, so the result is 875px tall, but the images are 825px tall. I am not sure if it would be better to have spacing add onto the height, so the images are each 875px tall and the overall layout is 925px tall.

View image

Vertical tiling with explicit width and equal border

scene.add(
    <SmartLinearLayout direction={Center.Vertical} width={450} border={25}>
        <Image image={pic1}/>
        <Image image={pic2}/>
        <Image image={pic3}/>
    </SmartLinearLayout>
)

View image

Horizontal tiling with explicit width and uneven border

scene.add(
    <SmartLinearLayout direction={Center.Horizontal} height={150} border={{top:35, left:35, bottom:15, right:15}}>
        <Image image={pic1}/>
        <Image image={pic2}/>
        <Image image={pic4}/>
    </SmartLinearLayout>
)

I don't see myself ever using an uneven border like this, but it might be useful. I think CSS specifies an uneven border as padding, so maybe it would be better to specify it as padding instead.

View image

Nested tiling in horizontal and vertical direction

scene.add(
    <SmartLinearLayout direction={Center.Horizontal} height={925} border={25} spacing={15}>
        <SmartLinearLayout direction={Center.Vertical} spacing={25}>
            <Image image={pic1}/>
            <Image image={pic2}/>
            <Image image={pic3}/>
        </SmartLinearLayout>
        <Image image={pic4}/>
    </SmartLinearLayout>
)

Note: because the child SmartLinearLayout does not define its own size, the spacing of 25 is 25 pixels at the HIGHEST level, not at the child level. If the child SmartLinearLayout were given a width or height, then the spacing would be applied before the child was scaled as a group. If the child SmartLinearLayout was given a height of 500 (which is not equal to the parent's height) the child should be constructed according to the height of 500 and then scaled to fit the parent.

View image

Considered alternatives
The NPM library https://github.com/naturalatlas/image-layout uses an algorithm for finding the optimal layout of images. While this has its uses, that algorithm solves for layout given a size, whereas this proposal solves for size given a layout.

Reactive Properties

There are two standard patterns of arbitrary reactivity. The first is a top-down pattern, as seen in SolidJS, Svelte, and Vue.js, which I call Computed Values.

rect.width(() => target.width() + 10)

Whenever rect.width is computed, the system monitors which dependencies are read. Whenever target.width is changed, rect.width is marked as "dirty", and whenever rect.width is read, if it is dirty, it is recomputed. This pattern works flawlessly as long as all mutable state is stored behind these special accessible properties. In order to allow for arbitrary variables, a new container type must be added.

const myValue = atom(10); // atom creates a Computed Value
const otherValue = atom(() => myValue() + 10);
myValue(20);
otherValue();
// 30

These properties would be used in combination with Yoga, which would absolve the system of having to deal with the messier parts of layouts, that being bidirectional data-binding and dimensions that depend on themselves. See #54 . The Computed Values could then be saved for unidirectional layouts outside of Yoga's wheelhouse, like arrows and pins.

These Computed Values make many patterns more intuitive beyond just layouts, however. A user in the discord once asked how to automatically update the color of a circle by its position. Reactive properties make that answer more intuitive.

circle.color(() => Math.hypot(circle.x(), circle.y()) < 30);

The second pattern of reactivity is a bottom-up pattern, taken from Haskell. Values are given a mapping function, which could be called pipe to avoid confusion with the existing map.

const myValue = atom(10);
// equivalent to atom(() => myValue() + 10)
const otherValue = myValue.pipe(v => v + 10) // v is a number
myValue(20);
otherValue();
// 30

This pattern would serve as a convenient alternative to constructing atom directly but would come in most useful when dealing with time.

If this system were implemented, time could become an observable property, fixing a potential quirk of time in Motion Canvas. Currently, when you use tween to set a value, and then access that value in between frames, the value is wrong.

yield tween(2, v => { 
  circle.x(30*v + 20)
})
yield* waitFor(0.02)
circle.x() // last frame value

This isn't usually a problem, but it can be. There's a common effect where a shape moves across the screen and leaves a trail of copies of itself. Trying to do this currently would result in misplaced copies, that may be noticed in their spacing. This result would be confusing to users and difficult to fix. If time were observable, and properties derived from time were reactive, however, this problem wouldn't exist

yield tween(2, v => {
  circle.x(() => 30*v() + 20)
})
yield* waitFor(0.02)
circle.x() // computed based on thread time

Of course, a tween isn't necessary here, but the same result would hold true when using the succinct API, circle.x(50, 2).

For time, piping would give a simpler API, especially in cases where a single time value is reused in multiple ways.

yield tween(2, v => {
  const eased = v.pipe(ease);
  const middle = v.pipe(v => remapClamp(0.5, 1.5, 0, 1, v));
  // use eased and middle in multiple places
})

Reactivity produces more to learn, but it simplifies the production of some animations by allowing people to centralize which values change imperatively and which values are just reactive byproducts. For example, in the original Motion Canvas video, it had to use a slightly custom keyframing system using an animator, so that it could play through the moving circle animation discretely, then continuously, and then repeatedly. With reactive values, you could create an observable frame value, derive the circle's position from it, and stick to editing the frame value.

// simplified code
const frame = atom(0)
const timelineFrame = atom(0)
circle.position(() => derivePosition(frame()))

yield* waitUntil('at frame ten')
timelineFrame(10)
yield* waitUntil('here')
frame(10) // jump to discrete points
// ...

timelineFrame(frame) // lock timeline to frame
yield* tween(2, v => frame(v.map(0, 100))) // map is the MC map function but as a method
// ...

The current MC animator is really simple, so I won't claim that reactivity would make that particular project any simpler, but animation is filled with moments where multiple items move in unison, and reactivity is a useful tool to have when synchronizing items to a central value for simplicity of control.

vite 4?

Description
How easy is it to update to vite 4?

Proposed solution
...

Considered alternatives
...

Additional context
...

Typo in flow section

It should be circle and not circle().

Inconsistent playhead behavior at composition end.

Describe the bug
Under certain conditions the playhead is not stopped/looped at the end of the composition.
Under other conditions, it does.

To Reproduce

1. Disable Looping.
2. Set the render range of the composition to not reach the end completely.
3. Play the composition.
4. When encountering the end of the composition, the playhead stops.
5. Set the render range of the composition to reach the end completely.
6. Play the composition.
7. When encountering the end of the composition, the playhead does not stop.

Expected behavior
I can't tell.

Package versions:

• core: 2.0.0
• ui: 2.0.0

Reactive Layouts (and possibly more)

I'm bringing this up now as it sounds like MC is going to get a new renderer.

Description

At the moment, Layout recomputation is handled through an isDirty/wasDirty cycle, which has some drawbacks.

The first is that it limits the number of dependent values that may exist in MC layouts. I believe the current cap is 10. This cap could be removed, but then layouts could hang forever in an infinite loop, and since the flags are handled imperatively, there's no apparent way for the system to detect the issue.

On top of this, it's very difficult if not impossible to change the flow of data in different layouts. For example, an image needs to maintain its aspect ratio, so the width and height must depend on each other. An image layout may want to set the width and the parent, then the width of the first child, then the height of the first child, then the position and width of the second child, etc, before eventually setting the height of the parent. This flow is difficult to do, however, if layouts assume by default that positions flow down and dimensions flow up.

Proposed solution

Build a reactive system for layouts that automatically tracks dependencies and hides the dirty/clean flag, similar to SolidJS. So, there could be a computed property with

// illustrative code
layout(() => {
  this.children().forEach(child => {
    child.left(() => this.left() + 5);
    child.width(() => this.width() - 10);
  });
});

Whenever this.width changes, child.width will be marked as dirty. During render, whenever the render function asks for the width of the component, this.width will pull the necessary values to generate itself, which will cause these properties to be marked as clean. No need for wasDirty as it's not a loop.

An issue arises when relationships change, however. If a parent component sets a property on its children, child.width(() => this.width() - 10), and then the array of children (this.children()) changes, the old width settings are outdated, and should be removed in order to avoid "zombie" bugs. It's not enough to just let the new writes stomp the old writes, since children may have disappeared from the children array. Therefore, the layout function should track writes as well as reads in order to be able to remove them.

This is different from how most reactive systems work. Usually, a property is given a function to compute itself, which is then used to detect the dependent values.

const B = reactive(() => C() + D())
// B depends on C and D

Here, the system is detecting the pull, C(), during computation to discover the dependencies C and D. The dependent value B is explicit. In the case of layout, the function both reads and writes values. It reads from this.children, and it writes to the children's width. The written value is then also reactive, as the child's width depends on the parents width. layout has created a reactive function that writes reactive functions, which is necessary for a layout engine, especially one robust enough to handle containers, arrows, and pins.

This does make tracking dependencies harder, as you may not detect a conditional write.

layout(() => {
  if (this.width() < 10) this.child.width(5) // not always detected
})

A system built this way could not always rely on a naive "push dirty, pull clean" setup, but would instead need to perform a best guess recomputation of values while occasionally throwing away inconsistent results.

Considerations

Motion Canvas doesn't need the best, most feature-rich layout system ever designed, but we can still pick its trade-offs. In particular, Motion Canvas runs supervised, where conventional user interfaces do not. If a Motion Canvas component clips its children, hiding some important information, the developer can just change the layout. This absolves MC of the need for certain conventional features like wrapping overflow elements. That being said, there are still some issues that could heavily affect MC's learning curve.

bidirectional data flow

One issue with the proposed approach is that not all layouts have a clear direction of data flow. Many components want to maintain an aspect ratio, which means either their width or height can be derived from the other. This design can show up in many places. For instance, when I set the width and left side position of a component, the right is derived from the left, but if I set the width and right side, the left is derived from the right. You can try to avoid this issue by always choosing to set one property, for instance,

// right align children (for argument, assume only `left` and `width` exists)
child.left(() => {
  const right = this.left() + this.width();
  return right - child.width();
})

but then anyone who creates a layout has to learn which properties exist and which don't, which could change given different child components, parent components, or settings, making compatibility an issue.

One option to fix this is to allow for bidirectional data flow. Like with algebraic structures.

this.left(this.right().sub(this.width()));

This would be interpreted as 0 = child.left - child.right + child.width. Then, if one of these values is requested after the other two values exist, the requested value would be computed from the others. Most users would never have to deal with this feature; it would mostly only be used by the base classes to set up convenient interrelations.

values can depend on themselves (sort of)

Occasionally, you'll have computed values that depend on themselves. Like a parent with vertically stacked children where the children stretch to match the widest child. First, the parent checks the width of the children to find the widest child. Then, the parent sets the width of the other children to match, but then the width of the children depends upon itself. This happens on the web in the Table and Flex layouts, and it would be difficult to pull off with the solution proposed above. I believe that it would also be difficult to pull off in the current system aswell, though.

One option to fix this is to use multiple values for dimensions, like fitWidth and width. This leaves users with more to learn and must be planned out in advance, but it's the simplest solution to the issue.

Another solution, however, is to just allow values to depend on themselves once, as long as the dependency happens in the same layout.

layout(() => {
  this.width(() => {
    return this.children.reduce((max, child) => Math.max(max, child.width()), 0);
  }); // first statement, given lowest priority
  this.children.forEach(child => child.width(2, () => this.width())) // second statement, given higher priority
});

The layout engine would then allow the width to be set twice, with the layout the read/writes running after the first setting. I'm oversimplifying both my code and my explanation here, but I think it captures the idea.

Text is invisible in Firefox

Describe the bug
By default, Firefox has dom.textMetrics.fontBoundingBox disabled, meaning that Text components dont render at all (this includes the HTML embeds of Motion Canvas animations)

To Reproduce
In a fresh install of Firefox with default settings, try to view a scene that uses a Text component

Expected behavior
The text is displayed

Additional context
A workaround for users is to manually enable dom.textMetrics.fontBoundingBox within Firefox's config by typing about:config in the address bar

Event times for scenes being reset.

Describe the bug
Twice now I've come back to my copy of animating-with-code to find that the times for just one of the scenes has been reset to their lowest possible setting. They've been reset in the metafile as well, so whatever is happening, the save code is being run as well. Luckily the metafile is protected by git, but if it wasn't checked in it could be frustrating.

To Reproduce
I'm not sure. I'm submitting this for logging purposes. Importantly, I've been actively developing Motion-Canvas core the whole time, and it seems to happen when I leave my computer and come back. It's possible it has something to do with rebuilding core, in which case it's low priority, but if anyone else experiences this while not developing core or UI, it would be nice to know about it.

Screenshot
Notice how the imperative Scene is wiped out. Before it was the Programming scene.

Package versions (please complete the following information):

• core: [e.g. 6.0.1]
• ui: [e.g. 1.6.0]

Editable signals

Description
Aside from time events, it's not possible to edit values through the editor.

Proposed solution
Introduce a new entity representing a value editable via the editor.
The API could be similar to signals:

const name = createProperty('Jacob');
const position = Vector2.createProperty([100, 200]);

All properties would be displayed on a dedicated timeline track, right under time events.
Each property would be represented by a pill, located at the time at which createProperty was called.
Selecting the pill would open up the inspector and allow editing of the property's value. This could be the same inspector as the one implemented in #169.

Certain types such as Vector2 and Rect should have dedicated gizmos displayed as an overlay on top of the canvas.
This would allow for editing them by dragging and resizing them instead of just typing the value in an input (which would be usless, since that's exactly the same as hardcoding a value in the code)

It should be possible to provide the matrix used for displaying the gizmo. For example, having the gizmo relative to a given node would look as follows:

const position = Vector2.createProperty([100, 200], node.worldToLocal());

These properties should be implemented using signals, this way the changes made through the editor could be immediately reflected in the preview, without the need for recalculation, this would provide a smooth dragging experience. After the drag is completed the scene should be recalculated regardless, to make sure that everything is correct.

Ending a scene with an waitUntil leaves it inaccessible on the timeline.

Describe the bug
The timeline ends when the scene ends, which leaves any final waitUntil inaccessible for to drag with the mouse. It's alleviated by having a waitFor at the end of the scene, but until then, the waitUntil is not even drawn.

In addition, if the waitUntil is near the end of the scene, it can be difficult to drag very far, as you can only drag to the end of the current scene before having to drop it to extend the scene and repeat the process.

To Reproduce
End any scene with a waitUntil.

yield* waitFor(15);
yield* waitUntil('end');
scene.canFinish();

Expected behavior
The waitUntil should be accessible and should be draggable beyond the current end of the timeline.

Possible Solution
I think the timeline should allow the user to scroll one screen length beyond the end of the scene. Then you could use zooming to drag any event as far as it needs to go.

Animate spreading colour in a Line

Description
I would like a way to draw a line, then fill it with a colour (blue) starting at one end and progressing to the other over time.

This would be an alternative to manually drawing this animation:

  const filled = createSignal(0);
  // (I couldn't figure out how to render Lines, they kept complaining with errors in vite :P)
  view.add(<Rect width={420} height={20} x={0} fill="white"/>)
  view.add(<Rect width={() => filled() * 420} height={20} x={() => (filled() - 1) * 210} fill="blue"/>)
  
  yield* filled(1, 2);

I would hope that a proper API would allow us to floodfill objects without rebuilding them with manual size/pos calculations.

Proposed solution

Nothing yet! I haven't got far enough into the app's guts to know how we'd need to implement this. It's easy if we can use shaders, but a bit of a PITA with html/css. The best option would be using gradients across the rendered boxes

Add code style docs, expand lint, reconfigure prettier.

Description
I'll have trouble remembering the code styles unless I take notes. This issue should allow us to keep track of the code styles that come up.

I would that whenever possible the code styles should go first into Prettier, as it has an autofix, then into lint, as it has autodetection, and then go into a CODE_STYLE.md (working name), as a fallback.

Proposed code styles so far

• Capitalize Interface names. (This can be set up in prettier)
• Use concise sentences in the imperative present tense for function JSDocs. "Run the function." "Do the thing." "Clear the items."
• Add eslint-plugin-tsdoc

For Jacob (the original dev) if you think of any more comment about them and I'll add them. If you are contributing this can operate as a work-in-progress code style guide, subject to change.

Scene Graph Tab

Description
There's currently no way to display the structure of the scene that would list all the nodes added to it.

Proposed solution
Introduce a new editor tab displaying the tree graph of the current scene.
The node inspector could be moved to the right and appear only when a node is selected - either by clicking on the canvas or by selecting the node in the scene graph.

Split basic API documentation from advanced documentation.

Description
There's a hard line between the API that would be used in scenes to produce videos and the API that would only be used if a user were to build a new UI, a new scene type, or to otherwise alter or extend core. I think the basic documentation needs its own area. There are several ways to do this.

Proposed solution
The first possibility is to aggressively mark items as @internal. For instance, the Player could be considered internal, as it is not intended to be used by most end-users. TypeDoc then allows you to generate documentation both with, and without internal elements. On the site, there could be a basic API section, without internal items, and an advanced API section, with internal items.

Alternatively, the "basic" API could all be documented with handwritten markdown pages, and the generated API documentation could be left as-is. This is how React does its documentation, see Hooks API Reference, and their documentation is better for it. The downside is that the Guides then have to be exhaustive and manually updated. This approach doesn't actually require a new ticket, so this one could be closed.

A third option is to define custom tags in TypeDoc and to write a little plugin to filter results on different pages. There could then be a @basic and @advanced tag instead of coopting the internal tag.

Question on introduction video code

Don't know is this issue appropriate but it seems I can't create a new discussion so I put it here anyway...

On video Animating with Code - Motion Canvas there are 3 pieces of code :

yield rect.value.radius(160, 0.6)
yield* rect.value.fill('#68ABDF', 0.6)

const task = yield rect.value.radius(160, 0.6)
yield* rect.value.fill('#68ABDF', 0.6)
yield* join(task)

yield* all(
  rect.value.radius(160, 0.6),
  rect.value.fill('#68ABDF', 0.6),
)

After watching this video, my understanding on motion-canvas behavior is something like this:

• write a generator which step the animation by one frame per yield
• yield this generator inside another generator to compose them
• use yield* to pull out all frames in a generator

The video says that this 3 pieces are identical on function, so based on my understanding, there are few questions bothering me:

• if the radius animation is yield for only once, then how this animation continued?
• If I just neglect the first question, I can see that the last piece do start two animation simultaneously, but I think the first and the second one start the radius animation one frame earlier than the color animation.

Please help! Thank You!

Better customization

Description
Currently, the only way to customize Motion Canvas is via command-line arguments which are very limited.
There's no way to add custom plugins or loaders for webpack, for instance.

Proposed solution
Introduce an optional motioncanvas.config.js file for a more fine-tuned configuration.
A sample config:

module.exports = {
  ui: {
    mode: 'server', // 'server' | 'path'
    url: 'http://localhost:8081/main.js',
  },
  output: './output/example/',
  webpack: {
    plugins: [
      new webpack.DefinePlugin({
        CUSTOM_CONST: `42`,
      }),
    ],
    devServer: {
      port: 8080,
    },
  },
};

ui and output would work analogically to the current command-line arguments.
The webpack property would contain a configuration object that would be merged with the default one using webpack-merge.

By default, the motion-canvas command should look for a motioncanvas.config.js file in the current directory.
A new command-line argument --config (-c) could be used for specifying custom config files.

In the future, this configuration file could also be used to add plugins.

Add guides to docs

Description
Current documentation is missing any sort of guides/tutorials.
typedoc-plugin-pages can be used to add them to the currently existing API docs.
Source files would be stored inside of the docs directory as markdown files.
This would make them available through GitHub for people who haven't set up Motion Canvas yet.

Attempting to open scene file fails.

Describe the bug
When clicking a scene link in the editor to open the file, the action fails.
Supposedly, due to the generation of malformed executable paths.

To Reproduce

1. Attempt to open scene file in the editor.
2. Action succeeds, creating a notepad instance with the opened file.
3. Start any program associated with the .tsx extension in Windows settings.
4. Attempt to open scene file in the editor.
5. Action fails, citing <Path> is not recognized as ... operable program due to the Path being malformed.

Expected behavior
The file should open in the editor successfully.

Package versions (please complete the following information):

• core: 2.0.0
• ui: 2.0.0

Additional context
Tested with both VSCode and Notepad++.

For VSCode:
Executable path: C:\Users\<user>\AppData\Local\Programs\Microsoft VS Code\Code.exe
Attempted access path: C:\Users\<user>\AppData\Local\Programs\Microsoft

For Notepad++:
Executable path: C:\Program Files\Notepad++\notepad++.exe
Attempted access path: C:\Program

Add a component that can render LaTeX equations and such

Description
LaTeX is a typesetting system, which includes a system for typesetting mathematical equations and such.
So being able to render LaTeX equations would be a great addition to the library to be able to visualize equations.

Proposed solution
There exists an npm package that can render LaTeX equations called KaTeX. I'd imagine using this would be the way to go to create a component that renders LaTeX equations.

Considered alternatives
There are some other libraries for rendering LaTeX, like MathJax. But from what I know it is slower & not the library to go to when you want to render mathematical equations.

Additional context
None that I can think of.

npm init project

From #28

Description
Motion-canvas should have a @motion-canvas/create package which initializes a new project.

Considered alternatives
There are many libraries that can initialize a project, like Yeoman. I have not researched any deeply.

Application settings

Description
There's currently no way to store application-wide settings.
Possible application settings include:

• Default project settings (size, background color)
• Grid configuration
• Theme color
• Future toggles for experimental features

Proposed solution

• Introduce application settings, stored somewhere in the user's home directory.
• Make them configurable through a dedicated editor tab

Per-thread signal caching

Description
As discussed in #58, it could be beneficial for signals that are undergoing a tween animation, to return values adjusted for the current time of the thread that requested them.
See this comment for more info on why this could be useful.

Proposed solution
Introduce a new type of signal representing the time. When this signal is used, it renders the entire dependency chain as "dependent on the time". Values of signals included in this chain would maintain a separate cache for each thread so that the time used to calculate them would be the time of the given thread.

Better project settings

Description
The current implementation of project settings is really crude. Things are configured through code. Some of them can be overridden through the editor, but the changes are saved in the localStorage making them impossible to share.
There are no application-wide settings.

Proposed solution

• Use the project's meta file to store project settings.
• Make all project-related options configurable through the editor: background color, audio offset, name, etc.
• Separate rendering settings from the preview settings.

Add logging middleware

Description
Errors and warnings are logged directly to the browser's console.
If users don't have their developer tools open they may not realize that something has gone wrong.

Proposed solution
Add a logging middleware, such as winston, that would allow the UI to display information about error logs.

To not overcomplicate things, the initial implementation should simply display the most recent error in a status bar beneath the timeline.
Example:

The meta file for scene.example is missing... | Open the console for more info CTRL+SHIFT+J [DISMISS]

Integrate commitlint.

Bring in Commitlint for both local pre-checks and a GitHub actions failure. They have a config for Angular-style commit messages that will catch the most common issues.

Considered alternatives
None. Commitlint appears to be the most popular commit message checker on npm by a wide margin.

Additional context
This will help to avoid having to "confront" people about their commit messages if they miss the guide.