Add a component that can render LaTeX equations and such

[motion-canvas]

Add a component that can render LaTeX equations and such

[hhenrichsen]

I'm down to look at turning my code into a component since it seems like a fairly frequent request.

[vvcaw]

Good point, coming from manim that is something I also considered to be missing here. As far as I know, KaTeX can either render to HTML (with CSS) or MathML, so I'm not sure how you would go about rendering it on canvas, or am I missing something here?

[AimarIbarra]

Another option to render formulas would be using svg images and the Image component.
If I recall correctly MathJax can export to svg formats.
If performance is a concern you could add a build step that would turn tex files to pdf, and then convert the pdf files to svg images.
On Linux I would use pdftex and pdf2svg to acomplish this, alongside a Makefile to automate the mentioned steps.

However vvcaw is right, KaTeX doesn't support canvas/image output by default.

[Aurora2500]

Ah, sadly I'm too high-level to ever have dwelved into actually rendering things in practice, so I didn't know Katex had these limitations.

[aarthificial]

Here's an example by @hhenrichsen on how to achieve something like this with MathJax.
The process of doing MathJax -> SVG -> base64 -> Image looks plausible and I'm okay with implementing it in Motion Canvas.
But I need to stress the fact that it won't be animatable like with Manim. For that, we'd need some lower-level renderer, possibly a WebGL one, which I'm not sure exists.

[t3ssel8r]

@aarthificial I've poked at the animation system in manim before and am roughly familiar with how it works. All equation animations in manim occur on SVG objects. It attempts (sometimes incorrectly) to reconstruct the symbol->path correspondence by rendering substrings and counting the number of paths in them, assuming that paths are exported sequentially from the renderer.

To replicate its functionality, I think all you would need fundamentally is the ability to compile SVG from LaTeX, and draw/interpolate SVG paths.

[hhenrichsen]

My PR does step 1, so the only other step would be the interpolation. Drawing is a bit awkward right now since we have to convert it to base64 to get it on the canvas, but I imagine drawing SVGs to a canvas is something that's been done before.

[AimarIbarra]

@hhenrichsen Assuming Motion Canvas draws using the context of the canvas, then the Path2D object would skip the step.

document.getElementById("canvas")
    .getContext("2d")
    .fill(new Path2D(svg_string));

Where svg_string is svg path data.
And the path data of an svg image can be extracted trough the following code.

<svg>
    <path id="img_path" p="M150 0 L75 200 L225 200 Z" />
</svg>

const svg_path_data = document.getElementById("img_path").getAttribute("p");

[hhenrichsen]

Latex isn't a single path, though, and it's not all composed of paths, which is why I went the base64 route. Here's an example from the quadratic formula, x = \frac{-b \pm \sqrt{b^2-4ac}{2a}, generated by MathJax:

<svg xmlns="http://www.w3.org/2000/svg" width="21.394ex" height="5.291ex" role="img" focusable="false" viewBox="0 -1642.5 9456 2338.5" xmlns:xlink="http://www.w3.org/1999/xlink" aria-hidden="true" style="vertical-align: -1.575ex;">
    <g stroke="currentColor" fill="currentColor" stroke-width="0" transform="scale(1,-1)">
        <g data-mml-node="math">
            <g data-mml-node="mi">
                <use data-c="1D465" xlink:href="#MJX-TEX-I-1D465"></use>
            </g>
            <g data-mml-node="mo" transform="translate(849.8,0)">
                <use data-c="3D" xlink:href="#MJX-TEX-N-3D"></use>
            </g>
            <g data-mml-node="TeXAtom" data-mjx-texclass="ORD" transform="translate(1905.6,0)">
                <g data-mml-node="mfrac">
                    <g data-mml-node="mrow" transform="translate(220,676)">
                        <g data-mml-node="mo">
                            <use data-c="2212" xlink:href="#MJX-TEX-N-2212"></use>
                        </g>
                        <g data-mml-node="mi" transform="translate(778,0)">
                            <use data-c="1D44F" xlink:href="#MJX-TEX-I-1D44F"></use>
                        </g>
                        <g data-mml-node="mo" transform="translate(1429.2,0)">
                            <use data-c="B1" xlink:href="#MJX-TEX-N-B1"></use>
                        </g>
                        <g data-mml-node="msqrt" transform="translate(2429.4,0)">
                            <g transform="translate(853,0)">
                                <g data-mml-node="msup">
                                    <g data-mml-node="mi">
                                        <use data-c="1D44F" xlink:href="#MJX-TEX-I-1D44F"></use>
                                    </g>
                                    <g data-mml-node="mn" transform="translate(462,289) scale(0.707)">
                                        <use data-c="32" xlink:href="#MJX-TEX-N-32"></use>
                                    </g>
                                </g>
                                <g data-mml-node="mo" transform="translate(1087.8,0)">
                                    <use data-c="2212" xlink:href="#MJX-TEX-N-2212"></use>
                                </g>
                                <g data-mml-node="mn" transform="translate(2088,0)">
                                    <use data-c="34" xlink:href="#MJX-TEX-N-34"></use>
                                </g>
                                <g data-mml-node="mi" transform="translate(2588,0)">
                                    <use data-c="1D44E" xlink:href="#MJX-TEX-I-1D44E"></use>
                                </g>
                                <g data-mml-node="mi" transform="translate(3117,0)">
                                    <use data-c="1D450" xlink:href="#MJX-TEX-I-1D450"></use>
                                </g>
                            </g>
                            <g data-mml-node="mo" transform="translate(0,106.5)">
                                <use data-c="221A" xlink:href="#MJX-TEX-N-221A"></use>
                            </g>
                            <rect width="3550" height="60" x="853" y="846.5"></rect>
                        </g>
                    </g>
                    <g data-mml-node="mrow" transform="translate(3121.7,-686)">
                        <g data-mml-node="mn">
                            <use data-c="32" xlink:href="#MJX-TEX-N-32"></use>
                        </g>
                        <g data-mml-node="mi" transform="translate(500,0)">
                            <use data-c="1D44E" xlink:href="#MJX-TEX-I-1D44E"></use>
                        </g>
                    </g>
                    <rect width="7032.4" height="60" x="120" y="220"></rect>
                </g>
            </g>
            <g data-mml-node="mo" transform="translate(9178,0)">
                <use data-c="2E" xlink:href="#MJX-TEX-N-2E"></use>
            </g>
        </g>
    </g>
</svg>

This doesn't work well with the Path2D element since it seems to be entirely made up of groups, so there's some additional parsing we'd have to do, just like t3ssel8r's solution. I think that might have merit, but there's a lot of processing to do there, still, maybe as a potential future improvement.

[AimarIbarra]

So it imports xlink svg data, base64 route then.

[aarthificial]

@t3ssel8r that's really good to know. I assumed they were doing the morphing on the level of meshes, but using SVGs seems so obvious in hindsight.