lume/lume

Create 3D web applications with HTML. Bring a new depth to your DOM!

A toolkit that simplifies the creation of rich and interactive 2D or 3D experiences.

Home  ·  Documentation  ·  Examples  ·  Forum  ·  Chat  ·  Source

npm install lume

Features

LUME is composed of several packages that can be used individually, or together as a whole:

lume - HTML elements for rich graphics

HTML elements for easily defining rich and interactive 2D or 3D applications powered by CSS3D, WebGL, or a combination of both.

This package uses and re-exports features from the below packages.

@lume/element - System for defining HTML elements

This is a web component system that allows you to create new, fast, and performant HTML elements in a simple way. It provides the foundation for LUME's HTML elements, and a standard pattern for building new elements that extend the features of LUME.

element-behaviors - Mix functionalities onto HTML elements

This allows you to augment HTML elements with features called "behaviors" that are similar to custom elements: each behavior is defined as a class that has the same lifecycle methods as custom elements. The difference is that an unlimited number of behaviors can be associated with an element.

glas - WebGL engine written in AssemblyScript (WIP)

This is a WebGL engine with the consistent performance of WebAssembly, written in AssemblyScript (a TypeScript-to-WebAssembly compiler).

LUMECraft

LUMECraft is a collection of applications made with LUME, showing what LUME can do, and serving as forkable starting points for further customization.

first-person-shooter - First-person shooter game

A first-person shooter game foundation made with LUME, Solid.js, and Meteor.

Getting involved

There are various ways to get involved!

• Visit the documentation and make something awesome!
• Submit fixes or new features to any packages or the website! See the contributing guide.
• Discuss LUME, get help, or help others in the forums or on our Discord chat server.

Status

lume's Issues

Implement scale, skew, and origin.

After trusktr/geometry-interfaces#4, then do this.

• There's currently no skew. Needs implementation on top of THREE.Matrix4.
  • Here's how to do skew/shear in Three.js
• Scale (built into THREE.Matrix4)
• finish mount-point on Z axis
• Origin is defaulted to 0.5, 0.5 via CSS. It needs a prop.
  • See notes in _calculateMatrix on where to put logic.
  • Perhaps ignore CSS origin property, we'll calc manually and only apply matrix3d only so that it matches with WebGL.
  • Here's a tool to visualize CSS transform-origin

Update README

It's a bit outdated.

• new examples using new API instead of the old stuff made with Famous.
• Replace gitter badge with link to site and forums.

motor-html: Make motor-scene and motor-node elements be the direct elements of Node instances.

For example, if we write this HTML:

<motor-scene ...>
  <motor-node ...>
  </motor-node>
</motor-scene>

then instead the imperative API creating a whole separate DOM tree where each DOM element is associated with a Node in the scene graph, have the imperative API adopt the <motor-scene> and <motor-node> elements directly as the elements associated with each Node of the scene graph.

For example, the current result of the above HTML is something like this (some details skipped for brevity):

<motor-scene ...>
  <motor-node ...>
  </motor-node>
  <div class="motor-scene" ...>
    <div class="motor-node" ...>
    </div>
  </div>
</motor-scene>

That new DOM (the <div> elements) is what is currently actually manipulated by the behind-the-scenes imperative API.

Instead, we want the DOM to remain the same, and for the imperative API (when used via the HTML API) to manage the user's actual elements, so the final result will just be

<motor-scene ...>
  <motor-node ...>
  </motor-node>
</motor-scene>

like the first example, with CSS styling (matrix transforms, etc) applied directly to those custom elements.

Motor.addRenderTask should return the render task, for reference.

For example, so we can easily remove it without manually creating a reference. Here's the current way, needing a manual reference before adding the task:

let task = function() {
    // manipulate some nodes in here...
}

Motor.addRenderTask(task)

// ...

Motor.removeRenderTask(task)

But, if Motor.addRenderTask return a reference to the task, then we can do something similar to the setInterval/clearInterval and setTimeout/clearTimeout pairs:

let task = Motor.addRenderTask(function() {
    // manipulate some nodes in here...
})

// ...

Motor.removeRenderTask(task)

which can be more convenient.

Avoid unnecessary re-rendering.

We can naively re-render a node and it's children by recursively calling render() on a node's children, but we might traverse some parts of the tree multiple times that way.

Instead, keep track of which nodes need to be re-rendered, and individually render those nodes, or find the top-most common ancestor of each subtree and re-render only those, recursing into their children.

Activity

Hey,

Is there still any activity going on in the famous 0.3.5 fork?

Manuel

Get it working in all browsers.

May need polyfills,etc.

Reset mount promise when child removed and re-mounted.

So, that when a child is removed and added to a new node (possibly in the future, not the same tick) that the mount promise is reset so it can be awaited again. f.e.

node.removeChild(child)
node2.addChild(child)
// ...
await child.mountPromise

possibly turn it into a function?:

await child.isMounted()

Considered "bug" because we should be able to use this API on subsequents mounts, not just the first.

• Also make sure to re-render one time after each time being re-mounted.

Scene does not need to extend from Node (same with HTML interface).

Scene can be a standalone thing that contains a single root Node within it, which is the top level Node of the world that the scene contains. Scene does not need transforms applied to it.

(This will also simplify module dependencies, reducing potential circular dependency problems.)

• Refactor so that Scene does not extend from Node (possibly both Node and Scene might extend from a base class if any functionality should be shared, possibly not if that shared API surface area is small enough and would merit individual implementation for each class).
• Do the same with motor-node/scene elements.

Work for #40 is waiting for this to be done first.

Move the scenePromise stuff into Node (from ImperativeBase).

Because Scene instances don't need to resolve their scene since they are the scene (it was an artifact of the ugly fact that Scene used to extend from Node, like in Famo.us 0.3). Also move the scene getter to Node.

Option to disable DOM elements when using WebGL.

Eventually, when we have WebGL, some people may want to skip having the DOM portion of the rendering entirely for performance (and will have to use the imperative API strictly). We can make an option to turn DOM off.

Docs broken?

Just tried to look a the docs for this but it appears that only the menu works? no content?

jQuery plugin for calling getters/setters

So that we can do stuff like

$('.some-node').rotation([20, 10, 40])

if using jQuery.

If we don't move away from mixin-based multiple-inheritance, then add mixin caching.

If we decide not to do, #49, then mixin applications need to be cached.

Get rid of forced reflows inside of the animation loop when calculating size.

getComputedStyle in Sizeable#actualSize is what causes it. Instead, let's listen to size changes on the root Scene, then we can calculate proportional sizes using plain math without hitting the DOM's getComputedStyle method. This is what we need to do anyways because when rendering to WebGL only (f.e. if DOM rendering were turned off when that option is available in the future), then there may be no DOM elements from which to call getComputedStyle on anyways.

Add ability to set properties (rotation, position, etc) to functions as a shortcut for adding a render task

For example, it could be something like

node.rotation = function(timestamp) {
  return [0, tween.get(timestamp), 0]
}

which would cause the supplied function to become a render task and will be executed repeatedly in the engine's animation loop.

Maybe there can also be a way to disable the function, something like

node.rotation = function(timestamp) {
  if (someCondition) {
    this.rotation = [0, tween.get(timestamp), 0] // set rotation back to a literal?
    return
  }
  return [0, tween.get(timestamp), 0]
}

or

node.rotation = function(timestamp) {
  if (someCondition) {
    this.rotation.stop() // a method attached to the render task, sets this.rotation back to a literal array.
    return
  }
  return [0, tween.get(timestamp), 0]
}

ShadowDOM compatibility: Use child-to-parent observation instead of parent-to-child observation for constructing the scene graph from the HTML elements, then observe slot distribution in ShadowDOM roots.

This will solve the problem of being able to detect when a <lume-node> element is improperly slotted/distributed into a ShadowDOM tree. Without this change (and with respect to elements that are distributed into a ShadowDOM tree), it is not possible to accurately construct our virtual scene graph because children elements observe their parent elements in order to determine how to attach virtual-scene-graph nodes, but closed ShadowDOM trees prohibit children from observing their parents (otherwise the ShadowDOM tree information would leak to the outer world which defeats ShadowDOM encapsulation).

For more background info:

• WICG/webcomponents#504
• aframevr/aframe#1492
• http://stackoverflow.com/questions/37377933

Currently, when a <lume-node> element is attached into the DOM, it checks to see if it's parent element that it was attached to ("connected" to in v1 API terms) is another <lume-node> or <lume-scene> element. If this is not the case, the child <lume-node> element throws an error.

However, this check is not possible when the <lume-node> element has been distributed into a ShadowDOM tree because from the perspective of the distributed <lume-node> element, it's parent is the parent it was attached to outside of the ShadowDOM tree, and it can not get a reference to it's containing <content> element (or <slot> element in v1 API), and therefore it cannot check that the parent of the <content> or <slot> element is another <lume-node> or <lume-scene> element.

To fix the problem we will have parent <lume-node> elements make the observation of their children instead of child <lume-node> elements looking at their parents. Parent <lume-node> elements that are inside of a ShadowDOM tree will have access to their <content> or <slot> element, and will be able to observe what gets distributed into there, and so the parent can throw the error based on the children it observes instead of the child throwing the error based on the parent it observes.

We also need to double check that our behind-the-scenes virtual scene graph is connected based on parent-to-child observation as well, so this way the API will be easier to understand if the flow of observation is the same direction in the virtual scene graph as well as the DOM. It already is.

cc @dmarcos, @cvan, @ngokevin, this will apply to a-frame elements too if they are to be ShadowDOM-compatible. I'm not sure if this is the case already (as far as parent-to-child observation). I know that a-frame isn't considering ShadowDOM yet, so HTML developers will run into this problem with A-Frame at some point when/if they decide to try using A-Frame in ShadowDOM trees.

mount/unmount methods for Scene

We should be able to mount and unmount a Scene instance from the HTMLElement where it is mounted. f.e.:

let scene = new Scene()

scene.mount('.scene-container')
scene.unmount()

Make sure that when setters are called outside of the animation loop that they trigger an animation update.

For example, someone might write:

const node = ...
let r = 0
setInterval(function() {
 node.rotation = [0,r++,0]
}, 5000)

The interval function might not execute during the engine's animation loop, and when that happens, setting node.rotate should trigger an update that re-renders the node from within the engine's animation loop.

Make perspective configurable.

Currently perspective is set to a constant 1000px via JS and passed to THREE.PerspectiveCamera in the form of position after some calculated values for aspect/fov/angle.

We can allow the user to set the perspective value on a Scene in order to have a CSS3D-like experience when not explicitly using a camera (i.e. not using a <i-perspective-camera> element in which case the default internal camera is one calculated to work like CSS3D with a perspective value of 1000px).

• Make a perspective setter/getter on the Scene class that can configure this perspective value.

Temporary visual glitch when a Node is removed and added back into a scene.

When a Node is removed then later added back into the scene graph, it will appear as a default element (top left, with display:block position) for a split second until the engine applies transforms, etc, after which it will move to it's actual position in 3D space. This happens really fast, so the user sees the Node in the display:block position for a split second, then the Node moves into position as expected.

We want to prevent this, so that when the Node is finally visible again it will appear in it's expected position without a flicker/"glitch".

Make rotation, position, etc, objects with x,y,z setters/getters.

Eventually, someone will want to animate individual axes from potentially different sources of code (different modules). This will be difficult without an abstraction on top of the current mechanism. Instead, we can implement this, so it works for everyone. Three.js has something like rotation.y, rotation.x, rotation.z, etc, for controlling each axis. With this change, we can import a node from some module, for example, apply a render task that animates one axis, and some other module can import the same node and add a render task that animates another axis, etc.

For example, we currently have to set a property to an array value:

node.rotation = [20,30,40]

With this change we'll instead do

node.rotation.x = 20
node.rotation.y = 30
node.rotation.z = 40

// or

node.rotation = { x: 20, y: 30, z: 40 }

node.rotation.x = 20
node.rotation.z = 30

// or

node.rotation = { x: 20, z: 30 }

This will make it easier to update Nodes when only the value of one axis changes without having to set new values for each axis at the same time, which will make Nodes easier to work with (for example one module might apply rotation on one axis, and some other module can apply rotation on a different axis, whereas before this wasn't possible unless those components were coupled together by a third mechanism).

When the app (window or tab) has lost focus, stop the loop.

We need a way to pause time when the window/tab loses focus. When a window/tab loses focus then the browser stops an rAF loop from looping, and when it starts back up again, the time difference from the last frame to the current frame could be large, which could cause things to sporadically jump to their new future positions. We want to avoid this, and to continue passing timestamps based on when the window/tab regains focus.

Clean up setters, move logic to render function.

Some of the setters do things like apply styles. We should move all of that to the Node#render method.

Don't re-render each node after each render task execution.

Currently, a node is re-rendered for each render task that it has in the central animation loop. Instead, we need to keep track of which nodes have registered a render task, then update those nodes in a single pass after all render tasks have been fired.

Node pooling for HTML interface?

Idea: Make a Node pool and re-use Nodes instead of creating new ones when possible. This is good, for example, when using React and switching motor-html nodes in and out of view because React may destroy the motor-html instances and instantiate new ones, in which case we can re-use Node instances. We'd have to reset the Node properties.

Add testing foundation (e2e, unit)

We need testing. Create a testing foundation.

I'm planning to use BrowserStack for automated browser testing.

The Ava test runner looks nice.

Node to signal Motor as needing render when a setter property changes.

Set a node as needing to be rendered when the node's properties change instead of basing it on the node having render tasks. (related: #17)

Currently, a node tells the motor that it needs rendering if it has tasks any existing render tasks. Instead, have a Node tell the Motor that it needs rendering when one of it's properties changes, regardless of whether tasks exist or not, causing the node to be rendered in the next frame.

[IDEA] Add API for detecting if Motor runs in current browser or not.

This will make it easy to display a message, for example, telling the user that they need to upgrade. Or developers can use it to detect when to provide a fallback UI. Etc...

Dont' require "[]" in motor-html element attributes.

For example, let's do

<motor-node rotation="10 20 30"></motor-node>

instead of

<motor-node rotation="[10 20 30]"></motor-node>

Detect when a web app is not in focus, then adjust update timestamps to reflect paused state.

For example, we don't want our animation loop to be naive and think that much time has passed between two frames due to the user switching away from the tab for a while. We want the time to continue counting only when the tab is in focus.

Update DOMMatrix to editor's draft.

DOMMatrix trusktr/geometry-interfaces#3

Move the central animation loop out of the Scene class to a separate class with one loop for all scenes.

Each scene currently has it's own animation loop. Instead, move that logic out into a separate class shared with all nodes and scenes, then have nodes register render tasks onto the singleton.

Having a central loop improves performance by reducing function calls and animation frames. Will also separate concerns away from the Scene class (cleaner code).

Dynamic animation loop

I'm working on a system so that animation frames only happen on elements that are receiving updates (in the dynamic-raf branch). The process of making this requires that certain logic be moved into DOMRenderer (no WebGLRenderer yet, but these steps will help plan for that).

• wrap logic in Node.render with requestAnimationFrame
• Move all add-child logic from Node.render to Node.addChild.
• Move all remove-child logic from Node.render to Node.removeChild.
• Make a central loop (currently in the Scene class, one per scene, but should be moved out of Scene so it can apply to any number of scenes.)
• Nodes should request updates, which places the requested update into a queue so it can be executed from the central loop.

Compatibility with both Custom Elements v0 and v1.

Make HTML elements use v1, fallback to v0.

Prevent the use of Node setters on Scene?

Scenes shouldn't be manipulated like Nodes (ie. rotation, translation, etc, should probably be ignored). Maybe we can prevent these things from working on a scene. For example, if we prevent size setters from doing anything, then the scene size can be set via CSS.

[PERFORMANCE] Keep track of Nodes that need rendering in a linked-list fashion to avoid allocating new memory?

Currently, Motor uses a Map to keep track of which Nodes need to be re-rendered (Set would've been better), but that may cause more GC than we want.

My hunch is that if we instead build a linked list with the Nodes, then that would prevent GC because we would not be deleting the placeholders in the list (the placeholders are the Nodes themselves, which we already hold references to).

Removing a node leaves animation functions (the node's render tasks) still running

When a node is removed, remove its render task, it does not need to be rendering anymore. When adding it back into the DOM, any property set to an animation function should start again.

In the first example at https://lume.io/docs/#/guide/install, change

<script type=module>
    box.rotation = (x, y, z) => [x, y + 1, z]
</script>

to

<script type=module>
    box.rotation = (x, y, z) => {
      console.log(1)
      return [x, y + 1, z]
    }
    setTimeout(() => box.remove(), 1000)
</script>

and notice that after removing the node, the console.log continues for each animation frame.

Handle Z axis for some Node properties.

Some only consider X and Y. It'll come handy once we start making 3D objects in WebGL or by combining multiple Nodes to make 3D objects out of their DOM elements.

Move observation of properties of Transformable, et al, out of those classes.

Perhaps come up with an external observation mechanism (f.e. an Observer class that can observe other objects, or perhaps an Observable class that we can mix in), or at least just add a hook to the Transformable and related classes that will call a supplied external function on property changes similar to the XYZValues class.

Then we can move the this._needsToBeRendered() calls out of the Transformable classes (probably place into ImperativeBase), which will decouple the Transformable classes from Motor stuff and make the classes simply re-usable anywhere (f.e. outside of Motor).

The motor-node styling loads after the motor-scene styling.

The motor-node styling should always load first, so that motor-scene styling can take advantage of the cascade in order to override motor-node style.

Switch to object with x,y,z, etc instead of arrays, for rotation, position, etc.

For example, we currently have to set a property to an array value:

node.rotation = [20,30,40]

With this change we'll instead do

node.rotation.x = 20
node.rotation.y = 30
node.rotation.z = 40

// or

node.rotation = { x: 20, y: 30, z: 40 }

node.rotation.x = 20
node.rotation.z = 30

// or

node.rotation = { x: 20, z: 30 }

This will make it easier to update Nodes when only the value of one axis changes without having to set new values for each axis at the same time, which will make Nodes easier to work with (for example one module might apply rotation on one axis, and some other module can apply rotation on a different axis, whereas before this wasn't possible unless those components were coupled together by a third mechanism).

Mirror getters and setters onto motor-html elements.

So that we can do stuff like

$('.some-node')[0].rotation = [20, 10, 40]

instead of currently:

$('.some-node').attr('rotation', '20, 10, 40')

(although the latter should continue to work)

JSS: Instead of extending with CSS specificity, extend via JS.

A Node creates a stylesheet and associates it with it's associated element. A Scene extends Node and creates a new stylesheet (new class name) and adds that class name next to what the base Node class already adds, and the Scene's style is supposed to override the super Node's style (the style should have higher specificity).

... But, it's hard to manage CSS specificity because classes can be added and removed in arbitrary order (and Scene or Node styles cleaned up in arbitrary order).

So, instead of dealing with CSS specificty, which morphs when styles are attached in the wrong order, let's just extend styles on the JavaScript side with good ol' object inheritance. Then the order will never matter!

Re-render child nodes if they are modified in a different Node's render task.

f.e.,

let r = 0
node1.addRenderTask(function() {
  r++
  node1.rotation = [0,r,0]
  node2.rotation = [0,r,0]
})

In the example and with the current API, node2 isn't rendered because this task is registered from node1 and we currently only render the node associated with the task.

Idea 1:

We might be able to find the top-most common ancestor of all nodes that were updated, then traverse the tree to render all children, in the process of which unsetting children from needing to be rendered.

Idea 2:

Or we could keep it as is, and make it clear that a separate render task has to be specified for each node that we wish to have updated. This would make things more verbose (we'd need to use the addRenderTask method a lot more), but that might possibly make it more explicit what updates. There may be performance implications when having a bigger list of render tasks to fire. The previous example would need to be written as follows (this works right now):

let r = 0
node1.addRenderTask(function() {
  r++
  node1.rotation = [0,r,0]
})
node2.addRenderTask(function() {
  node2.rotation = [0,r,0]
})

Related: #13

motor-html: Allow extension of existing elements, making it easy to integrate 3D into any existing HTML app.

(Not sure if this is feasible yet)

For example, suppose we take on a new project, and the new project has some existing HTML in it:

<div>
  <div id="foo">
    <h1>Hello</h1>
  </div>
</div>

Can we make an API so that we can enhance the app by applying the is="" attributes to the elements in order to introduce the 3D engine, by for example changing the markup to:

<div is="motor-scene" ...>
  <div id="foo" is="motor-node" ...>
    <h1>Hello</h1>
  </div>
</div>

If we can do that, then we can apply things to those elements like rotation, translate, animation, etc:

let node =  $('#foo')
let r = 0
requestAnimationFrame(function animation() {
  node.attr('rotation', `0, ${r++}, 0`)
  requestAnimationFrame(animation)
})

Possibly convert from mixin-based multiple-inheritance to an implementation using Proxy.

Based on my implementation of multiple().

Fix Scene default sizing.

Even though sizing is set to proportional modes and [1,1,1,], it still doesn't work and the Scene is not 100% width/height by default, but it works if we set the values after construction, f.e.

let scene = new Scene // doesn't work yet, but should

scene.sizeMode = ['proportional', 'proportional', 'proportional']
scene.proportionalSize = [1,1,1] // now it works, but shouldn't have to set this again.

Make sure global version works.

global.js should be usable via <script> tag.

[IDEA] WebAnimations compatibility: observe changes to motor-html element CSS transform, then apply to scene graph?

We can use MutationObserver (or something else?) to observe changes to a <motor-node> or <motor-scene> element's CSS transform style property, then we can apply those changes to the element's corresponding scene graph Node.

This would make it possible to use the WebAnimations API on motor-html elements to make changes to the scene graph.

A neat side-effect of this would be that WebAnimations could then be used to animate WebGL things simply by animating the transforms on the motor-html elements, once we introduce "mixed mode" (the term coined by Famous back when the project was open source).

Caveats:

• How would we synchronize with Motor's animation loop? Maybe in the doucmentation we just encourage developers to use the WebAnimations API inside of a Motor render task (Motor.addRenderTask()). Changes observed by MutationObserver would be synchronous, and would be applied before Motor API modifications. This would mean that it is possible to animate a rotation with WebAnimations, and animate a position using Motor API. If trying to animate position both with WebAnimations and Motor API, then the result would be silly: what I believe would happen is that the WebAnimation position would apply, then the Motor API position would apply, effectively overriding the WebAnimation changes. I am not sure if/when WebAnimations fire inside of a requeatAnimationFrame. Need to experiment...