Giter Site home page Giter Site logo

hydra-synth's Introduction

Hydra-Synth

Video synth engine for hydra.

Currently experimental / in-progress.

This is the main logic of hydra packaged as a javascript module, intended for use within javascript projects. If you are looking to get started with hydra quickly, visit the web editor or the main repo. To use hydra within atom, follow the instructions at https://github.com/ojack/hydra-examples.

image of hydra in webpage

To include in a webpage (bundled version):

Include the bundled version of this library in your html file:

<script src="https://unpkg.com/hydra-synth"></script>
<script>
      // create a new hydra-synth instance
      var hydra = new Hydra({ detectAudio: false })
      osc(4, 0.1, 1.2).out()
</script>

You can see and remix a live example here: https://glitch.com/edit/#!/hydra-webpage

To use as a module:

Download the module:

npm install --save hydra-synth

Include in your app:

import Hydra from 'hydra-synth'

const hydra = new Hydra({ detectAudio: false })
osc(4, 0.1, 1.2).out()

To use using cjs/require syntax:

const Hydra = require('hydra-synth')

The rest of this README is about configuring hydra-synth. For broader hydra documentation and usage, see getting started, interactive function documentation, and Hydra Book (by Naoto Hieda).

API:

const hydra = new Hydra([opts])

create a new hydra instance

If opts is specified, the default options (shown below) will be overridden.

{
  canvas: null, // canvas element to render to. If none is supplied, a canvas will be created and appended to the screen

  width: // defaults to canvas width when included, 1280 if not

  height: // defaults to canvas height when included, 720 if not

  autoLoop: true, // if true, will automatically loop using requestAnimationFrame.If set to false, you must implement your own loop function using the tick() method (below)

  makeGlobal: true, // if false, will not pollute global namespace (note: there are currently bugs with this)

  detectAudio: true, // recommend setting this to false to avoid asking for microphone

  numSources: 4, // number of source buffers to create initially

  numOutputs: 4, // number of output buffers to use. Note: untested with numbers other than 4. render() method might behave unpredictably

  extendTransforms: [] // An array of transforms to be added to the synth, or an object representing a single transform

  precision: null  // force precision of shaders, can be 'highp', 'mediump', or 'lowp' (recommended for ios). When no precision is specified, will use highp for ios, and mediump for everything else.

  pb = null, // instance of rtc-patch-bay to use for streaming
}

Custom render loop

You can use your own render loop for triggering hydra updates, instead of the automatic looping. To use, set autoLoop to false, and call

hydra.tick(dt)

where dt is the time elapsed in milliseconds since the last update

To develop:

npm run dev

Sets up an example using hydra-synth that is automatically updated when source files are updated. It is possible to write test code by editing /example/index.js or by writing hydra code into the developer console.

Non-global mode

If makeGlobal is set to false, buffers and functions can be accessed via the synth property of the hydra instance.

const h = new Hydra({ makeGlobal: false, detectAudio: false }).synth
h.osc().rotate().out()

In non-global mode, it is important to start all hydra functions, buffers, and variables by referencing the instance of hydra synth you are currently using.e.g.

const h = new Hydra({ makeGlobal: false, detectAudio: false }).synth
h.osc().diff(h.shape()).out()
h.gradient().out(h.o1)
h.render()

This also makes it possible to use more than one hydra canvas at once:

const h = new Hydra({ makeGlobal: false, detectAudio: false }).synth
h.osc().diff(h.shape()).out()
h.gradient().out(h.o1)
h.render()

const h2 = new Hydra({ makeGlobal: false, detectAudio: false }).synth
h2.shape(4).diff(h2.osc(2, 0.1, 1.2)).out()

See https://glitch.com/edit/#!/multi-hydra for a working example of multiple hydra canvases, created by Naoto Hieda.

If you would like to keep the same syntax as hydra in non-global mode, consider destructuring the object further:

const { src, osc, gradient, shape, voronoi, noise, s0, s1, s2, s3, o0, o1, o2, o3, render } = hydra
shape(4).diff(osc(2, 0.1, 1.2)).out()

hydra-ts is a fork of hydra-synth in Typescript maintained by @folz.

Known issues / troubleshooting

Vite

When using hydra with Vite, you might see the error Uncaught ReferenceError: global is not defined. This is an issue in hydra-synth dependency, which further depends on node's global.

  • To fully mitigate the issue: add a polyfill for global. (See ref)
  • To workaround: add the following snippet to vite.config.json. (See ref)
define: {
  global: {},
},

Autoplay on iOS

from issue #137

It seems on mobile safari, videos won't autoplay because of several reasons:

    <video style="position:static;top:1px;width:1px;height:1px" id="vid" autoplay loop muted playsinline crossorigin>
      <source src="https://cdn.glitch.global/8df667c3-e544-4cbb-8c16-f604238e8d2e/paper.mov?v=1682418858521">
    </video>
let v = document.getElementById("vid")
v.addEventListener('loadeddata', () => {
  s0.init({src: v})
})

Here is a live example: https://glitch.com/edit/#!/hydra-video-autoplay-ios

hydra-synth's People

Contributors

brucelane avatar cschenio avatar dependabot[bot] avatar echophon avatar geikha avatar luis11011 avatar micuat avatar misterinterrupt avatar munshkr avatar ojack avatar oscons avatar richardnias avatar rumblesan avatar ssssam avatar zachkrall avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

hydra-synth's Issues

Clear all buffers?

Hi there,

I've looked at all the documentation with no success so I am posting here for some help.
I was wondering if there is a command or a workaround to clear the defaults buffers o0, o1, o2, o3? I've seen that you can clear the source buffers like s0.clear(); but it doesn't seem to work for the output buffers.

Thank you for your help :)

Slow down (or speed up) time

It would be neat to be able to slow-down some of the faster moving animations, but keep them smooth. Ideally by a sort of slow down / speed up factor, ie 10x, 0.25x, etc. If it could be applied to everything running it could lead to cool effects if it were parameterized as well. I mainly want it to make flashier stuff into more atmospheric smooth stuff.

modulateHue doesn't seem to alter anything

This has been a bit of a mystery to me for a while now, so I'm not sure if I've just been using it wrong, but here goes.

I don't think I've ever gotten modulateHue to work to modulate one signal by another. I've been using it like this:
noise(3,1).color(1,0.5,0.5).modulateHue(osc(20),0.3).out()
and the modulateHue component isn't altering the visuals, despite not throwing any errors directly to the console. I've potentially been using this wrong, but I've never really seen this function being used at all, so I can't be certain.

This issue was present on hydra.ojack.xyz as of 09/07/2020 in Firefox 78.0.1 (Windows). I'm yet to try it with a local build.

Cell/Worley noise

Some cell noise might look pretty cool if implemented. Once I have a development environment working I could give this a shot.

`modulateRepeat` seems behaving similar to `modulate`

example: https://hydra.ojack.xyz/?code=c2hhcGUoKS5tb2R1bGF0ZVJlcGVhdChvc2MoMSUyQzEpLnBvc3Rlcml6ZSg4KS5waXhlbGF0ZSgxJTJDMSklMkMyJTJDMikub3V0KCklMEE=

here's repeat

   vec2 st = _st * vec2(repeatX, repeatY);
   st.x += step(1., mod(st.y,2.0)) * offsetX;
   st.y += step(1., mod(st.x,2.0)) * offsetY;
   return fract(st);

and modulateRepeat

   vec2 st = _st * vec2(repeatX, repeatY);
   st.x += step(1., mod(st.y,2.0)) + _c0.r * offsetX;
   st.y += step(1., mod(st.x,2.0)) + _c0.g * offsetY;
   return fract(st);

it seems like modulateRepeat is only changing the offset, which is basically the same feature as modulate. Instead repeatX repeatY should be multiplied by _c0.r _c0.g... is that correct?

The scroll function don't work well with shapes in the latest version

For example this: shape(3).scrollX(0.1, 0.1).out(). I would expect the triangle to come out on the other side, but it doesn't.
I am using the unpkg version of the lib, because I want to dynamically load hydra, but I've also checked this code in https://hydra.ojack.xyz/, with similar results:

<!DOCTYPE html>
<html lang="en">
<script src="https://unpkg.com/[email protected]/dist/hydra-synth.js"></script>
<body>
  <canvas id="hydra-canvas"></canvas>
  <script>
    new Hydra({ canvas: document.getElementById('hydra-canvas') });
    shape(3).scrollX(0.1, 0.1).out();
  </script>
</body>
</html>

non-global mode broken

I think this is a known issue that makeGlobal: false breaks something. I found that this case it ends up in an error

    let s = new Hydra({makeGlobal: false}).synth;
    let hydra2 = new Hydra();
    s.osc().add(s.shape()).out()
    osc().out()

I figured out that this is specific to combine / combineCoord functions. During the initialization, transforms' default inputs are somehow broken by the second instance. As a dirty fix, when I do

  // add extra input to beginning for backward combatibility @todo update compiler so this is no longer necessary
    if((obj.type === 'combine' || obj.type === 'combineCoord')&&obj.inputs[0]?.type!=='vec4') obj.inputs.unshift({
        name: 'color',
        type: 'vec4'
      })

in generator-factory.js, the 2 instances work properly. Of course this is not a fix - but I think somewhere in the code, obj.inputs is exposed globally and referenced twice.

How to properly stop a hydra-synth instance?

Hello!

I'm working on a project where I have multiple hydra-synth instances which appear and disappear. How can I clean up an instance properly after I don't need it any more?

I've put up a PR (#39) to expose the render loop which I'd then be able to stop, but I'd probably need to prune other objects/loops as well?

I'm happy to help researching and fixing :)

Thanks!

Hydra code as HTML backgroud - without Node?

Hello! Is is possible to have an Hydra scene running as a HTML background?
Is it possible to run an Hydra-Synth without Node.js on the server?
My hosting server does not support Node...
It does runs the Hydra-Editor alright. Maybe a way stripping down the editor to run just a specific Hydra code, without the Editor UI?

Thank you!

Uncaught TypeError: Cannot read property 'regl' of undefined

hi, im new to atom and to hydra. ive been using the web editor glitch tool and it's really a pleasure to spend my time there. so i decided to run it in atom to go deep, but after installing hydra-atom package and try to run toggle within atom i see this error and couldnt do anything.
i understand that this error could be already resolved in another post or issue but i didnt find it.
thanks anyway

details:

/Users/urbi/.atom/packages/atom-hydra/lib/main.js:229

TypeError: Cannot read property 'regl' of undefined
at Main.stop (/Users/urbi/.atom/packages/atom-hydra/lib/main.js:229:16)
at Object.toggle (/Users/urbi/.atom/packages/atom-hydra/lib/atom-hydra.js:40:24)
at HTMLElement.atomHydraToggle (/Users/urbi/.atom/packages/atom-hydra/lib/atom-hydra.js:18:39)
at CommandRegistry.handleCommandEvent (/Applications/Atom.app/Contents/Resources/app/static/:11:349123)
at CommandRegistry.dispatch (/Applications/Atom.app/Contents/Resources/app/static/:11:347598)
at AtomEnvironment.dispatchApplicationMenuCommand (/Applications/Atom.app/Contents/Resources/app/static/:1:718718)
at EventEmitter.t (/Applications/Atom.app/Contents/Resources/app/static/:1:726641)
at EventEmitter.emit (events.js:182:13)

don't automatically connect microphone

It'd be great to not ask users for mic permissions when a sketch doesn't require it (similar to how the camera currently works). I'm not sure exactly what the best way to do this would be without changing the API... perhaps add a getter method for a or a.fft with an if statement to check and see if the mic has been initialized... if it hasn't go ahead and ask for permission and initialize it, if it has just return the requested member variable.

layer behaves weirdly when nested

for example:

shape().r().color(.5,0,0)
  .layer(shape(4).r().scrollX(.1).color(0,0,.5)
    .layer(shape(99).r().scrollX(-.1).color(0,.5,.5)))
  .out()

https://hydra.ojack.xyz/?code=c2hhcGUoKS5yKCkuY29sb3IoLjUlMkMwJTJDMCklMEElMjAlMjAubGF5ZXIoc2hhcGUoNCkucigpLnNjcm9sbFgoLjEpLmNvbG9yKDAlMkMwJTJDLjUpJTBBJTIwJTIwJTIwJTIwLmxheWVyKHNoYXBlKDk5KS5yKCkuc2Nyb2xsWCgtLjEpLmNvbG9yKDAlMkMuNSUyQy41KSkpJTBBJTIwJTIwLm91dCgp

image

layer implementation:

   return vec4(mix(_c0.rgb, _c1.rgb, _c1.a), _c0.a+_c1.a);

https://github.com/ojack/hydra-synth/blob/master/src/glsl/glsl-functions.js#L707

I guess the overlapping part has excess alpha (a=2) and ended up mixing with weird value. Not sure if the return alpha should be capped, for example.

readme for dev needs update

https://github.com/ojack/hydra-synth/blame/master/README.md#L16

the command no longer exists. The proper sequence should be

npm install
npm run build

and I don't know why but in my environment, build fails with EACCES when outputting to a bundle with >. I had to change it to

    "build-hydra": "browserify hydra-synth.js --standalone Hydra | tee dist/hydra-synth.js",
    "build-shader-generator": "browserify shader-generator.js --standalone HydraShaders | tee dist/hydra-shaders.js",

not sure if this is related to the installation of browserify

allow setting filtering method for outputs

this could allow more analog-like visuals which many users may want. for example andrei jay's waaaves works on linear.

i'll do a PR if you see it fit.

this also adds up to the long-term idea for different,,, sort of,,, hydra profiles i'd call them. (toggling wrapping, a purely premultiplied mode, a purely straight alpha mode, a default mode etc)

issue sort of mentioned on hydra-synth/hydra#145

simple example:

setMode = function(mode='nearest'){
    mode == 'nearest' ? mode :
		(mode == 'linear' ? mode : 'nearest');
    this.fbos = (Array(2)).fill().map(() => this.regl.framebuffer({
		color: this.regl.texture({
		mag: mode,
		min: mode,
		width: width,
		height: height,
		format: 'rgba'
		}),
		depthStencil: false
  	}));
}

example in editor
another more beautiful example
a more more beautiful example
i mean just compare nearest and linear here

Dynamically setting canvas width & height

Hi ! I'm really enjoying this fantastic piece of work !

Any progress on dynamically setting the canvas width / height, or exposing these as options on the api ?

Thanks !

iOS initCam fix

currently initCam on iOS fails because of this

https://stackoverflow.com/questions/53483975/navigator-mediadevices-getusermedia-not-working-on-ios-12-safari

which means

    video.setAttribute('autoplay', '');
    video.setAttribute('muted', '');
    video.setAttribute('playsinline', '');

has to be added here

https://github.com/ojack/hydra-synth/blob/main/src/lib/webcam.js#L19

I can't make PR right now but quickly test that it works by editing the bundled hydra-synth

https://midnight-substantial-oxygen.glitch.me/

Black screen in some browsers

When opening https://hydra-editor.glitch.me/ with microsoft edge there appears only a black screen. The error the console gives me is:

SCRIPT438: SCRIPT438: Object doesn't support property or method 'captureStream' bundle.js (22526,7)

I know this issue exists also on some browsers on mobile phones, didn't check myself though.

Headless Hydra

Make Hydra runnable in a headless-gl context. Useful for offline rendering of Hydra sketches. Will elaborate as I keep experimenting with what could be the cleanest way to do this code-wise.

composable-glsl-transforms.js server version not updated when running locally

osc(...).color(...).sub is not a function

glsl-transforms.js

  sub: {
    transformType: 'color',
    inputs: [
      {
        name: 'src',
        type: 'image'
      }
    ],
    fragBody: `
      c -= texture2D(<0>, uv);
    `
  }

composable-glsl-functions.js

sub: {
    type: 'combine',
    inputs: [
      {
        name: 'color',
        type: 'vec4'
      },
      {
        name: 'amount',
        type: 'float',
        default: 0.5
      }
    ],
    glsl: `vec4 sub(vec4 c0, vec4 c1, float amount){
            return (c0-c1)*amount + c0*(1.0-amount);
          }`
  }

Issue

Running a local version of hydra referring to the current version of hydra-synth rather than the package. I've bee trying to add a subtract function, but despite me editing the glsltransforms section the editor doesn't recognise it as a function. On inspection the server version of composable-glsl-transforms isn't updated with my changes. Do I need to run anything else, or am I missing something?

Cheers ๐Ÿ‘

resolution.xy

put resolution.xy where necessary instead of resolution because sharing the shader to some apps resolution is a vec3:

  • in GeneratorFactory.js line 227 vec2 st = gl_FragCoord.xy/resolution; modify to resolution.xy
  • same in ShaderManager.js line 11
  • in composable-glsl-functions.js line 865

Media aspect ratio not preserved

When using either the video or audio inputs from the examples:

s0.initVideo("https://media.giphy.com/media/26ufplp8yheSKUE00/giphy.mp4", { flipY: true })
src(s0).out()
...
s0.initImage("https://upload.wikimedia.org/wikipedia/commons/2/25/Hydra-Foto.jpg")
src(s0).out()

the image or video gets fullscreened and the aspect ratio is not preserved, making them skewed.
Is there a parameter that can be used here that I am missing?
If not, is this something that I can add to the source? (For example: horizontal or vertical padds that would keep the aspect ratio)

Documentation Generator

Are there any plans/need to have an automated documentation generation system?

The docs I found at the following links seem enough for anyone to get started.

If we are looking to have more "professional" looking docs I'd like to help out with that. (Also a good chance for me to play with more sketches (: ).

ThreeJS maintains some neat documentation on their project. I like the search feature the most. They also explain some fundamental concepts really well.

Thoughts?

The alpha issue (premultiplication and overloading)

The alpha issue (premultiplication and overloading)

I create this issue after discussion in the Discord and various hours of testing different ideas. Thanks to @BenjaminHolland for the resources he shared in #107 which have been very helpful on finding weird behaviours while modifying alpha.


On premultiplication of alpha values

What is premultiplication?

If an alpha channel is used in an image, there are two common representations that are available: straight (unassociated) alpha and premultiplied (associated) alpha.

* With straight alpha, the RGB components represent the color of the object or pixel, disregarding its opacity.
* With premultiplied alpha, the RGB components represent the emission of the object or pixel, and the alpha represents the occlusion. 

source

Basically:

//straight alpha
return vec4(color, alpha)
//premultiflied alpha
return vec4(color*alpha, alpha)

Alpha compositing and different operations must be done consistently with the type of alpha being used.
Unfortunately, Hydra is inconsistent with the way of handling alpha.

The inconsistency

Functions that output / assume an input of premultiplied alpha:

  • luma (outputs & inputs)
  • mask (outputs & inputs)
  • layer (inputs, almost)
    • The current implementation is weird, it doesn't work correctly with neither actually. And also has this bug

Functions that output / input straight alpha:

  • Literally the whole rest of them that use color

This inconsistency can be seen in the following example, taken from @micuat 's 2nd tutorial on feedback.

src(o0)
	.modulateHue(src(o0).scale(1.01),1)
	.layer(
  		osc(10,.15,2)
  			.mask(
             			shape(4,0.5,.1)
            		)
  	)
	.out(o0)

open in editor

See how using pre-multiplied (abbreviated as pm) alpha versions of layer and mask gives the expected result. Actually, this is using the current version of mask.

await loadScript('https://hydra-extensions.glitch.me/hydra-alpha-issue-playground.js')

src(o0)
	.modulateHue(src(o0).scale(1.01),1)
	.pmlayer(
  		osc(10,.15,2)
  			.pmmask(
             			shape(4,0.5,.1)
            		)
  	)
	.out(o0)

open in editor

vec4 pmlayer(vec4 _c0, vec4 _c1) {
    return _c1+(_c0*(1.0-_c1.a));
}

vec4 pmmask(vec4 _c0, vec4 _c1) {
    float a = _luminance(_c1.rgb);
    return vec4(_c0.rgb*a, _c0.a*a);      
}

And now with straight (st) alpha:

await loadScript('https://hydra-extensions.glitch.me/hydra-alpha-issue-playground.js')

src(o0)
	.modulateHue(src(o0).scale(1.01),1)
	.stlayer(
  		osc(10,.15,2)
  			.stmask(
             			shape(4,0.5,.1)
            		)
  	)
	.out(o0)

open in editor

vec4 stlayer(vec4 _c0, vec4 _c1) {
    float a = _c1.a + (_c0.a*(1.0-_c1.a));
    vec3 color = (_c1.rgb*_c1.a) + 
                    (_c0.rgb*_c0.a*(1.0-_c1.a));
    color /= a + 0.0000001;
    return vec4(color,a);
}

vec4 stmask(vec4 _c0, vec4 _c1) {
    float a = _luminance(_c1.rgb*_c1.a);
    return vec4(_c0.rgb, _c0.a*a);   
}

Both layering methods were taken from the alpha composition wikipedia page

The screen lies to you

Edit: This behaviour can be different depending if you're on a Mac machine. As premult is default.

osc().color(1,1,1,0)
	.out()

There should be nothing in the screen, yet the bright colors are still showing. (If you change the website's background to red you'll see the oscillator with a red background instead of black). I think this is because by default WebGL assumes you'll use premultiplied alpha.

See this stackoverflow question.

Straight vs premultiplied, what should Hydra use internally?

EDIT: My opinion changed, keep reading the comments.

I believe Hydra should use straight alpha internally (at least by default), most of the functions already work that way and I believe that's the default mindset of any shaderer. Also because premultiplication is a lossy transformation (you can't get the exact same rgb colors back after premultiplying). In the long run we could build an option into Hydra to set which alpha mode to use, since there has also been talk about enabling/disabling wrapping. Also, [most] PNG images use straight alpha, so setting the default to be premultiplied would also imply taking care of that. This is also why PNG images look so weird in Hydra unless you layer them:

await loadScript("https://hydra-extensions.glitch.me/hydra-glsl.js")

s0.initImage('https://i.imgur.com/3KPzyjx.png')
src(s0)
	//.glslColor('float a = clamp(c0.a,0.0,1.0); vec4(c0.rgb*a,a)') //premultiplication
	.out(o0)

Proposed solution:

  • Set every function to work with straight alpha
  • Premultiply the final output set in renderFbo() and renderAll()

Output premultiplication could be as such:

void main () {
    [...]
    float alpha = clamp(gl_FragColor.a, 0.0, 1.0);
    gl_FragColor.rgb *= alpha;
    gl_FragColor.a = alpha;
}

Drawbacks

  • The changes in mask will have a visual impact, but this is technically a fix, as we see in the example above.
  • The changes in luma will be somewhat noticeable too (at least when layering over opaque backgrounds). luma currently leaves some strong dark edges (caused by the premultiplication) which various users already expect from luma. Correcting luma will lighten them a bit.

These also have to do with the correction of layer.

luma example:
await loadScript('https://hydra-extensions.glitch.me/hydra-alpha-issue-playground.js')

solid(1,.5,0)
	.stlayer(osc().stluma(.5,.1))
	.out(o0)
solid(1,.5,0)
	.layer(osc().luma(.5,.1))
	.out(o1)
solid(0,.5,1)
	.stlayer(osc(60,.1,2).stluma())
	.out(o2)
solid(0,.5,1)
	.layer(osc(60,.1,2).luma())
	.out(o3)

render()

I think the changes are light enough for it not to be too big of a deal. But we can vote on the Discord or here maybe.

On the overloading of alpha values

Using Hydra is very common to overload alpha values while using add or color. Or cancelling them out with sub (and not realizing because currently bright colors with 0 alpha still appear on the screen!).

What to do with add and sub?

add

Adding two images in Hydra also adds their alpha. In most cases this ends up with an image full of alpha values of 2.0.
This isn't much of a problem if we set on a specific way of handling alpha. For example, the straight alpha version of layer handles overloaded alpha with no problem:

await loadScript('https://hydra-extensions.glitch.me/hydra-alpha-issue-playground.js')

b = ()=> osc().add(noise())
b().out()
solid(0,0,0,0).stlayer(b()).out(o1)
src(o1).sub(o0).out(o2)
b().sub(solid(0,0,0,0).stlayer(b())).out(o3)

// change st layer to layer and see the original problem

render()

open in editor

Therefore in this scenario most users won't notice the overloading. No current Hydra function will suffer from this. However almost all of the functions proposed in #107 will. This could be fixed by clamping alpha before these specific functions. However the biggest intuition problem comes with alpha cancellation:

sub

Substracting an opaque image from another results in a blank image (and again, the user can't realize because bright colors still appear currently even if they have 0 alpha). This can be very couter-intuitive and the current way of dealing with this would be to add .color(1,1,1,0) to the input of sub, which is tedious. My first idea was to change both addition and substraction to:

void add(vec4 _c0, vec4 _c1, float amount, float alpha){
    _c1.a *= alpha;
    return _c0+(_c1*amount);
}
void sub(vec4 _c0, vec4 _c1, float amount, float alpha){
    _c1.a *= alpha;
    return _c0+(_c1*amount);
}

However this would mean that when alpha is set to 0, when adding, it wouldn't overload, but it also wouldn't add alpha at all. Which is counter-intuitive if the image to which the add function is being applied to has transparencies. It would be like saying a().add(b().mask(a())).

A solution could be to treat alpha in the same way that diff does:

void add(vec4 _c0, vec4 _c1, float amount){
    return vec4(_c0.rgb+(_c1.rgb*amount),max(_c0.a,_c1.a));
}
void sub(vec4 _c0, vec4 _c1, float amount){
    return vec4(_c0.rgb+(_c1.rgb*amount),max(_c0.a,_c1.a));
}

Return the maximum alpha value from each input. This could work, since it's super rare to want to use overloaded values from alpha. And these rare cases could be achived by doing .color(1,1,1,2) // or any n > 1, which is more intuitive than having to do .color(1,1,1,0) everytime images are substracted.

Other ideas

  • Adding a normalize or clamp function that simply clamps every channel to be between 0 and 1, could be useful for some arithmetics.
  • Actually add the functions in #107
    • Specially the source over, which I'd call unlayer. Basically layers the GlslSource to which it is being applied over the one provided in the argument.
  • Add a div function for more arithmetics.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.