elm-explorations / webgl Goto Github PK
View Code? Open in Web Editor NEWFunctional rendering with WebGL in Elm
Home Page: https://package.elm-lang.org/packages/elm-explorations/webgl/latest/
License: BSD 3-Clause "New" or "Revised" License
Functional rendering with WebGL in Elm
Home Page: https://package.elm-lang.org/packages/elm-explorations/webgl/latest/
License: BSD 3-Clause "New" or "Revised" License
Is there any way to use this package with the option of preserveDrawingBuffer True?
I've noticed that on two occasions (once in someone's Firefox browser and I don't recall which browser the other person was using), textures would have some kind of color correction applied. This would cause problems for this app I'm working on https://town-collab.app where certain texture colors (such as FF00FF) get replaced with user chosen colors at runtime. The color correction causes the game to look like this
The fix was to add gl.pixelStorei(gl.UNPACK_COLORSPACE_CONVERSION_WEBGL, false);
. Ideally the webgl package should do this automatically though?
Perhaps also gl.pixelStorei(gl.UNPACK_PREMULTIPLY_ALPHA_WEBGL, false);
should be used. I haven't encountered any problems with premultiplication but I suspect some browsers are going to randomly have different default values here too.
Edit: For reference, I got the two lines of code from here https://stackoverflow.com/questions/31500597/fragment-shader-wrong-color-value-from-texture
In order to alpha blend textures correctly, their color channel needs to be premultiplied by the alpha channel. I believe WebGL supports doing this when loading textures but that feature isn't exposed in this package.
For that reason I think premultiply : Bool
should be added as an additional field available when calling WebGL.Texture.loadWith
. For WebGL.Texture.load
the default value will be False
.
This was first reported at ianmackenzie/elm-3d-scene#87 but seems to be a general WebGL issue. On some versions of Microsoft Edge (tested on version 44.19041.1.0 on Windows 10), replacing a general HTML element with a WebGL canvas some time after the Elm app has started results in the WebGL scene not actually being displayed until a redraw is triggered (window resize, user drags the mouse over the canvas element, etc.).
SSCCE is at https://ellie-app.com/9tMP6X3FjVTa1. The expected behavior is that the page will display "Loading..." for 1 second and then switch to a blue square. On Edge, this can fail and the text disappears but the blue square does not appear until the page is resized or a redraw is otherwise forced. Replacing the Html.text "Loading..."
with a dummy WebGL scene like
WebGL.toHtml
[ Html.Attributes.width 300
, Html.Attributes.height 300
]
[]
fixes the issue, so it seems to be a problem when the virtual DOM replaces a non-WebGL element with a WebGL one.
Sometimes when hot-reloading the webpage, artifacts can get introduced into the scene such as where the red arrow is pointing
Hot-reloading sometimes also causes entities to not render.
In terms of frequency, the former seems to rarely happen (I delayed posting this issue because I couldn't reproduce it for a couple days) and the latter happens maybe 1 in 5 hot-reloads.
The mesh corruption artifacts don't seem to go away until I do a page refresh while the entities disappearing usually is fixed by doing one or two more hot-reloads.
This is occurring on Chrome both on my Mac and Windows computers. It might have also occurred on Firefox though I might be misremembering as I haven't tested that in a while.
Edit: Now I'm getting the mesh corruption repeatedly. I'm not sure what conditions cause it to happen more frequently.
Hi, I'm interested in rendering circuit boards similar to this
https://3c1703fe8d.site.internapcdn.net/newman/gfx/news/hires/2009/runningelect.jpg
Ideally I'd generate one large mesh for the entire circuit board. The layout of the circuit board isn't known at compile time though and as I understand it, all meshes need to be known at compile time, as generating them in the view causes a memory leak.
It seems like currently the best I can do then is have a rectangle and circle mesh at compile time and use a bunch of them to render the entire circuit board.
As a test to see if it would be feasible to do this I ran a quick benchmark by drawing the same circle mesh with varying transformation matrices. Around 600 circles per frame was the limit before the frame rate would start to drop. This seems too low to achieve what I'm aiming for.
So if possible, it would be great to support creating new meshes after compile time.
Edit: I realized that meshes can still be created during runtime so long as they are stored in the model and not modified. This works well enough for my purposes though it would still be nice to have some way to free/modify existing meshes.
Following up from elm-community/webgl#48, https://developer.mozilla.org/en-US/docs/Web/API/WebGL_API/WebGL_best_practices lists OES_standard_derivatives
as a "universally supported WebGL 1 extension". So it should be fairly safe to add
gl.getExtension('OES_standard_derivatives');
to the JS context initialization code and prepend
#extension GL_OES_standard_derivatives : enable
to each generated fragment shader. This would then open up a bunch of cool possibilities for things like edge highlighting.
A fragment shader like this:
fragmentShader : Shader {} Uniforms { vcoord : Vec2 }
fragmentShader =
[glsl|
uniform sampler2D texture;
varying vec2 vcoord;
// This ain't gonna work now
void main () {
gl_FragColor = texture2D(texture, vcoord);
}
|]
(The build uses Webpack, hence these messages)
ERROR in ./src/elm/Main.elm 12271:97
Module parse failed: Unexpected token (12271:97)
You may need an appropriate loader to handle this file type.
| };
| var author$project$GL$fragmentShader = {
> src: '\n uniform sampler2D texture;\n varying vec2 vcoord;\n\n // This ain't gonna compile now\n\n void main () {\n gl_FragColor = texture2D(texture, vcoord);\n }\n ',
| attributes: {},
| uniforms: {texture: 'texture'}
@ ./src/index.js 4:0-37 132:8-11
Note the src
of the author$project$GL$fragmentShader
object is delimited by '
, but the inline comment's '
is not escaped here thus breaking the JS (hence Webpack then failing).
If the same mesh is rendered many times in a row using the same shaders, the current behaviour would still call gl.bindBuffer
, gl.enableVertexAttribArray
and gl.vertexAttribPointer
for the same mesh. These extra calls could be potentially avoided.
This should improve the performance of 2D games with many sprites, that are rendered using the same quad mesh, the same shaders, but different uniforms. This should also improve the rendering of scenes like this http://unsoundscapes.com/elm-physics/examples/boxes/
It seems that gl.getProgramParameter
, gl.getActiveAttrib
, gl.getAttribLocation
can be cached for each program and not called for each draw.
This could improve the performance a little bit.
In your documentation, you said
Do not generate meshes in view, read more about this here
I didn't see any way of the alternative. All your example call WebGL.toHtml
that end up generating a mesh on the fly.
Did a miss something? Can you clarify what you mean by that? I supposed it's better to generate the mesh on a init
function and store them the application state somehow?
Thx!
Just pointing to the same issue in the old repo that is still relevant
elm-community/webgl#46
I have the following Elm type for uniforms:
type alias Uniforms = { cellColors : List Vec3 }
In a vertex shader I have the following declaration for an array of vec3 uniform:
uniform vec3 cellColors[9];
The GLSL parser, however, is inferring/parsing the type of the cellColors
variable to be Vec3
, and not Array Vec3
.
It seems as if this is a bug in the underlying GLSL parsing library: noteed/language-glsl#19
Is there a way to work around this bug?
Right now to clear the stencil or depth buffer it is necessary to do a hack like drawing a full-screen quad. It would be useful (and presumably more efficient) to have something like some special Entity
values WebGL.clearDepthBuffer
and WebGL.clearStencilBuffer
that would end up simply calling gl.clearDepth()
or gl.clearStencil()
when drawn.
We have several projects using Elm+WebGL for now and we produce generative graphics with that and we're very happy with using it. Thanks to Elm, we have type safety, and our code defines algorithms in a very clear way. Thanks to WebGL package and its simple and powerful API, doing shaders and passing complex structures to them is easy.
However, for more and more times we encounter the issue that it would be interesting to work with dynamic textures—for example, one would create a kaleidoscopic effect using the part of the viewport that is already rendered. Sure, we can use several canvases for that, but sometimes one layer is needs to be reflected and another is not, or the effect result itself is transforming inside the complex scene.
So, I would kindly ask to think once again on adding this functionality to the API. I did the Pull Request for the Elm 0.18 version, but now in 0.19 it's harder to have your own forks of Elm Packages which use JavaScript, except patching them on build.
The first frame may take a while to render because the WebGL scene may be composed out of different combinations of shaders.
WebGL best practices recommend that the shader compilation and program linking should be run in parallel.
This would probably require the following steps:
The OES_element_index_uint
extension is listed as a "universally supported extension", and enabling it should allow elm-explorations/webgl
to support indexed meshes with a basically unlimited number of vertices (theoretically billions instead of the current 65536 vertex limit).
This will also require updating the JS code to use Uint32Array
instead of Uint16Array
.
Support for instancing, aka being able to draw the same mesh many times with different uniforms, would be very useful for my project.
Here's an example of my current work around to be able to efficiently draw many instances of a single mesh (I've excluded some details but this is the general idea).
type alias ChipTileVertex =
{ position : Vec3
, instanceIndex : Float
}
type alias ChipTileUniforms =
{ perspective : Mat4
, camera : Mat4
, time : Second
, instanceOffset0 : Vec3
, instanceOffset1 : Vec3
, instanceOffset2 : Vec3
, instanceOffset3 : Vec3
, instanceOffset4 : Vec3
, instanceOffset5 : Vec3
, instanceOffset6 : Vec3
, instanceOffset7 : Vec3
, instanceOffset8 : Vec3
, instanceOffset9 : Vec3
, instanceOffset10 : Vec3
, instanceOffset11 : Vec3
, instanceOffset12 : Vec3
, instanceOffset13 : Vec3
, instanceOffset14 : Vec3
, instanceOffset15 : Vec3
, instanceOffset16 : Vec3
, instanceOffset17 : Vec3
, instanceOffset18 : Vec3
, instanceOffset19 : Vec3
}
chipTileUniforms : InGameModel -> Size Int -> Array Point2d -> ChipTileUniforms
chipTileUniforms inGameModel windowSize particles =
let
getOffset index =
Array.get index particles
|> Maybe.map
(\position ->
Vec3.vec3 (Point2d.xCoordinate position) (Point2d.yCoordinate position) 0
)
|> Maybe.withDefault (Vec3.vec3 -999999 -999999 0)
in
{ perspective = perspective windowSize
, camera = cameraMatrix inGameModel
, time = inGameModel.inGameTime
, instanceOffset0 = getOffset 0
, instanceOffset1 = getOffset 1
, instanceOffset2 = getOffset 2
, instanceOffset3 = getOffset 3
, instanceOffset4 = getOffset 4
, instanceOffset5 = getOffset 5
, instanceOffset6 = getOffset 6
, instanceOffset7 = getOffset 7
, instanceOffset8 = getOffset 8
, instanceOffset9 = getOffset 9
, instanceOffset10 = getOffset 10
, instanceOffset11 = getOffset 11
, instanceOffset12 = getOffset 12
, instanceOffset13 = getOffset 13
, instanceOffset14 = getOffset 14
, instanceOffset15 = getOffset 15
, instanceOffset16 = getOffset 16
, instanceOffset17 = getOffset 17
, instanceOffset18 = getOffset 18
, instanceOffset19 = getOffset 19
}
chipTileVS : Shader ChipTileVertex ChipTileUniforms { vcolor : Vec4 }
chipTileVS =
[glsl|
attribute vec3 position;
attribute float instanceIndex;
uniform mat4 perspective;
uniform mat4 camera;
uniform vec3 instanceOffset0;
uniform vec3 instanceOffset1;
uniform vec3 instanceOffset2;
uniform vec3 instanceOffset3;
uniform vec3 instanceOffset4;
uniform vec3 instanceOffset5;
uniform vec3 instanceOffset6;
uniform vec3 instanceOffset7;
uniform vec3 instanceOffset8;
uniform vec3 instanceOffset9;
uniform vec3 instanceOffset10;
uniform vec3 instanceOffset11;
uniform vec3 instanceOffset12;
uniform vec3 instanceOffset13;
uniform vec3 instanceOffset14;
uniform vec3 instanceOffset15;
uniform vec3 instanceOffset16;
uniform vec3 instanceOffset17;
uniform vec3 instanceOffset18;
uniform vec3 instanceOffset19;
uniform float time;
varying vec4 vcolor;
void main () {
mediump int index = int(instanceIndex);
mediump vec3 offset =
(instanceOffset0 * float(index == 0)) +
(instanceOffset1 * float(index == 1)) +
(instanceOffset2 * float(index == 2)) +
(instanceOffset3 * float(index == 3)) +
(instanceOffset4 * float(index == 4)) +
(instanceOffset5 * float(index == 5)) +
(instanceOffset6 * float(index == 6)) +
(instanceOffset7 * float(index == 7)) +
(instanceOffset8 * float(index == 8)) +
(instanceOffset9 * float(index == 9)) +
(instanceOffset10 * float(index == 10)) +
(instanceOffset11 * float(index == 11)) +
(instanceOffset12 * float(index == 12)) +
(instanceOffset13 * float(index == 13)) +
(instanceOffset14 * float(index == 14)) +
(instanceOffset15 * float(index == 15)) +
(instanceOffset16 * float(index == 16)) +
(instanceOffset17 * float(index == 17)) +
(instanceOffset18 * float(index == 18)) +
(instanceOffset19 * float(index == 19));
gl_Position = perspective * camera * vec4(position + vec3(offset.xy, 0.0), 1.0);
vcolor = vec4(0.0, 0.5, 0.0, 0.5);
}
|]
It's very repetative code and if I want a new shader that also draws multiple instances, I need to copy paste this code.
Also while it's faster than making one draw call per instance, having to do the following in the vertex shader is probably expensive.
mediump vec3 offset =
(instanceOffset0 * float(index == 0)) +
(instanceOffset1 * float(index == 1)) +
(instanceOffset2 * float(index == 2)) +
(instanceOffset3 * float(index == 3)) +
(instanceOffset4 * float(index == 4)) +
(instanceOffset5 * float(index == 5)) +
(instanceOffset6 * float(index == 6)) +
(instanceOffset7 * float(index == 7)) +
(instanceOffset8 * float(index == 8)) +
(instanceOffset9 * float(index == 9)) +
(instanceOffset10 * float(index == 10)) +
(instanceOffset11 * float(index == 11)) +
(instanceOffset12 * float(index == 12)) +
(instanceOffset13 * float(index == 13)) +
(instanceOffset14 * float(index == 14)) +
(instanceOffset15 * float(index == 15)) +
(instanceOffset16 * float(index == 16)) +
(instanceOffset17 * float(index == 17)) +
(instanceOffset18 * float(index == 18)) +
(instanceOffset19 * float(index == 19));
I admit, I haven't entirely thought out how the elm-webgl API should be modified in order to support instancing. My initial idea is to change the uniform parameter in entity
to a list. I'm happy to discuss how this might be organized.
I did some GPU profiling with Spector.js. It's reporting that there are some redundant calls made before and after each drawElements call.
In this screeshot for example, all the calls with orange text are apparently redundant.
Here is a capture I took of my game if you want to see more details.
capture 17_59_02.json.zip
Just ran into an issue where I was getting this error:
Uncaught Error: No info available for: 35676
at _WebGL_doBindAttribute (Main.elm:6110)
...
After a little digging I discovered it's because there's no support for mat4 attributes.
It seems like this should be possible to do, and pretty straightforward: https://stackoverflow.com/questions/38853096/webgl-how-to-bind-values-to-a-mat4-attribute
My use case is passing in a separate transform for each Entity as an attribute, in order to do the transformation on the GPU. I'm willing to make the change, just wanted to get some opinions first.
Uncaught Error: Trying to use
(==)
on functions. There is no way to know if functions are "the same" in the Elm sense. Read more about this at https://package.elm-lang.org/packages/elm/core/latest/Basics#== which describes why it is this way and what the better version will look like.
module Main exposing (main)
import Browser
import Html exposing (Html)
import Json.Decode exposing (Value)
import Task exposing (Task)
import WebGL.Texture as WebGL
init _ =
( ()
--here is problem
, Task.map2 (==) (WebGL.load image) (WebGL.load image)
|> Task.attempt identity
)
main : Program Value () (Result WebGL.Error Bool)
main =
Browser.element
{ init = init
, view = \_ -> Html.text "Success"
, subscriptions = \_ -> Sub.none
, update = \_ m -> ( m, Cmd.none )
}
image =
"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mPUDLS6DAAC6AGJTI9bHgAAAABJRU5ErkJggg=="
Apologies if this is already supported or due to be released, I couldn't find any references, but I think it's sort of related to the open PR about creating textures from bytes.
Textures can be loaded from a url, but I'd like to be able to read image data directly from an image I've already loaded, or a 2D canvas that is already on the page.
The following code (Ellie link) will show a green rectangle on Firefox and a black rectangle on Chrome and Safari. This behavior is consistent both on Windows and Mac OS.
If you change WebGL.entity
to WebGL.entityWith []
then all browsers will be consistent in showing a green rectangle.
module Main exposing (main)
import Browser.Events
import Browser
import Html exposing (Html, button, div, text)
import Html.Events exposing (onClick)
import Math.Matrix4 exposing (Mat4)
import Math.Vector2 as Vec2 exposing (Vec2)
import Math.Vector4 as Vec4 exposing (Vec4)
import WebGL exposing (Mesh, Shader)
main : Platform.Program () () ()
main =
Browser.element
{ init = \_ -> ((), Cmd.none)
, update = \_ _ -> ((), Cmd.none)
, view = view
, subscriptions =
\_ -> Browser.Events.onAnimationFrame (\_ -> ())
}
view model =
WebGL.toHtmlWith
[ WebGL.stencil 0
]
[]
[ WebGL.entity
vertexShader
fragmentShader
viewportSquare
{ color = Vec4.vec4 0 1 0 1
, matrix = Math.Matrix4.identity
}
]
viewportSquare : Mesh { position : Vec2 }
viewportSquare =
WebGL.triangleFan
[ { position = Vec2.vec2 -1 -1 }
, { position = Vec2.vec2 1 -1 }
, { position = Vec2.vec2 1 1 }
, { position = Vec2.vec2 -1 1 }
]
vertexShader : Shader { position : Vec2 } { a | matrix : Mat4 } {}
vertexShader =
[glsl|
attribute vec2 position;
uniform mat4 matrix;
void main () {
gl_Position = matrix * vec4(position, 0.0, 1.0);
}
|]
fragmentShader : Shader {} { a | color : Vec4 } {}
fragmentShader =
[glsl|
precision mediump float;
uniform vec4 color;
void main () {
gl_FragColor = color;
}
|]
Currently, the code in WebGL.js uses gl.drawElements
to draw all entities, constructing a 'dummy' index buffer for non-indexed meshes (which are in fact all current meshes except for those constructed using WebGL.indexedTriangles
).
It would be better to use gl.drawArrays
instead for non-indexed meshes; this would have a few advantages:
gl.drawArrays
, since they don't have explicit indicesThe documentation states
Is there any other way to build a Shader without the [glsl| ...|]
syntactical construct ? I would like to generate shader code in Elm, and therefore I don't have any shader code at compile time to put in there.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.