Comments (14)
Ok, digging back in, it seems mostly a firefox dev mailing list discussion (e.g., here
https://groups.google.com/forum/#!topic/mozilla.dev.platform/DcSi_wLG4fc/discussion and here https://support.mozilla.org/en-US/questions/1222676), and it's unclear where they are on their thinking here.
That said, when I read through the spec (https://w3c.github.io/ambient-light/), it feels like some of the concerns may be relevant if we enable this capability at the "lowest" level of permission. For example, a malicious script could potentially detect that multiple users are together by the ambient light sensing pattern, or even if they were in a certain place. These are the kind of threats that we need to be conscious of.
Now, if we only enable estimation when users have granted access to "world knowledge" of some form, I think many of these threats are less compelling, since there is so much other information to leverage.
from lighting-estimation.
I've seen a bunch of discussion about that API being removed (I think some browsers already have?). The issue (as with the devicemotion API) is that it can be accessed in the background without permission, and there were some crazy threat scenarios painted for how it can be abused.
from lighting-estimation.
To clarify, I wasn't necessarily proposing using the ambient light api! Just pointing out that we needed to pay attention to it's existence. Do you have a link to the discussion about it being avoided?
from lighting-estimation.
I want to implement the "Lighting estimation for WebXR" for my contents,
but IMHO, the current AmbientLightSensor API seems not suitable for AR.
The LightEstimation of the AR platforms are tightly coupled with the "frame",
because we can estimate the light of the AR scene with the background (camera preview ) of the scene, not with the additional sensor like illuminance sensor.
So, why don't we make separated API for WebXR, that are related to the XRFrame ?
(Light estimation for AR #401)
from lighting-estimation.
@kyungtae I think that was the rough conclusion (that we probably needed a separate API from the Ambient Light API), for the reason you stated (tied with frame).
from lighting-estimation.
IMHO, the LightEstimation API could be like below.
Because we can estimate the light of the AR scene with the environment (camera preview ) of the scene, we can add "XRLightEstimation" attribute in XRFrame.
[SecureContext, Exposed=Window] interface XRFrame { ... readonly attribute XRLightEstimation? lightEstimation; }
Additionally, the XRLightEstimation consists of the intensity and the color,
and the Application can apply them into the intensity and the color of the ambient light.
Color would be optional because some platforms may support only the intensity.
[SecureContext, Exposed=Window] interface XRLightEstimation { readonly attribute double intensity; readonly attribute double? color; };
from lighting-estimation.
When this came up before, the question raised was "are such simple values adequate"? Will platforms want to support this simple approach? Do we need to support a couple of variations or kinds of data?
I would be in favor of having this be a separate repo with a proposal that could be worked on, and it would be good if some of the platform engineers could chime in.
from lighting-estimation.
I think the simplicity of this proposal works as long as we specify that it's an ambient estimate. So if the above IDL is tweaked to say
[SecureContext, Exposed=Window] interface XRLightEstimation {
readonly attribute double ambientIntensity;
readonly attribute double? ambientColor;
};
I think that implicitly makes the lighting estimation object more extensible in the future. I personally have questions about whether or not we'd want to have the color be a single double (in which case it should probably be ambientColorTemperature
) or a full RGB value. I know that Google and Apple at least differ on this, and I'm not fully sure the reason why or what the benefits are one way or the other. @cwilso has said in the past that lighting temp is apparently a well known concept in photography, so I'm sure that played a part in Apple's design.
from lighting-estimation.
I received feedback that it's better to keep proposal Issues in here for search purposes, so I'm moving them back until we can think through how to handle it.
Ada, when you make the README for the repo, please add a link to this Issue.
from lighting-estimation.
@TrevorFSmith Did you close this issue intentionally? I think it might be helpful for searchability to keep it open until it is converted to a README or other representation for this spec component.
@toji about ambient color vs color temperature, I think an RGB ambient color seems strictly more powerful. A color temperature represents black body radiation that can be converted to RGB, i.e. via lookup tables or a calculation, but the inverse isn't really possible. For example, if the scene is lit with an unnatural light color such as green or purple, that can be represented as an RGB color, but color temperature is restricted to a one-dimensional red-to-white-to-blue path through the color space, so forcing it into that representation would potentially lose data.
from lighting-estimation.
@klausw Yes, once we create a feature repo all conversation should happen there. So, I recommend creating a new Issue over there and if necessary refer back to this Issue.
from lighting-estimation.
May I suggest that SH (Spherical Harmonics) be considered as part of the format to express the environment lighting in a directional dependent manner?
https://en.wikipedia.org/wiki/Spherical_harmonics
This has the advantage of being small (feasible to calculate and store hundreds on a mobile device), scalable (can easily access various levels of detail), and used ubiquitously in game engines for this purpose.
from lighting-estimation.
For background on use in lighting specifically:
https://en.wikipedia.org/wiki/Spherical_harmonic_lighting
from lighting-estimation.
Even more specifically, we could use PRT (Precomputed Radiance Transfer):
https://en.wikipedia.org/wiki/Precomputed_Radiance_Transfer
It should be possible to express a simple ambient / color temperature value in this format as well as extract such a value from PRT data that includes the higher frequency domain.
from lighting-estimation.
Related Issues (20)
- Describe the components of Float32Arrays that represent light with red, green, and blue components
- Refactor spec to be subscription based HOT 2
- Consider un-bounding the number of coefficients for XRLightProbe.sphericalHarmonicsCoefficients. HOT 2
- Update readme to point to explainer and add "unstable" notice.
- Consider forcing use of session feature flag to enable this API HOT 4
- Need to formally define the integration of irradiance in XRLightProbe's sphericalHarmonicsCoefficients and its relationship with the discrete samples accessed by XRReflectionProbe
- Describe temporal filtering required to avoid fingerprinting HOT 1
- Promises should not be returned by a function called on XRFrame
- createWebGLEnvironmentCube needs to know which GL context to create the texture on
- Consider removing `XRLightProbe.sphericalHarmonicsOrientation` and `XRReflectionProbe.orientation` HOT 1
- Consider exposing the XRReflectionProbe off the XRSession object HOT 3
- Describe method of quantization to reduce fingerprinting
- Determine internal format of the Reflection Cube Map HOT 4
- Restrict usage to one tab at a time?
- Update affiliation of 'Kearwood Gilbert'? HOT 1
- Broken references in WebXR Lighting Estimation API Level 1
- Current device support for lighting estimation in `immersive-ar`? HOT 2
- Consider renaming "Image Based Lighting" section to "Cube Map Textures"
- Describe simple renderers that could use HDR cubemap textures and SH lighting
- Consider adding ability to generate a low resolution HDR cube map from SH
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from lighting-estimation.