kfjahnke / envutil Goto Github PK
View Code? Open in Web Editor NEWutility to convert between lat/lon and cubemap environment maps, and to extract single images and image series in several projections and arbitrary orientation
License: MIT License
utility to convert between lat/lon and cubemap environment maps, and to extract single images and image series in several projections and arbitrary orientation
License: MIT License
Currently, envutil will process data as delivered by OIIO, without conversion to linear RGB if that is the native data format in the image file. This is not mathematically correct unless the data is in linear RGB already, and to be completely correct, the data should be scene-linear, and display-linear data would need to be converted back to scene-linear to do the math correctly.
The conversion of display-linear data or data in an output color space (like sRGB) may not be technically possible (if a 3D LUT was used, the conversion may not be unambiguously reversible, filters may be baked into the data) - but for the sake of processing, approaching scene-linear via a simple tone curve for all channels should produce better results, even if the correct reconstruction of the scene-linear data is not possible.
I think that the color space conversions could be done with OIIO or OCIO, but so far I don't have any code in place. Keep possibility of using EMoR in mind.
I need:
If a 3D LUT was used, inverting it may be possible under constraints, investigate.
This would be a path towards stitching: several source images could be mounted with additional orientation information (currently, the mount is conceptually head-on, and any reorientation is via the virtual camera's y/p/r orientation). The next step would be to do the 'voronoi thing' (like animated sequences of PTOs in lux), which would already be 'good enough' to apply patches. Eventually, even an implementation of the Burt&Adelson image splining algorithm might be slotted in for 'proper' stitching - or some other method to blend the images seamlessly.
ffmpeg is extremely capable when it comes to image processing - as it was pointed out by @chaosjug on hugin-ptx (see https://groups.google.com/g/hugin-ptx/c/C58NlKPYHbM/m/w02W0QD6BAAJ) the entire process of extracting a set of frames from an environment image and producing an output video from the result can be done using the ffmpeg v360 filter. Looking at this filter (see https://ffmpeg.org/ffmpeg-filters.html#v360), it offers a fair amount of extra options which go beyond what envutil currently provides, e.g. using double fisheyes as a substrate.
The v360 filter can also be used on single images (take care to work the data as TIFF to avoid compression artifacts), and I played with it a bit, and ascertained that the conversions compare well with what envutil produces, as long as the parameterization is compatible.
Can envutil be made to pass it's pixel pipeline through an ffmpeg filter? If the libffmpeg code can be leveraged to parse a filter specification (ffmpeg obviously can, the CL tool does just that) - it might be a matter of just passing through the textual spec and writing an interface which routes the data through the filter chain.
Having worked with the twining 'meta filter', I get the impression that the method is very powerful and versatile. It may be a good idea to decouple twining from the remainder of the code and make it into a stand-alone library. It would need only zimt as a dependency. It may also be a good idea to move the twining code to the zimt library, which as already accreting a fair amount of collateral code. The base idea is similar to what I proposed as 'meta filter' in vspline (see https://bitbucket.org/kfj/vspline/src/master/example/metafilter3.cc), but the twining code is more efficient, because it does not require rotation operations on the coordinates.
The twining code generates the basis vectors for the sub-pick-ups from two additional target coordinates calculated from two 'canonical' neighbours, optionally projected to the tangent plane for mathematical correctness (which is usually overkill, so it's optional, see --twine_precise). Is there maybe a better way to obtain the derivative? I also think that maybe reducing the width of the 'canonical' step to, say, 1%, may approach the derivative better, making the projection to the tangent plane even less necessary.
Additionally, providing a 2D version of the twining code would be nice. For both the 3D and the 2D case, an alternative route of parameterization might directly provide the basis vectors. This may be even more versatile and allow application code to come up with it's own ways of generating the basis vectors, rather than relying on the 'canonical neighbours'.
This is especially noticeable when OIIO's lookup code is used (the mounted image doesn't always wrap around correctly, the visibility of the flaw depending on the interpolator) - with twining it's masked by the reflect boundary condition, so there is indeed a discontinuity but it's usually too slight to be visible. With OIIO lookup, the problem can be mitigated by passing appropriate boundary treatment options, but this is not automatic.
envutil code would need special treatment of wrap-around for mounted images with 360 degrees hfov (and proper handling of zenith and nadir for full sphericals) - probably it would be best to 'elevate' such images to 'environment image status' and use that code path, rather than the mounted-image path. Cyclindrical images with 360 degree hfov would need special treatment, since the environment code path currently only accepts full sphericals and cubemaps ('fully covered' input in lux speak).
So far, generating video output from a 'seq file' doesn't produce generally viable output. Here on my debian system, the 'videos' app displays the video, but VLC does not. On other systems, VLC also doesn't seem to play the video. Is this a VLC problem (missing/wrong codec?) or is there something wrong with the video?
I have 'brutalized' an example file from the ffmpeg project to produce the output, and my handling of ffmpeg may not be correct. I suspect that the problem is my end, because emitting single frames and generating the video from that with CL ffmpeg produces a generally viable output.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.