ssteinberg/pltfalcor

Real-time physical light transport (PLT) framework

Real-time physical light transport (PLT) framework

This repository demonstrates interactive wave-optical rendering, and implements core concepts from the PLT series of publications. See relevant papers:

• A Generalized Ray Formulation For Wave-Optics Rendering
• Towards Practical Physical-Optics Rendering
• Physical Light-Matter Interaction in Hermite-Gauss Space
• A Generic Framework for Physical Light Transport

Path tracing is done unidirectionally with generalized rays (wave-optical constructs which generalize the classical rays typically used in rendering to wave optics). The renderer is spectral (with spectral MIS) and polarimetric.

All materials are diffractive, polarization-aware and optical coherence aware. The following PLT materials are implemented:

• Dielectrics
• Conductors
• Coated dielectrics
• Coated conductors
• Diffraction-grated dielectrics
• Diffraction-grated conductors
• Birefringent thin dielectrics
• Alternating multi-layered stacks on a dielectric

All surfaces allow arbitrary roughness (coherence-aware, first-order SPM with a K-Correlation PSD), modelled to match the "look-and-feel" of microfacet GGX roughness, but with correct physics. All conductors use tabulated spectral refractive-indices, and all light sources use spectral emission tables of real light sources (see 'Source/Tables' for tabulated data). Light emission data is from LSPDD.

Manifold sampling (MS) is implemented. MS enables exploring NEE connections through dielectrics, as well as NEE with a delta lobe. This is used to render the dispersive lobes of a diffraction grating, as demonstrated in the images below.

A sample scene with diffractive materials can be found here. Other PBRT, Falcor and mitsuba scenes can be loaded (PBRT and mitsuba importers with limited support), some materials are automatically converted to PLT materials, though probably some manual tuning will be required.

Builds upon the Falcor real-time rendering framework. For more information and additional resources, see the Falcor GitHub page.

Additional rendered images and videos available here.

Usage

• Run Mogwai
• Load script 'scripts/PLTPathTracer.py' (or 'scripts/PLTPathTracerNoDenoiser.py')
• Load a scene

To use the OptiX denoiser, OptiX and Cuda should be installed. See below.

Prerequisites

• Windows 10 version 20H2 (October 2020 Update) or newer, OS build revision .789 or newer
• Visual Studio 2019
• Windows 10 SDK (10.0.19041.0) for Windows 10, version 2004
• A GPU which supports DirectX Raytracing, such as the NVIDIA Titan V or GeForce RTX
• NVIDIA driver 466.11 or newer

** Note: There is a bug with some NVIDIA drivers (confirmed with 528.49) where manifold sampling (when enabled) fails for the first few hundred or thousand frames, and kicks in afterwards. This is under investigation.

Building

Falcor uses the CMake build system. Additional information on how to use Falcor with CMake is available in the CMake development documetation page.

Visual Studio

If you are working with Visual Studio, you can setup a native Visual Studio solution by running setup_vs2019.bat (or setup_vs2022.bat, same process) after cloning this repository. The solution files are written to build/windows-vs2019 and the binary output is located in build/windows-vs2019/bin.

Visual Studio Code

If you are working with Visual Studio Code, run setup.bat after cloning this repository. This will setup a VS Code workspace in the .vscode folder with sensible defaults (only if .vscode does not exist yet). When opening the project folder in VS Code, it will prompt to install recommended extensions. We recommend you do, but at least make sure that CMake Tools is installed. To build Falcor, you can select the configure preset by executing the CMake: Select Configure Preset action (Ctrl+Shift+P). Choose the Windows Ninja/MSVC preset. Then simply hit Build (or press F7) to build the project. The binary output is located in build/windows-ninja-msvc/bin.

Warning: Do not start VS Code from Git Bash, it will modify the PATH environment variable to an incompatible format, leading to issues with CMake.

Configure Presets

Falcor uses CMake Presets store in CMakePresets.json to provide a set of commonly used build configurations. You can get the full list of available configure presets running cmake --list-presets:

$ cmake --list-presets
Available configure presets:

  "windows-vs2019"           - Windows VS2019
  "windows-vs2022"           - Windows VS2022
  "windows-ninja-msvc"       - Windows Ninja/MSVC
  "linux-ninja-clang"        - Linux Ninja/Clang

Use cmake --preset <preset name> to generate the build tree for a given preset. The build tree is written to the build/<preset name> folder and the binary output files are in build/<preset name>/bin.

An existing build tree can be compiled using cmake --build build/<preset name>.

Note: Some render passes (RTXGI, RTXDI, DLSS in particular) are not fully working with the new Slang GFX backend.

Microsoft DirectX 12 Agility SDK

Falcor uses the Microsoft DirectX 12 Agility SDK to get access to the latest DirectX 12 features. Applications can enable the Agility SDK by putting FALCOR_EXPORT_D3D12_AGILITY_SDK in the main .cpp file. Mogwai, FalcorTest and RenderGraphEditor have the Agility SDK enabled by default.

NVAPI

To enable NVAPI support, head over to https://developer.nvidia.com/nvapi and download the latest version of NVAPI (this build is tested against version R520). Extract the content of the zip file into external/packman/ and rename R520-developer to nvapi.

CUDA

To enable CUDA support, download and install CUDA 11.6.2 or later and reconfigure the build.

See the CudaInterop sample application located in Source/Samples/CudaInterop for an example of how to use CUDA.

OptiX

If you want to use Falcor's OptiX functionality (specifically the OptixDenoiser render pass) download the OptiX SDK (Falcor is currently tested against OptiX version 7.6) After running the installer, link or copy the OptiX SDK folder into external/packman/optix (i.e., file external/packman/optix/include/optix.h should exist).

Note: You also need CUDA installed to compile the OptixDenoiser render pass, see above for details.

Resources

• Falcor: Falcor's GitHub page.
• Documentation: Additional information and tutorials.
  • Getting Started
  • Render Graph Tutorials
• Rendering Resources A collection of scenes loadable in Falcor (pbrt-v4 format).
• ORCA: A collection of scenes and assets optimized for Falcor.
• Slang: Falcor's shading language and compiler.

---

This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.

pltfalcor's Issues

Using mogwai?

Thanks for posting this project, I'm looking forward to having a go. I cloned the repo, and built without any trouble, and am trying to run on a machine with a 4090. I don't seem to be able to load projects into mogwai though, I just get a bright green screen.

All the sample data doesn't seem to report anything particularly amiss, a few bad meshes, duplicate textures, nothing startling ~

(Info) Falcor 6.0 (commit: c385a6d7, branch: master)
(Info) Created GPU device 'NVIDIA GeForce RTX 4090' using 'Direct3D 12' API.
(Info) Loaded 27 plugin(s) in 0.238s
(Info) Importing scene: D:\2023rtplt_bikescene\bikescene\scene_plt.xml
--- lots of warnings, no errors ---

I'm wondering if I'm on the right track, but missing a step (like hitting a "start rendering" hotkey or something) or if I should be opening a different program, other than mogwai?

I'd like to integrate with Unreal Engine

I'd like to integrate with Unreal Engine for learning purpose; is there any guidance on this topic ? @ssteinberg

Apparent broken links in README.md

Hey there. The hyperlinks at the end of these two sentences appear to be broken.

Additional rendered images and videos available here.

A few sample scenes with diffractive materials can be found here.

They redirect to https://github.com/ssteinberg/PLTFalcor/blob/master but that's a blank page.

Issues on Windows 11 error: MitsubaImporter.dll is not a shared library

Hello!
Thanks for posting such amazing work!
I am interested in this project and am trying to use it for trial, but got stuck in the initial step "Building/Configure Presets"
when I was trying to use the instruction "cmake --build build/windows-vs2019", the error below came.

ImportError: Failed to load plugin library from E:\PLTFalcor\build\windows-vs2019\bin\Debug\plugins\MitsubaImporter.d
ll. File is not a shared library.
Stacktrace:
0# 0x0000004dd11ec8b0 ( ?? )
1# 0x00007ffed800be60 (RtlGetCurrentServiceSessionId)

I am sure I have met all the conditions mentioned in your prerequisites except for the operation system (mine is Windows 11)
Have you tested your work on Windows 11? If so, could you let me know how to solve the above problem?
Thanks in advance.

pip package

could you provide a precompiled pip package for linux?

Issues running setup

Hi
I'm having trouble testing your PLTFalcor framework. I can successfully build the Falcor project from nvidia on my Ubuntu 22.04.
Running the setup.sh (Ubuntu 22.04) on a clean install returns the error pasted below.

./setup.sh Updating git submodules ... Synchronizing submodule url for 'external/args' Synchronizing submodule url for 'external/fmt' Synchronizing submodule url for 'external/glfw' Synchronizing submodule url for 'external/glm' Synchronizing submodule url for 'external/imgui' Synchronizing submodule url for 'external/pybind11' Synchronizing submodule url for 'external/vulkan-headers' Fetching dependencies ... Package 'python' at version '3.7.9-180.74b501b5-linux-x86_64' is missing from local storage. packman(ERROR): Unable to download file from 'https://d4i3qtqj3r0z5.cloudfront.net/python%403.7.9-180.74b501b5-linux-x86_64' (server returned 403) Failed to fetch dependencies!

Do you now how to solve this? I am not familiar with pacman as a package/dependency manager.