waldenlakes/awesome-illumination-estimation

A curated list of illumination estimation papers and resources.

Awesome-Illumination-Estimation

A curated list of illumination estimation papers and resources.

Table of Contents

• Papers
• Datasets
• Challenges

Papers

SV:Spatially-Varying, ID:Indoor, OD:Outdoor, E:Editable

Learning-based Methods

Indoor Scenes:

Year	Venue	Paper	Repo	Data	SV	ID	OD
2023	ArXiv	Towards a Perceptual Evaluation Framework for Lighting Estimation	Proj			✓	✓
2023	ICCV	EverLight: Indoor-Outdoor Editable HDR Lighting Estimation	Proj			✓	✓
2023	ICCV	Beyond the Pixel: a Photometrically Calibrated HDR Dataset for Luminance and Color Prediction	Proj & Code	LavalPhotometricIndoorHDR		✓
2023	ArXiv	Local-to-Global Panorama Inpainting for Locale-Aware Indoor Lighting Prediction			✓	✓
2022	ECCV	Stylelight: Hdr panorama generation for lighting estimation and editing	Code	LavalIndoorHDR		✓
2022	ECCV	Physically-based Editing of Indoor Scene Lighting from a Single Image	Proj & Code	OpenRooms	✓	✓
2022	ECCV	Editable indoor lighting estimation	Proj	LavalIndoorHDR		✓
2021	ICCV	Sparse needlets for lighting estimation with spherical transport loss	Code	LavalIndoorHDR		✓
2021	CVPR	HDR environment map estimation for real-time augmented reality	Code	LavalIndoorHDR		✓
2021	AAAI	Emlight: Lighting estimation via spherical distribution approximation	Code	LavalIndoorHDR		✓
2020	ECCV	PointAR: Efficient Lighting Estimation for Mobile Augmented Reality	Code	Matterport3D	✓	✓
2020	CVPR	Lighthouse: Predicting lighting volumes for spatially-coherent illumination	Code	InteriorNet	✓	✓
2019	ICCV	Deep parametric indoor lighting estimation	Proj	LavalIndoorHDR	✓	✓
2019	CVPR	Fast spatially-varying indoor lighting estimation	Proj	LavalIndoorSVHDR	✓	✓
2019	CVPR	Deeplight: Learning illumination for unconstrained mobile mixed reality	Proj	Capture		✓	✓
2019	CVPR	Neural illumination: Lighting prediction for indoor environments	Proj	Matterport3D	✓	✓
2017	SIGGRAPH	Learning to predict indoor illumination from a single image	Proj	LavalIndoorHDR	✓	✓

Ourdoor Scenes:

Year	Venue	Paper	Repo	Data	SV	ID	OD
2023	ArXiv	DiffusionLight: Light Probes for Free by Painting a Chrome Ball	Proj & Code			✓	✓
2022	ECCV	Neural Light Field Estimation for Street Scenes with Differentiable Virtual Object Insertion	Proj				✓
2022	ECCV	Estimating Spatially-Varying Lighting in Urban Scenes with Disentangled Representation	Code		✓		✓
2021	ICCV	Hierarchical disentangled representation learning for outdoor illumination estimation and editing			✓		✓
2021	CVPR	Spatially-varying outdoor lighting estimation from intrinsics	Video		✓		✓
2019	CVPR	Deep Sky Modeling for Single Image Outdoor Lighting Estimation	Proj				✓
2019	CVPR	All-weather deep outdoor lighting estimation	Proj				✓
2017	ICCV	Learning high dynamic range from outdoor panoramas	Proj				✓

Inverse-rendering based Methods:

Year	Venue	Paper	Repo	Data	SV	ID	OD
2023	ArXiv	Diffusion Reflectance Map: Single-Image Stochastic Inverse Rendering of Illumination and Reflectance	Proj			✓	✓
2023	T-PAMI	SPLiT: Single Portrait Lighting Estimation Via a Tetrad of Face Intrinsics				✓	✓
2023	SIGGRAPH Asia	Diffusion Posterior Illumination for Ambiguity-aware Inverse Rendering	Proj			✓	✓
2023	ArXiv	PSDR-Room: Single Photo to Scene using Differentiable Rendering			✓	✓
2023	ICCV	FIPT: Factorized Inverse Path Tracing for Efficient and Accurate Material-Lighting Estimation	Proj & Code	FIPT-data	✓	✓
2023	CVPR	MAIR: Multi-view Attention Inverse Rendering with 3D Spatially-Varying Lighting Estimation	Proj	OpenRoom FF	✓	✓
2023	CVPR	Multi-view Inverse Rendering for Large-scale Real-world Indoor Scenes	Proj & Code	Dataset	✓	✓
2022	SIGGRAPH Asia	Learning-based Inverse Rendering of Complex Indoor Scenes with Differentiable Monte Carlo Raytracing	Proj & Code	InteriorVerse	✓	✓
2022	ECCV	Physically-Based Editing of Indoor Scene Lighting from a Single Image	Proj & Code	OpenRoom	✓	✓
2022	CVPR	IRISformer: Dense Vision Transformers for Single-Image Inverse Rendering in Indoor Scenes	Code	OpenRoom	✓	✓
2022	CVPR	PhyIR: Physics-based Inverse Rendering for Panoramic Indoor Images	Proj	FutureHouse	✓	✓
2021	ICCV	Learning indoor inverse rendering with 3d spatially-varying lighting	Proj	InteriorNet	✓	✓
2020	CVPR	Inverse rendering for complex indoor scenes: Shape, spatially-varying lighting and svbrdf from a single image	Code	OpenRoom	✓	✓

Capture-based Methods

Year	Venue	Paper	Repo	Data	SV	ID	OD
2023	ArXiv	Accidental Light Probes	Proj	Capture		✓	✓
2022	ArXiv	ORCa: Glossy Objects as Radiance Field Cameras	Proj	Polarimetric		✓	✓
2021	ICCV	Objects as cameras: Estimating high-frequency illumination from shadows		hdrihaven.com & Capture		✓	✓
2021	CVPR	Indoor lighting estimation using an event camera		Capture	✓	✓
2020	ECCV	Object-based illumination estimation with rendering-aware neural networks	Proj	Collect		✓

Datasets

• Laval Photometric Indoor HDR Dataset [link]
• Laval Outdoor HDR Dataset [link]
• Laval Indoor HDR Dataset [link]
• Laval Indoor Spatially Varying HDR Dataset [link]
• Laval Face & Lighting HDR Dataset [link]
• OpenRooms Dataset [link]
• FutureHouse Dataset [link]
• InteriorVerse: Large-scale Photorealistic Indoor Scene Dataset [link]

Challenges

• AIM 2020: Scene Relighting and Illumination Estimation Challenge @ECCV [link] [Paper]
• Illumination Estimation Challenge 2019, 2020 @ISPA [link] [Paper]