HDR-GS logo

HDR-GS: Efficient High Dynamic Range Novel View Synthesis at 1000x Speed via Gaussian Splatting

NeurIPS 2024

GitHub arXiv Video ηŸ₯乎 Project Page AK HF Data Online Demo

High dynamic range Β· Exposure control Β· Gaussian splatting Β· Novel view synthesis

Online Demo

Explore all 12 scenes in the HDR-GS Gradio demo. Compare paired LDR and HDR videos, inspect both interactive Gaussian point clouds, and vary camera motion and exposure. Use a built-in scene or upload a compatible HDR-GS PLY with its matching tone mapper to render new views.

HDR-GS Studio showing paired LDR/HDR videos and interactive Gaussian point clouds

Novel View Synthesis Gallery

HDR-GS novel view synthesis of bathroom HDR-GS novel view synthesis of chair HDR-GS novel view synthesis of dog

Bathroom

Chair

Dog

HDR-GS novel view synthesis of bear HDR-GS novel view synthesis of desk HDR-GS novel view synthesis of sponza

Bear

Desk

Sponza

Demonstrations from the official HDR-GS project repository.

Model Description

HDR-GS (High Dynamic Range Gaussian Splatting) synthesizes novel HDR views and renders LDR views with a user-specified exposure time. The method introduces a Dual Dynamic Range (DDR) Gaussian point cloud model and Parallel Differentiable Rasterization (PDR). It represents HDR color with spherical harmonics and uses an MLP-based tone mapper to produce exposure-dependent LDR color. See the NeurIPS 2024 paper.

HDR-GS method overview

This repository provides scene-specific pretrained Gaussian point clouds and tone-mapping networks. Each pair represents a fitted scene and is loaded using the official HDR-GS code.

Available Models

The released checkpoints are grouped under synthetic/ and real/. Each scene folder contains its matching point_cloud.ply and tone_mapper.pth. Download both files together.

Group Scene Model directory Configuration
Synthetic Bathroom synthetic/bathroom/ bathroom.yaml
Synthetic Bear synthetic/bear/ bear.yaml
Synthetic Chair synthetic/chair/ chair.yaml
Synthetic Desk synthetic/desk/ desk.yaml
Synthetic Dining Room synthetic/diningroom/ diningroom.yaml
Synthetic Dog synthetic/dog/ dog.yaml
Synthetic Sofa synthetic/sofa/ sofa.yaml
Synthetic Sponza synthetic/sponza/ sponza.yaml
Real Flower real/flower/ flower.yaml
Real Computer real/computer/ computer.yaml
Real Box real/box/ box.yaml
Real Lucky Cat real/luckycat/ luckycat.yaml

Interactive preview files and videos are available in the Dataset visualization directory.

Release Records

The released Bathroom and Sofa checkpoints use the original HDR Gaussian geometry and updated tone mappers. The original revision provides the starting weights for these two scenes.

Release benchmark Β· Archived training code and scene YAMLs Β· Google Drive weights

Quick Start

1. Set Up the Environment

git clone https://github.com/caiyuanhao1998/HDR-GS --recursive
cd HDR-GS
conda env create --file environment.yml
conda activate hdr_gs

Follow the official setup instructions to configure a CUDA development toolkit (nvcc) and a compatible C++ compiler. On Linux, the tested combination is CUDA 11.8 and GCC 11. Then build both CUDA extensions after PyTorch is installed:

python -m pip install --no-build-isolation ./submodules/diff-gaussian-rasterization ./submodules/simple-knn

Set up the hdr_gs_demo environment as described in the same setup section to use the download commands below.

2. Download the Model and Scene Data

Run these commands from the project root to download the Bathroom checkpoint pair and matching scene data. The root config.json lists all checkpoint files and is used by the Hub for download statistics.

conda run --no-capture-output -n hdr_gs_demo hf download CaiYuanhao/HDR-GS --include "config.json" --include "synthetic/bathroom/**" --local-dir pretrained_weights
conda run --no-capture-output -n hdr_gs_demo hf download CaiYuanhao/HDR-GS --repo-type dataset --include "synthetic/bathroom/**" --local-dir data_hdr

The expected local layout is:

HDR-GS/
β”œβ”€β”€ config/bathroom.yaml
β”œβ”€β”€ data_hdr/synthetic/bathroom/
β”‚   β”œβ”€β”€ images/
β”‚   β”œβ”€β”€ exr/
β”‚   └── sparse/0/
└── pretrained_weights/
    β”œβ”€β”€ config.json
    └── synthetic/bathroom/
        β”œβ”€β”€ point_cloud.ply
        └── tone_mapper.pth

3. Test and Render

python3 train_synthetic.py \
    --config config/bathroom.yaml \
    --eval \
    --gpu_id 0 \
    --syn \
    --load_path pretrained_weights/synthetic/bathroom \
    --test_only

--load_path points to the directory containing both point_cloud.ply and tone_mapper.pth. The scene data supplies the cameras and reference images used by the evaluation pipeline. The test-only path runs video inference and evaluation; results are written to the run's output directory.

For Sofa, replace "synthetic/bathroom/**" with "synthetic/sofa/**" in both download commands above and run the matching evaluation command.

python3 train_synthetic.py \
    --config config/sofa.yaml \
    --eval \
    --gpu_id 0 \
    --syn \
    --load_path pretrained_weights/synthetic/sofa \
    --test_only

Training and Rendering Settings

Train a new scene model with its matching configuration:

# Synthetic scene
python3 train_synthetic.py --config config/bathroom.yaml --eval --gpu_id 0 --syn

# Real scene
python3 train_real.py --config config/flower.yaml --eval --gpu_id 0

The released bathroom and sofa configurations specify 30,000 training iterations, Gaussian optimization and densification settings, and separate tone-mapper learning rates. Their render_video and video_render_exps options configure the spiral video and exposure range. See the scene configurations and testing instructions.

Saved scene models contain point_cloud.ply and tone_mapper.pth together under the run's point_cloud/iteration_* directory. Point --load_path to that directory to render a trained model.

Visual Results

Synthetic Scenes

HDR-GS synthetic scene comparisons

Quantitative results on synthetic scenes

Paper-reported synthetic scene evaluation

Real Scenes

HDR-GS real scene comparisons

Quantitative results on real scenes

Paper-reported real scene evaluation

Figures are reproduced from the official project and report the original experiments.

Interactive comparisons: Bathroom Β· Chair Β· Diningroom Β· Dog Β· Sofa Β· Sponza

Citation

@inproceedings{hdr_gs,
  title={HDR-GS: Efficient High Dynamic Range Novel View Synthesis at 1000x Speed via Gaussian Splatting},
  author={Yuanhao Cai and Zihao Xiao and Yixun Liang and Minghan Qin and Yulun Zhang and Xiaokang Yang and Yaoyao Liu and Alan Yuille},
  booktitle={NeurIPS},
  year={2024}
}
Downloads last month
229
Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Dataset used to train CaiYuanhao/HDR-GS

Space using CaiYuanhao/HDR-GS 1

Paper for CaiYuanhao/HDR-GS