Differentiable Rendering

Implementations of three building blocks of modern differentiable rendering, built up from fitting a single image to training a neural radiance field. Each piece is written from scratch in PyTorch and is meant to be read as much as run.

1. Neural fields — fitting an image with a SIREN and a ReLU-MLP, including autograd-based gradient and Laplacian supervision.
2. Sphere tracing & volume rendering — two ways of turning an implicit 3D scene into an image.
3. NeRF — a positional-encoded coordinate MLP trained by volume rendering on a Blender-synthetic scene (e.g. lego).

neural_fields/   SIREN, ReLU-MLP, autograd operators, single-image training loop
renderers/       camera rays, sphere tracing, and volume rendering of analytic scenes
nerf/            positional encoding, NeRF field, dataset loader, volume renderer, trainer
scripts/         entry points for each part

Setup

python -m venv venv && source venv/bin/activate
pip install -r requirements.txt   # install the PyTorch build for your platform

1. Neural fields

A neural field maps continuous coordinates to signal values. The SIREN uses sine activations (with the paper's initialisation) so it can represent high-frequency detail and have well-behaved derivatives; the ReLU-MLP is the baseline. The training loop also visualises the field's gradient and Laplacian, computed with autograd.

python -m scripts.fit_image --model SIREN --steps 1000
python -m scripts.fit_image --model MLP --steps 2000

The SIREN reaches a sharp reconstruction in ~100 steps that the ReLU-MLP never reaches in 2000 — a direct consequence of spectral bias.

2. Sphere tracing & volume rendering

Two renderers for an implicit scene. Sphere tracing marches each ray by the signed-distance value to find a hard surface hit; volume rendering samples points along each ray and alpha-composites their density and colour — the same integral the NeRF uses, here on an analytic scene.

python -m scripts.render_demos   # writes sphere_tracing.png and volume_rendering.png

3. NeRF

A NeRF represents a scene as a function from a 3D point to a colour and a density. A point is Fourier-encoded, run through an MLP with a skip connection, and the resulting densities/colours along each ray are volume-rendered into a pixel. Training minimises the photometric error against posed images.

Download a scene from the NeRF-synthetic dataset and point --data at the scene directory (the one containing transforms_train.json):

python -m scripts.train_nerf --data data/nerf_synthetic/lego --iters 20000

Periodic renders and a final 360° orbit (spin.gif) are written to outputs/. Training is fast on a GPU and runs (more slowly) on CPU; results on a small MLP are blurred but clearly 3D.

Notes

• The volume renderer in nerf/ is the flat ray-batch counterpart of the one in renderers/ — same compositing integral, batched over randomly sampled rays and with stratified sampling for training.
• The NeRF field is position-only (no view-dependent effects), which keeps it small enough to train quickly on a single scene.

References

• Sitzmann et al., Implicit Neural Representations with Periodic Activation Functions (SIREN), 2020.
• Mildenhall et al., NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis, 2020.