Mini SDF-SRN —
Learning 3-D From Single Images.

Build the 3-D-from-2-D primitive from scratch, before scaling it.

The thesis line's image-to-3-D work — the Hypernet → DeepSDF archive (Topic 41), the Flow-SDF pipeline (Topic 44) — all rests on one primitive: a differentiable renderer that turns a 3-D implicit surface into a 2-D image, so gradients from a 2-D loss can teach a network 3-D geometry. Mini SDF-SRN is that primitive, built from scratch on the smallest problem that still exercises every component: 50 synthetic shapes, single-view silhouette supervision, a CNN encoder, a shared SDF decoder, a ray-marching renderer.

The point is not a competitive reconstruction model — it is to own the 3-D-from-2-D mechanism end to end before using it as a black box downstream. Same philosophy as the from-scratch transformer (Topic 13) and the from-scratch DDPM (Topic 6): build the 100-line version on a toy, watch every gradient, then scale.

Image → CNN encoder → latent → SDF decoder → renderer → silhouette loss.

CNN encoder, shared SDF decoder, fixed differentiable renderer.

CNN encoder. Four convolutional layers (3 → 32 → 64 → 128 → 256 channels, stride 2) followed by two fully-connected layers, compressing a 64 × 64 × 3 image into a 128-dimensional latent code.

SDF decoder. A 5-layer MLP with a skip connection, following the DeepSDF pattern. It takes the latent code concatenated with a 3-D query coordinate and outputs a scalar SDF value — the latent determines which shape, the coordinate determines where it is evaluated. It is shared across all shapes.

Differentiable renderer. A fixed (non-learned) ray-marching renderer — samples points along camera rays at fixed intervals, queries the SDF decoder at each, and computes a soft silhouette via sigmoid(−min_sdf / temperature). It is the bridge between 3-D and 2-D: not a neural network, but a mathematical operation that provides continuous gradients from 2-D pixel comparisons back through the SDF values to the latent codes.

Trained on 50 shapes, one view each. Predicts 6 novel angles.

Trained on 50 synthetic shapes for 300 epochs, ~70 minutes on an M4 iMac. Each shape was seen from only one viewpoint during training. The figures below are the actual epoch-300 outputs from the repository — the top row of each is predicted from the network, the bottom row is ground truth, across 6 novel angles the network never saw.

Sphere · epoch 300. Training view (red shaded) + 6 novel angles. Pred row vs GT row — the smooth shape is the easiest case and tracks closely.

Box · epoch 300. The hardest case — sharp corners. The network recovers the gross silhouette across angles; corners round off slightly.

Ellipsoid · epoch 300. Intermediate difficulty — the anisotropic silhouette is recovered at every novel angle.

Training convergence · epoch 300. Ground-truth silhouette / predicted silhouette / difference map. By epoch 300 the difference is a thin boundary band — the network has learned correct 3-D structure from single-view silhouette supervision alone.