MambaFlow3D —
Sparse-Voxel 3-D Generation.

Replicate SparC3D / TRELLIS, but faster on consumer GPUs.

The thesis line wants fast single-image-to-3-D — train on a consumer rig, infer at interactive rates. The published references at the time of this work — SparC3D and Microsoft's TRELLIS — both use transformer attention over sparse voxel token sequences. The token counts are short enough (~100–200 sparse cubes per scene) that transformer attention is not catastrophic, but the cost is still quadratic, and the per-step inference latency on consumer GPUs is the bottleneck for the interactive use-case the thesis targets.

The Mamba state-space block [1,2] is the obvious substitute. It is linear-time in sequence length, has constant memory under autoregressive rollouts, and on the MNIST validation (Topic 25) it matched or beat the transformer baseline on visual quality despite a slightly worse loss number. The question Topic 26 answers: does the Mamba-substitution survive the jump from 2-D MNIST to 3-D point clouds, and is the architecture spec coherent enough to commit to a full SparC3D-class implementation?

The honest scope: this is a design + Phase-2 validation page. The full SparC3D-class implementation is not yet built; what is built is the point-cloud generator end-to-end, the architecture spec for the sparse-voxel extension, and the documented bring-up on ModelNet10 including the PointNet++ channel-mismatch fix that blocked the first training run.

PointNet++ → 10 Pure-Mamba blocks → flow matching → FoldingNet.

The Phase-2 architecture is end-to-end deterministic in the encoder and decoder, with the generative work happening in latent space. The forward path is:

PointNet++ encoder. Three set-abstraction layers bringing 2 048 points down to 32 hierarchical features at 256 dimensions each. The three SA layers are (2048→512, mlp=[32,32,64]) → (512→128, mlp=[64,64,128]) → (128→32, mlp=[128,128,256]). The first layer takes no input features (only xyz coordinates); the next two take the previous layer's output. This is where the channel-mismatch bug appeared (§04).

Pure-Mamba body. Ten blocks at d_model = 256, d_state = 128, run over the 32 latent tokens. Pure-Mamba (not the Mamba+Attention hybrid) was the winner of the MNIST validation in Topic 25 on the speed–quality trade-off, so it is the substrate carried forward. The 10-block depth is taken from the MNIST setup; whether this is the right depth at 3-D scale is open.

Flow matching in latent space. Linear interpolation between noise and the latent code as the trajectory; the network predicts the velocity field v; loss is ‖v − v̂‖². Sampling is a 50-step Euler integrator from z₀ ∼ 𝒩(0, I) to the generated latent. The 50 steps is the nominal — the target band is 20–50 steps once the model is trained.

FoldingNet decoder. Two-step folding: the latent is broadcast across a fixed 2-D 45 × 45 ≈ 2 048 grid, the grid is concatenated with the latent and passed through a small MLP to produce the first folded point cloud, then concatenated again and folded a second time to produce the output point cloud. This is the standard FoldingNet pattern.

Total: ~7.25 M parameters — confirmed by parameter count at first training launch.

Single-image-to-3-D, MambaFlow3D arrangement.

The Phase-2 work documented here is the point-cloud generator — PointNet++ encoder + Pure-Mamba + flow matching + FoldingNet. The full Phase-3 / Phase-4 system swaps in a sparse-voxel tokeniser (SparseCubes, ~100–200 tokens per scene) and an image-conditioning branch (ViT image encoder, cross-attention into the Mamba blocks). The diagram below shows both — Phase-2 is the dark-bordered path, Phase-3+ is the light-bordered extension.

2–3× train speed-up, 5–12× inference speed-up vs SparC3D.

The target speed-up numbers are derived from two sources: the measured Mamba-vs-transformer ratios reported in [2] and an estimate of the inference advantage flow-matching gives over DDPM at 20–50 steps vs 250–1000 steps respectively.

The 5–12× inference-latency range is wide because two of the constituent factors (Mamba per-step speed-up on Ampere, FM step count that retains quality) are not yet measured at the SparC3D-scale token count. The 5× lower bound assumes the modest end of each; the 12× upper bound assumes the optimistic end of each. Neither has been verified at 3-D scale.

ModelNet10 dataset, 5 files, 7.25 M parameters.

The Phase-2 implementation is five files: flow_model.py (PointNet++ + Mamba + FoldingNet, the network), train.py (flow-matching loss, AdamW, gradient clipping), modelnet_loader.py (ModelNet10 dataloader, unit-sphere normalisation, 2 048-point sampling), evaluate.py (Chamfer Distance + 3-D plot), benchmark.py (timing harness for the 30 s / epoch target). All five live under 3d_point_cloud_Mamba/.

First training launch crashed at step 0. Cause was three layers up.

The launch crash was at the first forward pass through the encoder. Traceback bottomed out at a conv layer in the second set-abstraction (SA) layer of PointNet++ expecting 64 input channels and receiving 67. The factor-of-three surplus is the giveaway: the SA layer concatenates the grouped xyz coordinates (3 channels) onto the input features before the MLP, so the actual feature count entering each SA layer's MLP is in_channel + 3, not in_channel. The reference PointNet++ implementations handle this with an in_channel + 3 offset in the layer constructor; the initial code here did not.

Once the offset was applied uniformly to the three SA layers, the forward pass ran clean and training proceeded. The lesson, also saved as a feedback memory: PointNet++ derivative implementations are an old-favourite source of channel-count off-by-three bugs; the SA layer's in_channel argument means "input features", not "input features + xyz", and the +3 has to be added at the MLP head.