RLSplat: Resource-Linear Budget Control for Feed-forward 3D Gaussian Splatting

RLSplat. Resource-Linear Budget Control for Feed-forward 3D Gaussian Splatting. By Injae Kim and five others, from KAIST and Korea University. Published on the twentieth of October. Its code repository is worked out: about fifty-five source files, with a readme and a licence.

Feed-forward 3D Gaussian Splatting, abbreviated FF3DGS, reconstructs a three-D scene from images in a single forward pass, which suits it to applications that have to operate within limited resource budgets. Recent budget-controllable FF3DGS methods go a step further, letting the number of Gaussians be adjusted at inference so that the output fits a given storage limit or rendering-latency constraint.

What those methods overlook is the cost of inference itself. To produce a representation with the target number of Gaussians, they first predict a dense set and then prune or select from it. The output shrinks, but the work done to reach it does not: latency and peak memory stay nearly constant whatever the budget. A device asking for a small, light representation still has to pay the full price of the dense prediction first, which defeats much of the point of asking for less.

RLSplat is a resource-linear, budget-controllable FF3DGS framework. Its representation size, its inference latency, and its peak memory all decrease linearly as the Gaussian budget is reduced. A smaller budget becomes cheaper to produce, not only cheaper to store. The design principle is to never make more Gaussians than were asked for, and to keep every later step's cost tied to how many there are.

Two pieces deliver that. The first replaces pruning and selection with a learned Gaussian allocation policy, which distributes the budget across local regions of the scene, so that exactly the requested number of Gaussians is generated and nothing surplus has to be thrown away.

The second is resource-linear Gaussian decoding, which refines those Gaussians. Each Gaussian attends only to nearby Gaussians, found through an efficient hierarchical neighbor search. Because attention stays local, the decoding cost falls linearly as the number of Gaussians falls. Allocation fixes the count, and local decoding makes the refinement scale with that count, so neither stage quietly reintroduces the dense cost.

In zero-shot evaluation, RLSplat outperforms prior budget-controllable methods at every budget. It surpasses the strongest baseline even when using four times fewer Gaussians. At low budgets it runs up to four point one times faster, with nine point six times less peak memory. Quality and efficiency move together here: the method leads on quality at each budget, and those speed and memory figures are the ones reported at the low end of the range.

The shift is in what a budget controls. Earlier methods let it govern the size of the result. RLSplat lets it govern the whole cost of getting there, so that asking for less really does mean spending less, in time and in memory, at every step from allocation to decoding.