SMAGaussian: Stochastic Weight Averaging and Multi-Resolution Supervision for Egocentric Dynamic Scene Reconstruction

SMAGaussian reconstructs egocentric human-object interactions and renders high-quality novel views on both static and dynamic evaluation frames.

Architecture

Given an egocentric RGB video, scene reconstruction provides background and object Gaussians with per-frame object poses and cameras, all held fixed during the final joint refinement. Each iteration composes the scene, renders it, and evaluates a hand-masked photometric objective combining the full-resolution loss with the proposed multi-resolution supervision (MRPS). After a warmup phase, SWA-G maintains a gradient-free EMA shadow of the parameters and swaps it into the final model at save time. Both modules add no learnable parameters, keep the Gaussian count fixed, and incur no inference-time cost.

Abstract

3D Gaussian Splatting (3DGS) has emerged as a powerful explicit scene representation for real-time novel-view synthesis, and recent work has extended it to egocentric dynamic scene reconstruction. Existing methods jointly refine all Gaussian parameters in a final optimization stage, after estimating the static scene, object shape, and per-frame object poses from egocentric video. However, we observe that this stage often suffers from optimization instability: the optimizer oscillates within a flat basin of near-optimal solutions rather than converging to its centre, which degrades reconstruction fidelity on unseen views. Therefore, in this paper, we propose SMAGaussian, a simple yet effective framework to improve the stability and quality of this joint refinement stage. SMAGaussian introduces two complementary modules with zero additional learnable parameters, namely, Stochastic Weight Averaged Gaussians (SWA-G) and Multi-Resolution Photometric Supervision (MRPS). SWA-G maintains a detached exponential moving average of all Gaussian parameters throughout optimization and replaces the final parameters with their temporal average at convergence. This approach exploits the explicit and fixed-structure nature of 3DGS, where parameter-space averaging remains particularly meaningful, whereas the effectiveness of weight averaging in implicit neural representations is less predictable due to parameter entanglement. In parallel, MRPS augments the conventional full-resolution photometric loss with an auxiliary L1 loss evaluated at two downsampled scales, providing complementary low-frequency gradient signals to the standard full-resolution photometric loss. SWA-G and MRPS operate along complementary axes: SWA-G stabilizes the optimization trajectory in parameter-space, while MRPS reshapes the loss landscape through multi-scale supervision. Extensive experiments on the HOI4D and EPIC-KITCHENS benchmarks demonstrate that SMAGaussian achieves peak signal-to-noise ratio (PSNR) improvements of up to 2.07 dB and 3.56 dB, respectively, along with consistent gains in structural similarity index (SSIM) and learned perceptual image patch similarity (LPIPS). Ablation studies confirm that SWA-G provides the dominant improvement, while MRPS contributes a consistent additive refinement. Notably, all gains are achieved without additional learnable parameters, without increasing the Gaussian count, and without modifying the upstream pipeline, while incurring less than 4% training overhead and zero inference cost.

Method

SMAGaussian operates entirely within the final joint refinement stage, where the object poses are frozen and only the Gaussian parameters are optimized.

Main Results

Comparison with state-of-the-art methods. Static and dynamic evaluation frames are reported separately; best values in each column are in bold.

HOI4D

EPIC-KITCHENS

Qualitative Visualization

EPIC-KITCHENS

Novel-view reconstructions on EPIC-KITCHENS. SMAGaussian recovers sharper appearance and fewer artifacts than the baseline.

Ablation

Progressive effect of each module on a dynamic evaluation frame. Top: rendered images and ground truth. Bottom: per-pixel reconstruction error; SWA-G suppresses the large concentrated error present in the baseline, and MRPS further reduces the residual.

BibTeX

@article{smagaussian2026,
    title   = {SMAGaussian: Stochastic Weight Averaging and Multi-Resolution Supervision for Egocentric Dynamic Scene Reconstruction},
    author  = {Goshu, Hana L. and Wakjira, Tadesse G. and Fouda, Mostafa and Lam, Kin-Man},
    year    = {2026},
    note    = {Under Review}
}