Gaussian Splats part 3: There's always a bigger scene
magnopus.comThere's an interesting recent Siggraph paper on scaling up Gaussian splatting in a very different way, which offers some interesting benefits; Gaussian point splatting.
* Video: https://www.youtube.com/watch?v=2AWv_nIfp-U
* Paper: https://jorisar.nl/gaussian_point_splatting/gaussian_point_s...
Trivially portable to Vulkan as well
Does anyone know why triangle splatting never took off? I would think it would be much more suited to GPU rendering.
Not substantially faster as your still drawing roughly the same number of primitives (Gaussians are rendered as a quad covering the splat area), and the extra cost in shading isn't that expensive compared to all the fragment samples / depth tests. Triangles also tend to be less effective at modelling most real world 3D scenes. In most cases I tried, it required more primitives, longer training, and ultimately lower FPS and PSNR than 3DGS. Other methods like PSGR also helped push 3DGS to better fit real world geometry. Triangle splatting and its variants do not generate clean efficient meshes like what you might be used to with a modelled scene [0].
They address the lower PSNR for triangles compared with gaussians in the paper, basically PSNR rewards smoothness more than visual quality and gaussians are very smooth. I hadn't seen PSGR[0] before, that looks pretty good but it looks like it's pretty close to the triangle splatting. They flatten the gaussians to flat primitives with varying opacity like the triangles, but the authors of triangle splatting specifically call out that their window function is bounded by the triangles edges which produces better details in some areas. It kind of proves my point about 3DGS getting a lot of research and incremental improvement though, while I haven't seen any follow up research on triangle splatting. They do mention that their method would need additional steps in order to produce a connected mesh, but I think at least in their example images the surfaces looks pretty decent even thought their still made up of a triangle soup.
I just checked if they had released any follow up papers themselves, and one of the lead authors has published a paper about sparse geometry[1] that looks interesting. He's also done a couple of direct follow ups to triangle splatting called Triangle Splatting+[2] and Mesh splatting[3], that produces a connected mesh. It's at least good to see that there's still research into other techniques than just gaussians.
[0] https://zju3dv.github.io/pgsr/
[1] https://lessvrong.com/cs/nexels/
[2] https://trianglesplatting2.github.io/trianglesplatting2/
It still might, at least in some niches? The paper is not that old.
It's more than year old, that's pretty old in this fast moving space. I guess it's possible it will get more popular, but I haven't seen it get any traction at all, gaussian splats however are getting more and more development and adoption.
The "Kefermarkt Chruch" from Christoph Schindelar: https://www.youtube.com/watch?v=nC6Q2wN-vfM
With the simple equipements, the result is spectacle.
Thank you. The realtime version linked from the video is even more impressive!
I had no idea SOG was being used this way, so cool! I'm busy working on V3 if there's anything you think we should include.
Hey Donovan, keep up the great work :)
That reminds we, we had really weird results with splat-transform .sog output, when measuring the references for KISS-GS a couple months back. Why is the .sog curve dipping in quality when adding more primitives?
It's the "POPSpa + .sog" curve here, in the first Figure: https://arxiv.org/html/2608.26948v1
Always wanted to go back and measure with the latest splat-transform and find whats going on there.
The biggest update is adding alpha compensation (alpha > 1). With this in place we can addresses the gaps that appear after performing many rounds of decimation.
If anyone wants to just play and walk around a big splat: https://www.useblurry.com/shared/fONoiLF-8AiD
It has 50m splats, captured by XGRIDS K2 camera in roughly 1 hour of walking around.
"Hubris: Redux" is a pretty good description of my entire career.
The high points, anyway. The parts worth remembering.
Something I am taking away from this, is every optimization he makes here, takes away from the technologies supporting the overall dynamism of the scene, "dynamism" in terms of things interacting with things. The long solution might well be a very tightly described engine of "dynamism" - data formats, algorithms - that then feeds an optimized splat renderer, millisecond by millisecond.
Gaussian Splats are great, and putting them in Unreal is even better! Do you see splats cropping up in games/sport events? Where next?
Gaussian splats are surprisingly easy to make: https://superspl.at/scene/82eb1744 (I used a thermal camera - so I could also project the IR onto it too)https://superspl.at/scene/f9672a5e
They are not that easy to make in my experience. I struggled a bit with my phone camera images and moved onto my iPad images.
If I understand what you are saying, it might be that your phone had a better camera than your ipad? The depth of field and atmospheric lens effects aren't really what you want - a pinhole lens like old phones (and my flir) works better since blur is the enemy.
Thank you. I will try with that
I could swear I saw something about an NFL broadcast using them. They were part of the PGA championship broadcast.
Bit of a shameless plug: in August I built a demo [0] for our latest 3DGS compression paper (called KISS-GS). You can explore some standard Gaussian splatting scenes at different compression levels, see what breaks at really low bitrates, like 0.5 MB.
Scroll down the page a bit to unlock the controls under the viewer. I find the attribute toggles (for opacity, scale, rotation and so on) great fun for building an intuition on how splats and SOG compression works.
Really great write-up!
"lives in", "ledger" - The most obvious of the obvious claudisms :/