Consider integrating GSplat shader with lit shader features.

[querielo]

Currently, the GSplat shader is implemented as a custom shader. This means that if we want to add features from the Lit Shader (such as shadows or lighting) we have to duplicate code. By integrating the GSplat shaders into lit-shader we could simplify the process of enabling features like "skinning" and other light-based effects in the future.

This is just an idea for discussion.

Pros

• consistent feature set
• code reuse.

Cons

• increased complexity
• reduced modularity

Screen.Recording.2025-03-16.at.20.18.29.mov

[willeastcott]

If shadow casting works (as in your video), it would be very cool to combine a splat with a dome-projected HDRI and a shadow catcher!

[querielo]

Yes. But it requires to implement IBL. Probably, we might be able to mimic a lighting model with spherical harmonics. For the diffuse and ambient parts, low-order spherical harmonics can be used. For the specular part, we may need high-order spherical harmonics. I am not sure about the full theory yet.

For shadow mapping, we could use stochastic transparency, dithering (for large splats) similar to the Dithered transparency demo, or an alpha-test.

dither.mov

[mvaligursky]

I'm not sure adding it to LitShader at this point is something I would do, but perhaps in the future when we have prototypes of what can be used from there. In current engine I'm working on extracting all shader from from lit-shader to chunks (already done for the lighting part) and those can be used from different shaders now. I suspect this might be more of a way to integrate lighting with gsplats. Main reason is that the integration of lighting and feature set will be very different.
Similarly for skinning - those are already separate reusable chunks. This custom shader based on ShaderMaterial simply uses them to get skinning / morphing: https://playcanvas.vercel.app/#/graphics/shader-hatch. Gsplats use ShaderMaterial and can do the same in a few lines.

But definitely interesting topic and technology to explore.

[querielo]

Oh, that’s nice! I missed that. In that case, of course, it’s better to do it this way. It looks like these GSplatting limitations can be resolved with some limitations.

[mvaligursky]

#8872 addresses this with a different approach than merging the shader stacks: instead of integrating lit-shader features into the gsplat shader, the splat scene is relit using a proxy mesh that goes through the full lit pipeline.

The PR adds a gsplatModifyPS fragment chunk (paired with the existing gsplatModifyVS), allowing per-pixel customization of the final splat color via app.scene.gsplat.material. The new gaussian-splatting/relighting example and the GsplatRelighting script (scripts/esm/gsplat/gsplat-relighting.mjs) then demonstrate lit-shader features on splats: a proxy mesh of the splat scene is lit by IBL, a PCSS directional light and shadow-casting omni lights, rendered into a screen-aligned offscreen texture, and each splat fragment samples that lighting at its own screen position to modulate its color.

This delivers lighting/shadows on splats while keeping the gsplat and lit shaders modular, avoiding the complexity cost discussed above — so closing this in favor of that approach.