feat: add screen-space ambient occlusion (SSAO) by hubbardp · Pull Request #993 · google/neuroglancer

hubbardp · 2026-05-15T14:49:46Z

Summary

Screen-space ambient occlusion (SSAO) simulates shadows on 3D mesh surfaces by darkening crevices and concavities where ambient light would be occluded. It adds depth cues that help us perceive shapes, and it makes the display more appealing. SSAO is an efficient post-processing effect applied to the perspective view after opaque geometry is drawn. See src/ssao/README.md for more details and example images.

Screenshots

Usage

Press q to toggle SSAO on and off.
Use sliders in the "Settings" panel to adjust intensity and radius (softness).

Known limitations

SSAO is disabled in any perspective view that contains a volume-rendering layer; a one-time status banner notifies the user.
Translucent annotations covering a mesh suppress SSAO at the covered pixels.

Performance

The first use of SSAO triggers the creation of the NORMAL attachment, retained thereafter.
The first use of SSAO triggers mesh shader recompilation.
All uses of SSAO add three additional full-screen passes.
No user-perceptible change in performance, even when tested on older laptops with only integrated graphics.

Algorithm

GTAO (Ground Truth Ambient Occlusion): Jimenez et al., "Realtime Strategies for Accurate Indirect Occlusion", SIGGRAPH 2016

Testing

Manually verified with various datasets from Janelia, OpenOrganelle.
New browser tests in src/ssao/shaders.browser_test.ts cover composite math, GTAO and composite sentinel paths, blurring pass.

chrisj · 2026-05-15T16:10:32Z

@hubbardp this is very exciting! Looks great from a quick look so far

flyem_fib-25 DEMO

chrisj · 2026-05-15T16:18:14Z

+  // World→UV scale: wClip is -P.z under perspective and 1 under ortho.
+  float wClip = uProjection[2][3] * P.z + uProjection[3][3];
+  float screenRadius = uRadius * uProjection[1][1] / (2.0 * wClip);
+  screenRadius = min(screenRadius, MAX_KERNEL_FRACTION);


In the link I posted, I can't see any visible difference when changing the radius value, I'm wondering if the value is always larger than MAX_KERNEL_FRACTION

Thanks for the observation. I checked in what I think is an improvement; see more with your later comment, below.

hubbardp · 2026-05-15T16:33:24Z

For those who want to try the new functionality without building the code, it's running here:
http://neuroglancer-ssao.janelia.org/

chrisj · 2026-05-15T16:50:01Z

@hubbardp neuroglancer github actions are set up to create a deployment for every PR, available if you click "view details", it's definitely not obvious:

That's what I used to create the link I posted.

jbms · 2026-05-15T21:18:59Z

What is the reason for disabling the effect on highlighted segments? It makes the highlighting rather jarring, though I haven't tested with the alternative behavior.

hubbardp · 2026-05-15T21:43:56Z

What is the reason for disabling the effect on highlighted segments? It makes the highlighting rather jarring, though I haven't tested with the alternative behavior.

I tried not disabling the SSAO darkening on highlighted segments, and they were too dark to see when highlighted. Personally, I like the brighter highlighting. I found that before SSAO, sometimes the randomly-chosen segment color was bright enough that the highlighting seemed barely distinguishable.

fcollman · 2026-05-16T01:14:02Z

this look really cool! i'm excited by it.

I immediately went to a very busy microns dataset to see what it would look like

link

and I see thing looks reasonable when viewed from the +/-xz, +/-yz view, but rotating it +/- xy everything looks dim and muddy.

YZ looks good here

XY all the cells in the middle look dulled, when i was expecting some to look brighter.

hubbardp · 2026-05-16T16:45:13Z

this look really cool! i'm excited by it.
I immediately went to a very busy microns dataset to see what it would look like
and I see thing looks reasonable when viewed from the +/-xz, +/-yz view, but rotating it +/- xy everything looks dim and muddy.

Thanks for the test case. I pushed a fix for the anisotropy. I think it looks better from all view angles.

I have not yet updated http://neuroglancer-ssao.janelia.org/ so use the version from the GitHub action.

fcollman · 2026-05-16T16:52:59Z

awesome thank you agreed looks better on my localhost version!

seankmartin

Thanks for the changes! My comments are more related to the integration in neuroglancer, as I'll admit that I haven't dug into the SSAO implementation compared to the paper. Are there any known simplifications or deviations taken here over the paper description?

seankmartin · 2026-05-19T15:11:38Z

+
+SSAO is limited to mesh surfaces because only `MeshLayer` and
+`MultiscaleMeshLayer` supply a view-space normal via the three-argument
+`emit(color, pickId, viewNormal)` form. All other opaque geometry (skeletons,


I wouldn't see it as blocking, but in particular for the single meshes, I think it could be useful to expand here why they are not supported. Since they have normaIs, I imagine it's because the overhead in making the shader in the single mesh layer shader emitter aware and some duplication of the normal handling basically mimicking the changes made in mesh/frontend.ts

Thanks for the suggestion. I checked in a revised README that expands on the reasons for omitting annotations, skeletons and single-mesh layers. For the latter, I could add support with some refactoring, so my preference would be to leave it for future work. Let me know what you think.

seankmartin · 2026-05-19T15:24:31Z

+  // Referenced by glsl_perspectivePanelEmitWithNormals; declared here so any
+  // shader using this emitter (mesh, annotation, etc.) gets it without having
+  // to know the emit body's internal dependencies.
+  builder.addUniform("highp float", "uHighlighted");


I think the use of a uniform for highlighted to disable AO in the shader will up being a bit limited in scope. This assumes that in a single draw everything is either highlighted or unhighlighted, which is a bit limiting.

Perhaps this could move to the responsibility of the mesh layer if this functionality is needed. So meshes zero-out their view normal to the no AO sentinel version. And then the perspective shaders don't have to worry about this anymore.

If other layers want to disable AO dynamically, they can use the same pattern

Thanks for the suggestion. I checked in code that implements what I think is your idea.

seankmartin · 2026-05-19T15:29:49Z

+export const glsl_perspectivePanelEmitWithNormals = `
+void emit(vec4 color, highp uint pickId, vec3 viewNormal) {
+  out_color = color;
+  float zValue = 1.0 - gl_FragCoord.z;
+  out_z = vec4(zValue, zValue, zValue, 1.0);
+  float pickIdFloat = float(pickId);
+  out_pickId = vec4(pickIdFloat, pickIdFloat, pickIdFloat, 1.0);
+  // Highlighted objects collapse to the zero sentinel so SSAO leaves them alone.
+  vec3 packedNormal = (1.0 - uHighlighted) * (normalize(viewNormal) * 0.5 + 0.5);
+  out_normal = vec4(packedNormal, 1.0);
+}
+void emit(vec4 color, highp uint pickId) {
+  out_color = color;
+  float zValue = 1.0 - gl_FragCoord.z;
+  out_z = vec4(zValue, zValue, zValue, 1.0);
+  float pickIdFloat = float(pickId);
+  out_pickId = vec4(pickIdFloat, pickIdFloat, pickIdFloat, 1.0);
+  // Zero-RGB sentinel; alpha=1 so the source overwrites dst under
+  // blend(SRC_ALPHA, ONE_MINUS_SRC_ALPHA).
+  out_normal = vec4(0.0, 0.0, 0.0, 1.0);


I think it would be nice if we could reduce the duplication in the bodies here so changes propagate if ever needed and to clarify what's changing between emitters. Maybe something like

const glslEmitBase = ` out_color = color; float zValue = 1.0 - gl_FragCoord.z; out_z = vec4(zValue, zValue, zValue, 1.0); float pickIdFloat = float(pickId); out_pickId = vec4(pickIdFloat, pickIdFloat, pickIdFloat, 1.0);`;

and then it could be used in both the 2-arg and 3-arg emits, and also be replaced in glsl_perspectivePanelEmit. Don't feel strongly on the name of the var glslEmitBase

Thanks, I checked in this idea for factoring out shared code.

seankmartin · 2026-05-19T15:38:15Z

+    // Mixed mesh + volume scenes are not yet supported; the opaque pass and
+    // NORMAL writes already happened from the with-normals emitter (minor
+    // waste, not a correctness issue). Notify once per panel lifetime via
+    // both the console and a dismissable status banner.


I'm unsure why volume rendering disables SSAO. SSAO happens in opaque rendering only, so I don't see how the later transparent rendering is interfered with. Could be missing something of course

I did another test of the code modified to allow SSAO with volume rendering, and the result is not right. The problem is that SSAO involves a final compositing step that multiplies the existing color buffer by the AO darkening. By the time that step runs, the volume has been blended into the color buffer, so AO darkens the volume contribution, too, and not just the mesh underneath. The part of the volume in front of the mesh should hide those AO details, but instead it gets darkened along with them. It should be possible to restructure the pipeline so the volume rendering is added after the SSAO compositng, but that would add to the scale of the code changes. Let me know if you think it is worth doing.

seankmartin · 2026-05-19T15:42:58Z

+      );
+      const radius = ssaoRadius * this.navigationState.zoomFactor.value;
+
+      this.ssaoManager.render(


Just wanted to note that as the complexity of the panel.ts file has increased, moving part of the rendering responsibility out of this file, as done here, seems reasonable and could be a good idea for a later refactoring of the volume rendering path

seankmartin · 2026-05-19T15:45:04Z

+    // NORMAL writes already happened from the with-normals emitter (minor
+    // waste, not a correctness issue). Notify once per panel lifetime via
+    // both the console and a dismissable status banner.
+    if (ssaoRequested && hasVolumeRendering && !this.ssaoVolumeWarned) {


Related to https://github.com/google/neuroglancer/pull/993/changes#r3267587804, if we could leave AO on with volume rendering - maybe we could instead enable this warning message if you have an AO supported mesh layer but with transparent rendering for that layer?

seankmartin · 2026-05-19T15:56:52Z

@@ -0,0 +1,292 @@
+/**


As a general comment here, I could be missing it but I don't see a test for a case where ao is between 0 and 1, or in other words a non 1x1 size texture example to check sampling from nearby neighbours and having ao < 1.0 if the centre pixel is being occluded by neighbour pixels. Would that be possible to add?

chrisj · 2026-05-19T21:02:53Z

+        SSAO_RADIUS_RANGE.max,
+        Math.max(SSAO_RADIUS_RANGE.min, this.viewer.ssaoRadius.value),
+      );
+      const radius = ssaoRadius * this.navigationState.zoomFactor.value;


removing this operation causes the radius slider to have an effect on my device. It does mean it now becomes softer as you zoom out.

The old code scaled the radius incorrectly, so the clamping at MAX_KERNEL_FRACTION was happening too often. I checked in new code that clarifies how the slider value affects the sampling kernel and the falloff factor. I think it looks reasonable when zooming out, and lets the slider add some variation. The new default of 2 for the slider works well in every case I tried. I would not mind removing the slider altogether if it is confusing, but it also seems okay to leave it in for completeness.

seankmartin · 2026-05-19T21:05:17Z

One other thing that might be nice is if we add some kind of DEBUG_SSAO const to the ssao/shaders.ts which when enabled sets the fragment color to be the non-color composited ao value in the composite step (at the moment you can set fixed color to white in the segmentation layer instead for a similar effect) defineSSAOCompositeShader. For example:
AO debug:

While the regular result with color contribution is:

Could help for any later fine tuning, seems like there might be a bit of noise showing through in certain cases for e.g.
https://github.com/user-attachments/assets/432f450f-4b31-4fe4-928c-3c84fcf88b34

feat: add screen-space ambient occlusion (SSAO)

5b03482

chrisj reviewed May 15, 2026

View reviewed changes

Fixes a test failure due to Python line length.

f37900d

Fixes SSAO for anisotropic volumes.

bfd71cc

Fixes the scale bar with SSAO.

ceb02d0

seankmartin reviewed May 19, 2026

View reviewed changes

chrisj reviewed May 19, 2026

View reviewed changes

hubbardp added 3 commits May 20, 2026 10:44

Fixes use of the ssao radius slider value.

2c539a1

Updates documentation on types omitted from SSAO.

514097b

Improves how highlighted segments are excluded from SSAO.

ad4e877

seankmartin mentioned this pull request May 21, 2026

feat(panels): allow a debug visualisation for offscreen textures #1002

Open

Factors out shared code.

e799d44

Conversation

hubbardp commented May 15, 2026

Uh oh!

chrisj commented May 15, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

hubbardp commented May 15, 2026

Uh oh!

chrisj commented May 15, 2026

Uh oh!

jbms commented May 15, 2026

Uh oh!

hubbardp commented May 15, 2026

Uh oh!

fcollman commented May 16, 2026

Uh oh!

hubbardp commented May 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

fcollman commented May 16, 2026

Uh oh!

seankmartin left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

seankmartin May 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

seankmartin commented May 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

hubbardp commented May 16, 2026 •

edited

Loading

seankmartin May 19, 2026 •

edited

Loading

seankmartin commented May 19, 2026 •

edited

Loading