fix: raise default WebSocket payload limit to avoid dropping large result frames

[aicam]

What changes were proposed in this PR?

Raises the default maximum workflow WebSocket message size from 64 KB to 1024 KB (1 MB), in both places the default is defined:

• common/config/src/main/resources/application.conf — the base config default (web-server.max-workflow-websocket-request-payload-size-kb)
• bin/k8s/values.yaml — the Helm chart env default (MAX_WORKFLOW_WEBSOCKET_REQUEST_PAYLOAD_SIZE_KB), which overrides the base default in Kubernetes deployments

WebsocketPayloadSizeTuner applies this value to the Jetty WebSocket container's max text/binary message buffer. At the previous 64 KB default, the result/status frame pushed to the browser for a workflow with many operators or a sizable visualization payload (e.g. a Bar Chart) could exceed 65536 bytes; Jetty then drops the entire frame with MessageTooLargeException, so the final result/visualization never reaches the frontend even though the execution completes successfully on the backend. The overshoot is often only a couple KB, so even modest workflows can trip it. 1 MB gives ample headroom for large visualization payloads while remaining well within reasonable WebSocket frame sizes.

This changes only a default value; the limit remains overridable via the MAX_WORKFLOW_WEBSOCKET_REQUEST_PAYLOAD_SIZE_KB environment variable.

How was this PR tested?

Manually. Ran a workflow ending in a Bar Chart whose serialized result frame exceeded 64 KB:

• Before: the visualization did not render, and the computing unit logged MessageTooLargeException: ... too large for configured max of [65536] at completion.
• After: the frame is delivered and the chart renders.

Also verified with helm template that the rendered chart env value updates to 1024 and the chart still renders cleanly.

Was this PR authored or co-authored using generative AI tooling?

Generated-by: Claude Opus 4.8

Any related issues (either this closes or is dependent on)?

Closes #6857

[github-actions]

Automated Reviewer Suggestions

Based on the git blame history of the changed files, we recommend the following reviewers:

• Contributors with relevant context: @kunwp1
  You can notify them by mentioning @kunwp1 in a comment.

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 81.06%. Comparing base (06f8e42) to head (7de0224).
⚠️ Report is 50 commits behind head on main.

Additional details and impacted files

@@             Coverage Diff              @@
##               main    #6858      +/-   ##
============================================
+ Coverage     78.98%   81.06%   +2.08%     
+ Complexity     3785      256    -3529     
============================================
  Files          1160      532     -628     
  Lines         46105    30548   -15557     
  Branches       5115     3267    -1848     
============================================
- Hits          36414    24764   -11650     
+ Misses         8069     4984    -3085     
+ Partials       1622      800     -822     

Flag	Coverage Δ		*Carryforward flag
access-control-service	71.92% <ø> (+1.92%)	⬆️
agent-service	76.76% <ø> (ø)		Carriedforward from 214402f
amber	31.21% <ø> (-40.97%)	⬇️
computing-unit-managing-service	16.12% <ø> (-4.37%)	⬇️
config-service	63.57% <ø> (-3.10%)	⬇️
file-service	62.81% <ø> (-4.40%)	⬇️
frontend	82.99% <ø> (-0.01%)	⬇️	Carriedforward from 214402f
notebook-migration-service	78.94% <ø> (ø)
pyamber	95.38% <ø> (ø)		Carriedforward from 214402f
workflow-compiling-service	51.78% <ø> (+25.46%)	⬆️

*This pull request uses carry forward flags. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[github-actions]

⚠️ Benchmark changes need a look

🟢 0 better · 🔴 10 worse · ⚪ 5 noise (<±5%) · 0 without baseline

CI benchmark results are noisy; treat <±5% as noise unless repeated.

Dashboard · Run

	config	throughput	MB/s	latency	max Δ latest / 7d
🔴	bs=10 sw=10 sl=64	377	0.23	25,484/38,432/38,432 us	🔴 +150.7% / 🔴 +143.5%
🔴	bs=100 sw=10 sl=64	777	0.474	127,263/151,402/151,402 us	🔴 +29.4% / 🔴 +37.3%
⚪	bs=1000 sw=10 sl=64	921	0.562	1,080,958/1,135,252/1,135,252 us	⚪ within ±5% / 🔴 -8.0%

Baseline details

Latest main ebfd1db from 2026-07-30T14:07:03.708Z

config	metric	PR	latest main	7d avg	Δ latest	Δ 7d
bs=10 sw=10 sl=64	throughput	377 tuples/sec	732.84 tuples/sec	771.62 tuples/sec	-48.6%	-51.1%
bs=10 sw=10 sl=64	MB/s	0.23 MB/s	0.447 MB/s	0.471 MB/s	-48.6%	-51.2%
bs=10 sw=10 sl=64	p50	25,484 us	13,439 us	12,626 us	+89.6%	+101.8%
bs=10 sw=10 sl=64	p95	38,432 us	15,329 us	15,786 us	+150.7%	+143.5%
bs=10 sw=10 sl=64	p99	38,432 us	19,526 us	19,344 us	+96.8%	+98.7%
bs=100 sw=10 sl=64	throughput	777 tuples/sec	897.82 tuples/sec	971.56 tuples/sec	-13.5%	-20.0%
bs=100 sw=10 sl=64	MB/s	0.474 MB/s	0.548 MB/s	0.593 MB/s	-13.5%	-20.1%
bs=100 sw=10 sl=64	p50	127,263 us	110,657 us	103,463 us	+15.0%	+23.0%
bs=100 sw=10 sl=64	p95	151,402 us	116,972 us	110,296 us	+29.4%	+37.3%
bs=100 sw=10 sl=64	p99	151,402 us	132,496 us	118,690 us	+14.3%	+27.6%
bs=1000 sw=10 sl=64	throughput	921 tuples/sec	931.46 tuples/sec	1,001 tuples/sec	-1.1%	-8.0%
bs=1000 sw=10 sl=64	MB/s	0.562 MB/s	0.569 MB/s	0.611 MB/s	-1.1%	-8.0%
bs=1000 sw=10 sl=64	p50	1,080,958 us	1,071,100 us	1,008,988 us	+0.9%	+7.1%
bs=1000 sw=10 sl=64	p95	1,135,252 us	1,109,394 us	1,055,260 us	+2.3%	+7.6%
bs=1000 sw=10 sl=64	p99	1,135,252 us	1,125,922 us	1,074,689 us	+0.8%	+5.6%

Raw CSV

config_idx,batch_size,schema_width,string_len,num_batches,total_ms,total_tuples,total_bytes,tuples_per_sec,mb_per_sec,lat_p50_us,lat_p95_us,lat_p99_us
0,10,10,64,20,531.00,200,128000,377,0.230,25483.98,38431.68,38431.68
1,100,10,64,20,2574.11,2000,1280000,777,0.474,127262.87,151402.47,151402.47
2,1000,10,64,20,21711.01,20000,12800000,921,0.562,1080957.80,1135251.66,1135251.66

[kunwp1]

Is there any downside of increasing the default size to 1MB?

[aicam]

Is there any downside of increasing the default size to 1MB?

We were using this in a session with users with workflows with more than 10 operators, they all failed to run and get result of workflow, after increasing this number, all fixed. So I believe there is no downside since was tested in a live session with >10 users.

[Yicong-Huang]

@aicam do we need this in v1.2?

cc @xuang7

[aicam]

@aicam do we need this in v1.2?

cc @xuang7

@aicam do we need this in v1.2?

cc @xuang7

I think so, the symptom we had today was that users click on Run button and nothing happen. It happened in workflows which have heavy data for example a visualization operator which has more than 64kb data.

[kunwp1]

LGTM!

[chenlica]

Do we want to make the value dynamically configurable by the admins after the deployment?

[aicam]

Do we want to make the value dynamically configurable by the admins after the deployment?

I don't think so, it can't be changed on the fly and is also very deployment specific, something that should be set at the time system was deployed.

[xuang7]

I was wondering if this PR is ready to merge.