feat(ourlogs): add truncate RPC parameter for logs events query#116008
feat(ourlogs): add truncate RPC parameter for logs events query#116008JoshuaKGoldberg wants to merge 7 commits into
truncate RPC parameter for logs events query#116008Conversation
Adds a `truncate` query parameter to the organization events endpoint that truncates string column values at the RPC layer for the OurLogs dataset, reducing payload size for wide viewports.
github-actions
Bot
added
the
Scope: Backend
| "organizations:dynamic-sampling", | ||
| "organizations:on-demand-metrics-extraction", | ||
| "organizations:on-demand-metrics-extraction-widgets", | ||
| "organizations:on-demand-metrics-extraction-experimental", |
There was a problem hiding this comment.
🤔 some kind of merge oddity, will remove
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
JoshuaKGoldberg
changed the title
truncate RPC parameter for logs events query
| result_value = process_value(result_value) | ||
|
|
||
| # Note: post-query truncation may not be our preferred method long-term. | ||
| # We may want to set up a function that filters/truncates at the EAP side. |
There was a problem hiding this comment.
For my own learning, why is this not the preferred approach for this PR, time/scope?
There was a problem hiding this comment.
Please enjoy this 84-reply thread 😄 https://sentry.slack.com/archives/C08CR80T3RB/p1778786746769669
(Kevan and Will can speak to this better than me)
| truncate_str = request.GET.get("truncate") | ||
| if truncate_str is not None: | ||
| try: | ||
| max_string_length = int(truncate_str) |
There was a problem hiding this comment.
Ahh ok, this one does cast int here, so at least backend side is protected, sweet
adrianviquez
left a comment
adrianviquez
left a comment
There was a problem hiding this comment.
small q, otherwise 👍
![sentry[bot]](https://avatars.githubusercontent.com/in/12637?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 2 potential issues.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 215fa68. Configure here.
| additional_queries=additional_queries, | ||
| **( | ||
| {"max_string_length": max_string_length} | ||
| if scoped_dataset == OurLogs |
There was a problem hiding this comment.
Can we either error at the API level when this is passed on non logs datasets, or just enable it everywhere?
| sampling_mode=snuba_params.sampling_mode, | ||
| page_token=page_token, | ||
| additional_queries=additional_queries, | ||
| **( |
There was a problem hiding this comment.
Can you just pass this directly to run_table_query instead of a kwargs or put it in the config?
| assert response.status_code == 400 | ||
| assert response.data["detail"] == "Invalid per_page value. Must be between 1 and 9999." | ||
|
|
||
| @pytest.mark.querybuilder |
There was a problem hiding this comment.
I don't think this test needs to be part of the smoke suite
3 participants
Adds a
truncatequery parameter to the organization events endpoint for the OurLogs dataset. When provided, string column values are truncated at the RPC layer to the specified length. This can reduce response payload size nicely when there are huge log fields, such as -but not limited to-message.Split out of #115638. #116009 has the corresponding frontend changes.
Closes LOGS-816.