Skip to content

Add a query for the tool-input-size-to-memory-usage ratio.#139

Draft
natefoo wants to merge 6 commits into
galaxyproject:mainfrom
natefoo:input-mem-ratio
Draft

Add a query for the tool-input-size-to-memory-usage ratio.#139
natefoo wants to merge 6 commits into
galaxyproject:mainfrom
natefoo:input-mem-ratio

Conversation

@natefoo

@natefoo natefoo commented Nov 15, 2023

Copy link
Copy Markdown
Member

Hoping to use this for insight on TPV rules, although tools I've looked at so far tend not to have super meaningful correlations 🙃

@bgruening

Copy link
Copy Markdown
Member

@natefoo did you had a look at good-for-pulsar. Should we rename this one to fit into your naming schema and be a bit more explicit about what it does :) Should we harmonize here?

@natefoo

natefoo commented Nov 15, 2023

Copy link
Copy Markdown
Member Author

Ah thanks for the heads up, looks useful - although that is input-size-to-runtime, but they could certainly be combined.

@bgruening

Copy link
Copy Markdown
Member

or we rename good-for-pulsar to tool-input-size-to-runtime or something like that.

@hexylena

Copy link
Copy Markdown
Member

yes it's a good suggestion, please rename the pulsar function to match.

Comment thread CHANGELOG.md Outdated
Comment thread parts/22-query.sh
limit_clause="ORDER BY j.id DESC LIMIT $arg_last"
fi
if [[ -n "$arg_summary" ]]; then
summary="$(summary_statistics 'jmn.metric_value / isz.bytes' 0)"

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so glad stuff like this can get re-used

natefoo and others added 2 commits November 16, 2023 14:42
Co-authored-by: Helena <hexylena@galaxians.org>
@natefoo

natefoo commented Nov 22, 2023

Copy link
Copy Markdown
Member Author

The whole time I was writing this I was like "I swear I already wrote this," but I looked through the existing queries and didn't see it.

Well... #78 🤦

@natefoo natefoo marked this pull request as draft September 12, 2024 17:54
@natefoo

natefoo commented Sep 12, 2024

Copy link
Copy Markdown
Member Author

This is better than the other query IMO but needs a bit more work and to integrate the two of them.

@natefoo

natefoo commented Nov 7, 2024

Copy link
Copy Markdown
Member Author

Another thing to note - it would be great if we could determine (or barring that, estimate) the uncompressed input size, since that is probably wildly varying the data.

Related discussion in galaxyproject/total-perspective-vortex#141, but unlike TPV we are collecting historical data about inputs that are probably already deleted, so we can't check the actual size (and that would be impractically slow anyway).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants