Skip to content

added util file docs; reorged getting started#340

Merged
s-sajid-ali merged 9 commits intomainfrom
utils
Apr 15, 2026
Merged

added util file docs; reorged getting started#340
s-sajid-ali merged 9 commits intomainfrom
utils

Conversation

@RobJY
Copy link
Copy Markdown
Contributor

@RobJY RobJY commented Apr 8, 2026

Added documentation for some utilities we offer on Torch.

While working on that I had the idea about reorganizing the 'getting started' section, so I added that too. I thought it would be nice to give a little more information about the Slurm accounts since we still see questions about them.

I know the utilities documentation should be fleshed out a bit more, but I thought I'd make the PR to start the discussion.

Any suggestions?

@RobJY RobJY requested review from mdweisner and s-sajid-ali April 8, 2026 18:04
@RobJY RobJY self-assigned this Apr 8, 2026
@github-actions
Copy link
Copy Markdown

github-actions bot commented Apr 8, 2026

PR Preview Action v1.8.1
Preview removed because the pull request was closed.
2026-04-15 13:53 UTC

Comment thread docs/hpc/01_getting_started/03_Slurm_Accounts/01_intro_slurm_accounts.mdx Outdated
@RobJY
Copy link
Copy Markdown
Contributor Author

RobJY commented Apr 14, 2026

Ok, I think I've gotten that fleshed out, so it's ready to go, unless anyone would like any changes.

@RobJY
Copy link
Copy Markdown
Contributor Author

RobJY commented Apr 14, 2026

Looks great! Thanks for the fix and edits @s-sajid-ali !

@s-sajid-ali
Copy link
Copy Markdown
Member

s-sajid-ali commented Apr 14, 2026

Last suggestion is to simplify:

find $HOME -xdev | cut -d "/" -f 1-4 | sort | uniq -c | sort -nr | head -n 5

to:

du -sh --inodes ./* | sort -nr | head -n 5

unless we think this confuses our users as the former uses more steps

and maybe also add the following for space utilization:

du -sh ./* | sort -hr | head -n 5

@RobJY
Copy link
Copy Markdown
Contributor Author

RobJY commented Apr 14, 2026

du -sh --inodes ./* | sort -nr | head -n 5
Much faster, but seems to miss the dot directories. .cache is the largest for my home directory and it was left out.

( shopt -s dotglob nullglob; du -s --inodes -- * 2>&1 | sort -nr | head -n 5 )
seems to catch them all and is shorter, but maybe too much for users to run?

Great idea to also give a command for space, but I'm getting the same issue with dot dirs/files. This seems to work:
( shopt -s dotglob nullglob; du -sh -- * 2>/dev/null | sort -hr | head -n 5 )

Do you know of a cleaner solution? I'll add these if not.

@s-sajid-ali
Copy link
Copy Markdown
Member

Much faster, but seems to miss the dot directories. .cache is the largest for my home directory and it was left out.

Does this catch it?

du -sh --inodes . | sort -nr | head -n 5

@RobJY
Copy link
Copy Markdown
Contributor Author

RobJY commented Apr 14, 2026

Hmm, that just returns dot:

[rjy1@torch-login-b-1 ~]$ du -sh --inodes . | sort -nr | head -n 5
2.8K	.

@s-sajid-ali
Copy link
Copy Markdown
Member

This works:

~> du --inodes --max-depth=1 . | sort -nr | head -n 5
9010	.
6714	./.pixi
1729	./.cache
490	./.claude
25	./.local
~> du -h --max-depth=1 . | sort -hr | head -n 5
378M	.
370M	./.pixi
5.0M	./.claude
2.8M	./.cache
74K	./.local

@RobJY
Copy link
Copy Markdown
Contributor Author

RobJY commented Apr 14, 2026

Yes, it just needs to be sorted and have '.' filtered out:

[rjy1@torch-login-b-1 ~]$ du --inodes -h --max-depth=1
1	./.apptainer
2.7K	./.cache
2	./.conda
6	./.config
7	./.emacs.d
3	./.ipynb_checkpoints
1	./.ipython
3	./.jupyter
7	./.local
8	./.ssh
4	./doc_testing
9	./myquota_container
11	./ondemand
1	./ubi_sandbox
2.8K	.

this isn't bad:

[rjy1@torch-login-b-1 ~]$ du --inodes -h -s -- * .[!.]* ..?* 2>/dev/null | sort -hr | head -n 5
2.7K	.cache
11	ondemand
9	myquota_container
8	.ssh
7	.local

what do you think?

@s-sajid-ali
Copy link
Copy Markdown
Member

Looks good to me!

@RobJY
Copy link
Copy Markdown
Contributor Author

RobJY commented Apr 14, 2026

Thanks for working through that @s-sajid-ali ! It looks much better now.

Please let me know if you see anything else.

@s-sajid-ali s-sajid-ali merged commit 5fccc19 into main Apr 15, 2026
5 checks passed
@s-sajid-ali s-sajid-ali deleted the utils branch April 15, 2026 13:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants