Merge pull request #340 from NYU-RTS/utils

added util file docs; reorged getting started

Author: s-sajid-ali

docs/genai/02_onboarding/01_intro.mdx

@@ -2,7 +2,7 @@
 [research-workspace-request]: https://forms.gle/6DVA5dX4uPPrdX4Q9
 [member-onboarding-form]: https://forms.gle/wd3zcntnY94Q4LWq5
 
-This section deals with the eligibility for getting access to Pythia. To learn more about affiliate access (students after graduation, collaborators from other institutions, please refer to [this section](../../hpc/01_getting_started/02_getting_and_renewing_an_account.mdx))
+This section deals with the eligibility for getting access to Pythia. To learn more about affiliate access (students after graduation, collaborators from other institutions, please refer to [this section](../../hpc/01_getting_started/02_HPC_Accounts/01_getting_and_renewing_an_account.mdx))
 
 :::tip[workspaces]
 Access to Pythia is facilitated via workspaces. Beyond facilitating access, they also allow team members to collaboratively dvelop prompts.

docs/hpc/01_getting_started/01_intro.mdx

@@ -2,6 +2,10 @@
 
 Welcome to the Torch HPC documentation! If you do not have an HPC account, please proceed to the next section that explains how you may be able to get one.
 
+:::note
+Please be aware that a requirement to submit jobs on the Torch cluster is having a `SLURM` account (which is given to you once you are part of an active allocation associated a project within the HPC project management portal) as explained in the section on [Slurm Accounts](./03_Slurm_Accounts/01_intro_slurm_accounts.mdx)
+:::
+
 If you are an active user, you can proceed to one of the categories on the left.
 
 ## Announcements

…d/02_getting_and_renewing_an_account.mdx …s/01_getting_and_renewing_an_account.mdxdocs/hpc/01_getting_started/02_getting_and_renewing_an_account.mdx renamed to docs/hpc/01_getting_started/02_HPC_Accounts/01_getting_and_renewing_an_account.mdx docs/hpc/01_getting_started/02_getting_and_renewing_an_account.mdx renamed to docs/hpc/01_getting_started/02_HPC_Accounts/01_getting_and_renewing_an_account.mdx

@@ -23,7 +23,7 @@ This section deals with the eligibility for getting HPC accounts, the process to
 
 :::warning
 
-An active HPC account gives you access to Torch, but an active allocation within the HPC projects management portal gives you access to a SLURM account which is needed to run jobs on Torch. More information on using the HPC project management portal can be found [here](./07_hpc_project_management_portal.mdx)
+An active HPC account gives you access to Torch, but an active allocation within the HPC projects management portal gives you access to a SLURM account which is needed to run jobs on Torch. More information on using the HPC project management portal can be found [here](../03_Slurm_Accounts/02_hpc_project_management_portal.mdx)
 
 :::
 
@@ -36,11 +36,11 @@ NYU HPC clusters and related resources are available to full-time NYU faculty an
 
 ## Getting a New Account on the NYU HPC Clusters
 
-To request an NYU HPC account please log in to [NYU Identity Management service][nyu ims link] and follow the link to "Request HPC account" by clicking on the hamburger icon on the top left of the page and selecting "Manage Access" . We have a walkthrough of how to [request an account through IIQ](03_walkthrough_request_hpc_account.mdx). If you are a student, alumni or an external collaborator you will need an NYU faculty sponsor.
+To request an NYU HPC account please log in to [NYU Identity Management service][nyu ims link] and follow the link to "Request HPC account" by clicking on the hamburger icon on the top left of the page and selecting "Manage Access" . We have a walkthrough of how to [request an account through IIQ](02_walkthrough_request_hpc_account.mdx). If you are a student, alumni or an external collaborator you will need an NYU faculty sponsor.
 
 ## Renewing HPC Account
 
-Each year, non-faculty users must renew their HPC account by filling in the account renewal form from the [NYU Identity Management service][nyu ims link]. See [Renewing your HPC account](05_walkthrough_renew_hpc_account.md) for a walkthrough of the process.
+Each year, non-faculty users must renew their HPC account by filling in the account renewal form from the [NYU Identity Management service][nyu ims link]. See [Renewing your HPC account](04_walkthrough_renew_hpc_account.md) for a walkthrough of the process.
 
 ## Information for Faculty Who Sponsor HPC Users
 
@@ -72,7 +72,7 @@ HPC faculty sponsors are expected to:
 
 -   Respond promptly to account-related requests from HPC staff
 
-Each year, your sponsored users must renew their account. You will need to approve the renewal by logging into the [NYU Identity Management service][nyu ims link]. We have a [walkthrough of the approval process here](04_walkthrough_approve_hpc_account_request.md)
+Each year, your sponsored users must renew their account. You will need to approve the renewal by logging into the [NYU Identity Management service][nyu ims link]. We have a [walkthrough of the approval process here](03_walkthrough_approve_hpc_account_request.md)
 
 ## Bulk HPC Accounts for Courses
 
@@ -97,7 +97,7 @@ Accounts created for courses last until the end of the semester, rather than a f
 
 If you are collaborating with NYU researchers you will need to obtain **affiliate** status before applying for an NYU HPC account. A full-time NYU faculty member must sponsor a non-NYU collaborator for affiliate status.
 
-Please see [instructions for affiliate management][affiliate and account management link] (NYU NetID login is required to follow the link). [Please read instructions about sponsoring external collaborators here](06_hpc_accounts_external_collaborators.md).
+Please see [instructions for affiliate management][affiliate and account management link] (NYU NetID login is required to follow the link). [Please read instructions about sponsoring external collaborators here](05_hpc_accounts_external_collaborators.md).
 
 
 ## Access to Cluster After Graduation

…d/03_walkthrough_request_hpc_account.mdx …s/02_walkthrough_request_hpc_account.mdxdocs/hpc/01_getting_started/03_walkthrough_request_hpc_account.mdx renamed to docs/hpc/01_getting_started/02_HPC_Accounts/02_walkthrough_request_hpc_account.mdx docs/hpc/01_getting_started/03_walkthrough_request_hpc_account.mdx renamed to docs/hpc/01_getting_started/02_HPC_Accounts/02_walkthrough_request_hpc_account.mdx

@@ -4,7 +4,7 @@ import TabItem from '@theme/TabItem';
 # How to request an HPC account
 
 :::tip
-Make sure you don't already have an HPC account. You can check this by attempting to log in to the cluster, according to the instructions at [Connecting to the HPC Cluster](../02_connecting_to_hpc/01_connecting_to_hpc.mdx).
+Make sure you don't already have an HPC account. You can check this by attempting to log in to the cluster, according to the instructions at [Connecting to the HPC Cluster](../../02_connecting_to_hpc/01_connecting_to_hpc.mdx).
 :::
 
 Login to the URL given below, using your NetID/password, to create or manage HPC Account Requests:
@@ -16,37 +16,37 @@ You need to be on the NYU VPN to perform this task!
 <Tabs>
   <TabItem value="end-users" label="Researcher/End-User" default>
   Upon logging in, an end user’s landing page will look like this:
-  ![Request account landing page](./static/identity_landing.png)
+  ![Request account landing page](../static/identity_landing.png)
 
   Note that if the menu does not appear, select the "burger" menu on the top left hand corner:
-  ![Request burger](./static/request_burger.png)
+  ![Request burger](../static/request_burger.png)
 
   Navigate to Manage Accounts > Request  HPC Account:
 
-  ![Request account left hand button](./static/request_account.png)
+  ![Request account left hand button](../static/request_account.png)
 
   You will be prompted with a form. Please fill it out and Submit.
-  ![Request account form](./static/request_form.png)
+  ![Request account form](../static/request_form.png)
   </TabItem>
   <TabItem value="sponsors" label="Faculty/Sponsor/Approver">
   When an approver/sponsor/faculty logs in, it will look like this:
-  ![Faculty landing](./static/faculty_request_landing.png)
+  ![Faculty landing](../static/faculty_request_landing.png)
   with “QuickLinks” to both “Bulk HPC Account Request” and “Request HPC Account” and will show any recent or pending approvals or forms.
 
   If the **Request HPC Account** QuickLink is clicked, the following form appears:
-  ![Faculty request form](./static/faculty_request_form.png)
+  ![Faculty request form](../static/faculty_request_form.png)
 
   The user’s name will be pre populated, and the forms required fields must be completed (sponsor, reason for request, consent to terms of use).  After clicking “Submit” the chosen sponsor will be notified of the request and provisioning will only occur after approval.
 
   If the **Bulk HPC Account Request** QuickLink is clicked, the following form appears:
-  ![Bulk account request form](./static/bulk_acct_req_form.png)
+  ![Bulk account request form](../static/bulk_acct_req_form.png)
 
   The requestor’s name will be prepopulated, and the forms required fields must be completed (sponsor, list of netids, course identifier, reason for request, consent to terms of use).   These requests are auto-approved since they are usually submitted by the sponsor themselves or a member of the HPC Admin team.
   </TabItem>
 </Tabs>
 
 
 :::tip
--   An HPC account gives you access to Torch, but an active allocation within the HPC projects management portal gives you access to a SLURM account which is needed to run jobs on Torch. More information on using the HPC project management portal can be found [here](./07_hpc_project_management_portal.mdx)
+-   An HPC account gives you access to Torch, but an active allocation within the HPC projects management portal gives you access to a SLURM account which is needed to run jobs on Torch. More information on using the HPC project management portal can be found [here](../03_Slurm_Accounts/02_hpc_project_management_portal.mdx)
 :::
 

…lkthrough_approve_hpc_account_request.md …lkthrough_approve_hpc_account_request.mddocs/hpc/01_getting_started/04_walkthrough_approve_hpc_account_request.md renamed to docs/hpc/01_getting_started/02_HPC_Accounts/03_walkthrough_approve_hpc_account_request.md docs/hpc/01_getting_started/04_walkthrough_approve_hpc_account_request.md renamed to docs/hpc/01_getting_started/02_HPC_Accounts/03_walkthrough_approve_hpc_account_request.md

@@ -7,19 +7,19 @@ You need to be on the NYU VPN to perform this task!
 :::
 
 You can also [log into IIQ at any time](https://iiq.nyu.edu/identityiq), and if you have a request awaiting your approval, it will appear in your "Actions Items" box, as per the following screenshot:
-!["Actions Items" box](./static/work_item.png)
+!["Actions Items" box](../static/work_item.png)
 
 Another way to get to pending approvals is to click on the line item in the “Latest Approvals” section which will lead directly to the approval page. For new HPC Account Requests, the page will look like this:
-![“Latest Approvals” section](./static/latest_approvals.png)
+![“Latest Approvals” section](../static/latest_approvals.png)
 
 
 Here, the Approve or Deny button should be clicked, then confirmed, in order to complete the request.
 
 For HPC Account Renewals, the page will look like this:
-![Approver renewal](./static/approve_renewal.png)
+![Approver renewal](../static/approve_renewal.png)
 
 Here, all systems should be selected by clicking the check box in the menu bar, and choosing “Select Everything”.
-![Approve select everything](./static/approve_select_everything.png)
+![Approve select everything](../static/approve_select_everything.png)
 
 Then, the “Select Bulk Action” menu is used to Approve or Reject all items selected.  Please note that the line items may span multiple pages and all items must be acted upon in order to complete the request. Clicking “Complete” will complete the request.
 

…rted/05_walkthrough_renew_hpc_account.md …unts/04_walkthrough_renew_hpc_account.mddocs/hpc/01_getting_started/05_walkthrough_renew_hpc_account.md renamed to docs/hpc/01_getting_started/02_HPC_Accounts/04_walkthrough_renew_hpc_account.md docs/hpc/01_getting_started/05_walkthrough_renew_hpc_account.md renamed to docs/hpc/01_getting_started/02_HPC_Accounts/04_walkthrough_renew_hpc_account.md

@@ -10,22 +10,22 @@ https://identity.it.nyu.edu/
 
 Upon logging in, an end user’s landing page will look like this
 
-![Login screen](./static/renew_01.PNG)
+![Login screen](../static/renew_01.PNG)
 
 If the menu does not appear, select the "burger" menu on the top left hand corner:
-![View menu](./static/renew_02.PNG)
+![View menu](../static/renew_02.PNG)
 
 The burger menu will show an "Update/Renew HPC Account" option - select this.
-!["Update/Renew HPC Account" option](./static/renew_03.png)
+!["Update/Renew HPC Account" option](../static/renew_03.png)
 
 Next complete the form as instructed. Please note that all accounts require the sponsorship of a full-time NYU faculty member.
-![Select sponsor](./static/renew_04.png)
+![Select sponsor](../static/renew_04.png)
 
 The user’s name will be pre-populated, and the form's required fields must be completed (sponsor, reason for request, consent to terms of use).  After clicking “Submit” the chosen sponsor will be notified of the request and provisioning will only occur after approval.
 
 :::note
 If your HPC Account is due for renewal, you will get an update on your dashboard that will suggest that you fill out a form given in the "Latest form" widget for renewing your account:
-![Banner messages](./static/renew_05.png)
+![Banner messages](../static/renew_05.png)
 :::
 
 If you are not a full-time NYU faculty member, you will need an NYU faculty member to sponsor your application. This is probably your thesis supervisor, or NYU collaborator.

…6_hpc_accounts_external_collaborators.md …5_hpc_accounts_external_collaborators.mddocs/hpc/01_getting_started/06_hpc_accounts_external_collaborators.md renamed to docs/hpc/01_getting_started/02_HPC_Accounts/05_hpc_accounts_external_collaborators.md docs/hpc/01_getting_started/06_hpc_accounts_external_collaborators.md renamed to docs/hpc/01_getting_started/02_HPC_Accounts/05_hpc_accounts_external_collaborators.md

@@ -13,10 +13,10 @@ External (non-NYU) collaborators can access, with proper sponsorship, the NYU HP
 :::note
 Once the sponsoring faculty approves the account request, the HPC account is created within one hour.
 :::
--   Once the HPC account is created, the external collaborator can access HPC resources as described in our documentation page [Connecting to the HPC Cluster](../02_connecting_to_hpc/01_connecting_to_hpc.mdx).
+-   Once the HPC account is created, the external collaborator can access HPC resources as described in our documentation page [Connecting to the HPC Cluster](../../02_connecting_to_hpc/01_connecting_to_hpc.mdx).
 
 :::note
-As with all sponsored accounts, HPC accounts for external collaborators are valid for a period of 12 months, at which point a renewal process is required to continue access to the NYU HPC environment.  You can find information about renewing your account on our documentation page [Getting and Renewing an HPC Account](./02_getting_and_renewing_an_account.mdx).
+As with all sponsored accounts, HPC accounts for external collaborators are valid for a period of 12 months, at which point a renewal process is required to continue access to the NYU HPC environment.  You can find information about renewing your account on our documentation page [Getting and Renewing an HPC Account](../02_HPC_Accounts/01_getting_and_renewing_an_account.mdx).
 :::
 
 [nyu vpn link]: https://www.nyu.edu/life/information-technology/infrastructure/network-services/vpn.html

docs/hpc/01_getting_started/02_HPC_Accounts/_category_.json

@@ -0,0 +1,3 @@
+{
+  "label": "HPC Accounts"
+}

docs/hpc/01_getting_started/03_Slurm_Accounts/01_intro_slurm_accounts.mdx

@@ -0,0 +1,3 @@
+# Slurm Accounts
+
+Users are required to have at least one active Slurm acount to submit jobs on the cluster.  You will specify this with the `--account=` flag.  Accounts are associated with an active allocation within a project that PIs manage using the [HPC Project Portal](02_hpc_project_management_portal.mdx).

…ted/07_hpc_project_management_portal.mdx …nts/02_hpc_project_management_portal.mdxdocs/hpc/01_getting_started/07_hpc_project_management_portal.mdx renamed to docs/hpc/01_getting_started/03_Slurm_Accounts/02_hpc_project_management_portal.mdx docs/hpc/01_getting_started/07_hpc_project_management_portal.mdx renamed to docs/hpc/01_getting_started/03_Slurm_Accounts/02_hpc_project_management_portal.mdx

@@ -6,7 +6,7 @@ import TabItem from '@theme/TabItem';
 
 :::warning[Accessing the portal]
 -   You need to be connected to [NYU VPN](https://www.nyu.edu/life/information-technology/infrastructure/network-services/vpn.html).
--   You need to have an active [HPC account](./02_getting_and_renewing_an_account.mdx).
+-   You need to have an active [HPC account](../02_HPC_Accounts/02_walkthrough_request_hpc_account.mdx).
 :::
 
 :::tip[HTTP 431 Errors]
@@ -35,11 +35,10 @@ The HPC project and allocation management portal for Torch can be accessed by na
 
 
 How do I:
--   [Create a project?](08_creating_hpc_projects.mdx)
--   [Request an allocation?](09_requesting_an_allocation.mdx)
--   [View resources that are available to me?](09_requesting_an_allocation.mdx)
--   [Add publications and grants for your project](10_adding_publications_grants.mdx)
--   [Approve an allocation request?](11_approving_an_allocation_request.mdx)
-
+-   [Create a project?](03_creating_hpc_projects.mdx)
+-   [Request an allocation?](04_requesting_an_allocation.mdx)
+-   [Approve an allocation request?](05_approving_an_allocation_request.mdx)
+-   [View resources that are available to me?](04_requesting_an_allocation.mdx)
+-   [Add publications and grants for your project](06_adding_publications_grants.mdx)
 
 An active allocation in the HPC projects portal is needed to perform any work on Torch. The next sections describe the project and allocation request process in detail.