refactor: move step templates to source-first authoring under src

.github/CONTRIBUTING.md

@@ -5,32 +5,27 @@ Have a great custom step that other Octopus users will love? Here's how to get i
 
 1. [Fork](https://github.com/OctopusDeploy/Library/fork) the Library repository
 2. Clone your fork into a directory on your own machine
-3. _Export_ your template from the Octopus server
-4. Save the exported JSON to a file under `/step-templates`
+3. If you're adding a new template, create a new folder under `/src/step-templates/<template>/`
+4. Add or update `metadata.json` and any sibling script files such as `scriptbody.ps1`, `scriptbody.sh`, `scriptbody.py`, `predeploy.ps1`, `deploy.ps1`, or `postdeploy.ps1`
 5. Check that the `LastModifiedBy` username is one you're happy to use on the site (ideally your plain GitHub username)
 6. Add Id property and set it to a GUID using the following format `abcdef00-ab00-cd00-ef00-000000abcdef`, you can use [this site](https://www.guidgen.com/) to generate one automatically
 7. Optional: Assign your template to an existing category. Have a look at existing templates to find the category that matches your template. If you don't specify it your template will be assigned to 'other' category.
-   - If you add a new category, make sure that you add an icon in `.png` format with a size of 200x200px to the `logos` folder with the same name as your category. Also, the `switch` in the `humanize` function in [`gulpfile.babel.js`](https://github.com/OctopusDeploy/Library/blob/master/gulpfile.babel.js#L92) must have a `case` statement corresponding to it.
+   - If you add a new category, make sure that you add an icon in `.png` format with a size of 200x200px to the `src/step-templates/logos` folder with the same name as your category. Also, the `switch` in the `humanize` function in [`gulpfile.babel.js`](https://github.com/OctopusDeploy/Library/blob/master/gulpfile.babel.js#L92) must have a `case` statement corresponding to it.
 8. If you're updating an existing step template, make sure the `Version` property is incremented (e.g. by 1). If the `Version` doesn't change then the [Community Library Integration](http://docs.octopusdeploy.com/display/OD/Step+Templates#StepTemplates-TheCommunityLibrary) in Octopus won't see your changes.
-9. Commit and push your changes to your fork
-10. View your fork in GitHub to create a _pull request_
+9. Run `npm run build` to regenerate the packed compatibility JSON and validate the result
+10. Commit and push your changes to your fork
+11. View your fork in GitHub to create a _pull request_
 
 Someone from the Octopus team will review your request and help to make the step consistent with the others in the library. Once it's ready we'll merge it into the main repository and publish it to [the library site](http://library.octopusdeploy.com).
 
-**Note**: If you're editing an existing template we've got a tool you can use to help with packing and unpacking the scripts stored in the step template `*.json` file.
-
-* To unpack the step template scripts into separate files alongside the main step template file, run `powershell .\tools\_unpack.ps1`.
-* You can then edit the `*.ps1` files in the `.\step-templates` folder using your favourite PowerShell editor.
-* To pack the step template script files back into the main step template, run `powershell .\tools\_pack.ps1`. 
-
 Here's a **checklist** to consider:
 
 * Is the template a minor variation on an existing one? If so, please consider improving the existing template if possible.
 * Is the name of the template consistent with the examples already in the library, in style ("Noun - Verb"), layout and casing?
 * Are all parameters in the template consistent with the examples here, including help text documented with Markdown?
 * **To minimize the risk of step template parameters clashing with other variables in a project that uses the step template, ensure that you prefix your parameter names (e.g. an abbreviated name for the step template or the category of the step template**
 * Is the description of the template complete, correct Markdown?
-* Is the `.json` filename consistent with the name of the template?
+* Is the template folder name under `/src/step-templates` consistent with the name of the template?
 * Do scripts in the template validate required arguments and fail by returning a non-zero exit code when things go wrong?
 * Do scripts in the template produce worthwhile status messages as they execute?
 * Are you happy to contribute your template under the terms of the [license](https://github.com/OctopusDeploy/Library/blob/master/LICENSE.txt)? If you produced the template while working for your employer please obtain written permission from them before submitting it here.
@@ -63,7 +58,7 @@ We also accept contributions to improve the [library.octopusdeploy.com](http://l
 
 To build the library site you need to have `nodejs` installed on your system.
 
-Run the setup job to install `gulp` globally and install the npm dependencies:
+Run the setup job to install the npm dependencies:
 
 ```
 npm run setup
@@ -76,7 +71,7 @@ Congratulations, you are now ready to build and test the site locally.
 From the root of the repository, run the following command: 
 
 ```
-gulp build
+npm run build
 ```
 
 This will build a debuggable version of the library site and output it to `./build/`. To test the site, run the following command:
@@ -91,7 +86,7 @@ This will start the `express` server and you can browse the site at the followin
 While developing you can run the site in development mode so that as you make changes to the code your browser will refresh to reflect the changes you just made. To run the site in development mode, run the following command:
 
 ```
-gulp watch
+npm run dev
 ```
 
 This will start a `LiveServer` that is used as a proxy for the `express` server running on `http://localhost:9000` and `gulp` will watch for changes and when detected, refresh your browser window.

.github/pull_request_template.md

@@ -4,7 +4,7 @@
 * Is the name of the template consistent with the examples already in the library, in style ("Noun - Verb"), layout and casing?
 * Are all parameters in the template consistent with the examples here, including help text documented with Markdown?
 * Is the description of the template complete, with correct Markdown?
-* Is the `.json` filename consistent with the name of the template?
+* Is the template folder under `/src/step-templates` consistent with the name of the template?
 * Do scripts in the template validate required arguments and fail by returning a non-zero exit code when things go wrong?
 * Do scripts in the template produce worthwhile status messages as they execute?
 * Are you happy to contribute your template under the terms of the [license](https://github.com/OctopusDeploy/Library/blob/master/LICENSE.txt)? If you produced the template while working for your employer please obtain written permission from them before submitting it here.
@@ -37,11 +37,11 @@ _Before submitting your PR, please delete everything above the line below._
   - **NOTE** If you are modifying an existing step template, please make sure that you **do not** modify the `Id` property *(updating the `Id` will break the Library sync functionality in Octopus)*. 
 - [ ] `Version` should be incremented, otherwise the integration with Octopus won't update the step template correctly
 - [ ] Parameter names should not start with `$`
-- [ ] **Step template parameter names (the ones declared in the JSON, not the script body) should be prefixed with a namespace so that they are less likely to clash with other user-defined variables in Octopus** (see [this issue](https://github.com/OctopusDeploy/Issues/issues/2126)). For example, use an abbreviated name of the step template or the category of the step template).
+- [ ] **Step template parameter names (the ones declared in `metadata.json`, not the script body) should be prefixed with a namespace so that they are less likely to clash with other user-defined variables in Octopus** (see [this issue](https://github.com/OctopusDeploy/Issues/issues/2126)). For example, use an abbreviated name of the step template or the category of the step template).
 - [ ] `LastModifiedBy` field must be present, and (_optionally_) updated with the correct author
 - [ ] The best practices documented [here](https://github.com/OctopusDeploy/Library/wiki/Best-Practices) have been applied
 - [ ] If a new `Category` has been created:
-   - [ ] An image with the name `{categoryname}.png` must be present under the `step-templates/logos` folder
+   - [ ] An image with the name `{categoryname}.png` must be present under the `src/step-templates/logos` folder
    - [ ] The `switch` in the `humanize` function in [`gulpfile.babel.js`](https://github.com/OctopusDeploy/Library/blob/master/gulpfile.babel.js#L92) must have a `case` statement corresponding to it
 
 Fixes # . _If there is an open issue that this PR fixes add it here, otherwise just remove this line_

.gitignore

@@ -9,10 +9,9 @@ junitresults.xml
 scriptcs_packages/
 scriptcs_packages.config
 
-step-templates/*.ps1
-step-templates/*.sh
-step-templates/*.py
+step-templates/*
 diff-output/
+.tmp*/
 /.vs
 !.vscode
 *copy.ps1

README.md

@@ -6,9 +6,50 @@ A repository of step templates and other community-contributed extensions to Oct
 Organization
 ------------
 
-* *Step templates* are checked into `/step-templates` as raw JSON exports direct from Octopus Deploy
+* *Step templates* are authored under `/src/step-templates`
+* Generated compatibility JSON for the website and Octopus imports is written to `/step-templates`
 * The *library website* is largely under `/app`, with build artifacts at the root of the repository
-* The `/tools` folder contains utilities to help with editing step templates
+* The `/tools` folder contains the existing pack/unpack scripts plus the source-first generation helpers
+
+Source Layout
+-------------
+
+> [!IMPORTANT]
+> With `PRXXX`, how step templates are handled in this repository changed.
+>
+> Previously, step templates lived under `/step-templates` as `.json` files with packed source scripts embedded in them. That packed format is the format Octopus Deploy needs to consume step templates, but it made PR review difficult. Helper scripts for pack, unpack, and diff were added to assist with review, but Git history and blame still made it hard to understand how a template had changed over time.
+>
+> Starting with `PRXXX`, step templates are stored as source files under `/src/step-templates`. Each step template is now a collection of:
+> - `metadata.json`, which was `git mv`'d from `/step-templates` to retain Git history
+> - one or more unrolled script files such as `scriptbody.ps1`, `scriptbody.sh`, `scriptbody.py`, `predeploy.ps1`, `deploy.ps1`, or `postdeploy.ps1`
+>
+> A build step generates the `/step-templates/*.json` files so they remain available for compatibility with the website and Octopus consumers.
+
+The source-of-truth format for templates lives under `/src/step-templates`:
+
+* `/src/step-templates/<template>/metadata.json`
+* `/src/step-templates/<template>/scriptbody.ps1|sh|py`
+* `/src/step-templates/<template>/predeploy.ps1`
+* `/src/step-templates/<template>/deploy.ps1`
+* `/src/step-templates/<template>/postdeploy.ps1`
+* `/src/step-templates/logos`
+* `/src/step-templates/tests`
+
+`metadata.json` keeps the exported template metadata and uses placeholders for script-backed properties. The real script text lives in sibling files so normal GitHub diffs stay readable.
+
+Creating or updating a template
+-------------------------------
+
+To update an existing template, edit the files in `/src/step-templates/<template>/`.
+
+To add a new template:
+
+1. Create a new folder under `/src/step-templates/<template>/`
+2. Add a `metadata.json` file
+3. Add `scriptbody.ps1|sh|py` and any optional staged script files that the template needs
+4. Add a category logo to `/src/step-templates/logos` if you introduce a new category
+
+Generated files under `/step-templates` are compatibility output for the website and Octopus consumers. They should not be hand-edited or committed.
 
 Contributing step templates or to the website
 ---------------------------------------------
@@ -20,21 +61,7 @@ Reviewing PRs
 
 ### Reviewing script changes
 
-Step template JSON files embed scripts as single-line escaped strings, making diffs hard to read. Use the `_diff.ps1` tool to extract old and new scripts into separate files you can compare in your diff tool:
-
-```powershell
-# Compare ScriptBody against previous commit
-.\tools\_diff.ps1 -SearchPattern "template-name"
-
-# Compare against a specific commit or branch
-.\tools\_diff.ps1 -SearchPattern "template-name" -CompareWith "master"
-```
-
-This outputs readable files to `diff-output/`:
-- `template-name.ScriptBody.old.ps1`
-- `template-name.ScriptBody.new.ps1`
-
-Also handles `PreDeploy`, `Deploy`, and `PostDeploy` custom scripts if present.
+Review the files under `/src/step-templates/<template>/` directly.
 
 ### Checklist
 
@@ -44,6 +71,7 @@ When reviewing a PR, keep the following things in mind:
 * Parameter names should not start with `$`
 * The `DefaultValue`s of `Parameter`s should be either a string or null.
 * `LastModifiedBy` field must be present, and (_optionally_) updated with the correct author
+* Generated JSON under `/step-templates` should not be hand-edited or committed
 * If a new `Category` has been created:
-   * An image with the name `{categoryname}.png` must be present under the `step-templates/logos` folder
+   * An image with the name `{categoryname}.png` must be present under the `src/step-templates/logos` folder
    * The `switch` in the `humanize` function in [`gulpfile.babel.js`](https://github.com/OctopusDeploy/Library/blob/master/gulpfile.babel.js#L92) must have a `case` statement corresponding to it

app/components/TemplateItem.jsx

@@ -114,7 +114,7 @@ export default class TemplateItem extends React.Component {
                 </SyntaxHighlighter>
               </div>
               <p className="align-right">
-                <a className="faint" href={this.state.template.HistoryUrl}>
+                <a className="faint" href={this.state.template.HistoryUrl} rel="noopener noreferrer" target="_blank">
                   History &raquo;
                 </a>
               </p>

app/services/LibraryDb.js

@@ -1,12 +1,15 @@
 "use strict";
 
+import fs from "fs";
+import path from "path";
 import _ from "underscore";
 
-import StepTemplates from "./step-templates.json";
-
 class LibraryDb {
-  constructor() {
-    this._items = _.chain(StepTemplates.items)
+  loadTemplates() {
+    const templatePath = path.join(__dirname, "step-templates.json");
+    const stepTemplates = JSON.parse(fs.readFileSync(templatePath, "utf8"));
+
+    return _.chain(stepTemplates.items)
       .map(function (t) {
         if (t.Properties) {
           var script = t.Properties["Octopus.Action.Script.ScriptBody"];
@@ -41,16 +44,14 @@ class LibraryDb {
         return t.Name.toLowerCase();
       })
       .value();
-
-    this._all = _.indexBy(this._items, "Id");
   }
 
   list(cb) {
-    cb(null, this._items);
+    cb(null, this.loadTemplates());
   }
 
   get(id, cb) {
-    var item = this._all[id];
+    var item = _.indexBy(this.loadTemplates(), "Id")[id];
     cb(null, item);
   }
 }