|
| 1 | +--- |
| 2 | +id: developer-guidelines |
| 3 | +title: General Developer Notes |
| 4 | +sidebar_label: General Developer Notes |
| 5 | +--- |
| 6 | + |
| 7 | +Welcome to the ScriptManager developer wiki! |
| 8 | + |
| 9 | +This guide is designed to generally orient developers that plan to contribute to ScriptManager and establish some project-specific standards. There are some helpful checklists at the end for current developers. |
| 10 | + |
| 11 | +Quick Links: |
| 12 | +- [Picard Documentation][picard-javadocs] |
| 13 | +- [Picocli Documentation][picocli] |
| 14 | +- [SDK Man Documentation][sdkman-docs] |
| 15 | +- [Release Roadmap][release-roadmap] |
| 16 | +- [Gradle Documentation][gradle] |
| 17 | +- [Eclipse IDE][eclipse-ide] |
| 18 | +- [Lambda functions][lambda-tutorial] - we want to be using these more |
| 19 | + |
| 20 | +## ScriptManager Design Principles |
| 21 | + |
| 22 | +There are several principles to keep in mind during development, especially for new developers. |
| 23 | +- A single built ScripManager JAR file should run consistently across any OS with Java installed. |
| 24 | +- Every tool should have a graphical interface and every tool not based on an existing command line tool should have a command line interface. |
| 25 | +- The code for each tool should be isolated into a script object, a window object, and a command line object and organized into their appropriate packages in the `src/` library and use the local `util` and `objects` packages with [Picocoli][picocli], [HTSJDK][htsjdk], and [JFree][jfree] packages as appropriate. |
| 26 | + |
| 27 | +While there are plenty of developer tools available for Java developers, this page is provides some recommendations based on Olivia's setup as a starting point for new developers. |
| 28 | + |
| 29 | + |
| 30 | +## Java Development |
| 31 | +We write exclusively in Java or Java-compiled languages without any operating-system specific packages (to maintain portability across machines). |
| 32 | + |
| 33 | +### SDK Man |
| 34 | +Olivia recommends installing Java using [SDKMan][sdkman] for convenient flipping between Java versions. While ScriptManager is currently developed to the Java 8 SE standard, it is good practice to check for forward and backward compatibility between Java versions. We are constantly monitoring new Java releases and developing according to a standard that is consistent across Java versions makes our lives easier down the road when the Java version standard is incremented. |
| 35 | + |
| 36 | +### Integrated Development Environment (IDE) - Eclipse |
| 37 | +We recommend using [Eclipse][eclipse] to write Java code for ScriptManager because it supports both [Gradle][gradle] (see below) and [WindowBuilder][window-builder] for convenient building of JAR files and graphical interface development. |
| 38 | + |
| 39 | +### Gradle-based build |
| 40 | +Compiling Java classes and building JAR files could be handled manually, but for this project, we let Gradle juggle the process of compiling, retrieving dependencies, and building the final JAR file. The dependencies we use are a mix of downloaded JAR files (`scriptmanager/lib/*.jar` and dependencies retrieved directly from [Maven][maven]). |
| 41 | + |
| 42 | + |
| 43 | +## The Code Structure (Packages) |
| 44 | +``` |
| 45 | +scriptmanager/src/ |
| 46 | + |--charts |
| 47 | + |--cli |
| 48 | + |--main |
| 49 | + |--objects |
| 50 | + |--scripts |
| 51 | + |--util |
| 52 | + |--window_interface |
| 53 | +``` |
| 54 | + |
| 55 | +### window_interface |
| 56 | +There exists at least one window object for every tool. They are organized by tool groups (`scripts` and `cli` are similarly organized). The main `<ToolName>Window.java` object extends the JFrame class and manages the inputs from the user. Many tools also have a `<ToolName>Output.java` object which pops up a new window when the tool executes to show the user updates on the progress of the tool, stats on the output, or, for the tools with visual outputs, to display the generated charts/images. |
| 57 | + |
| 58 | +### main |
| 59 | +The main package includes the main class that parses subcommands to call the various CLI classes or initialize the main GUI (JFrame) class. |
| 60 | + |
| 61 | +### cli |
| 62 | +The structure of these classes is generally very simple. See [__Picocli documentation__][picocli] and check existing CLI class examples for more info. |
| 63 | + |
| 64 | +### scripts |
| 65 | +It is important that these classes do not extend JFrame. Since they can used by cli classes, we need to make sure there are no screen-rendering elements that would throw errors if run from the command line. This set of classes isolate out the job of taking inputs and performing the computations. |
| 66 | + |
| 67 | +### charts |
| 68 | +These classes are for making chart objects that visually display the data. They can be displayed on the monitor or saved as image files by the various tool objects. |
| 69 | + |
| 70 | +### objects |
| 71 | +The most used class is the `ToolDescriptions.java` which statically stores all the tool descriptions as Strings for easy updates and changes to the descriptions. It also stores the ScriptManager version so make sure that this is incremented appropriately. |
| 72 | + |
| 73 | +### util |
| 74 | +These include classes with generic methods that are used across multiple tools. |
| 75 | + |
| 76 | + |
| 77 | +## New Tool Checklist |
| 78 | + |
| 79 | +New tools should be written on branches. A pull request to the master branch can then be submitted and a reviewer will review the code and accept the merge. |
| 80 | + |
| 81 | +* Create __new issue ticket__ to associate commits with |
| 82 | + * [ ] spec out the tool input/output/parameters |
| 83 | + * [ ] decide on a tool group to add it to |
| 84 | +* Write __tests__ for Github Actions (automatic testing) |
| 85 | + * [ ] Write data with small storage footprint |
| 86 | + * [ ] Capture a variety of edge cases (different parameter combinations, adding extra input files as needed)) |
| 87 | + * [ ] Write tests into shell script for Github Actions |
| 88 | +* `objects.ToolDescriptions.java` |
| 89 | + * [ ] Add tool description String (used by main window and CLI help docs) |
| 90 | +* `window_interface.MyToolWindow.java` |
| 91 | + * [ ] Extends JFrame (see Java Swing documentation) |
| 92 | + * [ ] For particularly complex tool inputs, it may help to mock-up the window in Adobe Illustrator |
| 93 | +* `window_interface.MyToolOutput.java` (Optional) |
| 94 | + * [ ] Some tools do not have an output frame but rather pop up a simple `JDialog` window indicating the operation has completed. |
| 95 | + * [ ] Tools that have bigger outputs, esp figures/images/chargs, should create an output frame |
| 96 | + * [ ] Extends JFrame |
| 97 | +* `scripts.MyTool.java` |
| 98 | + * [ ] Make sure that the script object can be called and executed in a headless way (unit tests and CLI run) |
| 99 | + * [ ] Every tool should return a command line string for logging purposes. |
| 100 | +* `cli.MyToolCLI.java` |
| 101 | + * [ ] __Skip if re-implementing existing command line tool for the GUI__ |
| 102 | + * [ ] Use [Picocli][picocli] library to parse command line options |
| 103 | + * [ ] Create script object and call as appropriate |
| 104 | + * [ ] Return appropriate exit code |
| 105 | + * [ ] Import tool description from `ToolDescriptions` and add to appropriate help documentation fields |
| 106 | + * [ ] Test parameter constraints |
| 107 | +* `main.ScriptManagerGUI.java` |
| 108 | + * [ ] Add collapsible panel to appropriate tool group in tool three |
| 109 | + * [ ] Import title, description, and other appropriate tool information |
| 110 | +* `main.ScriptManager.java` |
| 111 | + * [ ] Create subcommand call for CLI (extend local abstract classs) |
| 112 | +* Update Docusaurus (documentation) |
| 113 | + * [ ] add screenshots and descriptions of input |
| 114 | + * [ ] use warnings and note boxes as appropriate |
| 115 | + * [ ] add to tool index, tool-group, and file-formats pages |
| 116 | + * [ ] make sure page renders appropriately |
| 117 | +* Write __Galaxy wrapper__ |
| 118 | +* Pull your changes into master! 🎉 |
| 119 | + |
| 120 | +:::tip |
| 121 | +The easiest way to write a new tool is to copy-paste the code from a similarly-structured tool and edit! |
| 122 | +::: |
| 123 | + |
| 124 | + |
| 125 | +## Version Incrementing Checklist |
| 126 | + |
| 127 | +The [Release Roadmap][release-roadmap] on Github organizes issue tickets and creates a projection of which issues should be addressed for each release. This helps when writing up the release notes and tagging all the appropriate issues as well as visually tracks what tasks are left to do in each release. When we are ready for a release, the following checklist should be followed to ensure that we update everything together without missing anything. |
| 128 | + |
| 129 | + |
| 130 | +* Check [Release Roadmap][release-roadmap] |
| 131 | + * [ ] Make sure all issues are closed and pulled into master |
| 132 | + * [ ] Remove/archive column so next version is first to display |
| 133 | +* Docusaurus updates |
| 134 | + * [ ] Make sure new tools have their own pages that thoroughly describe what they do |
| 135 | + * [ ] Affected tools have been updated accordingly (check commit log for list of tools) |
| 136 | + * [ ] Make sure `last updated` timestamps are appropriate/correct |
| 137 | + * [ ] Increment version across docs |
| 138 | +* Testing |
| 139 | + * [ ] Ideally some degree of user testing on the development version has been performed (ask the bench scientists). |
| 140 | + * [ ] Make sure latest Github Actions build ran successfully |
| 141 | +* `build.gradle` |
| 142 | + * [ ] Increment version (`version = ___`) and strip `dev` from JAR filename |
| 143 | +* `src/objects/ToolDescriptions.java` |
| 144 | + * [ ] Increment ScriptManager version constant (used by CLI tools, propogation will happen automatically) |
| 145 | +* Github version tag |
| 146 | + * [ ] Commit & pull request, review into master |
| 147 | + * [ ] Create version release & add version tag to the commit id |
| 148 | + * [ ] Compile JAR and save with source tar archive on release page |
| 149 | + * [ ] Write up summary for the version tag commit including links to resolved/relevant issue tickets |
| 150 | +* Switch naming back to `dev` |
| 151 | + * [ ] `build.gradle` file should switch naming JAR to use `dev` |
| 152 | + |
| 153 | + |
| 154 | + |
| 155 | + |
| 156 | + |
| 157 | + |
| 158 | + |
| 159 | + |
| 160 | +[eclipse]:https://www.eclipse.org/ide/ |
| 161 | +[eclipse-ide]:https://www.eclipse.org/eclipseide/ |
| 162 | +[gradle]:https://docs.gradle.org/current/userguide/userguide.html |
| 163 | +[htsjdk]:https://github.com/samtools/htsjdk |
| 164 | +[jfree]:https://github.com/jfree/jfreechart |
| 165 | +[maven]:https://maven.apache.org/ |
| 166 | +[picard-javadocs]:https://broadinstitute.github.io/picard/javadoc/picard/index.html |
| 167 | +[picocli]:https://picocli.info/ |
| 168 | +[sdkman]:https://sdkman.io/install |
| 169 | +[sdkman-docs]:https://sdkman.io/ |
| 170 | +[window-builder]:https://www.eclipse.org/windowbuilder/ |
| 171 | + |
| 172 | + |
| 173 | +[release-roadmap]:https://github.com/CEGRcode/scriptmanager/projects/6 |
| 174 | +[lambda-tutorial]:https://docs.oracle.com/javase/tutorial/java/javaOO/lambdaexpressions.html |
0 commit comments