Merge pull request #6 from JMU-CS/gui-and-tidy

Gui and tidy

Author: spragunr

README.md

@@ -11,41 +11,142 @@ Features include:
     * provides a `@required` annotation that makes a test required, so
       that subsequent tests will automatically fail if it fails.
     * provide several additional assertion methods, including an
-      `assertScriptOutputEqual` method that makes it possible to specify
+      `assertOutputEqual` method that makes it possible to specify
       stdin input and check for correct stdout output for a Python
       script.
 *    Some additional utility functions for checking PEP 8 compliance
      and applying regex searches to submitted source files.
 *    A script that makes it possible to test the autograder logic
      locally before uploading to the Gradescope server:
      `test_autograder.py`
+*    A graphical tool for testing autograders and building autograder zip
+     files.
 
-Documentation can be found here: <https://jmu-cs.github.io/jmu_python_gradescope_utils/>
+API Documentation can be found here: <https://jmu-cs.github.io/jmu_python_gradescope_utils/>
 
 ## Installation
 
 ```
+pip3 install wheel
 pip3 install git+https://github.com/JMU-CS/jmu_python_gradescope_utils.git
 ```
 
-## Instructions
+On Linux, this will probably add the GUI script to your PATH so that it can be executed as:
 
-See `examples/hello_world_autograder/` for a sample autograder.
+```
+$ jmu_gradescope_builder.py
+```
+
+If that doesn't work, you may need to execute the script using the Python interpreter:
+
+```
+python3 wherever/on/your/system/bin/jmu_gradescope_builder.py
+```
+
+Note that the GUI uses tk, which should be available in Windows, Linux
+and OSX. Unfortunately, the tk library distributed with some versions
+of OSX is broken, resulting in a blank black window. There are some
+reports that upgrading Python to 3.10 resolves the issue.
+
+## Autograder Format
+
+See `examples/hello_world/` and `hello_world_w_coverage` for a sample autograders.
+
+Autograder folders are organized as follows:
 
-To perform a test run: 
+```
+hello_world/
+│
+├── config.ini
+│
+├── configurations/
+│   ├── flake8.cfg
+│   ├── docstring.cfg
+│   └── requirements.txt
+│
+├── sample/
+│   └── hello_world.py
+│
+└── scaffolding/
+│
+└── tests/
+    └── test_hello.py
+```
+
+
+### `config.ini`
+
+This specifies the files that must be submitted by the student:
 
-```bash
-$ cd /your/autograder
-$ test_autograder.py /path/to/sample/submission/folder
 ```
+[SUBMIT]
+code: hello_world.py
+tests:
+```
+
+The `test` field should contain the names of student test files for assignments that require student-submitted unit tests.
+
+### `flake8.cfg`
+
+This is the `flake8` configuration file that will be used by
+[assertPassesPep8](https://jmu-cs.github.io/jmu_python_gradescope_utils/jmu_test_case.html#jmu_gradescope_utils.jmu_test_case._JmuTestCase.assertPassesPep8).
+
+The [flake8](https://flake8.pycqa.org/en/latest/index.html) tool is
+used by our library for Python style checking.  The possible error
+codes are listed in the following locations:
 
-The output of the test will be stored in a file named
-`test_results.json`.
+* [pycodestyle error codes](https://pycodestyle.pycqa.org/en/latest/intro.html#error-codes)
+* [flake8 error codes](https://flake8.pycqa.org/en/latest/user/error-codes.html)
 
-Once everything is working correctly, just zip the contents of the
-autograder folder and upload to Gradescope:
 
-```bash
-$ cd /your/autograder
-$ zip -r autograder.zip .
+The current default looks like this:
 ```
+[flake8]
+select = E, F, C90
+ignore = W, E226, E123, W504, N818, E704, E24, E121, E126
+max-line-length = 100
+```
+
+### `docstring.cfg`
+This is the `flake8` configuration file that will be used by [assertDocstringsCorrect](https://jmu-cs.github.io/jmu_python_gradescope_utils/jmu_test_case.html#jmu_gradescope_utils.jmu_test_case._JmuTestCase.assertDocstringsCorrect).  The current default looks like this:
+
+```
+[flake8]
+select = D,DAR
+ignore = D401
+
+[pydocstyle]
+convention=google
+
+[darglint]
+docstring_style=google
+```
+
+The available error codes for the two plugins can be found here:
+
+* `pydocstyle`: <http://www.pydocstyle.org/en/stable/error_codes.html>
+* `darglint`: <https://github.com/terrencepreilly/darglint#error-codes>
+
+
+### `requirements.txt`
+
+This file should list any python packages that are required for this
+assignment. They will be pip installed on the server by the autograder
+script. 
+
+### sample/
+
+This folder should contain a reference solution.  This isn't strictly
+necessary, but the autograder testing utility will expect to find a
+solution at this location.
+
+### scaffolding/
+
+Any files in this folder will be copied into the same folder as the
+student submission before testing.  This could be data files or Python
+modules.
+
+### tests/
+
+Instructor unit tests.
+

doc/jmu_test_case.rst

@@ -14,13 +14,3 @@ JmuTestCase
    :members:
    :private-members:
    :undoc-members:
-
-.. automodule:: jmu_gradescope_utils.utils
-   :members:
-   :private-members:
-   :undoc-members:
-
-.. automodule:: jmu_gradescope_utils.coverage_utils
-   :members:
-   :private-members:
-   :undoc-members:

examples/hello_world/config.ini

@@ -0,0 +1,6 @@
+[SUBMIT]
+# Student submitted code files, comma separated
+code: hello_world.py
+
+# Student submitted test files, comma separated
+tests:

examples/hello_world/configurations/docstring.cfg

@@ -0,0 +1,10 @@
+[flake8]
+select = D,DAR
+ignore = D401
+
+[pydocstyle]
+convention=google
+
+[darglint]
+docstring_style=google
+

examples/hello_world/configurations/flake8.cfg

@@ -0,0 +1,4 @@
+[flake8]
+select = E, F, C90
+ignore = W, E226, E123, W504, N818, E704, E24, E121, E126
+max-line-length = 100

…hello_world_autograder/tests/__init__.py …lo_world/configurations/requirements.txtexamples/hello_world_autograder/tests/__init__.py renamed to examples/hello_world/configurations/requirements.txt examples/hello_world_autograder/tests/__init__.py renamed to examples/hello_world/configurations/requirements.txt

@@ -0,0 +1 @@
+print("Hello World!")

examples/hello_world/sample/hello_world.py

@@ -1,31 +1,28 @@
 import unittest
 from gradescope_utils.autograder_utils.decorators import weight, number
-from jmu_gradescope_utils import JmuTestCase, required, check_submitted_files
+from jmu_gradescope_utils import JmuTestCase, required
 import jmu_gradescope_utils
 
+FILENAME = 'hello_world.py'
 
 class TestHelloWorld(JmuTestCase):
 
-    FILENAME = 'hello_world.py'
-
     @required()
     @weight(0)
     def test_submitted_files(self):
         """Check submitted files"""
-        self.assertRequiredFilesPresent([self.FILENAME])
+        self.assertRequiredFilesPresent([FILENAME])
 
-    
     @weight(2)
     def test_pep8(self):
         """PEP 8 checks. """
-        self.assertPassesPep8(self.FILENAME)
-        
+        self.assertPassesPep8(FILENAME)
+
     @weight(8)
-    def test_output(self):
-        """hello_world.py output checks. """
+    def test_functionality(self):
+        """Test for correct output."""
+
         string_in = ""
         expected = "Hello World!\n"
-        self.assertOutputCorrect(self.FILENAME, string_in, expected)
-
-
-
+        self.assertOutputEqual('hello_world.py', string_in, expected)
+        print('Correct output:\n' + expected)