Merge pull request #362 from carpentries-incubator/develop

Develop

Author: anenadic

README.md

@@ -69,7 +69,7 @@ This indicates that the maintainers will welcome pull requests fixing such issue
 
 Current maintainers of this lesson (in alphabetical order) are:
 
-* [Matthew Bluteau][matthew-bluteau] - Lead Maintainer for the period 1 May 2024 - 30 April 2024
+* [Matthew Bluteau][matthew-bluteau] - Lead Maintainer for the period 1 May 2024 - 31 October 2024
 * [Steve Crouch][steve-crouch]
 * [Kamilla Kopec-Harding][kamilla-kopec-harding]
 * [Doug Lowe][doug-lowe]

_episodes/23-continuous-integration-automated-testing.md

@@ -360,29 +360,40 @@ to refer to the values from the matrix.
 So, our `.github/workflows/main.yml` should look like the following:
 
 ~~~
-...
+# Same key-value pairs as in "Defining Our Workflow" section
+name: CI
+on: push
+jobs:
+  build:
+
+    # Here we add the matrices definition:
     strategy:
       matrix:
-        os: [ubuntu-latest, macos-latest, windows-latest]
+        os: ["ubuntu-latest", "macos-latest", "windows-latest"]
         python-version: ["3.10", "3.11"]
 
+    # Here we add the reference to the os matrix values
     runs-on: {% raw %}${{ matrix.os }}{% endraw %}
 
-...
-
-    # a job is a seq of steps
+    # Same key-value pairs as in "Defining Our Workflow" section
     steps:
 
-    # Next we need to checkout out repository, and set up Python
-    # A 'name' is just an optional label shown in the log - helpful to clarify progress - and can be anything
     - name: Checkout repository
       uses: actions/checkout@v4
 
     - name: Set up Python
       uses: actions/setup-python@v4
       with:
+        # Here we add the reference to the python-version matrix values
         python-version: {% raw %}${{ matrix.python-version }}{% endraw %}
-...
+    # Same steps as in "Defining Our Workflow" section
+    - name: Install Python dependencies
+      run: |
+        python3 -m pip install --upgrade pip
+        python3 -m pip install -r requirements.txt
+    - name: Test with PyTest
+      run: |
+        python3 -m pytest --cov=catchment.models tests/test_models.py
 ~~~
 {: .language-yaml}
 
@@ -410,4 +421,25 @@ which potentially saves us a lot of time waiting for testing results.
 Overall, this approach allows us to massively scale our automated testing
 across platforms we wish to test.
 
+> ## Failed CI Builds
+> A CI build can fail when, e.g. a used Python package no longer supports a particular version of 
+> Python indicated in a GitHub Actions CI build matrix. In this case, the solution is either to 
+> upgrade the Python version in the build matrix (when possible) or downgrade the package version (and not use the latest one like we have been doing in this course).
+>
+> Also note that, if one job fails in the build for any reason - all subsequent jobs will get cancelled because of the default behavior of
+> GitHub Actions. From [GitHub's documentation](https://docs.github.com/en/actions/using-jobs/using-a-matrix-for-your-jobs#handling-failures):
+>
+>   >*"GitHub will cancel all in-progress and queued jobs in the matrix if any job in the matrix fails."*
+>
+> This behaviour can be controlled by changing the value of the `fail-fast` property:
+> ~~~
+> ...
+>    strategy:
+>      fail-fast: false
+>      matrix:
+> ...
+> ~~~
+{: .language-yaml}
+{: .callout}
+
 {% include links.md %}

_episodes/24-diagnosing-issues-improving-robustness.md

@@ -120,37 +120,49 @@ to check that the normalisation function is correct for some test data.
     ])
 def test_patient_normalise(test, expected):
     """Test normalisation works for arrays of one and positive integers.
-       Assumption that test accuracy of two decimal places is sufficient."""
+       Test with a relative and absolute tolerance of 0.01."""
     from inflammation.models import patient_normalise
-    npt.assert_almost_equal(patient_normalise(np.array(test)), np.array(expected), decimal=2)
+    result = patient_normalise(np.array(test))
+    npt.assert_allclose(result, np.array(expected), rtol=1e-2, atol=1e-2)
 ~~~
 {: .language-python}
 
-Note that we are using the `assert_almost_equal()` Numpy testing function
+Note that we are using the `assert_allclose()` Numpy testing function
 instead of `assert_array_equal()`,
-since it allows us to test against values that are *almost* equal.
+since it allows us to test against values that are **close** to each other.
 This is very useful when we have numbers with arbitrary decimal places
 and are only concerned with a certain degree of precision,
-like the test case above,
-where we make the assumption that a test accuracy of two decimal places is sufficient.
+like the test case above.
+
+> ## Relative and absolute tolerance
+> **Relative tolerance** in unit testing means that the acceptable difference between the expected and actual results 
+> depends on the size of the expected result itself. So, if your expected result is 100, 
+> a relative tolerance of 0.1 (or 10%) means the actual result can be anywhere from 90 to 110 and still be considered correct.
+> 
+> **Absolute tolerance**, on the other hand, 
+> sets a fixed allowable difference regardless of the magnitude of the expected result. 
+> For example, if you set an absolute tolerance of 5, 
+> it means the actual result can be within 5 units of the expected result, 
+> regardless of whether the expected result is 10 or 1000.
+{: .callout}
 
 Run the tests again using `python -m pytest tests/test_models.py`
 and you will note that the new test is failing,
 with an error message that does not give many clues as to what went wrong.
 
 ~~~
-E       AssertionError:
-E       Arrays are not almost equal to 2 decimals
+E           AssertionError:
+E           Not equal to tolerance rtol=0.01, atol=0.01
 E
-E       Mismatched elements: 6 / 9 (66.7%)
-E       Max absolute difference: 0.57142857
-E       Max relative difference: 1.345
-E        x: array([[0.14, 0.29, 0.43],
-E              [0.5 , 0.62, 0.75],
-E              [0.78, 0.89, 1.  ]])
-E        y: array([[0.33, 0.67, 1.  ],
-E              [0.67, 0.83, 1.  ],
-E              [0.78, 0.89, 1.  ]])
+E           Mismatched elements: 6 / 9 (66.7%)
+E           Max absolute difference: 0.57142857
+E           Max relative difference: 0.57356077
+E            x: array([[0.142857, 0.285714, 0.428571],
+E                  [0.5     , 0.625   , 0.75    ],
+E                  [0.777778, 0.888889, 1.      ]])
+E            y: array([[0.33, 0.67, 1.  ],
+E                  [0.67, 0.83, 1.  ],
+E                  [0.78, 0.89, 1.  ]])
 
 tests/test_models.py:53: AssertionError
 ~~~
@@ -407,7 +419,7 @@ due to the division by zero as we predicted.
 
 ~~~
 E           AssertionError:
-E           Arrays are not almost equal to 2 decimals
+E           Not equal to tolerance rtol=0.01, atol=0.01
 E
 E           x and y nan location mismatch:
 E            x: array([[nan, nan, nan],
@@ -489,7 +501,8 @@ def patient_normalise(data):
 > > def test_patient_normalise(test, expected):
 > >     """Test normalisation works for arrays of one and positive integers."""
 > >     from inflammation.models import patient_normalise
-> >     npt.assert_almost_equal(patient_normalise(np.array(test)), np.array(expected), decimal=2)
+> >     result = patient_normalise(np.array(test))
+> >     npt.assert_allclose(result, np.array(expected), rtol=1e-2, atol=1e-2)
 > > ...
 > > ~~~
 > > {: .language-python}
@@ -569,11 +582,14 @@ but of an inappropriate value.
 def test_patient_normalise(test, expected, expect_raises):
     """Test normalisation works for arrays of one and positive integers."""
     from inflammation.models import patient_normalise
+        
     if expect_raises is not None:
         with pytest.raises(expect_raises):
-            npt.assert_almost_equal(patient_normalise(np.array(test)), np.array(expected), decimal=2)
+            result = patient_normalise(np.array(test))
+            npt.assert_allclose(result, np.array(expected), rtol=1e-2, atol=1e-2)
     else:
-        npt.assert_almost_equal(patient_normalise(np.array(test)), np.array(expected), decimal=2)
+        result = patient_normalise(np.array(test))
+        npt.assert_allclose(result, np.array(expected), rtol=1e-2, atol=1e-2)
 ~~~
 {: .language-python}
 
@@ -651,14 +667,17 @@ Be sure to commit your changes so far and push them to GitHub.
 > >         test = np.array(test)
 > >     if expect_raises is not None:
 > >         with pytest.raises(expect_raises):
-> >             npt.assert_almost_equal(patient_normalise(test), np.array(expected), decimal=2)
+> >           result = patient_normalise(test)
+> >           npt.assert_allclose(result, np.array(expected), rtol=1e-2, atol=1e-2)
+> >
 > >     else:
-> >         npt.assert_almost_equal(patient_normalise(test), np.array(expected), decimal=2)
+> >         result = patient_normalise(test)
+> >         npt.assert_allclose(result, np.array(expected), rtol=1e-2, atol=1e-2)
 > > ...
 > > ~~~
 > >
 > > Note the conversion from `list` to `np.array` has been moved
-> > out of the call to `npt.assert_almost_equal()` within the test function,
+> > out of the call to `npt.assert_allclose()` within the test function,
 > > and is now only applied to list items (rather than all items).
 > > This allows for greater flexibility with our test inputs,
 > > since this wouldn't work in the test case that uses a string.

_episodes/43-software-release.md

@@ -190,7 +190,7 @@ allowing us to distinguish between runtime and development dependencies.
 
 ~~~
 $ poetry add matplotlib numpy
-$ poetry add --dev pylint
+$ poetry add --group dev pylint
 $ poetry install
 ~~~
 {: .language-bash}

_episodes/51-managing-software.md

@@ -47,9 +47,11 @@ GitHub provides **Issues** -
 a framework for managing bug reports, feature requests, and lists of future work.
 
 Go back to the home page for your `python-intermediate-inflammation` repository in GitHub,
-and click on the **Issue** tab.
+and click on the `Issues` tab. 
 You should see a page listing the open issues on your repository -
-currently there should be none.
+currently there should be none. 
+If you do not see the `Issues` tab, you must first enable it in the settings of your repository: 
+go to the `Settings` tab, scroll down to the `Features` section and activate the checkmark on `Issues`.
 
 ![List of project issues in GitHub](../fig/github-issue-list.png){: .image-with-shadow width="1000px"}
 
@@ -269,7 +271,7 @@ representation of tasks and may not be as suitable for higher-level project mana
 prioritising tasks for future development, planning sprints and releases. Luckily,
 GitHub provides two project management tools for this purpose - **Projects** and **Milestones**.
 
-Both Projects and Milestones provide [agile development and project management systems](https://www.atlassian.com/agile)
+Both GitHub Projects and Milestones provide [agile development and project management systems](https://www.atlassian.com/agile)
 and ways of organising issues into smaller "sub-projects" (i.e.
 smaller than the "project" represented by the whole repository).
 Projects provide a way of visualising and organising work which is not time-bound and is on a higher level (e.g. more suitable for
@@ -287,7 +289,7 @@ for now, we will have a brief look at Projects.
 
 ### Projects
 
-A Project uses a "project board" consisting of columns and cards to keep track of tasks
+A GitHub Project uses a "project board" consisting of columns and cards to keep track of tasks
 (although GitHub now also provides a table view over a project's tasks).
 You break down your project into smaller sub-projects,
 which in turn are split into tasks which you write on cards,
@@ -299,7 +301,14 @@ such as issues and pull requests -
 cards can be added to track the progress of such tasks
 and automatically moved between columns based on their progress or status.
 
-> ## Project are a Cross-Repository Management Tool
+GitHub projects are "an adaptable, flexible tool for planning and tracking work on GitHub" -
+they now provide interchangeable spreadsheet, task-board, or roadmap views of your project 
+that integrates with your issues and pull requests on GitHub to help you 
+plan and track your work effectively.
+We recommend you to have a look at [GitHub’s documentation on Projects](https://docs.github.com/en/issues/planning-and-tracking-with-projects/learning-about-projects/about-projects) 
+and see if they are suitable for your software development workflow.
+
+> ## GitHub Projects are a Cross-Repository Management Tool
 > [Project in GitHub](https://docs.github.com/en/issues/planning-and-tracking-with-projects/learning-about-projects/about-projects)
 > are created on a user or organisation level,
 > i.e. they can span all repositories owned by a user or organisation in GitHub
@@ -308,32 +317,32 @@ and automatically moved between columns based on their progress or status.
 > to help you plan and track your team's work effectively.
 {: .callout}
 
-Let's create a Project in GitHub to plan the first release of our code.
+Let's have a quick look at how Projects are created in GitHub - we will not use them much in 
+this course but it is good to be aware of how to make use of them when suitable.
 
 1. From your GitHub account's home page (not your repository's home page!),
    select the "Projects" tab, then click the `New project` button on the right.
 
-    ![Adding a new project board in GitHub](../fig/github-new-project.png){: .image-with-shadow width="1000px"}
-
-2. In the "Select a template" pop-up window, select "Board" -
-   this will give you a classic "cards on a board" view of the project.
-   An alternative is the "Table" view,
-   which presents a spreadsheet-like and slightly more condensed view of a project.
-
-    ![Selecting a project board template in GitHub](../fig/github-board-template.png){: .image-with-shadow width="600px"}
-
-3. GitHub will create an unnamed project board for you.
-   You should populate the name and the description of the project from the project's Settings,
-   which can be found by clicking the `...` button in the top right corner of the board.
-
-   ![Project board setting in GitHub](../fig/github-project-settings.png){: .image-with-shadow width="800px"}
-
-4. We can, for example, use "Inflammation project - release v0.1"
-   and "Tasks for the v0.1 release of the inflammation project"
-   for the name and description of our project, respectively.
-   Or you can use anything that suits your project.
-
-   ![Naming a project in GitHub](../fig/github-name-project.png){: .image-with-shadow width="800px"}
+    ![Adding a new project board in GitHub](../fig/github-new-project.png){: .image-with-shadow width="800px"}
+
+2. In the "Create project" pop-up window, you can either start from one of the featured existing 
+project templates or create your project from scratch using one of the three standard project 
+types/views that you customise yourself: 
+      - Table - a spreadsheet-style table to filter, sort and group your issues and pull requests. 
+      - Board - a "cards on a board" view of the project, with issues and pull requests being 
+      spread across customizable columns as cards on kanban board
+      - Roadmap - suitable for a high-level visualisation of your project over time. 
+![Selecting a project board template in GitHub](../fig/github-project-template.png){: .image-with-shadow width="800px"}
+  Regardless of which project type/view you select, you can easily switch to a different 
+  project layout later on.
+
+3. For example, select the "Board" type for the project, fill in the name of your project
+(e.g. "Inflammation project - release v0.1"), and select `Create project`.
+4. After it is created, you should also populate the description of the project from the project's Settings,
+   which can be found by clicking the `...` button in the top right corner of the project.
+![Project board setting in GitHub](../fig/github-project-settings.png){: .image-with-shadow width="800px"}
+![Adding project description and metadata in GitHub](../fig/github-project-description.png){: .image-with-shadow width="800px"}
+   After adding a description, select `Save`.
 
 5. GitHub's default card board template contains
    the following three columns with pretty self-explanatory names:
@@ -349,6 +358,8 @@ Let's create a Project in GitHub to plan the first release of our code.
     if you have tasks that get held up by waiting on other people
     (e.g. to respond to your questions)
     then moving them to a separate column makes their current state clearer.
+    Another way to organise your table is to have a column for each quarter of the year - 
+    it is up to you to decide how you want to view your project's activities. 
 
     To add a new column,
     press the `+` button on the right;
@@ -375,7 +386,7 @@ Let's create a Project in GitHub to plan the first release of our code.
     or write more detailed comments
     (for that, use the `Convert to issue` option from the `...` menu on the card itself).
 
-   ![Coverting a task to issue](../fig/github-convert-task-to-issue.png){: .image-with-shadow width="800px"}
+   ![Converting a task to issue](../fig/github-convert-task-to-issue.png){: .image-with-shadow width="800px"}
 
 7.  In addition to creating new tasks as notes and converting them to issues -
 you can add an existing issue or pull request (from any repository visible to you)
@@ -386,6 +397,10 @@ and pressing the `Enter` key.
    Issues and pull requests on cards will automatically be moved to the `Done` column for you
    when you close the issue or merge the pull request -
    which is very convenient and can save you some project management time.
+9. Finally, you can change the way you view your project by adding another view. 
+For example, we can add a Table view to our Board view by clicking the `New button`
+and selecting it from the drop down menu.
+![Add another project view](../fig/github-project-add-view.png){: .image-with-shadow width="800px"}
 
 > ## Exercise: Working With Projects
 > Spend a few minutes planning what you want to do with your project as a bigger chunk of work