feat(ci): port status visibility and consistent component recursion

docs/ci.rst

@@ -194,6 +194,8 @@ Parameter                      Description                                   Env
 
 - ``process_group_id``, ``process_group_name``, ``state``, ``is_root``
 - Processor counts: ``total_processors``, ``running_processors``, ``stopped_processors``, ``invalid_processors``, ``disabled_processors``
+- Input port counts: ``total_input_ports``, ``running_input_ports``, ``stopped_input_ports``, ``invalid_input_ports``
+- Output port counts: ``total_output_ports``, ``running_output_ports``, ``stopped_output_ports``, ``invalid_output_ports``
 - Controller counts: ``total_controllers``, ``enabled_controllers``, ``disabled_controllers``
 - Queue stats: ``queued_flowfiles``, ``queued_bytes``, ``active_threads``
 - Version control: ``versioned``, ``version_id``, ``flow_id``, ``version_state``, ``modified``

docs/devnotes.rst

@@ -215,8 +215,69 @@ Release Process
 
 Streamlined release workflow using our modern build system. Assumes development environment is set up (``make dev-install`` completed).
 
-Pre-release Preparation
-~~~~~~~~~~~~~~~~~~~~~~~
+The key principle is **tag locally, build, verify, then push**. This avoids force pushes if something is wrong with the built distribution.
+
+Patch Release (bug fixes only)
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Use this when only maintained code has changed (no client regeneration, no new NiFi version).
+
+1. **Update Release Notes**:
+
+   Update ``docs/history.rst`` — move the ``Unreleased`` section to a dated version heading.
+
+2. **Commit Release Preparation**:
+
+   .. code-block:: shell
+
+       git add docs/history.rst
+       git commit -S -m "Prepare release X.Y.Z: brief summary"
+
+3. **Tag Locally** (do NOT push yet):
+
+   .. code-block:: shell
+
+       git tag -a -s vX.Y.Z -m "Release X.Y.Z"
+
+4. **Build and Verify**:
+
+   .. code-block:: shell
+
+       # Clean build artifacts (preserves generated clients)
+       make clean && make dist
+
+       # Verify clean version string (no .devN+gHASH suffix)
+       ls dist/
+       # Should show: nipyapi-X.Y.Z-py2.py3-none-any.whl and nipyapi-X.Y.Z.tar.gz
+
+   .. warning::
+
+       Do NOT use ``make clean-all`` for patch releases — it removes generated API clients
+       (``nipyapi/nifi/``, ``nipyapi/registry/``) and would require ``make gen-clients`` to restore them.
+
+5. **Push to GitHub** (triggers CI validation):
+
+   .. code-block:: shell
+
+       git push origin main && git push --tags
+
+6. **Wait for CI to pass** before publishing:
+
+   .. code-block:: shell
+
+       gh run list --limit 3
+       # Or check GitHub Actions UI
+
+7. **Publish to PyPI** (only after CI is green):
+
+   .. code-block:: shell
+
+       twine upload dist/*
+
+Full Release (new features, client regeneration)
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Use this for minor/major releases, especially when generated clients have changed.
 
 1. **Update Release Notes**:
 
@@ -237,39 +298,45 @@ Pre-release Preparation
    .. code-block:: shell
 
        git add docs/history.rst
-       git commit -S -m "Prepare release: update history and documentation"
+       git commit -S -m "Prepare release X.Y.Z: brief summary"
 
-Build and Quality Assurance
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+4. **Tag Locally** (do NOT push yet):
 
-.. code-block:: shell
+   .. code-block:: shell
 
-    # Build fresh distributions for release (rebuild-all already validated them)
-    make clean-all
-    make dist
+       git tag -a -s vX.Y.Z -m "Release X.Y.Z"
 
-Create Release
-~~~~~~~~~~~~~~
+5. **Build and Verify**:
 
-.. code-block:: shell
+   .. code-block:: shell
 
-    # Tag the release (triggers version detection via setuptools-scm)
-    git tag -a -s v1.0.0 -m "Release 1.0.0"
+       # Clean build artifacts and rebuild with the tagged version
+       make clean && make dist
 
-    # Push commit and tags to GitHub (triggers CI validation)
-    git push origin main
-    git push --tags
+       # Verify clean version string (no .devN+gHASH suffix)
+       ls dist/
+       # Should show: nipyapi-X.Y.Z-py2.py3-none-any.whl and nipyapi-X.Y.Z.tar.gz
 
-Publish to PyPI
-~~~~~~~~~~~~~~~
+6. **Push to GitHub** (triggers CI validation):
 
-.. code-block:: shell
+   .. code-block:: shell
+
+       git push origin main && git push --tags
+
+7. **Wait for CI to pass** before publishing:
+
+   .. code-block:: shell
+
+       gh run list --limit 3
+
+8. **Publish to PyPI** (only after CI is green):
+
+   .. code-block:: shell
 
-    # Upload to PyPI (requires PyPI API token configured)
-    twine upload dist/*
+       twine upload dist/*
 
-    # Alternative: Upload to TestPyPI first for validation
-    # twine upload --repository testpypi dist/*
+       # Alternative: Upload to TestPyPI first for validation
+       # twine upload --repository testpypi dist/*
 
 Post-release Verification
 ~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -292,3 +359,4 @@ Version Management Notes
 - **Development Versions**: Commits after tags get ``.devN+gHASH`` suffix automatically
 - **Release Versions**: Clean git tags (e.g., ``v1.0.0``) produce clean versions (``1.0.0``)
 - **Pre-releases**: Use tag patterns like ``v1.0.0rc1`` for release candidates
+- **If the build is wrong**: Delete the local tag (``git tag -d vX.Y.Z``), fix the issue, re-tag, and retry. No force push needed since nothing was pushed yet.

docs/history.rst

@@ -2,6 +2,29 @@
 History
 =======
 
+1.6.0 (2026-06-21)
+-------------------
+
+| Port status visibility and consistent component recursion for CI functions
+
+**Bug Fixes**
+
+- **get_status()**: Processor counts (``running_processors``, ``invalid_processors``, etc.) are now derived from actual processor enumeration rather than the Process Group's aggregate component counts. Previously these counts included all component types (ports, etc.), so invalid output ports were mis-reported as invalid processors.
+- **verify_config()**: Controller service verification now recurses into descendant Process Groups (``descendants=True``), consistent with processor and port verification. Ancestor-inherited services remain excluded. Previously controllers in child PGs were not verified, so a broken controller in a sub-flow could pass CI.
+
+**CI Module**
+
+- **get_status()**: Now reports input and output port counts: ``total_input_ports``, ``running_input_ports``, ``stopped_input_ports``, ``invalid_input_ports`` and the equivalent ``_output_ports`` fields. Invalid ports (e.g. an output port with no outgoing connection) are a real operational problem that was previously invisible to status checks. These are additive fields; existing output is unchanged.
+- **verify_config()**: New ``verify_ports`` parameter (default ``True``) adds input/output port validation. Ports with validation errors now appear in a new ``port_results`` key and cause verification to fail.
+
+**Canvas Module**
+
+- **list_invalid_ports()**: New function, parallel to ``list_invalid_processors``, returning input/output ports with validation errors across a Process Group and its descendants.
+
+**Documentation**
+
+- **Release process**: Reworked the developer release guide (``docs/devnotes.rst``) into separate Patch and Full release workflows, following a "tag locally, build, verify, then push" principle to avoid force pushes when a built distribution is wrong.
+
 1.5.1 (2026-06-02)
 -------------------
 

nipyapi/canvas.py

@@ -273,6 +273,38 @@ def list_invalid_processors(pg_id="root", summary=False):
     return out
 
 
+def list_invalid_ports(pg_id="root", summary=False):
+    """
+    Returns a flattened list of all Ports with Invalid Statuses
+
+    Args:
+        pg_id (str): The UUID of the Process Group to start from, defaults to
+            the Canvas root
+        summary (bool): True to return just the list of relevant
+            properties per Port, False for the full listing
+
+    Returns:
+        list[PortEntity]
+    """
+    assert isinstance(pg_id, str), "pg_id should be a string"
+    assert isinstance(summary, bool)
+    all_ports = list_all_input_ports(pg_id) + list_all_output_ports(pg_id)
+    port_list = [x for x in all_ports if x.component.validation_errors]
+    if summary:
+        out = [
+            {
+                "id": x.id,
+                "name": x.component.name,
+                "type": x.component.type,
+                "summary": x.component.validation_errors,
+            }
+            for x in port_list
+        ]
+    else:
+        out = port_list
+    return out
+
+
 def list_sensitive_processors(pg_id="root", summary=False):
     """
     Returns a flattened list of all Processors on the canvas which have

nipyapi/ci/get_status.py

@@ -26,6 +26,7 @@ def get_status(  # pylint: disable=too-many-locals,too-many-branches,too-many-st
         dict with status information including:
         - process_group_id, process_group_name, state, is_root
         - Processor counts (total, running, stopped, invalid, disabled)
+        - Port counts (total, running, stopped, invalid for input and output)
         - Controller counts (total, enabled, disabled)
         - Version control info (versioned, version_id, version_state, etc.)
         - Parameter context info
@@ -63,27 +64,63 @@ def get_status(  # pylint: disable=too-many-locals,too-many-branches,too-many-st
         "is_root": str(is_root).lower(),
     }
 
-    # Processor counts
-    running = pg.running_count or 0
-    stopped = pg.stopped_count or 0
-    invalid = pg.invalid_count or 0
-    disabled = pg.disabled_count or 0
+    # Processor counts (enumerated for accuracy — pg.*_count includes all component types)
+    processors = nipyapi.canvas.list_all_processors(process_group_id)
 
-    if running > 0:
+    def _proc_status(p):
+        return (p.status.run_status or "").upper() if p.status else ""
+
+    proc_running = sum(1 for p in processors if _proc_status(p) == "RUNNING")
+    proc_stopped = sum(1 for p in processors if _proc_status(p) == "STOPPED")
+    proc_invalid = sum(1 for p in processors if _proc_status(p) == "INVALID")
+    proc_disabled = sum(1 for p in processors if _proc_status(p) == "DISABLED")
+
+    # State reflects processor activity only (processor-centric: ports and
+    # controllers do not influence the PG state field)
+    if proc_running > 0:
         state = "RUNNING"
-    elif stopped > 0:
+    elif proc_stopped > 0:
         state = "STOPPED"
     else:
         state = "EMPTY"
 
     result["state"] = state
-    result["total_processors"] = str(running + stopped + invalid + disabled)
-    result["running_processors"] = str(running)
-    result["stopped_processors"] = str(stopped)
-    result["invalid_processors"] = str(invalid)
-    result["disabled_processors"] = str(disabled)
-
-    log.debug("State: %s (%d running, %d stopped)", state, running, stopped)
+    result["total_processors"] = str(len(processors))
+    result["running_processors"] = str(proc_running)
+    result["stopped_processors"] = str(proc_stopped)
+    result["invalid_processors"] = str(proc_invalid)
+    result["disabled_processors"] = str(proc_disabled)
+
+    log.debug("State: %s (%d running, %d stopped)", state, proc_running, proc_stopped)
+
+    # Port counts (enumerated from all descendant PGs)
+    input_ports = nipyapi.canvas.list_all_input_ports(process_group_id)
+    output_ports = nipyapi.canvas.list_all_output_ports(process_group_id)
+
+    def _port_run_status(p):
+        return (p.status.run_status or "").upper() if p.status else ""
+
+    result["total_input_ports"] = str(len(input_ports))
+    result["running_input_ports"] = str(
+        sum(1 for p in input_ports if _port_run_status(p) == "RUNNING")
+    )
+    result["stopped_input_ports"] = str(
+        sum(1 for p in input_ports if _port_run_status(p) == "STOPPED")
+    )
+    result["invalid_input_ports"] = str(
+        sum(1 for p in input_ports if _port_run_status(p) == "INVALID")
+    )
+
+    result["total_output_ports"] = str(len(output_ports))
+    result["running_output_ports"] = str(
+        sum(1 for p in output_ports if _port_run_status(p) == "RUNNING")
+    )
+    result["stopped_output_ports"] = str(
+        sum(1 for p in output_ports if _port_run_status(p) == "STOPPED")
+    )
+    result["invalid_output_ports"] = str(
+        sum(1 for p in output_ports if _port_run_status(p) == "INVALID")
+    )
 
     # Queue stats, active threads, and throughput
     if pg.status and pg.status.aggregate_snapshot: