docs pipeline standardization and minor doc fixes

Author: keighrim

.github/workflows/publish.yml

@@ -48,7 +48,4 @@ jobs:
       source_repo: clamsproject/clams-python
       source_ref: ${{ needs.check-pypi.outputs.version }}
       project_name: clams-python
-      build_command: 'echo "${{ needs.check-pypi.outputs.version }}" > VERSION && python3 build-tools/docs.py --output-dir docs'
-      docs_output_dir: 'docs'
-      python_version: '3.11'
     secrets: inherit

build-tools/docs.py

@@ -76,6 +76,13 @@ def main():
     parser = argparse.ArgumentParser(
         description="Build documentation for the clams-python project."
     )
+    parser.add_argument(
+        '--build-ver',
+        metavar='<version>',
+        default=None,
+        help='Accepted for CLI compatibility with other SDK repos. '
+             'Ignored by this script (clams-python uses '
+             'unversioned documentation).')
     parser.add_argument(
         '--output-dir', type=Path, default=None,
         help='Output directory for built docs (default: docs-test)')

clams/appmetadata/__init__.py

@@ -438,6 +438,18 @@ def add_input(self, at_type: Union[str, vocabulary.ThingTypesBase], required: bo
         return new
 
     def add_input_oneof(self, *inputs: Union[str, Input, vocabulary.ThingTypesBase]):
+        """
+        Helper method to add a ``oneOf`` (disjunctive) group to
+        the ``input`` list. When a single type is given, it is
+        added as a regular (conjunctive) input. When multiple
+        types are given, they are wrapped in a nested list to
+        indicate that exactly one of them is required.
+
+        :param inputs: one or more input types (as URI strings,
+            :class:`Input` objects, or vocabulary types)
+        :raises ValueError: if any input in a ``oneOf`` group is
+            optional, or if a duplicate input is detected
+        """
         newinputs = []
         if len(inputs) == 1:
             if isinstance(inputs[0], Input):
@@ -520,6 +532,13 @@ def add_more(self, key: str, value: str):
             raise ValueError("Key and value should not be empty!")
 
     def jsonify(self, pretty=False):
+        """
+        Serialize the app metadata to a JSON string.
+
+        :param pretty: if True, indent the output with 2 spaces
+        :returns: JSON string of the metadata
+        :rtype: str
+        """
         return self.model_dump_json(exclude_defaults=True, by_alias=True, indent=2 if pretty else None)
 
 

documentation/appmetadata.rst

@@ -13,7 +13,7 @@ Format
 
 A CLAMS App Metadata should be able to be serialized into a JSON string. 
 
-Input/Output type specification
+Input/Output Type Specification
 ===============================
 
 Essentially, all CLAMS apps are designed to take one MMIF file as input and produce another MMIF file as output. In this 
@@ -29,7 +29,7 @@ how that information should be formatted in terms of the App Metadata syntax, co
    additional information about submission. Visit the `CLAMS app directory <https://apps.clams.ai>`_ to see how the app 
    metadata is rendered.
 
-Annotation types in MMIF
+Annotation Types in MMIF
 ------------------------
 
 As described in the `MMIF documentation <https://mmif.clams.ai>`_, MMIF files can contain annotations of various types. 
@@ -56,8 +56,8 @@ needs to add additional information to the type definition, they can do so by ad
 definition in action. In such a case, the app developer is expected to provide the explanation of the extended type in
 the app metadata. See below for the syntax of I/O specification in the app metadata. 
 
-Syntax for I/O specification in App Metadata
---------------------------------------------
+Syntax for I/O Specification in App Metadata
+---------------------------------------------
 
 In the App Metadata, the input and output types are specified as lists of objects. Each object in the list should have
 the following fields:
@@ -69,7 +69,7 @@ the following fields:
   defaults to ``true``. Not applicable for output types.
 
 
-Simple case - using types as defined in the vocabularies
+Simple Case - Using Types as Defined in the Vocabularies
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 In the simplest case, where a developer merely re-uses an annotation type definition and pre-defined properties, an 
@@ -195,14 +195,14 @@ Note that in the actual output MMIF, more properties can be stored in the ``Time
 specification in the app metadata is a subset of the properties to be produced that are useful for type checking
 in the downstream apps, as well as for human readers to understand the output.
 
-Extended case - adding custom properties to the type definition
+Extended Case - Adding Custom Properties to the Type Definition
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 When the type definition is extended on the fly, developers are expected to provide the extended specification in the 
 form of key-value pairs in the ``properties`` field. The grammar of the JSON object does not change, but developers are
 expected to provide a verbose description of the type extension in the ``description`` field. 
 
-Runtime parameter specification
+Runtime Parameter Specification
 ===============================
 
 CLAMS apps designed to be run as HTTP servers, preferably as `stateless <https://en.wikipedia.org/wiki/Stateless_protocol>`_.
@@ -219,7 +219,7 @@ can be specified as ``multivalued=True`` to accept multiple values as a list. Fo
 parameter value parsing works, please refer to the App Metadata json scheme (in the `below <#clams-app-runtime-parameter>`_ 
 section). 
 
-Syntax for parameter specification in App Metadata
+Syntax for Parameter Specification in App Metadata
 --------------------------------------------------
 
 Metadata Schema

documentation/cli.rst

@@ -3,13 +3,12 @@
 ``clams`` shell command
 =======================
 
-``clams-python`` comes with a command line interface (CLI) that allows you to
+``clams-python`` comes with a command line interface (CLI) for creating a new CLAMS app from a template (``develop`` subcommand).
 
-#. create a new CLAMS app from a template
-#. create a new MMIF file with selected source documents and an empty view
+The CLI is installed as the ``clams`` shell command. For backward compatibility, it also exposes all ``mmif`` subcommands (``source``, ``rewind``, ``describe``, ``summarize``). See the `mmif-python CLI documentation <https://clams.ai/mmif-python/cli>`_ for details on those commands.
 
-The CLI is installed as ``clams`` shell command. To see the available commands, run
+To see the available commands, run
 
-.. code-block:: bash 
+.. code-block:: bash
 
     clams --help

documentation/conf.py

@@ -11,11 +11,14 @@
 # documentation root, use os.path.abspath to make it absolute, like shown here.
 
 import datetime
-from pathlib import Path
-import shutil
-import sys
 import inspect
+import json
 import os
+import re
+import shutil
+import subprocess
+import sys
+from pathlib import Path
 
 import mmif
 
@@ -134,40 +137,50 @@ def linkcode_resolve(domain, info):
 
 
 def generate_whatsnew_rst(app):
-    import subprocess
+    """
+    Generate whatsnew.md by fetching the latest release PR body
+    from GitHub via ``gh pr list``.
+
+    Falls back gracefully if ``gh`` is unavailable (local builds).
+    """
     output_path = (proj_root_dir / 'documentation'
                    / 'whatsnew.md')
-    content = None
+    repo = f'clamsproject/{project}'
+
     try:
-        jq_expr = (
-            'sort_by(.mergedAt)'
-            ' | if length > 0 then .[-1].body'
-            ' else "" end'
-        )
         result = subprocess.run(
             ['gh', 'pr', 'list',
-             '-R', f'clamsproject/{project}',
-             '-s', 'merged',
-             '--search',
-             f'releasing {version} in:title',
-             '--json', 'body,mergedAt',
-             '--jq', jq_expr],
-            capture_output=True, text=True, timeout=10
+             '-s', 'merged', '-B', 'main',
+             '-L', '100',
+             '--json', 'title,body',
+             '--repo', repo],
+            capture_output=True, text=True, timeout=15,
         )
-        if result.returncode == 0:
-            content = result.stdout.strip() or None
-    except (FileNotFoundError, subprocess.TimeoutExpired):
-        pass
+        if result.returncode != 0:
+            raise RuntimeError(result.stderr)
+
+        prs = json.loads(result.stdout)
+        pr = next(
+            (p for p in prs
+             if p['title'].startswith('releasing ')),
+            None,
+        )
+        if pr is None:
+            raise RuntimeError("No release PR found")
+        title = pr['title']
+        body = pr.get('body', '')
 
-    with open(output_path, 'w') as f:
-        if content:
-            f.write(
-                f"## What's New in {version}\n\n"
-                f"(Full changelog available in the "
-                f"[CHANGELOG.md]({blob_base_url}"
-                f"/main/CHANGELOG.md))\n"
-            )
-            f.write(content)
+        with open(output_path, 'w') as f:
+            f.write(f"## {title}\n\n")
+            f.write(f"(Full changelog: "
+                    f"[CHANGELOG.md]"
+                    f"({blob_base_url}/main/CHANGELOG.md))\n\n")
+            if body:
+                f.write(body)
+
+    except Exception as e:
+        with open(output_path, 'w') as f:
+            f.write("")
 
 
 def generate_jsonschema(app):
@@ -186,6 +199,9 @@ def update_target_spec(app):
     target_vers_csv = Path(__file__).parent / 'target-versions.csv'
     with open(proj_root_dir / "VERSION", 'r') as version_f:
         version = version_f.read().strip()
+    # Skip dev/dummy versions to avoid dirtying the git-tracked CSV
+    if 'dev' in version or not re.match(r'^\d+\.\d+\.\d+$', version):
+        return
     mmifver = mmif.__version__
     specver = mmif.__specver__
     with open(target_vers_csv) as in_f, open(f'{target_vers_csv}.new', 'w') as out_f:

documentation/index.rst

@@ -1,5 +1,5 @@
-Welcome to CLAMS Python SDK documentation!
-==========================================
+CLAMS Python SDK
+================
 
 .. mdinclude:: ../README.md
 
@@ -28,9 +28,8 @@ Welcome to CLAMS Python SDK documentation!
 
   modules
 
-Indices and tables
+Indices and Tables
 ==================
 
 * :ref:`genindex`
 * :ref:`modindex`
-* :ref:`search`

documentation/introduction.rst

@@ -1,6 +1,6 @@
 .. _introduction: 
 
-Getting started
+Getting Started
 ===============
 
 Overview