Plan next steps for making candidate selection stage plugable

[faridseifi]

Here are potential next steps:
☑ Update dependency management for SCAFFOLD based on uv instead of poetry

Clean up esteemer base class

☑ Move loading environment variables from esteemer base class to pipeline startup
☑ Move loading of preferences and history from esteemer base class to MPM candidate selector
☑ Base class should be dictating the select method
☑ Move MPM_prioritization package back to SCAFFOLD
☑ Rename MPM_prioritization to MPM_candidate_selector

Make the esteemer plugin more complete and robust

☑ Update esteemer to load scaffold.esteemer plugins and choose one (configured one) and use its select method. IT IS IMPORTANT TO ONLY USE THE SPECIFIED PLUGIN RATHER THAN A DEFAULT. Allow an override via environment variable.
☑ load the configuration for selection plugin from knowledge base with the version which will be check by plugin loader. Either using package-lib and/or add a version() method to the base class
☑ Make mpm_candidate_selector a python plugin for scaffold.esteemr and set it as the default plugin for esteemer, update tests as needed.
☑ Move back and update MPM_candidate_selector tests from external package
☑ Clean up prioritization algorithms added to the knowledge base

☑ Create and test a random candidate selector plugin for scaffold.esteemer plugins

☑ Add JSON inputs to the MPM_candidate_selector to load preferences and history
☑ Cleaning up preformer_graph used instead of subject_graph
☑ Pass context to the init instead of passing which parts to pass

☐ Build SCAFFOLD model (core) and include things like namespaces, esteemer base class, may be signal detectors.

☐ Create a manual for creating and using a new selection algorithm.

[pboisver]

Yes, you can absolutely do this in a single repository. In fact, this is exactly what uv workspaces were designed for. It gives you the "Monorepo" experience: you have one folder and one git repo, but you maintain two distinct, versionable Python packages.

1. The Workspace Layout

With uv, your repository structure will look like this:

scaffold-workspace/
├── pyproject.toml         # Workspace configuration
├── uv.lock                # Shared lockfile for consistency
└── packages/
    ├── scaffold-core/     # The lightweight library
    │   ├── pyproject.toml
    │   └── src/
    │       └── scaffold_core/
    │           ├── __init__.py
    │           ├── models.py
    │           └── base.py
    └── scaffold/          # The heavy CLI application
        ├── pyproject.toml
        └── src/
            └── scaffold/
                ├── __init__.py
                ├── main.py
                └── generator.py

2. Configuring the Workspace (scaffold-workspace/pyproject.toml)

This root file tells uv that it should manage these two packages together.

[tool.uv.workspace]
members = ["packages/*"]

[project]
name = "scaffold-workspace"
version = "0.1.0"
requires-python = ">=3.12"

3. Linking the Packages

In the scaffold (the app) pyproject.toml, you define a dependency on scaffold-core using a relative path. This ensures that when you develop the app, it uses your local core code.

# packages/scaffold/pyproject.toml
[project]
name = "scaffold"
version = "0.1.0"
dependencies = [
    "scaffold-core",  # Named dependency
    "polars",
    "typer",
]

[tool.uv.sources]
scaffold-core = { path = "../scaffold-core", editable = true }

[project.scripts]
scaffold = "scaffold.main:app"

---

4. Building the Executable

Since you want an executable for the App, you can use PyInstaller or Nuitka. uv makes this easy because it handles the environment for you.

To build a standalone binary for the CLI:

1. Add PyInstaller to the workspace: uv add --dev pyinstaller
2. Run the build:

   uv run pyinstaller --onefile --name scaffold packages/scaffold/src/scaffold/main.py

This will bundle the app logic, the core library logic, and all dependencies (Polars, Typer) into a single file.

---

5. Packaging the Core Library

When you are ready to let plugin developers use the core, you simply build and publish just that sub-package.

From the root:

# Build the core library into a distributable .whl and .tar.gz
uv build --package scaffold-core

This generates a small, clean wheel file in a dist/ folder. A plugin developer can then install it with pip install scaffold-core. They will not get Polars, Typer, or any of your heavy CLI logic—only the models and interfaces.

Pros of this Setup

• Physical Separation: You cannot accidentally import a "heavy" app module into the core library because they are in different packages.
• Atomic Updates: You can update the CLI (App) 50 times without ever needing to release a new version of the Core, keeping things stable for plugin authors.
• Developer Experience: Running uv sync at the root sets up a single virtual environment where both packages are available and editable.

A Wit's Warning

Just remember: while the workspace makes it feel like one project, they are two different packages. If you change a method signature in scaffold-core, don't forget to update the corresponding calls in scaffold before you build that executable, or you'll have a very fast, very broken binary!

Does this workspace structure feel like it provides the right amount of "guardrails" for your team?

[pboisver]

Yes, moving the loading logic into the base class is a clean architectural move. It centralizes the "plugin discovery" logic and makes the base class responsible for its own implementations.

In this setup, CandidateSelectorBase acts as both the Interface and the Factory.

1. The Base Class (base.py)

We use importlib.metadata inside a @classmethod. Note that we keep the imports inside the method or at the top of the file; since it's the standard library, it won't add overhead.

import logging
from abc import ABC, abstractmethod
from importlib.metadata import entry_points
from typing import Self
from .models import Candidate

logger = logging.getLogger(__name__)

class CandidateSelectorBase(ABC):
    
    @abstractmethod
    def select(self, candidates: list[Candidate]) -> list[Candidate]:
        """Method to be implemented by all plugins."""
        pass

    @classmethod
    def load_active(cls) -> Self:
        """
        Factory method to discover and return the active selector plugin.
        """
        group_name = "my_app.selectors"
        eps = entry_points(group=group_name)
        
        names = eps.names
        non_default_names = [n for n in names if n != "default"]

        # Rule 1: Only one entry exists total
        if len(names) == 1:
            selected_name = names[0]
            logger.info(f"Using only available selector: {selected_name}")
            return eps[selected_name].load()()

        # Rule 3: Multiple non-default entries
        if len(non_default_names) > 1:
            error_msg = (
                "More than one non-default candidate selector found. "
                f"Check your configuration. Found: {', '.join(non_default_names)}"
            )
            raise RuntimeError(error_msg)

        # Rule 2: Exactly one non-default entry exists (ignore default)
        if len(non_default_names) == 1:
            selected_name = non_default_names[0]
            logger.info(f"Ignoring 'default' selector. Using: {selected_name}")
            return eps[selected_name].load()()

        # Fallback: Default
        if "default" in names:
            logger.info("Using default selector.")
            return eps["default"].load()()

        raise RuntimeError(f"No candidate selectors registered in {group_name}")

2. Implementation (plugins/custom.py)

Your plugins just need to inherit from the base class.

from my_app.base import CandidateSelectorBase

class CustomSelector(CandidateSelectorBase):
    invocation_order = 10 # Accessible property as requested
    
    def select(self, candidates):
        # Implementation logic...
        return candidates[:3]

3. Usage in main.py

The CLI code becomes much more descriptive. You are asking the Base class to provide the active implementation.

import typer
from .base import CandidateSelectorBase
from .generator import CandidateGenerator

app = typer.Typer()

@app.command()
def run(file_path: str):
    try:
        # The Base class acts as the factory
        selector = CandidateSelectorBase.load_active()
        
        # Standard flow
        df = pl.read_csv(file_path)
        candidates = CandidateGenerator().generate(df)
        results = selector.select(candidates)

        for c in results:
            typer.echo(f"Candidate: {c.id}")

    except RuntimeError as e:
        typer.secho(str(e), fg="red", err=True)
        raise typer.Exit(1)

---

Why this is a great pattern:

• Encapsulation: The knowledge of how a selector is found (entry points, naming rules, etc.) is hidden inside the CandidateSelectorBase class.
• Type Hinting: By using -> Self (available in Python 3.11+) or -> "CandidateSelectorBase", IDEs will correctly provide autocomplete for the select() method on the returned object.
• Readability: CandidateSelectorBase.load_active() reads almost like a sentence, making the intent of the code clear to other developers.

One small detail: ensure that CandidateSelectorBase is imported from a module that doesn't cause a circular import (e.g., don't import a plugin inside base.py since the plugin already imports base.py). Using importlib inside the method prevents this, as the import happens at runtime only when called.