Include pipeline settings in the NWB epochs table

[Sam-NT]

This PR adds a feature to store pipeline settings into the NWB epochs table. If the pipeline_settings setting in the NWBSinkSettings class is set to an ez.Settings instance, it will create a flattened dictionary of the configurable settings in the instance and automatically add them to the epochs table each time a new entry is added.

I thought this would be the most convenient way to store the settings, since get_epoch()[setting] (pseudocode) from the NWB file seems like a pretty easy interface. Definitely open to feedback on that though, as the settings can be added to the NWB file in a number of ways.

Quick pipeline to test with an Orion source:

import logging

import ezmsg.core as ez
from ezmsg.nwb.writer import NWBSink, NWBSinkSettings
from intent.orion.orion_source import OrionClientInfo, OrionSource, OrionSourceSettings

from intent.pipelines.component.spike_rate_bandpower import (
    NeuralFeaturesFromRaw,
    NeuralFeaturesFromRawSettings,
)

logging.basicConfig(
    level=logging.INFO, format="%(asctime)s [%(levelname)s] %(name)s: %(message)s"
)


class NwbTestSettings(ez.Settings):
    """
    Settings to test NWB file storage.
    """

    file_path: str = "/Users/nasonsr1/projects/intent/scratch/"
    """Where to store NWB files"""

    orion_host: str = "127.0.0.1"
    """Orion's hostname or IP address"""

    orion_port: str = 5111
    """Orion's port"""


class NwbTest(ez.Collection):
    """
    Collection for testing NWB file storage.
    """

    SETTINGS = NwbTestSettings

    ORION = OrionSource()
    FEATS = NeuralFeaturesFromRaw()
    NWB = NWBSink()

    def configure(self) -> None:
        self.ORION.apply_settings(
            OrionSourceSettings(
                client_info=OrionClientInfo(
                    orion_host=self.SETTINGS.orion_host,
                    orion_port=self.SETTINGS.orion_port,
                )
            )
        )
        self.FEATS.apply_settings(NeuralFeaturesFromRawSettings())
        self.NWB.apply_settings(
            NWBSinkSettings(filepath=self.SETTINGS.file_path, pipeline_settings=self.SETTINGS)
        )

    def network(self) -> ez.NetworkDefinition:
        return (
            (self.ORION.OUTPUT_SIGNAL, self.FEATS.INPUT_SIGNAL),
            (self.FEATS.OUTPUT_SIGNAL_SPK, self.NWB.INPUT_SIGNAL),
            (self.FEATS.OUTPUT_SIGNAL_SBP, self.NWB.INPUT_SIGNAL),
        )


def main(
    graph_ip: str | None = None,
    graph_port: int | None = None,
):
    if graph_ip is None and graph_port is None:
        graph_address = None
    else:
        graph_address = (graph_ip or "127.0.0.1", graph_port or 25978)
    ez.run(
        components={"SYSTEM": NwbTest(settings=NwbTestSettings())},
        graph_address=graph_address,
    )


if __name__ == "__main__":
    import typer

    typer.run(main)

[Sam-NT]

Recent commits refactor NWB settings logging to use upcoming ezmsg built-in features to query graph settings (see ezmsg core PR #234). This expands support for settings logging to include all settings included in the running pipeline. This is accomplished by the NWBSink unit creating its own GraphContext, subscribing to settings events, and asynchronously sending events to the NWBSinkConsumer for storage. Additionally, settings are now stored in a separate pynwb.epoch.TimeIntervals table called "pipeline_settings" (instead of the epochs table) for ease of access. Lastly, this PR includes tests for this updated functionality.

Note that settings logging is attempted by default. If it fails, it reports a warning that settings logging cannot proceed, then continues with standard NWB file storage (without logging settings).

Remaining to-dos prior to marking this PR as ready for review:

• Merge ezmsg core PR #234
• Update pyproject.toml to specify the new minimum ezmsg core version