Skip to content

Updated DynamicTable configuration to support row-based recording#305

Open
oruebel wants to merge 12 commits into
add_event_tablefrom
rowbased_table_acquisition
Open

Updated DynamicTable configuration to support row-based recording#305
oruebel wants to merge 12 commits into
add_event_tablefrom
rowbased_table_acquisition

Conversation

@oruebel

@oruebel oruebel commented Jul 5, 2026

Copy link
Copy Markdown
Contributor

Fix #303

Problem: The current implementation of DynamicTable is mainly designed for the purpose of metadata tables, e.g., ElectrodeTable, where we can usually fill in the whole table before the actual acquisition. With the new EventTable, however, we need enhanced support for acquiring the data in the table as part of the acquisition workflow.

Goal: Update DynamicTable and EventTable to simplify row-based recording and configuration of the table

Changes:

  • Support configuration of the columns of a DynamicTable directly via initialize, rather than just incremental viaaddColumn
    • Added Data::DataSpecBase and Data::DataSpec<T> to support runtime configuration of DynamicTable column layouts and to decouple table schema definition from object initialization. Each Data subclass (e.g., VectorData, ElementIdentifiers, TimestampVectorData, DurationVectorData) now exposes a nested DataSpec struct that bundles the dataset configuration, description, and any type-specific parameters needed to create and initialize the column.
    • Added DynamicTable::createDefaultDataSpecs static factory to return the default ordered list of DataSpec objects for a table (currently just the id column). Subclasses (ElectrodesTable, EventsTable, MeaningsTable) override this to append their own required columns, allowing callers to retrieve, inspect, and customize the default column configuration before passing it to initialize().
    • Updated DynamicTable::initialize, ElectrodesTable::initialize, EventsTable::initialize, and MeaningsTable::initialize to accept a std::vector<DataSpecPtr> column-spec list instead of individual rowChunkSize / resolution / flag parameters.
    • Added DynamicTable::addColumn(DataSpecPtr) overload consistent with the new DataSpec-based API used by initialize() and createDefaultDataSpecs().
  • Support row-based write of DynamicTable across all columns of the table
    • Added DynamicTable::addRow and DynamicTable::addRows methods to support row-based data insertion into a DynamicTable. Rows are specified as DynamicTable::RowData (an unordered_map from column name to CellValue variant), enabling type-safe, column-keyed writes without requiring callers to manage per-column buffers directly. Row IDs are auto-generated if not provided.
    • Added BaseDataType::BaseDataVariant scalar variant type and BaseDataType::createEmptyVectorVariant helper to support runtime-typed single-cell and buffer operations needed by addRow/addRows. (@cline, @oruebel, #305)
  • Update Documentation
    • Added tutorial to demonstrate acquisition workflow for DynamicTable. The tutorial uses EventsTable as an example and shows: 1) row-based appending of data and 2)column-based write and addition of full columns to a table, and 3) adding of a MeaningsTable to a columns
    • Updated the CHANGELOG
  • TODO:
    • Appending rows for reference columns is not yet working. Currently reference columns are only used in ElectrodesTable which is setup before recording, so append is not necessary there. For main data acquisition this not yet critial, however, to clean up handling and to fully support reference columns will require: 1) support for creation of chunked reference columns (much like regular array datasets) , and 2) support in the DynamicTable column configuration mechanism to allow configuration of reference columns directly. This should be handled separately in a follow-up issue.

@oruebel oruebel added this to the 0.4.0 milestone Jul 5, 2026
@oruebel oruebel self-assigned this Jul 5, 2026
@oruebel oruebel added the category: enhancement proposed enhancements or new features label Jul 5, 2026
@oruebel oruebel marked this pull request as ready for review July 5, 2026 22:15
@oruebel oruebel requested a review from rly July 5, 2026 22:15
@oruebel oruebel linked an issue Jul 5, 2026 that may be closed by this pull request
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

category: enhancement proposed enhancements or new features

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support row-wise recording of DynamicTable

1 participant