Merge pull request #82 from VEZY/attributes-in-table -> Columnar Attribute Backend + Traversal Integration

### Summary

This PR migrates MTG attributes to a **typed, per-symbol columnar backend** and updates traversal/query paths to leverage typed data while preserving current semantics (`nothing` in traversal outputs for absent attributes, `ignore_nothing` behavior, etc.).

---

### 1. New attribute backend (table-like, per symbol)

Implemented core types:

- `Column{T}`
- `SymbolBucket`
- `MTGAttributeStore`
- `NodeAttrRef`
- `ColumnarAttrs`
- `ColumnarStore` (marker type)

Design:

- Attributes are stored in typed columns (`Vector{T}`) grouped by symbol (e.g. `:Leaf`, `:Internode`).
- Node-to-row and row-to-node mappings provide O(1) access from node to its data row.
- This improves type stability and memory locality compared to per-node `Dict{...,Any}`.

---

### 2. Default backend switched to columnar

- `read_mtg` now always uses `ColumnarAttrs`.
- Parsed MTG attributes are converted into the typed columnar store by default.
- Manual node construction still accepts `Dict`/`NamedTuple` input, which is auto-converted to `ColumnarAttrs`.

---

### 3. New explicit attribute API

Added/standardized:

- `attribute(node, key; default=nothing)`
- `attribute!(node, key, value)`
- `attributes(node; format=:namedtuple | :dict)`
- `attribute_names(node)`

Schema operations (per symbol):

- `add_column!(mtg, symbol, key, T; default=...)`
- `drop_column!(mtg, symbol, key)`
- `rename_column!(mtg, symbol, from, to)`

Note: schema changes are **symbol-bucket scoped** (apply to all nodes of that symbol).

---

### 4. Tables.jl integration

Added tabular sources:

- `symbol_table(mtg, symbol)` (per-symbol view)
- `mtg_table(mtg)` (unified view in traversal order)

Behavior:

- Unified view uses `missing` for attributes absent on some symbols.
- Works directly with `DataFrame(...)`.

---

### 5. Descendants/ancestors: typed inference + optimized paths

- Return eltypes are now inferred from typed columns.
- `type=` kw for `descendants`/`ancestors` is deprecated.
- `ignore_nothing` semantics preserved.

Implemented:

- Multi-key descendants (`descendants(node, [:k1, :k2], ...)`) with correct grouped row output and key order.
- In-place and allocating variants parity.
- Fast columnar lookup path (`unsafe_getindex` with query plan).

---

### 6. Hybrid descendants strategy

Added strategy control:

- `descendants_strategy(mtg)`
- `descendants_strategy!(mtg, :pointer | :indexed | :auto)`

Backend behavior:

- `:pointer`: direct tree traversal (as before)
- `:indexed`: DFS interval index (`tin/tout/dfs_order`) for fast repeated subtree queries
- `:auto`: switches based on query/mutation dynamics (the default)

Structural mutations mark the index as dirty and trigger a rebuild when needed.

---

### 7. Structural mutation integration

Updated mutation paths to keep store/index consistent:

- insert binds newly created nodes into the columnar store
- delete/prune removes nodes from store rows
- node structure changes mark subtree index dirty

This keeps traversal and attribute access coherent after growth operations.

---

### 8. Benchmark suite expansion (ASV)

Benchmarks now cover:

- traversal updates (node indexing vs explicit attribute API)
- descendants/ancestors one-key and multi-key
- mixed-symbol queries with `ignore_nothing=true/false`
- repeated small queries (`children`, `parent`, `ancestors`, `descendants`)
- mutation API surface (`insert/delete/prune/select/transform/write`)
- table conversion (`symbol_table`, `mtg_table`)

Also made benchmark harness cross-version robust for Symbol/String filters/keys.

---

### 9. Tests/docs updates

Tests added/extended for:

- columnar backend correctness
- schema operations add/drop/rename
- descendants multi-key correctness and parity
- traversal strategy behavior
- Tables.jl views and DataFrame conversion

Docs updated to explain:

- new attribute API
- columnar backend behavior
- strategy modes (`:pointer`, `:indexed`, `:auto`)
- performance guidance for growth-heavy vs query-heavy workloads

---

### Important notes for reviewers

- The large speedups in some descendants workloads are real for typed/multi-attr columnar retrieval.
- Mutation-heavy and some short-query workloads are still areas of active optimization.
- Public traversal semantics were preserved (notably `nothing` handling in mixed-symbol traversals).

Author: VEZY

AGENT.md

@@ -21,6 +21,7 @@
 - `src/compute_MTG/indexing.jl`
 - `src/compute_MTG/check_filters.jl`
 - `src/types/Node.jl`
+- `src/types/Attributes.jl`
 - `src/compute_MTG/node_funs.jl`
 
 ## Practical Optimization Rules
@@ -30,4 +31,14 @@
   - `descendants!(buffer, node, key; ...)`
 - Keep filter checks branch-light when no filters are provided.
 - Keep key access on typed attribute containers (`NamedTuple`, `MutableNamedTuple`, typed dicts) in specialized methods when possible.
+- Prefer explicit attribute APIs:
+  - `attribute(node, key)`
+  - `attribute!(node, key, value)`
+  - `attributes(node; format=:namedtuple|:dict)`
+  - `add_column!` / `drop_column!` / `rename_column!`
 - Preserve API behavior and add tests for every optimization that changes internals.
+
+## Benchmark Tiers
+- `small` (~10k nodes): full matrix including API-surface benchmarks
+- `medium` (~100k nodes): hot-path matrix
+- `large` (~300k nodes): critical hot paths only

Project.toml

@@ -14,6 +14,7 @@ MutableNamedTuples = "af6c499f-54b4-48cc-bbd2-094bba7533c7"
 OrderedCollections = "bac558e1-5e72-5ebc-8fee-abe8a469f55d"
 Printf = "de0858da-6303-5e67-8744-51eddeeeb8d7"
 SHA = "ea8e919c-243c-51af-8825-aaa63cd721ce"
+Tables = "bd369af6-aec1-5ad0-b16a-f7cc5008161c"
 XLSX = "fdbf4ff8-1666-58a4-91e7-1b58723a45e0"
 
 [compat]
@@ -25,6 +26,7 @@ MetaGraphsNext = "0.5, 0.6, 0.7"
 MutableNamedTuples = "0.1"
 OrderedCollections = "1.4"
 SHA = "0.7, 1"
+Tables = "1"
 XLSX = "0.7, 0.8, 0.9, 0.10"
 julia = "1.6"
 

benchmark/Project.toml

@@ -1,3 +1,7 @@
 [deps]
 BenchmarkTools = "6e4b80f9-dd63-53aa-95a3-0cdb28fa8baf"
 MultiScaleTreeGraph = "dd4a991b-8a45-4075-bede-262ee62d5583"
+Tables = "bd369af6-aec1-5ad0-b16a-f7cc5008161c"
+
+[sources]
+MultiScaleTreeGraph = {path = ".."}

benchmark/README.md

@@ -10,6 +10,21 @@ julia --project=benchmark benchmark/benchmarks.jl
 
 Workloads currently covered:
 
-- full-tree traversal
-- traversal + data extraction from descendants
-- repeated many-small queries (`children`, `parent`, `ancestors`)
+- tiered datasets: `small` (~10k nodes), `medium` (~100k), `large` (~300k)
+- full-tree traversal and traversal updates
+  - one/multi-attribute updates using node indexing (`node[:k]`)
+  - one/multi-attribute updates using explicit API (`attribute`/`attribute!`)
+  - one symbol (`:Leaf`) and mixed symbols (`[:Leaf, :Internode]`)
+- descendants queries
+  - one/many attributes, one symbol
+  - one/many attributes, mixed symbols
+  - `ignore_nothing=true/false`
+  - in-place and allocating variants
+- repeated many-small queries
+  - `children`, `parent`, `ancestors`, `ancestors!`
+  - repeated descendants retrieval with and without in-place buffers
+- API surface (small tier)
+  - insertion/deletion/pruning
+  - `transform!`, `select!`
+  - `symbol_table` / `mtg_table`
+  - `write_mtg`