feat(mem-wal): ShardWriter::abort and Session::invalidate_dataset for drop-table

[hamersaw]

What

Two small mem-wal primitives that sophon's WAL drop-table teardown needs:

• ShardWriter::abort(&self) — shut down the background flush tasks (task_executor.shutdown_all()) without flushing, discarding buffered memtable state. Unlike close(self) it takes &self (so it can be called through the Arc<ShardWriter> callers hold) and performs no object-store IO. The caller must quiesce writes first (documented). Idempotent.
• Session::invalidate_dataset(uri) — evict a dataset's metadata- and index-cache entries via the existing LanceCache::invalidate_prefix primitive, scoped with a trailing slash ("{uri}/") so a sibling whose URI shares the prefix (t.lance vs t.lance2) is left untouched.

Why

When a WAL-enabled table is dropped, the owning pod must tear down its in-memory ShardWriter before the dataset directory is removed:

• abort keeps the background flush task from re-creating _mem_wal/ files under a just-deleted directory (a graceful close() would flush them back).
• invalidate_dataset lets a same-name/same-URI recreate cold-read fresh state instead of serving the dropped incarnation's cached manifests/indices from the shared Session.

Both build on machinery that already exists (shutdown_all, invalidate_prefix, per-dataset URI cache prefixing); this is a thin exposure, not new infrastructure.

Tests

• test_abort_discards_without_flushing_and_is_idempotent — abort leaves no new L0 generation (contrast with close), idempotent on a second call.
• test_invalidate_dataset_clears_only_that_uri — clears the target dataset's metadata + index entries while leaving a prefix-sharing sibling intact (guards the trailing-slash scoping).

🤖 Generated with Claude Code

[github-actions]

ACTION NEEDED
Lance follows the Conventional Commits specification for release automation.

The PR title and description are used as the merge commit message. Please update your PR title and description to match the specification.

For details on the error please inspect the "PR Title Check" action.