Skip to content

Add Lance namespace read/write support#35

Draft
FANNG1 wants to merge 1 commit into
daft-engine:mainfrom
FANNG1:namespace
Draft

Add Lance namespace read/write support#35
FANNG1 wants to merge 1 commit into
daft-engine:mainfrom
FANNG1:namespace

Conversation

@FANNG1

@FANNG1 FANNG1 commented Jun 19, 2026

Copy link
Copy Markdown

Summary

  • add a shared Lance namespace resolver for URI-or-namespace validation, namespace client construction, table resolution, and storage option merging
  • wire namespace table support through daft.read_lance, DataFrame.write_lance, LanceDataSink, and worker scan reconstruction
  • add dir namespace coverage plus a gated Gravitino Lance REST namespace integration test

Validation

  • make check-format
  • .venv/bin/python -m py_compile daft_lance/namespace.py daft_lance/_lance.py daft_lance/lance_data_sink.py daft_lance/lance_scan.py daft_lance/utils.py tests/io/lancedb/test_namespace_rest_e2e.py
  • .venv/bin/python -m pytest tests/io/lancedb/test_namespace.py tests/io/lancedb/test_namespace_rest_e2e.py -v
  • DAFT_LANCE_REST_URI=http://127.0.0.1:9101/lance NO_PROXY='' no_proxy='' AWS_ACCESS_KEY_ID=minioadmin AWS_SECRET_ACCESS_KEY=minioadmin AWS_ENDPOINT_URL=http://127.0.0.1:9000 AWS_REGION=us-east-1 AWS_DEFAULT_REGION=us-east-1 AWS_ALLOW_HTTP=true .venv/bin/python -m pytest tests/io/lancedb/test_namespace_rest_e2e.py -v

@FANNG1 FANNG1 marked this pull request as draft June 19, 2026 12:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant