feat: abstract controller storage layer and add postgre backend#79
Open
zdravko61 wants to merge 18 commits intoTHCLab:masterfrom
Open
feat: abstract controller storage layer and add postgre backend#79zdravko61 wants to merge 18 commits intoTHCLab:masterfrom
zdravko61 wants to merge 18 commits intoTHCLab:masterfrom
Conversation
…tecture Gate redb behind `storage-redb` feature flag (default ON) in keri-core and teliox so the core protocol logic can compile and run without redb. This enables future alternative storage backends (e.g., DynamoDB for serverless). Key changes: - Split EventStorage constructors: generic `new()` (no mailbox) vs `new_redb()` (RedbDatabase with mailbox) vs `new_with_mailbox()` (inject) - Make mailbox_data an Option<MailboxData> to support non-redb backends - Remove Any bound from EventValidator - Gate TelLogDatabase, teliox EscrowDatabase, and escrow module behind storage-redb feature - Genericize teliox escrow structs over K: EventDatabase for KEL storage - Add in-memory MemoryDatabase implementing all database traits for validation and testing - Move rkyv_adapter to database::rkyv_adapter (not under database::redb) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Introduce `storage-postgres` feature and dependencies - Implement PostgresDatabase, log, escrow, and KSN log modules - Add SQL migrations for all KERI tables - Integrate with existing database traits and event flows Signed-off-by: Zdravko Iliev <zdravko.iliev@vereign.com>
Signed-off-by: Zdravko Iliev <zdravko.iliev@vereign.com>
Signed-off-by: Zdravko Iliev <zdravko.iliev@vereign.com>
Signed-off-by: Zdravko Iliev <zdravko.iliev@vereign.com>
storage Signed-off-by: Zdravko Iliev <zdravko.iliev@vereign.com>
Signed-off-by: Zdravko Iliev <zdravko.iliev@vereign.com>
- Replace dual cfg-gated OobiManager struct definitions with a single generic struct and per-backend type aliases (RedbOobiManager, PostgresOobiManager) - Make process_reply and process_signed_oobi generic over OobiStorageBackend - Propagate CBOR deserialization errors instead of silently dropping corrupted rows - Replace unwrap() on serde_json/serde_cbor serialization with ? via new Cbor/Json variants on PostgresError Signed-off-by: Zdravko Iliev <zdravko.iliev@vereign.com>
- Split EndRoleAdd and EndRoleCut into separate arms (was incorrectly
merged)
- Add TODO comment on EndRoleCut matching redb's pending delete
semantics
- Replace silent _ => {} catch-all with explicit Ksn => todo!()
- Add Cbor and Json error variants to PostgresError and propagate
serialization/deserialization errors instead of panicking or silently
dropping failures
Signed-off-by: Zdravko Iliev <zdravko.iliev@vereign.com>
Extract TelEventDatabase and TelEscrowDatabase traits to decouple teliox from redb. Make escrow structs and default_escrow_bus generic over the new traits. Implement PostgresTelDatabase and PostgresTelEscrowDatabase backed by sqlx/PgPool, sharing the schema and migrations from keriox_core. Add postgres integration tests. Signed-off-by: Zdravko Iliev <zdravko.iliev@vereign.com>
- Refactor get_events_from_index to return Result, propagating SQL and CBOR deserialization errors instead of silently returning empty results - Replace unwrap() in escrow error branches with map_err(...)? in missing_issuer and missing_registry processors Signed-off-by: Zdravko Iliev <zdravko.iliev@vereign.com>
Identifier generic over storage backends Introduces three type parameters <D, T, S> across the controller crate, where D: EventDatabase + EscrowCreator, T: TelEventDatabase, and S: OobiStorageBackend. Adds RedbController and RedbKnownEvents type aliases behind the storage-redb feature flag (default). Replaces the hardcoded RedbError variant in ControllerError with a generic DatabaseError(String). Signed-off-by: Zdravko Iliev <zdravko.iliev@vereign.com>
Adds PostgresController and PostgresKnownEvents type aliases behind the storage-postgres feature flag, with PostgresController::new_postgres and PostgresKnownEvents::with_postgres async constructors. Adds postgres integration tests matching the full redb test suite (KEL, TEL, group inception, delegated inception). Fixes a bug in PostgresSnKeyDatabase where get_greater_than used sn > $3 instead of sn >= $3, causing sn=0 events (inception) to be excluded from the partially witnessed escrow. Signed-off-by: Zdravko Iliev <zdravko.iliev@vereign.com>
- Export new type aliases from controller and lib modules - Update tests and helpers to use RedbIdentifier - Ensure test databases are unique per test binary to avoid conflicts Signed-off-by: Zdravko Iliev <zdravko.iliev@vereign.com>
Signed-off-by: Zdravko Iliev <zdravko.iliev@vereign.com>
Signed-off-by: Zdravko Iliev <zdravko.iliev@vereign.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Completes the storage abstraction across the full KERI stack for the
controller crate, following the same pattern established in keriox_core
and teliox to allow any other storage implementation.
Changes
Abstraction
KnownEvents,Communication,Controller, andIdentifierare nowgeneric over
<D: EventDatabase + EscrowCreator, T: TelEventDatabase, S: OobiStorageBackend>RedbControllerandRedbKnownEventstype aliases added behindstorage-redbfeature (default, no breaking change)ControllerError::RedbErrorreplaced withDatabaseError(String)PostgreSQL backend
PostgresControllerandPostgresKnownEventstype aliases behindstorage-postgresfeaturePostgresController::new_postgres(url, config)async constructorTEL management, group inception, delegated inception