Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 36 additions & 0 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
name: Tests

on:
push:
branches: ["**"]
pull_request:
branches: ["**"]

jobs:
test-plain:
name: "SQLite (no encryption)"
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.11"
- run: pip install -e ".[dev]" || pip install -e .
- run: pip install pytest
- run: pytest tests/test_sqlitedb.py -q -p no:ovoscope

test-cipher:
name: "SQLite + SQLCipher"
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.11"
- name: Install sqlcipher system library
run: |
sudo apt-get update
sudo apt-get install -y libsqlcipher-dev
- run: pip install -e ".[cipher]"
- run: pip install pytest
- run: pytest tests/test_sqlitedb.py -q -p no:ovoscope
51 changes: 3 additions & 48 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,57 +1,12 @@
# Changelog

## [0.2.1a4](https://github.com/JarbasHiveMind/hivemind-sqlite-database/tree/0.2.1a4) (2026-04-15)
## [0.3.0a1](https://github.com/JarbasHiveMind/hivemind-sqlite-database/tree/0.3.0a1) (2026-04-15)

[Full Changelog](https://github.com/JarbasHiveMind/hivemind-sqlite-database/compare/0.2.1a3...0.2.1a4)
[Full Changelog](https://github.com/JarbasHiveMind/hivemind-sqlite-database/compare/0.2.1...0.3.0a1)

**Merged pull requests:**

- ci: sync publish workflows from shared templates [\#22](https://github.com/JarbasHiveMind/hivemind-sqlite-database/pull/22) ([JarbasAl](https://github.com/JarbasAl))

## [0.2.1a3](https://github.com/JarbasHiveMind/hivemind-sqlite-database/tree/0.2.1a3) (2026-04-15)

[Full Changelog](https://github.com/JarbasHiveMind/hivemind-sqlite-database/compare/0.2.1a2...0.2.1a3)

## [0.2.1a2](https://github.com/JarbasHiveMind/hivemind-sqlite-database/tree/0.2.1a2) (2026-04-15)

[Full Changelog](https://github.com/JarbasHiveMind/hivemind-sqlite-database/compare/0.2.1a1...0.2.1a2)

**Merged pull requests:**

- Update actions/setup-python action to v6 [\#15](https://github.com/JarbasHiveMind/hivemind-sqlite-database/pull/15) ([renovate[bot]](https://github.com/apps/renovate))
- Update actions/checkout action to v6 [\#9](https://github.com/JarbasHiveMind/hivemind-sqlite-database/pull/9) ([renovate[bot]](https://github.com/apps/renovate))

## [0.2.1a1](https://github.com/JarbasHiveMind/hivemind-sqlite-database/tree/0.2.1a1) (2026-04-15)

[Full Changelog](https://github.com/JarbasHiveMind/hivemind-sqlite-database/compare/0.2.0a1...0.2.1a1)

**Merged pull requests:**

- fix: move pytest\_plugins to top-level conftest for pytest 9 compatibility [\#18](https://github.com/JarbasHiveMind/hivemind-sqlite-database/pull/18) ([JarbasAl](https://github.com/JarbasAl))

## [0.2.0a1](https://github.com/JarbasHiveMind/hivemind-sqlite-database/tree/0.2.0a1) (2026-04-15)

[Full Changelog](https://github.com/JarbasHiveMind/hivemind-sqlite-database/compare/0.0.4a2...0.2.0a1)

**Merged pull requests:**

- feat: release 0.1.0 — tests, thread-safety, SQL injection fix [\#14](https://github.com/JarbasHiveMind/hivemind-sqlite-database/pull/14) ([JarbasAl](https://github.com/JarbasAl))

## [0.0.4a2](https://github.com/JarbasHiveMind/hivemind-sqlite-database/tree/0.0.4a2) (2025-12-19)

[Full Changelog](https://github.com/JarbasHiveMind/hivemind-sqlite-database/compare/0.0.4a1...0.0.4a2)

**Merged pull requests:**

- chore\(deps\): update actions/setup-python action to v6 [\#12](https://github.com/JarbasHiveMind/hivemind-sqlite-database/pull/12) ([renovate[bot]](https://github.com/apps/renovate))

## [0.0.4a1](https://github.com/JarbasHiveMind/hivemind-sqlite-database/tree/0.0.4a1) (2025-12-18)

[Full Changelog](https://github.com/JarbasHiveMind/hivemind-sqlite-database/compare/0.0.3...0.0.4a1)

**Merged pull requests:**

- chore: Configure Renovate [\#7](https://github.com/JarbasHiveMind/hivemind-sqlite-database/pull/7) ([renovate[bot]](https://github.com/apps/renovate))
- feat: add SQLCipher encryption support \(password kwarg\) [\#24](https://github.com/JarbasHiveMind/hivemind-sqlite-database/pull/24) ([JarbasAl](https://github.com/JarbasAl))



Expand Down
77 changes: 76 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -1 +1,76 @@
# HiveMind SQLite Database
# HiveMind SQLite Database

SQLite database plugin for [hivemind-core](https://github.com/JarbasHiveMind/HiveMind-core).

Implements the `AbstractDB` interface via `hivemind-plugin-manager` and stores HiveMind
client records (API keys, crypto keys, access-control lists) in a local SQLite file.

## Installation

```bash
pip install hivemind-sqlite-database
```

### With encryption support (SQLCipher)

```bash
# 1. Install the SQLCipher system library
# Debian/Ubuntu:
sudo apt install libsqlcipher0

# 2. Install the Python binding via the optional extra
pip install "hivemind-sqlite-database[cipher]"
```

> The `sqlcipher3` wheel on PyPI ships its own libsqlcipher for x86_64 Linux, so the
> `apt` step may be optional on that platform. On ARM or Alpine you must build from
> source and will need the system library.

## Usage

### Plain (unencrypted) database — default

```python
from hivemind_sqlite_database import SQLiteDB

db = SQLiteDB() # stores data in XDG_DATA_HOME/hivemind-core/clients.db
```

### Encrypted database (SQLCipher / AES-256)

```python
from hivemind_sqlite_database import SQLiteDB

db = SQLiteDB(password="your-strong-passphrase")
```

Pass the same `password` every time you open the database. The encryption is
transparent — all existing methods (`add_item`, `search_by_value`, etc.) work
identically.

> **Data-loss warning**: There is no password recovery. If you lose the passphrase
> the database is permanently unrecoverable. Back up your passphrase securely.

### hivemind-core configuration

```json
{
"database": {
"module": "hivemind-sqlite-db-plugin",
"hivemind-sqlite-db-plugin": {
"name": "clients",
"subfolder": "hivemind-core",
"password": "your-strong-passphrase"
}
}
}
```

Leave `"password"` out (or set it to `null`) to use an unencrypted database.

## Notes

- An encrypted database cannot be opened by the plain `sqlite3` CLI or stdlib module.
- A plaintext database cannot be opened as encrypted. There is no automatic migration.
- The `password` field maps directly to SQLCipher's `PRAGMA key`.
- WAL journal mode is enabled for both encrypted and unencrypted databases.
32 changes: 29 additions & 3 deletions hivemind_sqlite_database/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
import os.path
import sqlite3
import threading
from typing import List, Union, Iterable
from typing import List, Optional, Union, Iterable

from ovos_utils.log import LOG
from ovos_utils.xdg_utils import xdg_data_home
Expand All @@ -24,17 +24,43 @@ class SQLiteDB(AbstractDB):
"""Database implementation using SQLite."""
name: str = "clients"
subfolder: str = "hivemind-core"
password: Optional[str] = None

def __post_init__(self):
"""
Initialize the SQLiteDB connection.

When *password* is set the database is opened via ``sqlcipher3`` and
encrypted with AES-256 (SQLCipher). The system library
``libsqlcipher0`` must be installed and ``sqlcipher3`` must be
available (``pip install hivemind-sqlite-database[cipher]``).

When *password* is ``None`` (default) the standard ``sqlite3`` module
is used and the database file is unencrypted.
"""
db_path = os.path.join(xdg_data_home(), self.subfolder, self.name + ".db")
LOG.debug(f"sqlite database path: {db_path}")
os.makedirs(os.path.dirname(db_path), exist_ok=True)

self.conn = sqlite3.connect(db_path, check_same_thread=False)
self.conn.row_factory = sqlite3.Row
if self.password is not None:
if self.password == "":
raise ValueError("password must be non-empty when encryption is enabled")
try:
import sqlcipher3 as _sqlcipher
except ImportError:
raise ImportError(
"sqlcipher3 is required to open an encrypted SQLite database. "
"Install the system library (e.g. 'apt install libsqlcipher-dev') "
"then: pip install hivemind-sqlite-database[cipher]"
)
self.conn = _sqlcipher.connect(db_path, check_same_thread=False)
self.conn.row_factory = _sqlcipher.Row
escaped_password = self.password.replace("'", "''")
self.conn.execute(f"PRAGMA key='{escaped_password}'")
else:
self.conn = sqlite3.connect(db_path, check_same_thread=False)
self.conn.row_factory = sqlite3.Row

self.conn.execute("PRAGMA journal_mode=WAL")
self._write_lock = threading.Lock()
self._initialize_database()
Expand Down
6 changes: 3 additions & 3 deletions hivemind_sqlite_database/version.py
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
# START_VERSION_BLOCK
VERSION_MAJOR = 0
VERSION_MINOR = 2
VERSION_BUILD = 1
VERSION_ALPHA = 4
VERSION_MINOR = 3
VERSION_BUILD = 0
VERSION_ALPHA = 1
# END_VERSION_BLOCK

__version__ = f"{VERSION_MAJOR}.{VERSION_MINOR}.{VERSION_BUILD}" + (f"a{VERSION_ALPHA}" if VERSION_ALPHA else "")
3 changes: 3 additions & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,9 @@ dependencies = [
]


[project.optional-dependencies]
cipher = ["sqlcipher3"]

[project.urls]
Homepage = "https://github.com/JarbasHiveMind/hivemind-sqlite-database"

Expand Down
87 changes: 87 additions & 0 deletions tests/test_sqlitedb.py
Original file line number Diff line number Diff line change
Expand Up @@ -302,5 +302,92 @@ def test_replace_item(self):
self.assertEqual(db.search_by_value("api_key", "revoked")[0].client_id, 1)


try:
import sqlcipher3 as _sqlcipher3 # noqa: F401
_SQLCIPHER_AVAILABLE = True
except ImportError:
_SQLCIPHER_AVAILABLE = False


@unittest.skipUnless(_SQLCIPHER_AVAILABLE, "sqlcipher3 not installed")
class TestSQLiteDBEncrypted(unittest.TestCase):
"""Tests for the SQLCipher-encrypted path. Skipped when sqlcipher3 is absent."""

def _make_encrypted_db(self, path: str, password: str = "hunter2") -> SQLiteDB:
import unittest.mock as mock
db = SQLiteDB.__new__(SQLiteDB)
db.name = os.path.splitext(os.path.basename(path))[0]
db.subfolder = ""
db.password = password
with mock.patch(
"hivemind_sqlite_database.xdg_data_home",
return_value=os.path.dirname(path),
):
db.__post_init__()
return db

def test_encrypted_file_unreadable_by_stdlib_sqlite3(self):
"""A file created with a password must be opaque to plain sqlite3."""
with tempfile.NamedTemporaryFile(suffix=".db", delete=False) as f:
path = f.name
os.unlink(path)
try:
db = self._make_encrypted_db(path, password="secret123")
db.add_item(make_client(1, "enc-key"))
db.commit()
# stdlib sqlite3 should not be able to read it
plain_conn = sqlite3.connect(path)
with self.assertRaises(sqlite3.DatabaseError):
plain_conn.execute("SELECT * FROM clients").fetchall()
plain_conn.close()
finally:
if os.path.exists(path):
os.unlink(path)

def test_encrypted_round_trip(self):
"""add_item then search_by_value works through the encryption layer."""
with tempfile.NamedTemporaryFile(suffix=".db", delete=False) as f:
path = f.name
os.unlink(path)
try:
db = self._make_encrypted_db(path, password="roundtrip")
db.add_item(make_client(1, "enc-api-key", name="alice"))
db.commit()
# Reopen with same password
db2 = self._make_encrypted_db(path, password="roundtrip")
results = db2.search_by_value("api_key", "enc-api-key")
self.assertEqual(len(results), 1)
self.assertEqual(results[0].name, "alice")
finally:
if os.path.exists(path):
os.unlink(path)

def test_sqlitedb_password_kwarg_raises_importerror_without_sqlcipher3(self):
"""Confirmed separately in test_sqlitedb_no_sqlcipher.py; skip here."""
pass


class TestSQLiteDBMissingCipher(unittest.TestCase):
"""Verify ImportError is raised when sqlcipher3 is absent and password is given."""

def test_importerror_when_sqlcipher3_missing(self):
import sys
import unittest.mock as mock

# Simulate sqlcipher3 not being installed
with mock.patch.dict(sys.modules, {"sqlcipher3": None}):
with tempfile.TemporaryDirectory() as tmpdir:
with self.assertRaises(ImportError) as ctx:
db = SQLiteDB.__new__(SQLiteDB)
db.name = "test"
db.subfolder = tmpdir
db.password = "secret"
# Patch xdg_data_home so db_path resolves inside tmpdir
with mock.patch("hivemind_sqlite_database.xdg_data_home",
return_value=tmpdir):
db.__post_init__()
self.assertIn("sqlcipher3", str(ctx.exception))


if __name__ == "__main__":
unittest.main()
Loading