diff --git a/.github/workflows/benchmark.yml b/.github/workflows/benchmark.yml index 6c9908d0c..1202d2dd9 100644 --- a/.github/workflows/benchmark.yml +++ b/.github/workflows/benchmark.yml @@ -48,7 +48,7 @@ jobs: - name: Install protobuf-compiler run: | sudo apt-get update - sudo apt-get install -y protobuf-compiler + sudo apt-get install -y protobuf-compiler libtss2-dev pkg-config - uses: bencherdev/bencher@main diff --git a/.github/workflows/benchmark_fork_run.yml b/.github/workflows/benchmark_fork_run.yml index a0eba471e..757b11596 100644 --- a/.github/workflows/benchmark_fork_run.yml +++ b/.github/workflows/benchmark_fork_run.yml @@ -21,7 +21,7 @@ jobs: - name: Install protobuf-compiler run: | sudo apt-get update - sudo apt-get install -y protobuf-compiler + sudo apt-get install -y protobuf-compiler libtss2-dev pkg-config - name: Install Rust uses: dtolnay/rust-toolchain@6d653acede28d24f02e3cd41383119e8b1b35921 # stable diff --git a/.github/workflows/benchmark_pr.yml b/.github/workflows/benchmark_pr.yml index 7713de527..73ea7c7f0 100644 --- a/.github/workflows/benchmark_pr.yml +++ b/.github/workflows/benchmark_pr.yml @@ -42,7 +42,7 @@ jobs: - name: Install protobuf-compiler run: | sudo apt-get update - sudo apt-get install -y protobuf-compiler + sudo apt-get install -y protobuf-compiler libtss2-dev pkg-config - name: Install Rust uses: dtolnay/rust-toolchain@6d653acede28d24f02e3cd41383119e8b1b35921 # stable diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index c35747245..72e60e3d1 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -70,6 +70,32 @@ jobs: sudo apt-get update sudo apt-get install -y protobuf-compiler + # Build-time dependency of the `tpm` feature (`storage-crypto-tpm`'s + # `tss-esapi` needs `libtss2-dev` + `pkg-config` to compile at all, even + # without a real or virtual TPM present — `cargo nextest run + # --all-features` below builds it). No TPM device is required in CI: + # per ADR 0016-v2 §2.5.2's test/sample scope, `storage-crypto-tpm` has + # no CI-gated tests, only the example built in "Build TPM KEK example" + # below. + - name: Install libtss2-dev + run: | + sudo apt-get update + sudo apt-get install -y libtss2-dev pkg-config + + # Run-time dependency of the `pkcs11` feature's SoftHSM2-backed + # integration tests (`storage-crypto-pkcs11/tests/softhsm.rs`, + # `storage/tests/test_pkcs11_cluster.rs`). Each test provisions its own + # isolated token via `SOFTHSM2_CONF`, so no system-wide token + # initialization is needed here — only the module itself + # (`libsofthsm2.so`) needs to be present. Tests skip themselves + # (rather than fail) if it isn't, so this step is not a hard + # requirement for CI to pass, only for the pkcs11 tests to actually run + # instead of skipping. + - name: Install SoftHSM2 + run: | + sudo apt-get update + sudo apt-get install -y softhsm2 + - name: Install Rust uses: dtolnay/rust-toolchain@6d653acede28d24f02e3cd41383119e8b1b35921 # stable with: @@ -108,6 +134,13 @@ jobs: - name: Run tests (unit) run: cargo nextest run --all-features --profile ci + # Compile-only: catches rot in the TPM sample without requiring a + # real/virtual TPM in CI (ADR 0016-v2 §2.5.2 test/sample scope decision + # — real/virtual TPM availability in CI runners isn't reliable enough + # to gate merges on, so `swtpm` is intentionally not installed here). + - name: Build TPM KEK example + run: cargo build -p openstack-keystone-storage-crypto-tpm --example tpm_kek_demo + - name: Run integration tests (sqlite) run: cargo nextest run -p test_integration --profile ci @@ -168,7 +201,7 @@ jobs: - name: Install protobuf-compiler run: | sudo apt-get update - sudo apt-get install -y protobuf-compiler + sudo apt-get install -y protobuf-compiler libtss2-dev pkg-config - name: Rust Cache uses: swatinem/rust-cache@98c8021b550208e191a6a3145459bfc9fb29c4c0 # v2.8.0 diff --git a/.github/workflows/functional.yml b/.github/workflows/functional.yml index cb3561e83..eefd331ce 100644 --- a/.github/workflows/functional.yml +++ b/.github/workflows/functional.yml @@ -81,7 +81,7 @@ jobs: - name: Install protobuf-compiler run: | sudo apt-get update - sudo apt-get install -y protobuf-compiler + sudo apt-get install -y protobuf-compiler libtss2-dev pkg-config - name: Build Keystone run: cargo build --release diff --git a/.github/workflows/release-plz.yml b/.github/workflows/release-plz.yml index c4cf542d8..269788576 100644 --- a/.github/workflows/release-plz.yml +++ b/.github/workflows/release-plz.yml @@ -29,10 +29,10 @@ jobs: fetch-depth: 0 persist-credentials: false - - name: Install protobuf-compiler + - name: Install build dependencies run: | sudo apt update - sudo apt install -y protobuf-compiler + sudo apt-get install -y protobuf-compiler libtss2-dev pkg-config - name: Install Rust toolchain run: rustup update stable diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 5ed6f6c43..0ae2f9565 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -192,6 +192,9 @@ When submitting a PR: | `crates/core-types` | Shared data structures across the workspace | | `crates/api-types` | API request/response models and conversions from `core-types` | | `crates/storage` | Distributed storage implementation (OpenRaft) | +| `crates/storage-crypto` | KEK/DEK primitives shared by the storage engine: `KekProvider` trait, `EnvKek` (dev-mode), AES-GCM cipher/nonce/audit helpers | +| `crates/storage-crypto-pkcs11` | `Pkcs11Kek` — production `KekProvider` backed by a non-extractable AES key on a PKCS#11 token/HSM (e.g. SoftHSM2, vendor HSMs) | +| `crates/storage-crypto-tpm` | `TpmKek` — production `KekProvider` backed by a TPM 2.0 resident AES key | | `crates/*-sql` | SQL-backed persistence drivers (e.g. `identity-sql`, `catalog-sql`) using Sea-ORM | | `crates/*-raft` | Raft-backed persistence drivers for distributed storage | | `crates/config` | Configuration parsing | diff --git a/Cargo.lock b/Cargo.lock index de8b9a567..1e8c6547d 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -606,6 +606,26 @@ dependencies = [ "serde", ] +[[package]] +name = "bitfield" +version = "0.19.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "21ba6517c6b0f2bf08be60e187ab64b038438f22dd755614d8fe4d4098c46419" +dependencies = [ + "bitfield-macros", +] + +[[package]] +name = "bitfield-macros" +version = "0.19.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f48d6ace212fdf1b45fd6b566bb40808415344642b76c3224c07c8df9da81e97" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.118", +] + [[package]] name = "bitflags" version = "1.3.2" @@ -1406,7 +1426,7 @@ dependencies = [ "rustc-hash", "serde", "smallvec", - "target-lexicon", + "target-lexicon 0.13.5", "wasmtime-internal-core", ] @@ -1459,7 +1479,7 @@ dependencies = [ "cranelift-codegen", "log", "smallvec", - "target-lexicon", + "target-lexicon 0.13.5", ] [[package]] @@ -1476,7 +1496,7 @@ checksum = "c3ec0cc1a54e22925eacf4fc3dc815f907734d3b377899d19d52bec04863e853" dependencies = [ "cranelift-codegen", "libc", - "target-lexicon", + "target-lexicon 0.13.5", ] [[package]] @@ -1660,6 +1680,28 @@ dependencies = [ "rand_core 0.10.1", ] +[[package]] +name = "cryptoki" +version = "0.12.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ff765b99fc49f3116c9a908484486a2b92fd73c48da45c3a69716471c6cc56c6" +dependencies = [ + "bitflags 2.13.0", + "cryptoki-sys", + "libloading", + "log", + "secrecy", +] + +[[package]] +name = "cryptoki-sys" +version = "0.5.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f1fd850498411e4057f1cba79e6e2bc7cbe960544c1046ab46d4685c403a1121" +dependencies = [ + "libloading", +] + [[package]] name = "ctr" version = "0.10.1" @@ -2164,6 +2206,26 @@ dependencies = [ "syn 2.0.118", ] +[[package]] +name = "enumflags2" +version = "0.7.12" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1027f7680c853e056ebcec683615fb6fbbc07dbaa13b4d5d9442b146ded4ecef" +dependencies = [ + "enumflags2_derive", +] + +[[package]] +name = "enumflags2_derive" +version = "0.7.12" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "67c78a4d8fdf9953a5c9d458f9efe940fd97a0cab0941c075a813ac594733827" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.118", +] + [[package]] name = "equivalent" version = "1.0.2" @@ -2947,6 +3009,12 @@ dependencies = [ "windows-sys 0.61.2", ] +[[package]] +name = "hostname-validator" +version = "1.1.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f558a64ac9af88b5ba400d99b579451af0d39c6d360980045b91aac966d705e2" + [[package]] name = "http" version = "1.4.2" @@ -3663,6 +3731,16 @@ version = "0.2.186" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "68ab91017fe16c622486840e4c83c9a37afeff978bd239b5293d61ece587de66" +[[package]] +name = "libloading" +version = "0.8.9" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d7c4b02199fee7c5d21a5ae7d8cfa79a6ef5bb2fc834d6e9058e89c825efdc55" +dependencies = [ + "cfg-if", + "windows-link", +] + [[package]] name = "libm" version = "0.2.16" @@ -3849,6 +3927,16 @@ version = "0.3.4" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "4facc753ae494aeb6e3c22f839b158aebd4f9270f55cd3c79906c45476c47ab4" +[[package]] +name = "mbox" +version = "0.7.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "26d142aeadbc4e8c679fc6d93fbe7efe1c021fa7d80629e615915b519e3bc6de" +dependencies = [ + "libc", + "stable_deref_trait", +] + [[package]] name = "md-5" version = "0.10.6" @@ -4177,6 +4265,15 @@ dependencies = [ "memchr", ] +[[package]] +name = "oid" +version = "0.2.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9c19903c598813dba001b53beeae59bb77ad4892c5c1b9b3500ce4293a0d06c2" +dependencies = [ + "serde", +] + [[package]] name = "oid-registry" version = "0.7.1" @@ -4684,6 +4781,7 @@ dependencies = [ "byteorder", "chrono", "criterion", + "cryptoki", "dashmap", "eyre", "fjall", @@ -4697,6 +4795,8 @@ dependencies = [ "openstack-keystone-config", "openstack-keystone-storage-api", "openstack-keystone-storage-crypto", + "openstack-keystone-storage-crypto-pkcs11", + "openstack-keystone-storage-crypto-tpm", "prost", "rand 0.10.1", "rcgen", @@ -4936,6 +5036,32 @@ dependencies = [ "zeroize", ] +[[package]] +name = "openstack-keystone-storage-crypto-pkcs11" +version = "0.1.0" +dependencies = [ + "cryptoki", + "openstack-keystone-storage-crypto", + "rand 0.10.1", + "temp-env", + "tempfile", + "tracing", + "zeroize", +] + +[[package]] +name = "openstack-keystone-storage-crypto-tpm" +version = "0.1.0" +dependencies = [ + "openstack-keystone-storage-crypto", + "rand 0.10.1", + "subtle", + "tempfile", + "tracing", + "tss-esapi", + "zeroize", +] + [[package]] name = "openstack-keystone-token-driver-fernet" version = "0.1.1" @@ -5767,6 +5893,41 @@ dependencies = [ "siphasher", ] +[[package]] +name = "picky-asn1" +version = "0.10.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2ff038f9360b934342fb3c0a1d6e82c438a2624b51c3c6e3e6d7cf252b6f3ee3" +dependencies = [ + "oid", + "serde", + "serde_bytes", +] + +[[package]] +name = "picky-asn1-der" +version = "0.5.6" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d413165e4bf7f808b9a27cbaba657657a2921f0965db833f488c4d4be96dcd2e" +dependencies = [ + "picky-asn1", + "serde", + "serde_bytes", +] + +[[package]] +name = "picky-asn1-x509" +version = "0.15.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "859d4117bd1b1dc5646359ee7243c50c5000c0920ea2d1fb120335a2f4c684b8" +dependencies = [ + "base64 0.22.1", + "oid", + "picky-asn1", + "picky-asn1-der", + "serde", +] + [[package]] name = "pin-project" version = "1.1.13" @@ -8019,6 +8180,12 @@ dependencies = [ "xattr", ] +[[package]] +name = "target-lexicon" +version = "0.12.16" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "61c41af27dd6d1e27b1b16b489db798443478cef1f06a660c96db617ba5de3b1" + [[package]] name = "target-lexicon" version = "0.13.5" @@ -8720,6 +8887,39 @@ version = "0.2.5" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "e421abadd41a4225275504ea4d6566923418b7f05506fbc9c0fe86ba7396114b" +[[package]] +name = "tss-esapi" +version = "7.7.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3f10b25a84912b894d0e6d68f4a3771c923e9c44ddaaed7920cde92ed28aa84e" +dependencies = [ + "bitfield", + "enumflags2", + "getrandom 0.4.3", + "hostname-validator", + "log", + "mbox", + "num-derive", + "num-traits", + "oid", + "picky-asn1", + "picky-asn1-x509", + "regex", + "serde", + "tss-esapi-sys", + "zeroize", +] + +[[package]] +name = "tss-esapi-sys" +version = "0.6.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a7f972672926a3d3d18ecc04524720e4d20b7d1664a3fb73dbf7d4274196dbd9" +dependencies = [ + "pkg-config", + "target-lexicon 0.12.16", +] + [[package]] name = "twox-hash" version = "2.1.2" @@ -9292,7 +9492,7 @@ dependencies = [ "serde_derive", "serde_json", "smallvec", - "target-lexicon", + "target-lexicon 0.13.5", "tempfile", "wasm-compose", "wasm-encoder 0.245.1", @@ -9336,7 +9536,7 @@ dependencies = [ "serde_derive", "sha2 0.10.9", "smallvec", - "target-lexicon", + "target-lexicon 0.13.5", "wasm-encoder 0.245.1", "wasmparser 0.245.1", "wasmprinter", @@ -9415,7 +9615,7 @@ dependencies = [ "object 0.38.1", "pulley-interpreter", "smallvec", - "target-lexicon", + "target-lexicon 0.13.5", "thiserror 2.0.18", "wasmparser 0.245.1", "wasmtime-environ", @@ -9497,7 +9697,7 @@ dependencies = [ "gimli 0.33.0", "log", "object 0.38.1", - "target-lexicon", + "target-lexicon 0.13.5", "wasmparser 0.245.1", "wasmtime-environ", "wasmtime-internal-cranelift", @@ -9780,7 +9980,7 @@ dependencies = [ "gimli 0.33.0", "regalloc2", "smallvec", - "target-lexicon", + "target-lexicon 0.13.5", "thiserror 2.0.18", "wasmparser 0.245.1", "wasmtime-environ", diff --git a/Cargo.toml b/Cargo.toml index 06818cadb..e03827cb1 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -25,6 +25,8 @@ members = [ "crates/storage", "crates/storage-api", "crates/storage-crypto", + "crates/storage-crypto-pkcs11", + "crates/storage-crypto-tpm", "crates/token-driver-fernet", "crates/token-restriction-driver-sql", "crates/trust-driver-sql", @@ -113,6 +115,8 @@ openstack-keystone-distributed-storage = { version = "0.1.0", path = "crates/sto openstack-keystone-dynamic-plugin-runtime = { version = "0.1.0", path = "crates/dynamic-plugin-runtime" } openstack-keystone-storage-api = { version = "0.1.0", path = "crates/storage-api" } openstack-keystone-storage-crypto = { version = "0.1.0", path = "crates/storage-crypto" } +openstack-keystone-storage-crypto-pkcs11 = { version = "0.1.0", path = "crates/storage-crypto-pkcs11" } +openstack-keystone-storage-crypto-tpm = { version = "0.1.0", path = "crates/storage-crypto-tpm" } openstack-keystone-federation-driver-sql = { version = "0.1", path = "crates/federation-driver-sql" } openstack-keystone-k8s-auth-driver-sql = { version = "0.1", path = "crates/k8s-auth-driver-sql/" } openstack-keystone-k8s-auth-driver-raft = { version = "0.1", path = "crates/k8s-auth-driver-raft/" } diff --git a/crates/storage-crypto-pkcs11/Cargo.toml b/crates/storage-crypto-pkcs11/Cargo.toml new file mode 100644 index 000000000..2a9f42573 --- /dev/null +++ b/crates/storage-crypto-pkcs11/Cargo.toml @@ -0,0 +1,32 @@ +[package] +name = "openstack-keystone-storage-crypto-pkcs11" +description = "PKCS#11 (Cryptoki) HSM-backed KEK provider for Keystone distributed storage (ADR 0016-v2 §2.5.1)." +version = "0.1.0" +edition.workspace = true +license.workspace = true +authors.workspace = true +rust-version.workspace = true +repository.workspace = true +homepage.workspace = true +exclude.workspace = true + +[lints.rust] +unsafe_code = "forbid" + +[lints.clippy] +enum_glob_use = "deny" +expect_used = "deny" +print_stdout = "deny" +unwrap_used = "deny" +doc_paragraphs_missing_punctuation = "warn" + +[dependencies] +cryptoki = "0.12" +openstack-keystone-storage-crypto.workspace = true +rand.workspace = true +tracing.workspace = true +zeroize = "1" + +[dev-dependencies] +tempfile.workspace = true +temp-env.workspace = true diff --git a/crates/storage-crypto-pkcs11/src/lib.rs b/crates/storage-crypto-pkcs11/src/lib.rs new file mode 100644 index 000000000..89417bd9c --- /dev/null +++ b/crates/storage-crypto-pkcs11/src/lib.rs @@ -0,0 +1,244 @@ +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. +// +// SPDX-License-Identifier: Apache-2.0 +//! # PKCS#11 KEK provider +//! +//! Implements [`Pkcs11Kek`], a [`KekProvider`] backed by a non-extractable +//! AES-256 key object on a PKCS#11 token (ADR 0016-v2 §2.5.1). Wrap/unwrap +//! use `CKM_AES_GCM` directly against the token key, producing the same +//! `[12-byte nonce][ciphertext][16-byte tag]` wire format [`EnvKek`] uses — +//! nothing downstream of [`KekProvider`] needs to know the DEK is wrapped by +//! an HSM rather than in-process. +//! +//! Kept in its own crate (not a module of `storage-crypto`) so the FFI-heavy +//! `cryptoki` dependency stays out of the crate that owns the workspace's +//! `unsafe_code = "deny"` core primitives — this crate itself never needs +//! `unsafe` (`cryptoki` is a safe wrapper over the C API). +//! +//! [`EnvKek`]: openstack_keystone_storage_crypto::kek::EnvKek + +use std::path::Path; +use std::sync::Mutex; + +use cryptoki::context::{CInitializeArgs, CInitializeFlags, Pkcs11}; +use cryptoki::mechanism::Mechanism; +use cryptoki::mechanism::aead::GcmParams; +use cryptoki::object::{Attribute, KeyType, ObjectClass, ObjectHandle}; +use cryptoki::session::{Session, UserType}; +use cryptoki::slot::Slot; +use cryptoki::types::AuthPin; +use openstack_keystone_storage_crypto::error::CryptoError; +use openstack_keystone_storage_crypto::kek::KekProvider; +use rand::RngExt; +use zeroize::Zeroizing; + +/// Associated data used for DEK wrapping, matching [`EnvKek`]'s binding +/// context so ciphertext produced by either provider is distinguishable only +/// by which KEK unwraps it, not by format. +/// +/// [`EnvKek`]: openstack_keystone_storage_crypto::kek::EnvKek +const DEK_WRAP_AD: &[u8] = b"keystone-dek-wrap-v1"; + +/// Selects which token slot to open a session on. +#[derive(Debug, Clone)] +pub enum SlotSelector { + /// Open the slot with this numeric id. + Id(u64), + /// Open the slot whose token label matches exactly. + Label(String), +} + +/// Parameters needed to open (and, if missing, provision) the PKCS#11 KEK. +pub struct Pkcs11KekParams<'a> { + /// Path to the PKCS#11 module (`.so`) to `dlopen`. + pub module_path: &'a Path, + /// Which slot to open a session on. + pub slot: SlotSelector, + /// `CKA_LABEL` of the AES key object to use as the KEK. + pub key_label: &'a str, + /// User PIN for the token, as raw bytes (must be valid UTF-8). + pub pin: &'a [u8], + /// If no key with `key_label` exists on the token, generate a new + /// non-extractable AES-256 key with that label instead of failing. + /// + /// Left as an explicit constructor parameter rather than an implicit + /// default: whether first-run auto-provisioning is acceptable is an + /// operator/deployment decision (regulated environments may require an + /// out-of-band key ceremony instead), so the caller must opt in. + pub auto_generate: bool, +} + +/// PKCS#11 HSM-backed KEK (ADR 0016-v2 §2.5.1). +/// +/// Wraps/unwraps the DEK via `CKM_AES_GCM` against a non-extractable +/// (`CKA_EXTRACTABLE=false`, `CKA_SENSITIVE=true`) AES-256 key object on the +/// token — the KEK never enters process memory (invariant 13). +pub struct Pkcs11Kek { + // A PKCS#11 session is stateful (at most one active operation of a given + // kind at a time) and `Session` is `Send` but not `Sync`; the mutex + // serializes concurrent wrap/unwrap calls and makes the provider `Sync` + // as required by `KekProvider`. + session: Mutex, + key: ObjectHandle, +} + +impl Pkcs11Kek { + /// Open a session against the configured token and resolve (or, if + /// `auto_generate` is set, create) the AES KEK object. + pub fn open(params: Pkcs11KekParams<'_>) -> Result { + let pkcs11 = Pkcs11::new(params.module_path) + .map_err(|e| CryptoError::Pkcs11(format!("loading PKCS#11 module: {e}")))?; + pkcs11 + .initialize(CInitializeArgs::new(CInitializeFlags::OS_LOCKING_OK)) + .map_err(|e| CryptoError::Pkcs11(format!("initializing PKCS#11 module: {e}")))?; + + let slot = resolve_slot(&pkcs11, ¶ms.slot)?; + let session = pkcs11 + .open_rw_session(slot) + .map_err(|e| CryptoError::Pkcs11(format!("opening PKCS#11 session: {e}")))?; + + let pin_str = std::str::from_utf8(params.pin) + .map_err(|_| CryptoError::Pkcs11("PKCS#11 PIN is not valid UTF-8".into()))?; + session + .login(UserType::User, Some(&AuthPin::new(pin_str.into()))) + .map_err(|e| CryptoError::Pkcs11(format!("PKCS#11 login failed: {e}")))?; + + let key = find_key(&session, params.key_label)?; + let key = match key { + Some(key) => key, + None if params.auto_generate => generate_key(&session, params.key_label)?, + None => { + return Err(CryptoError::Pkcs11(format!( + "no AES key object with label {:?} on token and auto_generate is disabled", + params.key_label + ))); + } + }; + + Ok(Self { + session: Mutex::new(session), + key, + }) + } +} + +fn resolve_slot(pkcs11: &Pkcs11, selector: &SlotSelector) -> Result { + match selector { + SlotSelector::Id(id) => Slot::try_from(*id) + .map_err(|e| CryptoError::Pkcs11(format!("invalid PKCS#11 slot id {id}: {e}"))), + SlotSelector::Label(label) => { + let slots = pkcs11 + .get_slots_with_token() + .map_err(|e| CryptoError::Pkcs11(format!("listing PKCS#11 slots: {e}")))?; + for slot in slots { + let info = pkcs11 + .get_token_info(slot) + .map_err(|e| CryptoError::Pkcs11(format!("reading token info: {e}")))?; + if info.label() == label { + return Ok(slot); + } + } + Err(CryptoError::Pkcs11(format!( + "no PKCS#11 slot with token label {label:?}" + ))) + } + } +} + +fn find_key(session: &Session, label: &str) -> Result, CryptoError> { + let template = vec![ + Attribute::Class(ObjectClass::SECRET_KEY), + Attribute::KeyType(KeyType::AES), + Attribute::Label(label.as_bytes().to_vec()), + ]; + let mut found = session + .find_objects(&template) + .map_err(|e| CryptoError::Pkcs11(format!("finding PKCS#11 key object: {e}")))?; + Ok(found.pop()) +} + +fn generate_key(session: &Session, label: &str) -> Result { + let template = vec![ + Attribute::Token(true), + Attribute::Private(true), + Attribute::Sensitive(true), + Attribute::Extractable(false), + Attribute::Encrypt(true), + Attribute::Decrypt(true), + Attribute::Label(label.as_bytes().to_vec()), + Attribute::ValueLen(32.into()), + ]; + session + .generate_key(&Mechanism::AesKeyGen, &template) + .map_err(|e| CryptoError::Pkcs11(format!("generating PKCS#11 AES key: {e}"))) +} + +impl KekProvider for Pkcs11Kek { + fn wrap_dek(&self, dek: &[u8; 32]) -> Result, CryptoError> { + let mut nonce_bytes: [u8; 12] = rand::rng().random(); + let gcm_params = GcmParams::new(&mut nonce_bytes, DEK_WRAP_AD, 128.into()) + .map_err(|e| CryptoError::Pkcs11(format!("building GCM parameters: {e}")))?; + + let session = self + .session + .lock() + .unwrap_or_else(std::sync::PoisonError::into_inner); + let ciphertext_and_tag = session + .encrypt(&Mechanism::AesGcm(gcm_params), self.key, dek) + .map_err(|_| CryptoError::AesEncrypt)?; + drop(session); + + // ciphertext_and_tag is [32-byte ciphertext][16-byte tag] per the + // PKCS#11 CKM_AES_GCM convention (tag appended to the output). + if ciphertext_and_tag.len() != 32 + 16 { + return Err(CryptoError::AesEncrypt); + } + + let mut out = Vec::with_capacity(12 + 32 + 16); + out.extend_from_slice(&nonce_bytes); + out.extend_from_slice(&ciphertext_and_tag); + Ok(out) + } + + fn unwrap_dek(&self, wrapped: &[u8]) -> Result, CryptoError> { + // Layout: [12-byte nonce][32-byte ciphertext][16-byte tag] + if wrapped.len() != 12 + 32 + 16 { + return Err(CryptoError::WrappedDekSize); + } + let mut nonce_bytes: [u8; 12] = wrapped[..12] + .try_into() + .map_err(|_| CryptoError::WrappedDekSize)?; + let ciphertext_and_tag = &wrapped[12..]; + + let gcm_params = GcmParams::new(&mut nonce_bytes, DEK_WRAP_AD, 128.into()) + .map_err(|e| CryptoError::Pkcs11(format!("building GCM parameters: {e}")))?; + + let session = self + .session + .lock() + .unwrap_or_else(std::sync::PoisonError::into_inner); + let plaintext = Zeroizing::new( + session + .decrypt(&Mechanism::AesGcm(gcm_params), self.key, ciphertext_and_tag) + .map_err(|_| CryptoError::AesDecrypt)?, + ); + drop(session); + + if plaintext.len() != 32 { + return Err(CryptoError::InvalidKeyLength); + } + let mut out = Zeroizing::new([0u8; 32]); + out.copy_from_slice(&plaintext); + Ok(out) + } +} diff --git a/crates/storage-crypto-pkcs11/tests/softhsm.rs b/crates/storage-crypto-pkcs11/tests/softhsm.rs new file mode 100644 index 000000000..e928833fb --- /dev/null +++ b/crates/storage-crypto-pkcs11/tests/softhsm.rs @@ -0,0 +1,331 @@ +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. +// +// SPDX-License-Identifier: Apache-2.0 +//! SoftHSM2-backed integration test for [`Pkcs11Kek`] (ADR 0016-v2 §2.5.1, +//! implementation plan step 3/6). +//! +//! Requires a SoftHSM2 module to be installed (`apt-get install softhsm2`, +//! same install step CI already runs for SPIRE). The module path is taken +//! from `TEST_PKCS11_MODULE` if set — matching the `cryptoki` crate's own +//! convention — otherwise the common Debian/Ubuntu path is tried. +//! +//! Each test initializes its own token in an isolated temp directory and +//! runs its whole body under `temp_env::with_var("SOFTHSM2_CONF", ...)`, +//! which serializes access to that process-global variable across the crate +//! (`temp_env` internally locks) so tests can still run concurrently without +//! clobbering each other's token directory. + +use std::path::PathBuf; + +use cryptoki::context::{CInitializeArgs, CInitializeFlags, Pkcs11}; +use cryptoki::session::UserType; +use cryptoki::types::AuthPin; +use openstack_keystone_storage_crypto::error::CryptoError; +use openstack_keystone_storage_crypto::kek::KekProvider; +use openstack_keystone_storage_crypto_pkcs11::{Pkcs11Kek, Pkcs11KekParams, SlotSelector}; +use tempfile::TempDir; + +const SO_PIN: &str = "1234567890"; +const USER_PIN: &str = "fedcba0987"; +const TOKEN_LABEL: &str = "keystone-test"; + +fn module_path() -> Option { + if let Ok(p) = std::env::var("TEST_PKCS11_MODULE") { + return Some(PathBuf::from(p)); + } + [ + "/usr/lib/softhsm/libsofthsm2.so", + "/usr/lib/x86_64-linux-gnu/softhsm/libsofthsm2.so", + "/usr/local/lib/softhsm/libsofthsm2.so", + ] + .into_iter() + .map(PathBuf::from) + .find(|p| p.exists()) +} + +/// `Result::unwrap()`/`expect()` are denied by the crate's clippy lints for +/// `#[test]`-attributed functions (matched by name/attribute), but these +/// setup helpers aren't themselves `#[test]` functions, so clippy doesn't +/// recognize them as test code. Panicking on setup failure is still exactly +/// what's wanted here — this just spells it without the denied methods. +fn ok(result: Result, context: &str) -> T { + match result { + Ok(v) => v, + Err(e) => panic!("{context}: {e}"), + } +} + +fn skip_without_softhsm() -> bool { + if module_path().is_none() { + eprintln!( + "skipping: no SoftHSM2 module found (set TEST_PKCS11_MODULE or `apt-get install softhsm2`)" + ); + return true; + } + false +} + +/// A freshly initialized SoftHSM2 token, isolated in its own temp directory. +struct TestToken { + module: PathBuf, +} + +impl TestToken { + /// Initialize a new token (SO PIN + user PIN set) inside `dir`. + /// Must be called with `SOFTHSM2_CONF` already pointed at a config file + /// under `dir`. + fn init(module: PathBuf) -> Self { + let pkcs11 = ok(Pkcs11::new(&module), "load module"); + ok( + pkcs11.initialize(CInitializeArgs::new(CInitializeFlags::OS_LOCKING_OK)), + "initialize", + ); + let slot = ok(pkcs11.get_slots_with_token(), "slots")[0]; + ok( + pkcs11.init_token(slot, &AuthPin::new(SO_PIN.into()), TOKEN_LABEL), + "init token", + ); + { + let session = ok(pkcs11.open_rw_session(slot), "so session"); + ok( + session.login(UserType::So, Some(&AuthPin::new(SO_PIN.into()))), + "so login", + ); + ok( + session.init_pin(&AuthPin::new(USER_PIN.into())), + "init user pin", + ); + } + Self { module } + } + + fn open( + &self, + key_label: &str, + pin: &[u8], + auto_generate: bool, + ) -> Result { + Pkcs11Kek::open(Pkcs11KekParams { + module_path: &self.module, + slot: SlotSelector::Label(TOKEN_LABEL.into()), + key_label, + pin, + auto_generate, + }) + } +} + +/// Set up an isolated SoftHSM2 token directory/config, run `f` with +/// `SOFTHSM2_CONF` pointed at it for the duration, and tear down afterwards. +/// Returns `None` if no SoftHSM2 module is available (caller should treat +/// that as a skipped test). +fn with_token(f: impl FnOnce(&TestToken) -> R) -> Option { + let module = module_path()?; + let dir = ok(TempDir::new(), "tempdir"); + let token_dir = dir.path().join("tokens"); + ok(std::fs::create_dir_all(&token_dir), "mkdir tokens"); + let conf_path = dir.path().join("softhsm2.conf"); + ok( + std::fs::write( + &conf_path, + format!("directories.tokendir = {}\n", token_dir.display()), + ), + "write conf", + ); + + let result = temp_env::with_var("SOFTHSM2_CONF", Some(conf_path.as_os_str()), || { + let token = TestToken::init(module); + f(&token) + }); + Some(result) +} + +#[test] +fn test_wrap_unwrap_roundtrip() { + if skip_without_softhsm() { + return; + } + with_token(|token| { + let kek = token + .open("roundtrip-kek", USER_PIN.as_bytes(), true) + .expect("open KEK"); + + let dek = [0xABu8; 32]; + let wrapped = kek.wrap_dek(&dek).expect("wrap"); + assert_eq!(wrapped.len(), 60); // 12-byte nonce + 32-byte ciphertext + 16-byte tag + let unwrapped = kek.unwrap_dek(&wrapped).expect("unwrap"); + assert_eq!(unwrapped.as_ref(), &dek); + }); +} + +#[test] +fn test_wrap_produces_different_nonces() { + if skip_without_softhsm() { + return; + } + with_token(|token| { + let kek = token + .open("nonce-kek", USER_PIN.as_bytes(), true) + .expect("open KEK"); + + let dek = [0xCDu8; 32]; + let w1 = kek.wrap_dek(&dek).expect("wrap 1"); + let w2 = kek.wrap_dek(&dek).expect("wrap 2"); + assert_ne!(w1, w2); + // Same DEK, different nonces -> different ciphertext, but both + // unwrap back to the same plaintext. + assert_eq!(kek.unwrap_dek(&w1).expect("unwrap 1").as_ref(), &dek); + assert_eq!(kek.unwrap_dek(&w2).expect("unwrap 2").as_ref(), &dek); + }); +} + +#[test] +fn test_unwrap_tampered_tag_fails() { + if skip_without_softhsm() { + return; + } + with_token(|token| { + let kek = token + .open("tamper-kek", USER_PIN.as_bytes(), true) + .expect("open KEK"); + + let dek = [0x11u8; 32]; + let mut wrapped = kek.wrap_dek(&dek).expect("wrap"); + *wrapped.last_mut().expect("non-empty") ^= 0xFF; + assert!(matches!( + kek.unwrap_dek(&wrapped), + Err(CryptoError::AesDecrypt) + )); + }); +} + +#[test] +fn test_unwrap_tampered_ciphertext_fails() { + if skip_without_softhsm() { + return; + } + with_token(|token| { + let kek = token + .open("tamper-ct-kek", USER_PIN.as_bytes(), true) + .expect("open KEK"); + + let dek = [0x22u8; 32]; + let mut wrapped = kek.wrap_dek(&dek).expect("wrap"); + wrapped[20] ^= 0xFF; // inside the ciphertext region + assert!(matches!( + kek.unwrap_dek(&wrapped), + Err(CryptoError::AesDecrypt) + )); + }); +} + +#[test] +fn test_open_reuses_existing_key_across_sessions() { + if skip_without_softhsm() { + return; + } + with_token(|token| { + let kek1 = token + .open("reuse-kek", USER_PIN.as_bytes(), true) + .expect("open first session"); + + let dek = [0x33u8; 32]; + let wrapped = kek1.wrap_dek(&dek).expect("wrap with first session"); + drop(kek1); + + // A second provider opened against the same token/label must + // resolve to the same underlying key object and be able to unwrap + // the first session's ciphertext. + let kek2 = token + .open("reuse-kek", USER_PIN.as_bytes(), false) + .expect("reopen without auto_generate"); + let unwrapped = kek2 + .unwrap_dek(&wrapped) + .expect("unwrap with second session"); + assert_eq!(unwrapped.as_ref(), &dek); + }); +} + +#[test] +fn test_open_wrong_pin_fails() { + if skip_without_softhsm() { + return; + } + with_token(|token| { + // Establish the key first so a wrong PIN is the only failure mode. + token + .open("wrong-pin-kek", USER_PIN.as_bytes(), true) + .expect("provision key"); + + let result = token.open("wrong-pin-kek", b"not-the-pin", false); + assert!(matches!(result, Err(CryptoError::Pkcs11(_)))); + }); +} + +#[test] +fn test_open_missing_key_without_auto_generate_fails() { + if skip_without_softhsm() { + return; + } + with_token(|token| { + let result = token.open("nobody-generated-this-label", USER_PIN.as_bytes(), false); + assert!(matches!(result, Err(CryptoError::Pkcs11(_)))); + }); +} + +#[test] +fn test_open_slot_by_label_matches_slot_by_id() { + if skip_without_softhsm() { + return; + } + // SoftHSM only tolerates one live `C_Initialize`'d context per process + // for a given module, so the two providers are opened sequentially + // (the first is dropped, closing its session, before the second opens) + // rather than held open simultaneously. + with_token(|token| { + let wrapped = { + let kek_by_label = token + .open("slot-id-kek", USER_PIN.as_bytes(), true) + .expect("open by label"); + let dek = [0x44u8; 32]; + kek_by_label.wrap_dek(&dek).expect("wrap via label session") + }; + + // Slot ids handed out by SoftHSM are only valid within the + // `C_Initialize` session that produced them, so re-resolve the id + // fresh here (the label-keyed session above has already been + // dropped and finalized) rather than reusing the id captured during + // `TestToken::init`. + let slot_id = { + let pkcs11 = Pkcs11::new(&token.module).expect("load module for slot lookup"); + pkcs11 + .initialize(CInitializeArgs::new(CInitializeFlags::OS_LOCKING_OK)) + .expect("initialize for slot lookup"); + pkcs11.get_slots_with_token().expect("slots")[0].id() + }; + + let kek_by_id = Pkcs11Kek::open(Pkcs11KekParams { + module_path: &token.module, + slot: SlotSelector::Id(slot_id), + key_label: "slot-id-kek", + pin: USER_PIN.as_bytes(), + auto_generate: false, + }) + .expect("open by slot id"); + + let unwrapped = kek_by_id + .unwrap_dek(&wrapped) + .expect("unwrap via id session"); + assert_eq!(unwrapped.as_ref(), &[0x44u8; 32]); + }); +} diff --git a/crates/storage-crypto-tpm/Cargo.toml b/crates/storage-crypto-tpm/Cargo.toml new file mode 100644 index 000000000..eeafbea63 --- /dev/null +++ b/crates/storage-crypto-tpm/Cargo.toml @@ -0,0 +1,32 @@ +[package] +name = "openstack-keystone-storage-crypto-tpm" +description = "TPM 2.0-backed KEK provider for Keystone distributed storage (ADR 0016-v2 §2.5.2)." +version = "0.1.0" +edition.workspace = true +license.workspace = true +authors.workspace = true +rust-version.workspace = true +repository.workspace = true +homepage.workspace = true +exclude.workspace = true + +[lints.rust] +unsafe_code = "forbid" + +[lints.clippy] +enum_glob_use = "deny" +expect_used = "deny" +print_stdout = "deny" +unwrap_used = "deny" +doc_paragraphs_missing_punctuation = "warn" + +[dependencies] +openstack-keystone-storage-crypto.workspace = true +rand.workspace = true +subtle = "2.6" +tracing.workspace = true +tss-esapi = "7.7" +zeroize = "1" + +[dev-dependencies] +tempfile.workspace = true diff --git a/crates/storage-crypto-tpm/examples/tpm_kek_demo.rs b/crates/storage-crypto-tpm/examples/tpm_kek_demo.rs new file mode 100644 index 000000000..4dbe8dc48 --- /dev/null +++ b/crates/storage-crypto-tpm/examples/tpm_kek_demo.rs @@ -0,0 +1,103 @@ +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. +// +// SPDX-License-Identifier: Apache-2.0 +//! Runnable sample for the TPM 2.0 KEK provider (ADR 0016-v2 §2.5.2, +//! implementation plan step 4). +//! +//! This is a doc sample, not a CI-gated test (see the implementation plan's +//! "Test/sample scope" decision) — CI only compiles it to catch rot, it is +//! not executed. Run it locally against a software TPM: +//! +//! ```sh +//! # 1. Start a software TPM (swtpm) listening on the default MSSIM-style +//! # port pair used below. +//! mkdir -p /tmp/swtpm-state +//! swtpm socket --tpmstate dir=/tmp/swtpm-state \ +//! --ctrl type=tcp,port=2322 --server type=tcp,port=2321 \ +//! --tpm2 --flags not-need-init & +//! +//! # 2. Send TPM2_Startup (tpm2-tools' tpm2_startup, or any TSS client). +//! TPM2TOOLS_TCTI="swtpm:host=127.0.0.1,port=2321" tpm2_startup -c +//! +//! # 3. Run this example. TPM_KEK_CONTEXT_FILE defaults to a temp path if +//! # unset; delete it between runs to re-provision fresh keys. +//! cargo run -p openstack-keystone-storage-crypto-tpm --example tpm_kek_demo +//! ``` +//! +//! First run creates the AES + HMAC child keys (`auto_generate: true`) and +//! saves their blobs to the context file pair; subsequent runs reload the +//! same keys and prove a DEK wrapped by one process invocation unwraps +//! correctly in the next. + +use std::path::PathBuf; + +use openstack_keystone_storage_crypto::kek::KekProvider; +use openstack_keystone_storage_crypto_tpm::{KeyReference, TpmKek, TpmKekParams}; + +#[allow(clippy::print_stdout)] +fn main() -> Result<(), Box> { + let tcti = std::env::var("TPM_KEK_TCTI") + .unwrap_or_else(|_| "swtpm:host=127.0.0.1,port=2321".to_string()); + let context_file = std::env::var("TPM_KEK_CONTEXT_FILE") + .map(PathBuf::from) + .unwrap_or_else(|_| std::env::temp_dir().join("keystone-tpm-kek-demo.ctx")); + + println!("connecting to TPM via TCTI {tcti:?}"); + println!("AES key context file: {}", context_file.display()); + println!("HMAC key context file: {}.hmac", context_file.display()); + + let already_provisioned = context_file.exists(); + let kek = TpmKek::open(TpmKekParams { + tcti: &tcti, + key_reference: KeyReference::ContextFile(context_file.clone()), + auth: None, + auto_generate: true, + })?; + println!( + "{} KEK (create the files with `rm {} {}.hmac` to reset)", + if already_provisioned { + "loaded existing" + } else { + "provisioned new" + }, + context_file.display(), + context_file.display(), + ); + + let dek = [0x42u8; 32]; + let wrapped = kek.wrap_dek(&dek)?; + println!( + "wrapped a 32-byte DEK into {} bytes: {}", + wrapped.len(), + hex_encode(&wrapped) + ); + + let unwrapped = kek.unwrap_dek(&wrapped)?; + assert_eq!(unwrapped.as_ref(), &dek, "round-trip must recover the DEK"); + println!("unwrapped successfully — round-trip verified"); + + let mut tampered = wrapped.clone(); + if let Some(last) = tampered.last_mut() { + *last ^= 0xFF; + } + match kek.unwrap_dek(&tampered) { + Err(e) => println!("tampered ciphertext correctly rejected: {e}"), + Ok(_) => return Err("tampered ciphertext was NOT rejected".into()), + } + + Ok(()) +} + +fn hex_encode(bytes: &[u8]) -> String { + bytes.iter().map(|b| format!("{b:02x}")).collect() +} diff --git a/crates/storage-crypto-tpm/src/lib.rs b/crates/storage-crypto-tpm/src/lib.rs new file mode 100644 index 000000000..75f89e78d --- /dev/null +++ b/crates/storage-crypto-tpm/src/lib.rs @@ -0,0 +1,528 @@ +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. +// +// SPDX-License-Identifier: Apache-2.0 +//! # TPM 2.0 KEK provider +//! +//! Implements [`TpmKek`], a [`KekProvider`] backed by two TPM-resident, +//! non-duplicable keys (ADR 0016-v2 §2.5.2): an AES-256-CFB key for +//! confidentiality and an HMAC-SHA256 key for integrity. TPM 2.0 has no +//! native AES-GCM command (`TPM2_EncryptDecrypt2` only supports +//! CFB/CBC/CTR/OFB/ECB), so this provider uses Encrypt-then-MAC instead, +//! checking the MAC (constant-time) before ever attempting decryption +//! (invariant 15). +//! +//! Wire format: `[16-byte iv][32-byte ciphertext][32-byte HMAC tag]` — 80 +//! bytes for a 32-byte DEK. Distinct from [`EnvKek`]'s GCM format, but still +//! opaque `Vec` behind [`KekProvider`]. +//! +//! Kept in its own crate for the same reason as `storage-crypto-pkcs11`: the +//! FFI-heavy `tss-esapi` dependency (and its `tpm2-tss` system library +//! requirement) stays out of the crate that owns the workspace's +//! `unsafe_code = "deny"` core primitives. +//! +//! ## Key provisioning +//! +//! Both TPM keys are children of a primary key that is *not* persisted: TPM +//! 2.0 primary key derivation is deterministic given the same hierarchy, +//! template and (empty) `unique` field, so recreating it on every [`open`] +//! reproduces the same parent without needing storage. It exists only to +//! load/create the two children and is flushed immediately afterwards. +//! +//! The AES and HMAC child keys themselves need durable identity across +//! process restarts, selected by [`KeyReference`]: +//! +//! * [`KeyReference::PersistentHandle`] — the AES key lives at the given +//! persistent handle, the HMAC key at `handle + 1`. Both must already be +//! provisioned (via [`Pkcs11Kek`]-style `auto_generate`, see below) unless +//! `auto_generate` is set. +//! * [`KeyReference::ContextFile`] — the AES key's `(public, private)` blobs +//! (TPM-encrypted under the primary, safe to store outside the TPM) are +//! marshalled to the given path; the HMAC key's blobs go to `.hmac`. +//! +//! [`EnvKek`]: openstack_keystone_storage_crypto::kek::EnvKek +//! [`open`]: TpmKek::open +//! [`Pkcs11Kek`]: ../openstack_keystone_storage_crypto_pkcs11/struct.Pkcs11Kek.html + +use std::convert::TryFrom; +use std::path::{Path, PathBuf}; +use std::str::FromStr; +use std::sync::Mutex; + +use openstack_keystone_storage_crypto::error::CryptoError; +use openstack_keystone_storage_crypto::kek::KekProvider; +use rand::RngExt; +use subtle::ConstantTimeEq; +use tss_esapi::Context; +use tss_esapi::abstraction::cipher::Cipher; +use tss_esapi::attributes::ObjectAttributesBuilder; +use tss_esapi::handles::{KeyHandle, ObjectHandle, PersistentTpmHandle, TpmHandle}; +use tss_esapi::interface_types::algorithm::{HashingAlgorithm, PublicAlgorithm, SymmetricMode}; +use tss_esapi::interface_types::dynamic_handles::Persistent; +use tss_esapi::interface_types::key_bits::RsaKeyBits; +use tss_esapi::interface_types::resource_handles::{Hierarchy, Provision}; +use tss_esapi::interface_types::session_handles::AuthSession; +use tss_esapi::structures::{ + Auth, Digest, InitialValue, KeyedHashScheme, MaxBuffer, Private, Public, PublicBuilder, + PublicKeyedHashParameters, RsaExponent, SymmetricCipherParameters, +}; +use tss_esapi::tcti_ldr::TctiNameConf; +use tss_esapi::traits::{Marshall, UnMarshall}; +use tss_esapi::utils::create_restricted_decryption_rsa_public; +use zeroize::Zeroizing; + +/// Associated data used for DEK wrapping, matching [`EnvKek`]'s binding +/// context. +/// +/// [`EnvKek`]: openstack_keystone_storage_crypto::kek::EnvKek +const DEK_WRAP_AD: &[u8] = b"keystone-dek-wrap-v1"; + +/// Selects how the AES/HMAC child key pair is identified and persisted. +#[derive(Debug, Clone)] +pub enum KeyReference { + /// AES key at this persistent TPM handle, HMAC key at `handle + 1`. + PersistentHandle(u32), + /// AES key blobs at this path, HMAC key blobs at `.hmac`. + ContextFile(PathBuf), +} + +/// Parameters needed to open (and, if missing, provision) the TPM KEK. +pub struct TpmKekParams<'a> { + /// TCTI connection string (e.g. `"device:/dev/tpmrm0"` or + /// `"swtpm:host=127.0.0.1,port=2321"`). + pub tcti: &'a str, + /// How to locate/persist the AES and HMAC child keys. + pub key_reference: KeyReference, + /// Auth value applied to both child keys, as raw bytes. `None`/empty + /// means no auth is required to use them. + pub auth: Option<&'a [u8]>, + /// If the referenced key(s) don't exist yet, generate them instead of + /// failing. See [`Pkcs11KekParams::auto_generate`] for the same + /// caller-opts-in rationale. + /// + /// [`Pkcs11KekParams::auto_generate`]: ../openstack_keystone_storage_crypto_pkcs11/struct.Pkcs11KekParams.html#structfield.auto_generate + pub auto_generate: bool, +} + +/// TPM 2.0-backed KEK (ADR 0016-v2 §2.5.2). +pub struct TpmKek { + // `Context` is `Send` but its FFI session/handle state means concurrent + // use from multiple threads is unsound; the mutex serializes wrap/unwrap + // calls and makes the provider `Sync` as `KekProvider` requires. + context: Mutex, + aes_key: KeyHandle, + hmac_key: KeyHandle, +} + +impl TpmKek { + /// Open a TPM context, resolve (or, if `auto_generate` is set, create) + /// the AES and HMAC child keys, and apply `auth` to both. + pub fn open(params: TpmKekParams<'_>) -> Result { + let tcti = TctiNameConf::from_str(params.tcti) + .map_err(|e| CryptoError::Tpm(format!("invalid TCTI {:?}: {e}", params.tcti)))?; + let mut context = Context::new(tcti) + .map_err(|e| CryptoError::Tpm(format!("opening TPM context: {e}")))?; + + let auth_value = match params.auth { + Some(bytes) if !bytes.is_empty() => Some( + Auth::try_from(bytes.to_vec()) + .map_err(|e| CryptoError::Tpm(format!("invalid TPM auth value: {e}")))?, + ), + _ => None, + }; + + context + .tr_set_auth(Hierarchy::Owner.into(), Auth::default()) + .map_err(|e| CryptoError::Tpm(format!("setting owner auth: {e}")))?; + let primary = context + .execute_with_session(Some(AuthSession::Password), |ctx| { + ctx.create_primary( + Hierarchy::Owner, + create_restricted_decryption_rsa_public( + Cipher::aes_128_cfb() + .try_into() + .map_err(|e| CryptoError::Tpm(format!("primary cipher: {e}")))?, + RsaKeyBits::Rsa2048, + RsaExponent::default(), + ) + .map_err(|e| CryptoError::Tpm(format!("building primary template: {e}")))?, + None, + None, + None, + None, + ) + .map_err(|e| CryptoError::Tpm(format!("creating TPM primary key: {e}"))) + })? + .key_handle; + + let (aes_key, hmac_key) = match ¶ms.key_reference { + KeyReference::PersistentHandle(handle) => ( + resolve_persistent( + &mut context, + primary, + *handle, + aes_key_public, + auth_value.clone(), + params.auto_generate, + )?, + resolve_persistent( + &mut context, + primary, + handle + .checked_add(1) + .ok_or_else(|| CryptoError::Tpm("TPM handle overflow".into()))?, + hmac_key_public, + auth_value.clone(), + params.auto_generate, + )?, + ), + KeyReference::ContextFile(path) => ( + resolve_context_file( + &mut context, + primary, + path, + aes_key_public, + auth_value.clone(), + params.auto_generate, + )?, + resolve_context_file( + &mut context, + primary, + &hmac_sibling_path(path), + hmac_key_public, + auth_value.clone(), + params.auto_generate, + )?, + ), + }; + + context + .execute_without_session(|ctx| ctx.flush_context(primary.into())) + .map_err(|e| CryptoError::Tpm(format!("flushing TPM primary key: {e}")))?; + + if let Some(auth) = &auth_value { + context + .tr_set_auth(aes_key.into(), auth.clone()) + .map_err(|e| CryptoError::Tpm(format!("setting AES key auth: {e}")))?; + context + .tr_set_auth(hmac_key.into(), auth.clone()) + .map_err(|e| CryptoError::Tpm(format!("setting HMAC key auth: {e}")))?; + } + + Ok(Self { + context: Mutex::new(context), + aes_key, + hmac_key, + }) + } +} + +fn hmac_sibling_path(path: &Path) -> PathBuf { + let mut s = path.as_os_str().to_owned(); + s.push(".hmac"); + PathBuf::from(s) +} + +fn aes_key_public() -> Result { + let object_attributes = ObjectAttributesBuilder::new() + .with_fixed_tpm(true) + .with_fixed_parent(true) + .with_sensitive_data_origin(true) + .with_user_with_auth(true) + .with_sign_encrypt(true) + .with_decrypt(true) + .build() + .map_err(|e| CryptoError::Tpm(format!("building AES key attributes: {e}")))?; + PublicBuilder::new() + .with_public_algorithm(PublicAlgorithm::SymCipher) + .with_name_hashing_algorithm(HashingAlgorithm::Sha256) + .with_object_attributes(object_attributes) + .with_symmetric_cipher_parameters(SymmetricCipherParameters::new( + Cipher::aes_256_cfb() + .try_into() + .map_err(|e| CryptoError::Tpm(format!("AES key cipher: {e}")))?, + )) + .with_symmetric_cipher_unique_identifier(Default::default()) + .build() + .map_err(|e| CryptoError::Tpm(format!("building AES key template: {e}"))) +} + +fn hmac_key_public() -> Result { + let object_attributes = ObjectAttributesBuilder::new() + .with_fixed_tpm(true) + .with_fixed_parent(true) + .with_sensitive_data_origin(true) + .with_user_with_auth(true) + .with_sign_encrypt(true) + .build() + .map_err(|e| CryptoError::Tpm(format!("building HMAC key attributes: {e}")))?; + PublicBuilder::new() + .with_public_algorithm(PublicAlgorithm::KeyedHash) + .with_name_hashing_algorithm(HashingAlgorithm::Sha256) + .with_object_attributes(object_attributes) + .with_keyed_hash_parameters(PublicKeyedHashParameters::new( + KeyedHashScheme::HMAC_SHA_256, + )) + .with_keyed_hash_unique_identifier(Digest::default()) + .build() + .map_err(|e| CryptoError::Tpm(format!("building HMAC key template: {e}"))) +} + +/// Resolve a child key at a persistent handle, creating and evicting it +/// there if missing and `auto_generate` is set. +fn resolve_persistent( + context: &mut Context, + primary: KeyHandle, + handle: u32, + public_template: fn() -> Result, + auth_value: Option, + auto_generate: bool, +) -> Result { + let persistent_handle = PersistentTpmHandle::new(handle) + .map_err(|e| CryptoError::Tpm(format!("invalid persistent handle {handle:#x}: {e}")))?; + + if let Ok(object_handle) = context.tr_from_tpm_public(TpmHandle::Persistent(persistent_handle)) + { + return Ok(KeyHandle::from(object_handle)); + } + + if !auto_generate { + return Err(CryptoError::Tpm(format!( + "no TPM key at persistent handle {handle:#x} and auto_generate is disabled" + ))); + } + + let transient = context + .execute_with_session(Some(AuthSession::Password), |ctx| { + ctx.create( + primary, + public_template()?, + auth_value.clone(), + None, + None, + None, + ) + .map_err(|e| CryptoError::Tpm(format!("creating TPM key: {e}"))) + }) + .and_then(|created| { + context + .execute_with_session(Some(AuthSession::Password), |ctx| { + ctx.load(primary, created.out_private, created.out_public) + }) + .map_err(|e| CryptoError::Tpm(format!("loading created TPM key: {e}"))) + })?; + + let evicted = context + .execute_with_session(Some(AuthSession::Password), |ctx| { + ctx.evict_control( + Provision::Owner, + transient.into(), + Persistent::Persistent(persistent_handle), + ) + }) + .map_err(|e| CryptoError::Tpm(format!("persisting TPM key at {handle:#x}: {e}")))?; + + Ok(KeyHandle::from(evicted)) +} + +/// Resolve a child key from a context file, creating and saving it there if +/// missing and `auto_generate` is set. +fn resolve_context_file( + context: &mut Context, + primary: KeyHandle, + path: &Path, + public_template: fn() -> Result, + auth_value: Option, + auto_generate: bool, +) -> Result { + if let Some((private, public)) = read_key_blob(path)? { + return context + .execute_with_session(Some(AuthSession::Password), |ctx| { + ctx.load(primary, private, public) + }) + .map_err(|e| CryptoError::Tpm(format!("loading TPM key from {path:?}: {e}"))); + } + + if !auto_generate { + return Err(CryptoError::Tpm(format!( + "no TPM key context file at {path:?} and auto_generate is disabled" + ))); + } + + let created = context.execute_with_session(Some(AuthSession::Password), |ctx| { + ctx.create(primary, public_template()?, auth_value, None, None, None) + .map_err(|e| CryptoError::Tpm(format!("creating TPM key: {e}"))) + })?; + + write_key_blob(path, &created.out_private, &created.out_public)?; + + context + .execute_with_session(Some(AuthSession::Password), |ctx| { + ctx.load(primary, created.out_private, created.out_public) + }) + .map_err(|e| CryptoError::Tpm(format!("loading newly created TPM key: {e}"))) +} + +/// Context file format: `[4-byte LE public len][public][4-byte LE private +/// len][private]`, each half TPM-marshalled. Returns `None` if the file +/// doesn't exist. +fn read_key_blob(path: &Path) -> Result, CryptoError> { + let bytes = match std::fs::read(path) { + Ok(b) => b, + Err(e) if e.kind() == std::io::ErrorKind::NotFound => return Ok(None), + Err(e) => { + return Err(CryptoError::Tpm(format!( + "reading TPM key context file {path:?}: {e}" + ))); + } + }; + let bad_format = || CryptoError::Tpm(format!("malformed TPM key context file {path:?}")); + + let public_len = bytes.first_chunk::<4>().ok_or_else(bad_format)?; + let public_len = u32::from_le_bytes(*public_len) as usize; + let rest = bytes.get(4..).ok_or_else(bad_format)?; + let public_bytes = rest.get(..public_len).ok_or_else(bad_format)?; + let rest = rest.get(public_len..).ok_or_else(bad_format)?; + + let private_len = rest.first_chunk::<4>().ok_or_else(bad_format)?; + let private_len = u32::from_le_bytes(*private_len) as usize; + let rest = rest.get(4..).ok_or_else(bad_format)?; + let private_bytes = rest.get(..private_len).ok_or_else(bad_format)?; + + let public = Public::unmarshall(public_bytes) + .map_err(|e| CryptoError::Tpm(format!("unmarshalling TPM public blob: {e}")))?; + // `Private` is a plain length-prefixed buffer type (TPM2B_PRIVATE), not a + // `Marshall`/`UnMarshall` structure — its byte representation is exactly + // what `Private::try_from` / `.value()` produce. + let private = Private::try_from(private_bytes.to_vec()) + .map_err(|e| CryptoError::Tpm(format!("decoding TPM private blob: {e}")))?; + Ok(Some((private, public))) +} + +fn write_key_blob(path: &Path, private: &Private, public: &Public) -> Result<(), CryptoError> { + let public_bytes = public + .marshall() + .map_err(|e| CryptoError::Tpm(format!("marshalling TPM public blob: {e}")))?; + let private_bytes = private.value(); + + let mut out = Vec::with_capacity(8 + public_bytes.len() + private_bytes.len()); + out.extend_from_slice(&(public_bytes.len() as u32).to_le_bytes()); + out.extend_from_slice(&public_bytes); + out.extend_from_slice(&(private_bytes.len() as u32).to_le_bytes()); + out.extend_from_slice(private_bytes); + + std::fs::write(path, out) + .map_err(|e| CryptoError::Tpm(format!("writing TPM key context file {path:?}: {e}"))) +} + +impl KekProvider for TpmKek { + fn wrap_dek(&self, dek: &[u8; 32]) -> Result, CryptoError> { + let iv_bytes: [u8; 16] = rand::rng().random(); + let mut context = self + .context + .lock() + .unwrap_or_else(std::sync::PoisonError::into_inner); + + let iv = InitialValue::try_from(iv_bytes.to_vec()).map_err(|_| CryptoError::AesEncrypt)?; + let data = MaxBuffer::try_from(dek.to_vec()).map_err(|_| CryptoError::AesEncrypt)?; + + let (ciphertext, _) = context + .execute_with_session(Some(AuthSession::Password), |ctx| { + ctx.encrypt_decrypt_2(self.aes_key, false, SymmetricMode::Cfb, data, iv) + }) + .map_err(|_| CryptoError::AesEncrypt)?; + let ciphertext: Vec = ciphertext.to_vec(); + if ciphertext.len() != 32 { + return Err(CryptoError::AesEncrypt); + } + + let tag = compute_tag(&mut context, self.hmac_key, &iv_bytes, &ciphertext) + .map_err(|_| CryptoError::AesEncrypt)?; + + let mut out = Vec::with_capacity(16 + 32 + 32); + out.extend_from_slice(&iv_bytes); + out.extend_from_slice(&ciphertext); + out.extend_from_slice(&tag); + Ok(out) + } + + fn unwrap_dek(&self, wrapped: &[u8]) -> Result, CryptoError> { + // Layout: [16-byte iv][32-byte ciphertext][32-byte HMAC tag] + if wrapped.len() != 16 + 32 + 32 { + return Err(CryptoError::WrappedDekSize); + } + let iv_bytes: [u8; 16] = wrapped[..16] + .try_into() + .map_err(|_| CryptoError::WrappedDekSize)?; + let ciphertext = &wrapped[16..48]; + let expected_tag = &wrapped[48..]; + + let mut context = self + .context + .lock() + .unwrap_or_else(std::sync::PoisonError::into_inner); + + // Authenticate before decrypting (ADR invariant 15): compute and + // constant-time-compare the tag before ever calling + // encrypt_decrypt_2 on unauthenticated ciphertext. + let actual_tag = compute_tag(&mut context, self.hmac_key, &iv_bytes, ciphertext) + .map_err(|_| CryptoError::AesDecrypt)?; + if actual_tag.ct_eq(expected_tag).unwrap_u8() != 1 { + return Err(CryptoError::AesDecrypt); + } + + let iv = InitialValue::try_from(iv_bytes.to_vec()).map_err(|_| CryptoError::AesDecrypt)?; + let data = MaxBuffer::try_from(ciphertext.to_vec()).map_err(|_| CryptoError::AesDecrypt)?; + let (plaintext, _) = context + .execute_with_session(Some(AuthSession::Password), |ctx| { + ctx.encrypt_decrypt_2(self.aes_key, true, SymmetricMode::Cfb, data, iv) + }) + .map_err(|_| CryptoError::AesDecrypt)?; + drop(context); + + // `plaintext` (a `MaxBuffer`) already wraps a `Zeroizing>` + // internally — copy straight out of it via `.value()` rather than + // `.to_vec()`-ing into a throwaway plain `Vec` that would hold + // the unwrapped DEK without being zeroized on drop. + if plaintext.value().len() != 32 { + return Err(CryptoError::InvalidKeyLength); + } + let mut out = Zeroizing::new([0u8; 32]); + out.copy_from_slice(plaintext.value()); + Ok(out) + } +} + +/// `TPM2_HMAC(hmac_key, iv ++ ciphertext ++ DEK_WRAP_AD)`. +fn compute_tag( + context: &mut Context, + hmac_key: KeyHandle, + iv: &[u8], + ciphertext: &[u8], +) -> Result, CryptoError> { + let mut buf = Vec::with_capacity(iv.len() + ciphertext.len() + DEK_WRAP_AD.len()); + buf.extend_from_slice(iv); + buf.extend_from_slice(ciphertext); + buf.extend_from_slice(DEK_WRAP_AD); + let buffer = MaxBuffer::try_from(buf).map_err(|e| CryptoError::Tpm(format!("{e}")))?; + + let digest = context + .execute_with_session(Some(AuthSession::Password), |ctx| { + ctx.hmac( + ObjectHandle::from(hmac_key), + buffer, + HashingAlgorithm::Sha256, + ) + }) + .map_err(|e| CryptoError::Tpm(format!("computing HMAC tag: {e}")))?; + Ok(digest.to_vec()) +} diff --git a/crates/storage-crypto/src/error.rs b/crates/storage-crypto/src/error.rs index a3b23b9a4..38617591c 100644 --- a/crates/storage-crypto/src/error.rs +++ b/crates/storage-crypto/src/error.rs @@ -17,13 +17,22 @@ use thiserror::Error; /// Errors produced by the storage cryptographic layer. #[derive(Error, Debug)] pub enum CryptoError { - /// AES-256-GCM encryption failed (internal AES error). - #[error("AES-256-GCM encryption error")] + /// AES encryption (wrap) failed, whether the AEAD construction is + /// AES-256-GCM ([`EnvKek`], PKCS#11) or AES-256-CFB + HMAC + /// Encrypt-then-MAC (TPM). + /// + /// [`EnvKek`]: crate::kek::EnvKek + #[error("AES encryption error")] AesEncrypt, - /// AES-256-GCM decryption failed — GCM tag mismatch indicates data - /// corruption or tampering. - #[error("AES-256-GCM decryption error: GCM tag verification failed")] + /// AES decryption (unwrap) failed — tag mismatch indicates data + /// corruption or tampering. Covers both AES-256-GCM ([`EnvKek`], + /// PKCS#11) and AES-256-CFB + HMAC Encrypt-then-MAC (TPM); the shared + /// variant intentionally makes the two constructions' failures + /// indistinguishable to callers. + /// + /// [`EnvKek`]: crate::kek::EnvKek + #[error("AES decryption error: authentication tag verification failed")] AesDecrypt, /// DEK epoch has not been loaded; `bootstrap_dek` must be called first. @@ -81,7 +90,17 @@ pub enum CryptoError { #[error("DEK epoch {version} was revoked; decryption refused")] RevokedDek { version: u32 }, - /// PKCS#11 backend is not yet implemented. - #[error("PKCS#11 HSM backend is not implemented in this build")] - Pkcs11NotImplemented, + /// A PKCS#11 setup or session operation failed (module load, slot lookup, + /// login, key generation/lookup). Wrap/unwrap operation failures use + /// [`CryptoError::AesEncrypt`] / [`CryptoError::AesDecrypt`] instead, so + /// this variant is confined to provider construction. + #[error("PKCS#11 operation failed: {0}")] + Pkcs11(String), + + /// A TPM 2.0 setup or session operation failed (context open, primary + /// creation, key generation/lookup, persistence). Wrap/unwrap operation + /// failures use [`CryptoError::AesEncrypt`] / [`CryptoError::AesDecrypt`] + /// instead, so this variant is confined to provider construction. + #[error("TPM operation failed: {0}")] + Tpm(String), } diff --git a/crates/storage-crypto/src/kek.rs b/crates/storage-crypto/src/kek.rs index 239c4b891..1554b316b 100644 --- a/crates/storage-crypto/src/kek.rs +++ b/crates/storage-crypto/src/kek.rs @@ -14,19 +14,18 @@ //! # Key Encryption Key (KEK) providers //! //! A `KekProvider` wraps and unwraps the Data Encryption Key (DEK) using an -//! external key source. Two implementations are provided: +//! external key source. This crate provides [`EnvKek`], a dev-mode-only +//! implementation that reads a hex-encoded 256-bit key from the +//! `KEYSTONE_DEV_KEK` environment variable — requires `--dev-mode` and +//! `KEYSTONE_ALLOW_ENV_KEK=1`. After reading, the variable is removed from +//! the Rust environment map (via `unsafe env::remove_var`). Zeroing the +//! underlying bytes in `/proc//environ` was attempted but is currently a +//! no-op (see [`zero_environ_entry`]) — the raw `environ` bytes are not +//! scrubbed, so this is a best-effort dev-mode control, not a guarantee. //! -//! * [`EnvKek`] — reads a hex-encoded 256-bit key from the `KEYSTONE_DEV_KEK` -//! environment variable. Requires `--dev-mode` and -//! `KEYSTONE_ALLOW_ENV_KEK=1`. After reading, the variable is removed from -//! the Rust environment map (via `unsafe env::remove_var`). Zeroing the -//! underlying bytes in `/proc//environ` was attempted but is currently a -//! no-op (see [`zero_environ_entry`]) — the raw `environ` bytes are not -//! scrubbed, so this is a best-effort dev-mode control, not a guarantee. -//! -//! * [`Pkcs11KekStub`] — placeholder that always returns -//! [`CryptoError::Pkcs11NotImplemented`]. Reserves the production interface -//! so the abstraction boundary is locked in before the HSM is wired up. +//! Production KEK sources — PKCS#11 and TPM 2.0 — live in the separate +//! `storage-crypto-pkcs11` and `storage-crypto-tpm` crates (ADR 0016-v2 +//! §2.5), each implementing the same [`KekProvider`] trait defined here. use std::env; @@ -203,27 +202,6 @@ impl KekProvider for EnvKek { } } -// --------------------------------------------------------------------------- -// Pkcs11KekStub -// --------------------------------------------------------------------------- - -/// PKCS#11 HSM-backed KEK — interface stub only. -/// -/// All calls return [`CryptoError::Pkcs11NotImplemented`]. Reserves the -/// production abstraction boundary so callers can be written against the trait -/// without a live HSM. -pub struct Pkcs11KekStub; - -impl KekProvider for Pkcs11KekStub { - fn wrap_dek(&self, _dek: &[u8; 32]) -> Result, CryptoError> { - Err(CryptoError::Pkcs11NotImplemented) - } - - fn unwrap_dek(&self, _wrapped: &[u8]) -> Result, CryptoError> { - Err(CryptoError::Pkcs11NotImplemented) - } -} - // --------------------------------------------------------------------------- // Helpers // --------------------------------------------------------------------------- @@ -315,24 +293,6 @@ mod tests { assert!(decode_hex("abc").is_err()); } - #[test] - fn test_pkcs11_stub_wrap_not_implemented() { - let stub = Pkcs11KekStub; - assert!(matches!( - stub.wrap_dek(&[0u8; 32]), - Err(CryptoError::Pkcs11NotImplemented) - )); - } - - #[test] - fn test_pkcs11_stub_unwrap_not_implemented() { - let stub = Pkcs11KekStub; - assert!(matches!( - stub.unwrap_dek(&[0u8; 60]), - Err(CryptoError::Pkcs11NotImplemented) - )); - } - // `EnvKek::from_env` reads and removes a process-global environment // variable, so tests that touch it must be serialized against each other // to avoid racing on shared state. diff --git a/crates/storage-crypto/src/lib.rs b/crates/storage-crypto/src/lib.rs index 041f16f14..2c85c2d9b 100644 --- a/crates/storage-crypto/src/lib.rs +++ b/crates/storage-crypto/src/lib.rs @@ -16,8 +16,9 @@ //! Implements the cryptographic barrier described in ADR 0016-v2 §2: //! //! * **Key Encryption Key (KEK):** provided by [`kek::KekProvider`]. -//! Development mode uses [`kek::EnvKek`]; production uses -//! [`kek::Pkcs11KekStub`] (stub — full HSM integration is a future phase). +//! Development mode uses [`kek::EnvKek`]; production uses the PKCS#11 or +//! TPM 2.0 providers in the separate `storage-crypto-pkcs11` / +//! `storage-crypto-tpm` crates (ADR 0016-v2 §2.5). //! //! * **Data Encryption Key (DEK) hierarchy:** [`dek::DekEpoch`] holds the //! current epoch version and its HKDF-derived sub-keys ([`dek::LogDek`], @@ -55,6 +56,6 @@ pub use cipher::{ }; pub use dek::{BackupDek, DekEpoch, LogDek, StateDek, generate_dek}; pub use error::CryptoError; -pub use kek::{EnvKek, KekProvider, Pkcs11KekStub}; +pub use kek::{EnvKek, KekProvider}; pub use mlock::LockedKey; pub use nonce::{NonceManager, NoncePersistence}; diff --git a/crates/storage/Cargo.toml b/crates/storage/Cargo.toml index e5cb2919a..4c1cc37c8 100644 --- a/crates/storage/Cargo.toml +++ b/crates/storage/Cargo.toml @@ -53,6 +53,8 @@ openraft = { workspace = true, features = ["serde", "type-alias"] } openstack-keystone-config.workspace = true openstack-keystone-storage-api.workspace = true openstack-keystone-storage-crypto.workspace = true +openstack-keystone-storage-crypto-pkcs11 = { workspace = true, optional = true } +openstack-keystone-storage-crypto-tpm = { workspace = true, optional = true } prost.workspace = true rand.workspace = true rmp.workspace = true @@ -78,6 +80,7 @@ x509-parser.workspace = true [dev-dependencies] criterion = { workspace = true, features = ["async_tokio"] } +cryptoki = "0.12" rcgen.workspace = true reserve-port = "2.5.0" rustls.workspace = true @@ -99,3 +102,5 @@ tonic-prost-build.workspace = true default = [] bench_internals = [] mock = [] +pkcs11 = ["dep:openstack-keystone-storage-crypto-pkcs11"] +tpm = ["dep:openstack-keystone-storage-crypto-tpm"] diff --git a/crates/storage/src/app.rs b/crates/storage/src/app.rs index 196ac8c0c..206120e3e 100644 --- a/crates/storage/src/app.rs +++ b/crates/storage/src/app.rs @@ -19,7 +19,7 @@ use dashmap::DashMap; use eyre::eyre; use openraft::Config; use openraft::async_runtime::WatchReceiver; -use openstack_keystone_storage_crypto::{DekEpoch, EnvKek, KekProvider, Pkcs11KekStub}; +use openstack_keystone_storage_crypto::{DekEpoch, EnvKek, KekProvider}; use crate::protobuf as pb; use openraft::ReadPolicy; @@ -213,6 +213,150 @@ async fn verify_node_id_uniqueness_live( Ok(false) } +/// Build the Key Encryption Key provider selected by `ds_config.kek_provider` +/// (ADR 0016-v2 §2.1 / §2.5). +fn build_kek( + ds_config: &openstack_keystone_config::DistributedStorageConfiguration, +) -> Result, StoreError> { + use openstack_keystone_config::KekProvider as KekProviderKind; + + match ds_config.kek_provider { + KekProviderKind::Env => { + if !ds_config.dev_mode { + return Err(StoreError::Other(eyre!( + "kek_provider = \"env\" is only valid when dev_mode = true \ + (ADR 0016-v2 invariant 6)" + ))); + } + if std::env::var("KEYSTONE_ALLOW_ENV_KEK").as_deref() != Ok("1") { + return Err(StoreError::Other(eyre!( + "dev_mode is enabled but KEYSTONE_ALLOW_ENV_KEK=1 is not set; \ + refusing to start with an environment-provided KEK \ + (ADR 0016-v2 §2.1, invariant 6)" + ))); + } + let kek = EnvKek::from_env() + .map_err(|e| StoreError::Other(eyre!("failed to load KEYSTONE_DEV_KEK: {e}")))?; + Ok(Arc::new(kek)) + } + KekProviderKind::Pkcs11 => build_pkcs11_kek(ds_config), + KekProviderKind::Tpm => build_tpm_kek(ds_config), + } +} + +/// Open the PKCS#11 KEK from `ds_config.pkcs11` (ADR 0016-v2 §2.5.1). +/// +/// `auto_generate` is hardcoded to `false`: whether first-run +/// auto-provisioning of the AES key on the token is acceptable is an +/// operator/deployment decision (regulated environments may require an +/// out-of-band key ceremony instead), so it is not offered as an implicit +/// default here. A dedicated `keystone-manage` provisioning path is a +/// candidate follow-up if auto-provisioning in-process turns out to be +/// wanted. +#[cfg(feature = "pkcs11")] +fn build_pkcs11_kek( + ds_config: &openstack_keystone_config::DistributedStorageConfiguration, +) -> Result, StoreError> { + use secrecy::ExposeSecret; + + use openstack_keystone_storage_crypto_pkcs11::{Pkcs11Kek, Pkcs11KekParams, SlotSelector}; + + let cfg = ds_config.pkcs11.as_ref().ok_or_else(|| { + StoreError::Other(eyre!( + "kek_provider = \"pkcs11\" requires a [distributed_storage.pkcs11] section" + )) + })?; + let pin = cfg.pkcs11_pin_content.as_ref().ok_or_else(|| { + StoreError::Other(eyre!("PKCS#11 PIN was not loaded from pkcs11_pin_file")) + })?; + let slot = match (&cfg.pkcs11_slot_label, cfg.pkcs11_slot_id) { + (Some(label), _) => SlotSelector::Label(label.clone()), + (None, Some(id)) => SlotSelector::Id(id), + (None, None) => { + return Err(StoreError::Other(eyre!( + "[distributed_storage.pkcs11] requires either pkcs11_slot_id or \ + pkcs11_slot_label" + ))); + } + }; + + let kek = Pkcs11Kek::open(Pkcs11KekParams { + module_path: &cfg.pkcs11_module_path, + slot, + key_label: &cfg.pkcs11_key_label, + pin: pin.expose_secret(), + auto_generate: false, + }) + .map_err(|e| StoreError::Other(eyre!("failed to open PKCS#11 KEK: {e}")))?; + Ok(Arc::new(kek)) +} + +#[cfg(not(feature = "pkcs11"))] +fn build_pkcs11_kek( + _ds_config: &openstack_keystone_config::DistributedStorageConfiguration, +) -> Result, StoreError> { + Err(StoreError::Other(eyre!( + "kek_provider = \"pkcs11\" was selected but this build was compiled without the \ + `pkcs11` feature" + ))) +} + +/// Open the TPM KEK from `ds_config.tpm` (ADR 0016-v2 §2.5.2). +/// +/// `auto_generate` is hardcoded to `false` for the same reason as +/// [`build_pkcs11_kek`]. The AES/HMAC child-key pair is located via +/// `tpm_key_handle` (HMAC key at `handle + 1`) or `tpm_key_context_file` +/// (HMAC blobs at `.hmac`) — a convention owned by the +/// `storage-crypto-tpm` crate rather than a second config field, since +/// `TpmKekConfiguration` already carries exactly one key reference and +/// splitting it in two would only matter if an operator needed the two +/// child keys at independently chosen locations, which no deployment has +/// asked for. +#[cfg(feature = "tpm")] +fn build_tpm_kek( + ds_config: &openstack_keystone_config::DistributedStorageConfiguration, +) -> Result, StoreError> { + use secrecy::ExposeSecret; + + use openstack_keystone_storage_crypto_tpm::{KeyReference, TpmKek, TpmKekParams}; + + let cfg = ds_config.tpm.as_ref().ok_or_else(|| { + StoreError::Other(eyre!( + "kek_provider = \"tpm\" requires a [distributed_storage.tpm] section" + )) + })?; + let key_reference = match (cfg.tpm_key_handle, &cfg.tpm_key_context_file) { + (Some(handle), _) => KeyReference::PersistentHandle(handle), + (None, Some(path)) => KeyReference::ContextFile(path.clone()), + (None, None) => { + return Err(StoreError::Other(eyre!( + "[distributed_storage.tpm] requires either tpm_key_handle or \ + tpm_key_context_file" + ))); + } + }; + let auth = cfg.tpm_auth_content.as_ref().map(|s| s.expose_secret()); + + let kek = TpmKek::open(TpmKekParams { + tcti: &cfg.tpm_tcti, + key_reference, + auth, + auto_generate: false, + }) + .map_err(|e| StoreError::Other(eyre!("failed to open TPM KEK: {e}")))?; + Ok(Arc::new(kek)) +} + +#[cfg(not(feature = "tpm"))] +fn build_tpm_kek( + _ds_config: &openstack_keystone_config::DistributedStorageConfiguration, +) -> Result, StoreError> { + Err(StoreError::Other(eyre!( + "kek_provider = \"tpm\" was selected but this build was compiled without the `tpm` \ + feature" + ))) +} + /// Initialize storage services backed by the raft. /// /// # Parameters @@ -245,27 +389,11 @@ pub async fn init_storage(config_manager: &Arc) -> Result = if ds_config.dev_mode { - if std::env::var("KEYSTONE_ALLOW_ENV_KEK").as_deref() != Ok("1") { - return Err(StoreError::Other(eyre!( - "dev_mode is enabled but KEYSTONE_ALLOW_ENV_KEK=1 is not set; \ - refusing to start with an environment-provided KEK \ - (ADR 0016-v2 §2.1, invariant 6)" - ))); - } - Arc::new( - EnvKek::from_env() - .map_err(|e| StoreError::Other(eyre!("failed to load KEYSTONE_DEV_KEK: {e}")))?, - ) - } else { - Arc::new(Pkcs11KekStub) - }; + // `ds_config.kek_provider` drives the choice; config-time validation + // (`validate_kek_selection`) already rejects `env` outside `dev_mode` and + // missing `pkcs11`/`tpm` sections, but that is a fail-fast, not the sole + // enforcement point — `build_kek` re-checks at construction time too. + let kek: Arc = build_kek(&ds_config)?; // Run OS-level security pre-flight checks after key material is cleared. // In production mode (dev_mode = false) any failure is fatal per ADR diff --git a/crates/storage/tests/test_cluster.rs b/crates/storage/tests/test_cluster.rs index 987c5cea5..e77491919 100644 --- a/crates/storage/tests/test_cluster.rs +++ b/crates/storage/tests/test_cluster.rs @@ -93,10 +93,11 @@ fn test_node_restart_with_address_format_change() { TypeConfig::run(test_node_restart_inner()).unwrap(); } -/// `init_storage` must refuse to start in production mode (`dev_mode = -/// false`): there is currently no production `KekProvider` (HSM/PKCS#11/KMS) -/// wired up, so falling back to an environment-provided KEK would silently -/// violate ADR 0016-v2 §2.1 / invariant 6. +/// `init_storage` must refuse to start with `kek_provider = "env"` (the +/// default) when `dev_mode = false`: the dev-mode `EnvKek` is never a valid +/// production KEK source, so falling back to it would silently violate ADR +/// 0016-v2 §2.1 / invariant 6. Production deployments select `"pkcs11"` or +/// `"tpm"` instead (`test_pkcs11_cluster.rs` covers that path end-to-end). #[serial_test::serial] #[tracing_test::traced_test] #[test] diff --git a/crates/storage/tests/test_pkcs11_cluster.rs b/crates/storage/tests/test_pkcs11_cluster.rs new file mode 100644 index 000000000..1e2cc8c0d --- /dev/null +++ b/crates/storage/tests/test_pkcs11_cluster.rs @@ -0,0 +1,452 @@ +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. +// +// SPDX-License-Identifier: Apache-2.0 +#![cfg(feature = "pkcs11")] +#![allow(clippy::uninlined_format_args)] +#![allow(clippy::unwrap_used)] +#![allow(clippy::expect_used)] +#![allow(clippy::print_stdout)] +//! SoftHSM2-backed end-to-end integration test (ADR 0016-v2 §2.5.1, +//! implementation plan step 6). +//! +//! `crates/storage-crypto-pkcs11/tests/softhsm.rs` already covers +//! `Pkcs11Kek::wrap_dek`/`unwrap_dek` in isolation. This file instead boots a +//! real single-node cluster through `init_storage` with +//! `kek_provider = "pkcs11"`, exercising the whole path config -> `build_kek` +//! -> `Pkcs11Kek` -> the Raft state machine's DEK wrap/unwrap, and does an +//! end-to-end write/read of sensitive-tier data plus a restart that reopens +//! the same real token. +//! +//! Requires a SoftHSM2 module to be installed (`apt-get install softhsm2`, +//! same install step CI already runs for SPIRE). The module path is taken +//! from `TEST_PKCS11_MODULE` if set, otherwise the common Debian/Ubuntu path +//! is tried; tests skip (rather than fail) when no module is found. + +use std::path::PathBuf; +use std::time::Duration; + +use cryptoki::context::{CInitializeArgs, CInitializeFlags, Pkcs11}; +use cryptoki::session::UserType; +use cryptoki::types::AuthPin; +use eyre::Result; +use openraft::type_config::TypeConfigExt; +use rcgen::{ + BasicConstraints, CertificateParams, DistinguishedName, DnType, ExtendedKeyUsagePurpose, IsCa, + Issuer, KeyPair, KeyUsagePurpose, SanType, +}; + +use openstack_keystone_config::{ + Config, ConfigManager, DistributedStorageConfiguration, KekProvider, Pkcs11KekConfiguration, + RaftTlsConfiguration, TlsConfiguration, TlsConfigurationBuilder, +}; +use openstack_keystone_distributed_storage::TypeConfig; +use openstack_keystone_distributed_storage::app::init_storage; +use openstack_keystone_distributed_storage::{ + DataTier, Metadata, StorageApi, StoreDataEnvelope, StoreError, +}; +use openstack_keystone_storage_crypto_pkcs11::{Pkcs11Kek, Pkcs11KekParams, SlotSelector}; + +const SO_PIN: &str = "1234567890"; +const USER_PIN: &str = "fedcba0987"; +const TOKEN_LABEL: &str = "keystone-storage-test"; +const KEY_LABEL: &str = "keystone-kek"; + +fn make_sensitive_env( + value: &T, +) -> Result>, StoreError> { + Ok(StoreDataEnvelope { + data: rmp_serde::to_vec(value)?, + metadata: Metadata::with_tier(DataTier::Sensitive), + }) +} + +fn module_path() -> Option { + if let Ok(p) = std::env::var("TEST_PKCS11_MODULE") { + return Some(PathBuf::from(p)); + } + [ + "/usr/lib/softhsm/libsofthsm2.so", + "/usr/lib/x86_64-linux-gnu/softhsm/libsofthsm2.so", + "/usr/local/lib/softhsm/libsofthsm2.so", + ] + .into_iter() + .map(PathBuf::from) + .find(|p| p.exists()) +} + +/// `Result::unwrap()`/`expect()` are denied by the crate's clippy lints for +/// `#[test]`-attributed functions, but these setup helpers aren't themselves +/// `#[test]` functions, so clippy doesn't recognize them as test code. +/// Panicking on setup failure is still exactly what's wanted here — this +/// just spells it without the denied methods. +fn ok(result: Result, context: &str) -> T { + match result { + Ok(v) => v, + Err(e) => panic!("{context}: {e}"), + } +} + +fn skip_without_softhsm() -> bool { + if module_path().is_none() { + eprintln!( + "skipping: no SoftHSM2 module found (set TEST_PKCS11_MODULE or `apt-get install softhsm2`)" + ); + return true; + } + false +} + +/// Initialize a fresh SoftHSM2 token and provision the AES-256 KEK key +/// object on it. `build_pkcs11_kek` (`crates/storage/src/app.rs`) always +/// opens with `auto_generate: false` — first-run key creation must not +/// happen implicitly in-process (ADR 0016-v2 review decision, see +/// `doc/plans/0016-v2-pkcs11-tpm-kek.md`) — so the key must already exist +/// before `init_storage` runs, exactly as an operator's out-of-band key +/// ceremony would provision it. +fn provision_token(module: &PathBuf) { + // SoftHSM only tolerates one live `C_Initialize`'d context per process + // for a given module, so the admin context used for token/PIN setup is + // scoped to this block and dropped (finalizing it) before `Pkcs11Kek` + // opens its own context to provision the AES key below. + { + let pkcs11 = ok(Pkcs11::new(module), "load module"); + ok( + pkcs11.initialize(CInitializeArgs::new(CInitializeFlags::OS_LOCKING_OK)), + "initialize", + ); + let slot = ok(pkcs11.get_slots_with_token(), "slots")[0]; + ok( + pkcs11.init_token(slot, &AuthPin::new(SO_PIN.into()), TOKEN_LABEL), + "init token", + ); + let session = ok(pkcs11.open_rw_session(slot), "so session"); + ok( + session.login(UserType::So, Some(&AuthPin::new(SO_PIN.into()))), + "so login", + ); + ok( + session.init_pin(&AuthPin::new(USER_PIN.into())), + "init user pin", + ); + } + + // Provision the AES key itself, as an operator's out-of-band ceremony + // would; `init_storage` will only ever open it with `auto_generate: false`. + ok( + Pkcs11Kek::open(Pkcs11KekParams { + module_path: module, + slot: SlotSelector::Label(TOKEN_LABEL.into()), + key_label: KEY_LABEL, + pin: USER_PIN.as_bytes(), + auto_generate: true, + }), + "provision KEK key", + ); +} + +fn make_certificates() -> Result { + let mut ca_params = CertificateParams::default(); + ca_params.is_ca = IsCa::Ca(BasicConstraints::Unconstrained); + ca_params.key_usages = vec![ + KeyUsagePurpose::KeyCertSign, + KeyUsagePurpose::DigitalSignature, + KeyUsagePurpose::CrlSign, + ]; + + let mut ca_dn = DistinguishedName::new(); + ca_dn.push(DnType::CommonName, "CA"); + ca_params.distinguished_name = ca_dn; + + let ca_key = KeyPair::generate()?; + let ca_cert = ca_params.self_signed(&ca_key)?; + let ca = Issuer::new(ca_params, ca_key); + + let mut peer_cert_params = CertificateParams::default(); + let now = time::OffsetDateTime::now_utc(); + peer_cert_params.not_before = now - time::Duration::days(1); + peer_cert_params.not_after = now + time::Duration::days(28); + + let client_ip: std::net::IpAddr = "127.0.0.1".parse()?; + peer_cert_params.subject_alt_names = vec![SanType::IpAddress(client_ip)]; + peer_cert_params.key_usages = vec![KeyUsagePurpose::DigitalSignature]; + peer_cert_params.extended_key_usages = vec![ + ExtendedKeyUsagePurpose::ServerAuth, + ExtendedKeyUsagePurpose::ClientAuth, + ]; + let peer_key = KeyPair::generate()?; + let peer_cert = peer_cert_params.signed_by(&peer_key, &ca)?; + + Ok(TlsConfigurationBuilder::default() + .tls_client_ca_content(ca_cert.pem().as_bytes().to_vec()) + .tls_cert_content(peer_cert.pem().as_bytes().to_vec()) + .tls_key_content(peer_key.serialize_pem().as_bytes().to_vec()) + .build()?) +} + +/// Config for a single, unnetworked node backed by the real SoftHSM2 token +/// at `module`. No gRPC server is started — as in `test_cluster.rs`'s +/// `test_quarantine_committed_via_raft`, the Raft handle is exercised +/// directly, which is enough to drive the state machine's DEK wrap/unwrap +/// path without needing a live network listener. +fn pkcs11_ds_config( + node_id: u64, + db_path: PathBuf, + tls_config: TlsConfiguration, + module: PathBuf, +) -> DistributedStorageConfiguration { + let addr = format!("127.0.0.1:{}", 22000 + node_id as u16); + DistributedStorageConfiguration { + node_cluster_addr: format!("https://{addr}").parse().expect("valid address"), + node_listener_addr: addr.parse().expect("valid address"), + node_id, + path: db_path, + tls_configuration: RaftTlsConfiguration::Tls(tls_config), + dev_mode: false, + retry_join_nodes: vec![], + kek_provider: KekProvider::Pkcs11, + pkcs11: Some(Pkcs11KekConfiguration { + pkcs11_key_label: KEY_LABEL.to_string(), + pkcs11_module_path: module, + pkcs11_pin_content: Some(USER_PIN.as_bytes().to_vec().into()), + // Unused: `build_pkcs11_kek` reads `pkcs11_pin_content` (already + // populated above), not this file — `Config::load_all` is the + // only caller that reads `pkcs11_pin_file`, and this test builds + // `Config` directly, bypassing it. + pkcs11_pin_file: PathBuf::new(), + pkcs11_slot_id: None, + pkcs11_slot_label: Some(TOKEN_LABEL.to_string()), + }), + tpm: None, + } +} + +/// Set up an isolated SoftHSM2 token directory/config, provision the KEK key +/// on it, and run `f` — a synchronous closure that may itself block on an +/// async runtime via [`TypeConfig::run`] — with `SOFTHSM2_CONF` pointed at +/// the token for the duration. `temp_env::with_var` requires a synchronous +/// closure, so `f` must fully block on any async work itself rather than +/// return a `Future`; this mirrors every other test in this crate, which +/// calls `TypeConfig::run` directly from a plain `#[test]` fn rather than +/// nesting async runtimes. Returns `None` if no SoftHSM2 module is available +/// (caller should treat that as a skipped test). +fn with_provisioned_token(f: impl FnOnce(&PathBuf) -> R) -> Option { + let module = module_path()?; + let dir = ok(tempfile::TempDir::new(), "tempdir"); + let token_dir = dir.path().join("tokens"); + ok(std::fs::create_dir_all(&token_dir), "mkdir tokens"); + let conf_path = dir.path().join("softhsm2.conf"); + ok( + std::fs::write( + &conf_path, + format!("directories.tokendir = {}\n", token_dir.display()), + ), + "write conf", + ); + + let result = temp_env::with_var("SOFTHSM2_CONF", Some(conf_path.as_os_str()), || { + provision_token(&module); + f(&module) + }); + Some(result) +} + +/// Boots a single-node cluster with `kek_provider = "pkcs11"` and does an +/// end-to-end write/read of sensitive-tier data — the DEK that encrypts the +/// value is itself wrapped/unwrapped by the real `Pkcs11Kek` against a real +/// SoftHSM2 token, not the dev-mode `EnvKek`. +#[serial_test::serial] +#[tracing_test::traced_test] +#[test] +fn test_pkcs11_backed_cluster_write_read() { + if skip_without_softhsm() { + return; + } + let provider = rustls::crypto::aws_lc_rs::default_provider(); + let _ = rustls::crypto::CryptoProvider::install_default(provider); + + with_provisioned_token(|module| { + TypeConfig::run(test_pkcs11_backed_cluster_write_read_inner(module.clone())).unwrap(); + }); +} + +async fn test_pkcs11_backed_cluster_write_read_inner(module: PathBuf) -> Result<()> { + let storage_dir = tempfile::TempDir::new().unwrap(); + let tls_configuration = make_certificates()?; + let ds_config = pkcs11_ds_config( + 301, + storage_dir.path().to_path_buf(), + tls_configuration, + module, + ); + let config = Config { + distributed_storage: Some(ds_config), + ..Default::default() + }; + + let storage = init_storage(&ConfigManager::not_watched(config)) + .await + .expect("init_storage with a real SoftHSM2-backed KEK"); + + storage + .initialize( + [( + 301u64, + openstack_keystone_storage_api::Node { + node_id: 301, + rpc_addr: "127.0.0.1:0".to_string(), + }, + )] + .into_iter() + .collect(), + ) + .await?; + for _ in 0..50 { + if storage.current_leader() == Some(301) { + break; + } + TypeConfig::sleep(Duration::from_millis(50)).await; + } + assert_eq!(storage.current_leader(), Some(301)); + + let key = "pkcs11-e2e-key".to_string(); + let value = "pkcs11-e2e-value".to_string(); + storage + .set_value(key.clone(), make_sensitive_env(&value)?, None, None) + .await?; + + let got = storage + .get_by_key(key.as_bytes(), None) + .await? + .expect("value should be readable back through the real PKCS#11 KEK"); + assert_eq!( + value, + got.try_deserialize::()?.data, + "round-tripped value must match what was written" + ); + + storage.raft.shutdown().await.ok(); + Ok(()) +} + +/// A wrapped DEK persisted while the token was open must still unwrap after +/// a full storage shutdown and restart that reopens the same SoftHSM2 token +/// from scratch — proving the wrapped-DEK-on-disk format doesn't depend on +/// anything beyond the token itself (no in-memory session state leaking +/// into what gets persisted). +#[serial_test::serial] +#[tracing_test::traced_test] +#[test] +fn test_pkcs11_backed_cluster_restart_reopens_token() { + if skip_without_softhsm() { + return; + } + let provider = rustls::crypto::aws_lc_rs::default_provider(); + let _ = rustls::crypto::CryptoProvider::install_default(provider); + + with_provisioned_token(|module| { + TypeConfig::run(test_pkcs11_backed_cluster_restart_reopens_token_inner( + module.clone(), + )) + .unwrap(); + }); +} + +async fn test_pkcs11_backed_cluster_restart_reopens_token_inner(module: PathBuf) -> Result<()> { + let storage_dir = tempfile::TempDir::new().unwrap(); + let tls_configuration = make_certificates()?; + let key = "pkcs11-restart-key".to_string(); + let value = "pkcs11-restart-value".to_string(); + + { + let ds_config = pkcs11_ds_config( + 302, + storage_dir.path().to_path_buf(), + tls_configuration.clone(), + module.clone(), + ); + // Unlike the single-shot quarantine/rotation tests, this test + // restarts against the same config, so the registered rpc_addr must + // match `node_cluster_addr` (mod scheme) for + // `check_node_id_uniqueness` to recognize the restart as the same + // node rather than a conflicting one — a bare "127.0.0.1:0" + // placeholder (fine for a test that never restarts) would not. + let rpc_addr = ds_config.node_listener_addr.to_string(); + let config = Config { + distributed_storage: Some(ds_config), + ..Default::default() + }; + let storage = init_storage(&ConfigManager::not_watched(config)) + .await + .expect("first init_storage"); + + storage + .initialize( + [( + 302u64, + openstack_keystone_storage_api::Node { + node_id: 302, + rpc_addr, + }, + )] + .into_iter() + .collect(), + ) + .await?; + for _ in 0..50 { + if storage.current_leader() == Some(302) { + break; + } + TypeConfig::sleep(Duration::from_millis(50)).await; + } + assert_eq!(storage.current_leader(), Some(302)); + + storage + .set_value(key.clone(), make_sensitive_env(&value)?, None, None) + .await?; + + // Fjall's file lock is only released once RaftCore's background + // task actually stops (its Drop impl doesn't do this for us) — same + // requirement documented on + // `test_node_restart_with_address_format_change`. + storage.raft.shutdown().await.ok(); + } + TypeConfig::sleep(Duration::from_millis(200)).await; + + let ds_config_restart = pkcs11_ds_config( + 302, + storage_dir.path().to_path_buf(), + tls_configuration, + module, + ); + let config_restart = Config { + distributed_storage: Some(ds_config_restart), + ..Default::default() + }; + let storage_restart = init_storage(&ConfigManager::not_watched(config_restart)) + .await + .expect("restart must reopen the same SoftHSM2 token and unwrap the persisted DEK"); + + assert!( + storage_restart.is_initialized().await?, + "storage should detect persisted cluster state" + ); + + let got = storage_restart + .get_by_key(key.as_bytes(), None) + .await? + .expect("value written before restart should still be readable after it"); + assert_eq!(value, got.try_deserialize::()?.data); + + storage_restart.raft.shutdown().await.ok(); + Ok(()) +} diff --git a/deny.toml b/deny.toml index e4579b399..c62b6aa35 100644 --- a/deny.toml +++ b/deny.toml @@ -204,6 +204,13 @@ allow = [ # due to Cargo feature unification (reqwest 0.12 enables __rustls-ring). # When openidconnect or its deps update to reqwest 0.13 with aws-lc-rs support, # this exception can be reconsidered. +# +# NOTE: ADR 0016-v2 §2.5's `cryptoki` (storage-crypto-pkcs11) and `tss-esapi` +# (storage-crypto-tpm) dependency trees were reviewed for this addendum: every +# new transitive crate they bring in (bitfield, enumflags2, mbox, num-derive, +# oid, picky-asn1[-x509], secrecy, libloading, etc.) is MIT, Apache-2.0, or +# ISC — already covered by [licenses] allow above — so no new bans or +# licenses.exceptions entries were needed. deny = [ #"ansi_term@0.11.0", #{ crate = "ansi_term@0.11.0", reason = "you can specify a reason it is banned" }, diff --git a/doc/plans/0016-v2-pkcs11-tpm-kek.md b/doc/plans/0016-v2-pkcs11-tpm-kek.md index b3039e4d7..af1111c17 100644 --- a/doc/plans/0016-v2-pkcs11-tpm-kek.md +++ b/doc/plans/0016-v2-pkcs11-tpm-kek.md @@ -1,6 +1,6 @@ # Implementation plan: PKCS#11 and TPM KEK providers (ADR 0016-v2 §2.5) -Status: **steps 1-2 complete**, steps 3-10 not started. +Status: **steps 1-9 complete**. Plan finished. This is a working implementation plan, not user-facing documentation — it is intentionally not linked from `doc/src/SUMMARY.md`. The durable design record @@ -11,10 +11,13 @@ how that design gets built. ADR 0016-v2 §2.1 named "HSM / PKCS#11 / Cloud KMS" as the production KEK source but never specified a mechanism. `crates/storage-crypto/src/kek.rs` -already reserves the abstraction boundary: the `KekProvider` trait, a working -`EnvKek` (dev-mode only), and a `Pkcs11KekStub` that always returns -`CryptoError::Pkcs11NotImplemented`. No TPM path exists at all. This plan -replaces the stub with real PKCS#11 and TPM 2.0 providers. +already reserved the abstraction boundary: the `KekProvider` trait, a working +`EnvKek` (dev-mode only), and (until step 4) a `Pkcs11KekStub` placeholder +that always returned `CryptoError::Pkcs11NotImplemented`. No TPM path existed +at all. This plan replaced the stub with real PKCS#11 and TPM 2.0 providers; +`Pkcs11KekStub` and `CryptoError::Pkcs11NotImplemented` were deleted in step 5 +once `crates/storage/src/app.rs` no longer needed a placeholder to fall back +to. ## Decisions taken @@ -69,42 +72,163 @@ Full mechanism detail, wire formats, and new invariants (13-15) are in reference is exactly one of handle/context-file) + file-based secret loading wired into `Config::load_all`. ✅ done (`crates/config/src/distributed_storage.rs`, `crates/config/src/lib.rs`). -3. **`storage-crypto-pkcs11` crate**: `cryptoki` dependency, `Pkcs11Kek` - implementing `KekProvider`, optional auto-generate-key-if-missing on first - startup, unit tests against SoftHSM2 (token init, wrap/unwrap round-trip, - tamper/tag-corruption rejection, wrong-PIN failure). ⬜ not started -4. **`storage-crypto-tpm` crate**: `tss-esapi` dependency, `TpmKek` - implementing `KekProvider` (Encrypt-then-MAC per §2.5.2), example under - `examples/tpm_kek_demo.rs` targeting `swtpm`. ⬜ not started -5. **Wire into `crates/storage/src/app.rs`**: replace the hardcoded - `dev_mode ? EnvKek : Pkcs11KekStub` selection with one driven by - `ds_config.kek_provider`; remove the implicit stub fallback so production - requires an explicit, valid provider selection (already enforced at - config-validation time by step 2, but must also be enforced at runtime - construction). Feature flags `pkcs11` / `tpm` on the `storage` crate pull - in the new provider crates. ⬜ not started -6. **SoftHSM-backed integration test**: new file in `crates/storage/tests/` - (feature-gated `pkcs11`) that boots a single-node cluster with a - SoftHSM-backed KEK and does an end-to-end write/read, beyond the - unit-level wrap/unwrap coverage in step 3. ⬜ not started -7. **CI**: add a "Install SoftHSM2 + init test token" step to - `.github/workflows/ci.yml` (apt-get, same pattern as the SPIRE binary - install) ahead of running the `pkcs11`-featured tests. Compile (but do not - run) the TPM example in CI to catch rot, without adding `swtpm` as a hard - CI dependency. ⬜ not started -8. **Docs**: update `doc/src/raft_storage.md`'s crate-layout tree and key - hierarchy section, and `CONTRIBUTING.md`'s crate-purpose table, with the - two new crates. Add the TPM sample walkthrough. ⬜ not started -9. **Supply chain**: add `cryptoki` and `tss-esapi` (pinned versions) to the - extended cargo-vet coverage list in ADR §1, and check `deny.toml` for any - new transitive-dependency rules needed. ⬜ not started - -## Open questions for step 3 onward - -- Exact `cryptoki` and `tss-esapi` version pins, and whether either requires - system packages beyond what CI/dev containers currently install - (`libp11-kit`, `tpm2-tss` headers, etc.) — needs a spike before locking - `Cargo.toml` entries. -- Whether the PKCS#11 auto-generate-key-on-first-use convenience (step 3) - should be gated behind its own config flag, given it changes the operator - key-ceremony story for regulated deployments. +3. **`storage-crypto-pkcs11` crate**: `cryptoki` 0.12 dependency (pure-Rust, + `dlopen`s the module at runtime — no system PKCS#11 headers needed to + build), `Pkcs11Kek` implementing `KekProvider` over `CKM_AES_GCM`, with + `auto_generate: bool` as an explicit `Pkcs11KekParams` field (caller opts + in rather than it being an implicit default — resolves the corresponding + open question below), slot selection by numeric id or token label. 8 + SoftHSM2-backed integration tests (round-trip, distinct nonces per wrap, + tampered-tag rejection, tampered-ciphertext rejection, key reuse across + sessions, wrong-PIN failure, missing-key-without-auto-generate failure, + slot-by-id vs slot-by-label equivalence) — all passing against a real + `libsofthsm2.so`. ✅ done + (`crates/storage-crypto-pkcs11/`). +4. **`storage-crypto-tpm` crate**: `tss-esapi` 7.7 dependency (requires the + system `tpm2-tss` library + headers via `pkg-config`, unlike `cryptoki` — + resolves the corresponding open question below), `TpmKek` implementing + `KekProvider` via Encrypt-then-MAC (AES-256-CFB `TPM2_EncryptDecrypt2` + + HMAC-SHA256 `TPM2_HMAC`, constant-time tag comparison before decrypting, + per §2.5.2). Both child keys are `fixedTPM | fixedParent` and + `sensitiveDataOrigin` (TPM-generated, non-extractable), children of a + primary that's recreated deterministically each `open()` rather than + persisted. `KeyReference::PersistentHandle(u32)` provisions the AES key at + that handle and the HMAC key at `handle + 1`; `KeyReference::ContextFile` + stores each key's TPM-encrypted `(public, private)` blob pair at a path / + `.hmac`. `examples/tpm_kek_demo.rs` exercises both key-reference + modes and tamper rejection — not CI-gated (compiled only, per the decided + test/sample scope), but manually verified against a real `swtpm` instance + for this change, including cross-process-restart reload and wrong-auth + rejection. ✅ done + (`crates/storage-crypto-tpm/`). +5. **Wire into `crates/storage/src/app.rs`**: replaced the hardcoded + `dev_mode ? EnvKek : Pkcs11KekStub` selection with `build_kek`, driven by + `ds_config.kek_provider` (`Env`/`Pkcs11`/`Tpm`), with per-provider builder + functions. `Pkcs11KekStub` and `CryptoError::Pkcs11NotImplemented` were + deleted outright (dead once nothing falls back to them) rather than kept + around as an unused placeholder. New `pkcs11` / `tpm` feature flags on the + `storage` crate gate the optional dependency on the corresponding provider + crate; selecting a provider whose feature isn't compiled in fails at + `build_kek` construction time with a clear error rather than failing to + compile or silently falling back. `auto_generate` is hardcoded `false` for + both providers (resolves the corresponding open question below); PIN/auth + secrets come from the config's already-loaded `SecretSlice` content + (`Config::load_all` reads `pkcs11_pin_file` / `tpm_auth_file` before + `app.rs` ever sees the config). ✅ done + (`crates/storage/src/app.rs`, `crates/storage/Cargo.toml`). +6. **SoftHSM-backed integration test**: `crates/storage/tests/test_pkcs11_cluster.rs` + (feature-gated `pkcs11`, whole file behind `#![cfg(feature = "pkcs11")]`) + provisions a real SoftHSM2 token (via the `cryptoki` crate directly, now a + dev-dependency of `storage`, matching `storage-crypto-pkcs11`'s own test + style) and the AES-256 KEK key on it — out-of-band, as `build_pkcs11_kek`'s + hardcoded `auto_generate: false` requires — then boots a real single-node + cluster through `init_storage` with `kek_provider = "pkcs11"` and: + - `test_pkcs11_backed_cluster_write_read`: end-to-end write/read of + sensitive-tier data, exercising config -> `build_kek` -> `Pkcs11Kek` -> + the Raft state machine's DEK wrap/unwrap, beyond the crate-level + wrap/unwrap unit coverage in step 3. + - `test_pkcs11_backed_cluster_restart_reopens_token`: writes data, does a + full storage shutdown (`raft.shutdown().await`, same Fjall-lock-release + requirement as the existing `test_node_restart_with_address_format_change`), + then restarts against the same on-disk state and the same real token — + proving the wrapped-DEK-on-disk format survives a full process restart + and doesn't depend on anything the first PKCS#11 session held in memory. + + Also corrected a stale doc comment on + `test_kek_gating_production_mode_rejected` (`test_cluster.rs`) that still + claimed no production `KekProvider` existed — no longer true since step 5. + ✅ done (`crates/storage/tests/test_pkcs11_cluster.rs`, + `crates/storage/Cargo.toml`, `crates/storage/tests/test_cluster.rs`). +7. **CI**: `cargo nextest run --all-features --profile ci` in + `.github/workflows/ci.yml`'s `test` job already builds both new crates on + every run (it was silently relying on the runner happening to have the + right system packages — true in this dev environment, not guaranteed on a + stock `ubuntu-latest` runner). Added: + - **`libtss2-dev` + `pkg-config`** (apt-get, unconditional): build-time + requirement for `tss-esapi` to compile at all — needed regardless of + whether any TPM test runs, since `--all-features` always compiles the + `tpm` feature. + - **`softhsm2`** (apt-get): provides `libsofthsm2.so` so the `pkcs11` + integration tests (`storage-crypto-pkcs11/tests/softhsm.rs`, + `storage/tests/test_pkcs11_cluster.rs`) actually run instead of + self-skipping. No system-wide token/PIN setup step needed — every test + already provisions its own isolated token via `SOFTHSM2_CONF` (step + 3/6), so this is purely a package install. + - **"Build TPM KEK example"** step: `cargo build -p + openstack-keystone-storage-crypto-tpm --example tpm_kek_demo`. + `cargo nextest run` doesn't build `examples/` on its own, so without + this the sample could silently rot; per the decided test/sample scope + it's compiled only, not executed — no `swtpm` install in CI. + + Verified with `cargo check --workspace --all-features --tests`, which + exercises the same build surface `nextest` does. ✅ done + (`.github/workflows/ci.yml`). +8. **Docs**: `doc/src/raft_storage.md`'s crate-layout tree now lists + `storage-crypto-pkcs11` and `storage-crypto-tpm` alongside their key + files/tests; the intro paragraph and Key Hierarchy diagram's KEK-source + line were updated to name both production providers instead of a generic + "HSM / Cloud KMS". Added a new "PKCS#11 and TPM KEK Providers" subsection + (linked from the ToC and from a new Prerequisites bullet) documenting the + `[distributed_storage.pkcs11]` / `[distributed_storage.tpm]` config blocks, + the out-of-band key-provisioning requirement (`auto_generate: false` per + step 5), how to run the SoftHSM2-backed tests locally, and a walkthrough of + `crates/storage-crypto-tpm/examples/tpm_kek_demo.rs` against `swtpm` + (mirroring the example's own doc comment). `CONTRIBUTING.md`'s workspace + structure table gained rows for `storage-crypto`, `storage-crypto-pkcs11`, + and `storage-crypto-tpm` (none had an entry before). ✅ done + (`doc/src/raft_storage.md`, `CONTRIBUTING.md`). +9. **Supply chain**: extended the ADR §1 Supply Chain paragraph's + `cargo-vet` coverage list with `cryptoki` (pinned `0.12.0`) and + `tss-esapi` (pinned `7.7.0`), explaining why they meet criteria (a)/(b) + (both transiently hold the unwrapped DEK during `wrap_dek`/`unwrap_dek` + and sit directly in the KEK call path). Checked every new transitive + dependency `cryptoki`/`tss-esapi` bring in (via `cargo metadata` + + `cargo tree --all-features`) against `deny.toml`'s license allow-list — + all are MIT/Apache-2.0/ISC, already allowed, so no new `deny.toml` rules + were needed; added a comment recording that review so it isn't silently + re-derived later. ✅ done + (`doc/src/adr/0016-v2-raft-storage.md`, `deny.toml`). + +## Open questions resolved in step 5 + +- `auto_generate`: hardcoded `false` in both `build_pkcs11_kek` and + `build_tpm_kek` (`crates/storage/src/app.rs`) rather than given a config + key. First-run auto-provisioning bypasses any out-of-band key ceremony a + regulated deployment might require, so it's not offered as an implicit + production default; revisit only if an actual deployment asks for it (a + dedicated `keystone-manage` provisioning command is the more likely shape + for that, not a config flag toggling behaviour inside `init_storage`). +- `storage-crypto-tpm::KeyReference::PersistentHandle`'s `handle + 1` (AES / + HMAC) convention was carried into `app.rs`'s config-to-params translation + as-is, not promoted to a second `TpmKekConfiguration` field. + `TpmKekConfiguration` continues to carry exactly one key reference; nothing + about the ADR or the initial deployment target needs the two child keys at + independently chosen locations. + +## Open questions resolved in step 7 + +- CI system-package needs, resolved as anticipated: `cryptoki` needed nothing + beyond the module itself (pure Rust, `dlopen`s at runtime, so `pkcs11`'s + only CI need is `softhsm2` for the module file); `tss-esapi` needed + `libtss2-dev` + `pkg-config` at build time, installed unconditionally since + `--all-features` always compiles it. `swtpm` was not added to CI, matching + the decided test/sample scope (TPM has no CI-gated tests). + +## Open questions resolved in step 8 + +None arose — the docs work was mechanical (crate-layout tree, a new config +walkthrough section, a workspace-structure table row) with no design +decisions to make. + +## Open questions resolved in step 9 + +None arose — every transitive dependency `cryptoki`/`tss-esapi` bring in was +already under a license `deny.toml` already allows, so no bans/exceptions +needed adding. + +## Plan complete + +All 9 steps are done: ADR addendum, config schema, `storage-crypto-pkcs11`, +`storage-crypto-tpm`, `app.rs` wiring, the SoftHSM2-backed cluster test, CI +system-package wiring, docs, and supply-chain coverage. No further steps are +planned. diff --git a/doc/src/adr/0016-v2-raft-storage.md b/doc/src/adr/0016-v2-raft-storage.md index 4301458ae..2472a4b60 100644 --- a/doc/src/adr/0016-v2-raft-storage.md +++ b/doc/src/adr/0016-v2-raft-storage.md @@ -91,7 +91,12 @@ dependency. `cargo-vet` or equivalent is used in CI for these two crates specifically. `cargo deny` rules reject any transitive dependency that stores key material without implementing `ZeroizeOnDrop`. `cargo-vet` coverage is extended to all crates that directly handle key material or ciphertext: the -AES-GCM provider, the mlock wrapper, and the HKDF implementation. Any new +AES-GCM provider, the mlock wrapper, the HKDF implementation, and — per the +§2.5 PKCS#11/TPM addendum — the two production KEK-provider crates' HSM/TPM +client libraries: `cryptoki` (pinned `0.12.0`, `storage-crypto-pkcs11`) and +`tss-esapi` (pinned `7.7.0`, `storage-crypto-tpm`). Both hold the unwrapped DEK +transiently during `wrap_dek`/`unwrap_dek` and sit directly in the KEK call +path, so they meet criteria (a) and (b) below. Any new dependency falls under extended cargo-vet coverage if it: (a) receives or stores a `Zeroizing` value; (b) is in the AES-HKDF-KMS call path; (c) provides `mlock` or `VirtualLock` functionality; or (d) processes raw ciphertext before @@ -260,10 +265,14 @@ data, and a 128-bit tag). The resulting wire format is byte-identical to this is an ergonomic convenience for fresh clusters, not a substitute for operator-controlled key ceremony in regulated deployments. - **PIN handling:** the token PIN is read once at startup from - `pkcs11_pin_file` into a `Zeroizing` buffer, used for `C_Login`, and - zeroed immediately after. The PIN is never accepted via environment - variable or inline config value — only a file path, consistent with the - existing `tls_key_file`/`tls_cert_file` convention (§4.2). + `pkcs11_pin_file` into a `SecretSlice` (zeroized on `Drop`), used for + `C_Login`. It is held for the duration of the `init_storage` call — not + scrubbed the instant `C_Login` returns, since the same in-memory config + clone is threaded through the rest of node startup — and zeroized once + that call completes and the clone is dropped. The PIN is never accepted + via environment variable or inline config value — only a file path, + consistent with the existing `tls_key_file`/`tls_cert_file` convention + (§4.2). - **Failure handling:** a login failure, missing key object, or GCM tag mismatch on unwrap is fatal to node startup (or, post-startup, treated the same as any other GCM tag failure under invariant 5's quarantine logic). @@ -307,11 +316,13 @@ non-duplicable, non-extractable attributes above. Provisioning itself (via `tpm2_create`/`tpm2_evictcontrol` or equivalent) is an out-of-band operator step, documented alongside the PKCS#11 key ceremony. - **Auth handling:** if the key was provisioned with `userWithAuth`, the auth - value is read once from `tpm_auth_file` into a `Zeroizing` buffer and used - to authorize the TPM session; zeroed immediately after. As with PKCS#11, - only a file path is accepted, never an environment variable or inline - value. A key relying purely on PCR/policy session authorization may omit - `tpm_auth_file`. + value is read once from `tpm_auth_file` into a `SecretSlice` (zeroized on + `Drop`) and used to authorize the TPM session. As with the PKCS#11 PIN + (§2.5.1), it is held for the duration of the `init_storage` call rather + than scrubbed immediately after use, and zeroized once that call + completes. Only a file path is accepted, never an environment variable or + inline value. A key relying purely on PCR/policy session authorization may + omit `tpm_auth_file`. - **Sample scope:** the TPM provider ships with a runnable example targeting a software TPM (`swtpm`) for local exploration, and is not part of the required CI gate — real and virtual TPM availability in CI runners is not diff --git a/doc/src/raft_storage.md b/doc/src/raft_storage.md index d471a80ec..32fd495c9 100644 --- a/doc/src/raft_storage.md +++ b/doc/src/raft_storage.md @@ -3,8 +3,11 @@ This guide covers the architecture, cryptographic design, and operational procedures for the Keystone-RS distributed storage engine. The design is specified in [ADR 0016-v2](adr/0016-v2-raft-storage.md) and implemented across -two crates: `openstack-keystone-distributed-storage` (consensus, state machine, -gRPC) and `openstack-keystone-storage-crypto` (all cryptographic primitives). +four crates: `openstack-keystone-distributed-storage` (consensus, state +machine, gRPC), `openstack-keystone-storage-crypto` (shared cryptographic +primitives and the `KekProvider` trait), and the two production KEK +providers, `openstack-keystone-storage-crypto-pkcs11` and +`openstack-keystone-storage-crypto-tpm`. ## Table of Contents @@ -23,6 +26,7 @@ gRPC) and `openstack-keystone-storage-crypto` (all cryptographic primitives). 9. [DEK Rotation](#dek-rotation) 10. [Deployment Guide](#deployment-guide) - [Configuration Reference](#configuration-reference) + - [PKCS#11 and TPM KEK Providers](#pkcs11-and-tpm-kek-providers) - [First-Time Cluster Bootstrap](#first-time-cluster-bootstrap) - [Adding Nodes](#adding-nodes) - [TLS Certificate Management](#tls-certificate-management) @@ -126,13 +130,23 @@ crates/ ├── storage-crypto/ # All cryptographic primitives │ └── src/ │ ├── lib.rs # Public re-exports -│ ├── kek.rs # KekProvider trait, EnvKek, Pkcs11KekStub +│ ├── kek.rs # KekProvider trait, EnvKek (production +│ │ # providers: storage-crypto-pkcs11, -tpm) │ ├── dek.rs # DekEpoch, LogDek, StateDek, BackupDek, generate_dek │ ├── cipher.rs # log_encrypt/decrypt, state_encrypt/decrypt, │ │ # backup_encrypt/decrypt │ ├── nonce.rs # NonceManager — durable monotonic counter │ └── audit.rs # AuditHmacKey │ +├── storage-crypto-pkcs11/ # Production KekProvider: PKCS#11 HSM/token +│ ├── src/lib.rs # Pkcs11Kek, Pkcs11KekParams, SlotSelector +│ └── tests/softhsm.rs # SoftHSM2-backed wrap/unwrap round-trip test +│ +├── storage-crypto-tpm/ # Production KekProvider: TPM 2.0 resident key +│ ├── src/lib.rs # TpmKek, TpmKekParams, KeyReference +│ └── examples/ +│ └── tpm_kek_demo.rs # Runnable sample against a software TPM (swtpm) +│ └── storage/ # Consensus, gRPC, state machine └── src/ ├── lib.rs # StorageApi impl, DEK bootstrap @@ -156,7 +170,7 @@ crates/ ## Key Hierarchy ```text - HSM / Cloud KMS (production) + PKCS#11 HSM/token, or TPM 2.0 resident key (production) │ or KEYSTONE_DEV_KEK env var (dev mode only) │ @@ -446,7 +460,9 @@ node restarts mid-rotation, it resumes from the last checkpoint. - Rust toolchain (see `rust-toolchain.toml`) - A SPIRE deployment, or TLS certificates from a dedicated Intermediate CA -- For production: HSM or Cloud KMS for KEK storage +- For production: a PKCS#11 HSM/token (`kek_provider = "pkcs11"`) or a TPM 2.0 + chip (`kek_provider = "tpm"`) for KEK storage — see + [PKCS#11 and TPM KEK Providers](#pkcs11-and-tpm-kek-providers) - For development: set `KEYSTONE_DEV_KEK` and `KEYSTONE_ALLOW_ENV_KEK=1` ### Configuration Reference @@ -477,6 +493,11 @@ dek_rotation_days = 90 # Per-record write version threshold before blocking further writes (default: 2^30). write_rate_threshold = 1073741824 +# Selects the production KEK source. "env" (default) is dev-mode only and is +# rejected unless dev_mode = true. See "PKCS#11 and TPM KEK Providers" below. +# kek_provider = "pkcs11" +# kek_provider = "tpm" + # --- Transport: SPIFFE (default) --- trust_domains = "example.org" @@ -501,6 +522,97 @@ trust_domains = "example.org" > in production Dockerfiles, Kubernetes manifests, or systemd units. The CI gate > `tools/check_no_dev_mode.sh` enforces this. +### PKCS#11 and TPM KEK Providers + +Production deployments select one of the two hardware-backed `KekProvider` +implementations (ADR 0016-v2 §2.5). Both wrap/unwrap the DEK with +`CKM_AES_GCM`/TPM2 AES-GCM directly against a non-extractable AES-256 key +object — the key material never leaves the token or chip. + +#### PKCS#11 (HSM or token) + +```toml +[distributed_storage] +kek_provider = "pkcs11" + +[distributed_storage.pkcs11] +# Path to the vendor's (or SoftHSM2's) Cryptoki shared library. +pkcs11_module_path = "/usr/lib/softhsm/libsofthsm2.so" + +# CKA_LABEL of the AES-256 key object. Must have CKA_EXTRACTABLE = false +# (ADR 0016-v2 §10 invariant 13). init_storage never creates this key — +# it must already exist via an operator's out-of-band provisioning step. +pkcs11_key_label = "keystone-kek" + +# Either the slot id or the token label must be given; label is preferred +# since slot ids can shift across token re-initialisation. +pkcs11_slot_label = "keystone-storage" +# pkcs11_slot_id = 0 + +# File containing the token PIN. Never accepted inline or via env var +# (ADR 0016-v2 §10 invariant 14). +pkcs11_pin_file = "/etc/keystone/storage/pkcs11.pin" +``` + +Provisioning the key (a one-time operator ceremony, run once per token before +the cluster's first boot) uses any Cryptoki client capable of generating a +non-extractable AES-256 key under the target label — for example, +`pkcs11-tool --keygen --key-type AES:32 --label keystone-kek`, or the +provisioning helper in `crates/storage-crypto-pkcs11/tests/softhsm.rs` and +`crates/storage/tests/test_pkcs11_cluster.rs`, which do the equivalent +programmatically against SoftHSM2 for local testing. + +**Local testing against SoftHSM2:** + +```sh +# libsofthsm2.so and the softhsm2-util CLI: +sudo apt-get install -y softhsm2 + +# Run the SoftHSM2-backed tests (skip themselves if the module isn't found): +cargo test -p openstack-keystone-storage-crypto-pkcs11 +cargo test -p openstack-keystone-distributed-storage --features pkcs11 --test test_pkcs11_cluster +``` + +#### TPM 2.0 + +```toml +[distributed_storage] +kek_provider = "tpm" + +[distributed_storage.tpm] +# TCTI connection string: a hardware TPM's resource manager device, or a +# software TPM (swtpm) for testing. +tpm_tcti = "device:/dev/tpmrm0" + +# Exactly one of the following identifies the pre-provisioned AES-256 key: +tpm_key_handle = "0x81000001" # persistent handle, or: +# tpm_key_context_file = "/etc/keystone/storage/tpm-kek.ctx" + +# Optional: file containing the key's auth value (userWithAuth keys only). +# tpm_auth_file = "/etc/keystone/storage/tpm.auth" +``` + +As with PKCS#11, `init_storage` only ever opens the key with +`auto_generate: false` — provisioning is a separate operator step. +`crates/storage-crypto-tpm/examples/tpm_kek_demo.rs` is a runnable sample that +provisions and exercises a KEK against a software TPM: + +```sh +# 1. Start a software TPM: +mkdir -p /tmp/swtpm-state +swtpm socket --tpmstate dir=/tmp/swtpm-state \ + --ctrl type=tcp,port=2322 --server type=tcp,port=2321 \ + --tpm2 --flags not-need-init & +TPM2TOOLS_TCTI="swtpm:host=127.0.0.1,port=2321" tpm2_startup -c + +# 2. Run the sample (first run provisions the key, later runs reload it): +cargo run -p openstack-keystone-storage-crypto-tpm --example tpm_kek_demo +``` + +This example is compiled in CI on every run to catch rot, but is not executed +there — real/virtual TPM availability isn't reliable enough on shared CI +runners to gate merges on (ADR 0016-v2 §2.5.2). + ### First-Time Cluster Bootstrap **Step 1 — Start each node** (do not initialize yet):