Stateful encoders: LevelEncoder and RandomProjection#64
Open
MichielStock wants to merge 2 commits into
Open
Conversation
Completes the stateful AbstractEncoder pair begun with LevelEncoder:
- RandomProjection{HV, T}: fixed D×d projection matrix (gaussian /
bipolar / sparse_ternary), per-type nonlinearities, scalar-or-vector
ternary threshold θ with a data-driven target_sparsity constructor,
supplied-matrix constructor, rethreshold sharing R, and
nearest-neighbour-only decode (lossy encoding: no analytic inverse).
- phase_encode(z, β): single shared FHRR phase-encoding helper; both
LevelEncoder's fractional power path and RandomProjection's FHRR
nonlinearity (random Fourier features) route through it.
- Regression tests for two known sharp edges: the ternary constructor's
positional collision (supplied matrix vs training data; the residual
square-matrix ambiguity and silently ignored kwargs are recorded as
TODO §2.6, fix is a follow-up rename) and BipolarHV's raw-bits /
zero-state polarity traps, pinned as behaviour-locking tests so the
obvious-but-wrong idioms trip red instead of shipping corruption.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Collaborator
|
This looks good, I would merge this to main, merge it against #63, update tutorials and then merge that to main. LMKWYT |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Introduces the stateful encoder layer (
AbstractEncoder{HV}): encoder objects hold hypervector state built once at construction and slot in as the first argument ofencode, withdecodeas the inverse. Two members land here.LevelEncoder— scalars (breaking)level/encodelevel/decodelevel/convertlevelfamily, no deprecation shims (package is unregistered). The old design rebuilt the level set per call, so separate calls produced incomparable encodings (TODO §1.4/§1.4b); the struct builds the level set once, fixing this by construction.bandwidth= flip fraction per step, exposed as a keyword) and fractional power encoding for FHRR (continuous,βbandwidth; explicit level count requests a ladder instead).decodeis nearest-neighbour by similarity (robust); FHRR additionally offersmethod = :analyticfor clean vectors.RandomProjection— feature vectorsx ∈ ℝᵈ ↦ nonlinearity(R·x)with a fixedD × dmatrix (:gaussian,:bipolar,:sparse_ternary), per-type nonlinearities, and a parametric scalar-or-vector ternary thresholdθ(plus a data-driventarget_sparsityconstructor andrethresholdsharingR).fit; supplied-matrix constructor for reproducible/saved projections.decode(rp, hv, references)is nearest-neighbour clean-up only — a random projection is lossy, so no fake analytic inverse is offered; calling without a codebook errors.:gaussianis exactly a Rahimi–Recht random Fourier feature map (similarity ≈ Gaussian kernel with bandwidthβ), locked by a kernel-identity test.Shared machinery
phase_encode(z, β)is the single FHRR phase-encoding primitive; both LevelEncoder's fractional power path and RandomProjection's FHRR nonlinearity route through it (equality-locked in tests).Test hardening
Test plan
doctest(fix = true)and verified in a fresh process (same invocation as the CI doctests job)runic --checkclean on all touched files🤖 Generated with Claude Code