- Fixed metadata filtering issue in memory retrieval:
- Corrected the
_matches_metadata_filtersmethod to properly evaluate metadata fields - Improved handling of importance score filtering with proper normalization
- Added debug logging for metadata filter evaluation process
- Fixed validation tests to use correct metadata fields in test memories
- Enhanced metadata comparison logic to handle missing fields gracefully
- Corrected the
- Fixed ImportanceStrategy implementation:
- Resolved issue with importance value parsing from different memory formats
- Fixed importance comparison logic for non-normalized importance values
- Corrected sorting behavior to properly prioritize memories with higher importance
- Added proper handling for memory objects with 'id' field vs 'memory_id' field
- Fixed query handling for different query formats (direct value vs. dictionary)
- Enhanced debug output in validation scripts for better troubleshooting
- Improved error handling for missing metadata during filtering operations
- Added checksum functionality to memory metadata:
- New
checksumfield inMemoryMetadatafor data integrity verification - Added utility functions in
memory/utils/checksums.pyfor generating and validating checksums - Implemented automatic checksum generation when storing memories
- Added checksum validation when retrieving memories
- Added integrity verification flags in memory metadata
- New
- Enhanced data integrity through checksum verification across all memory tiers
- Added support for different hashing algorithms in checksum generation
- Implemented comprehensive test suite for checksum functionality
- Extracted duplicate code into common utilities in
utils.py:cosine_similarity: Moved fromvector_store.pyfor vector comparisonflatten_dict: Moved fromautoencoder.pyfor dictionary flatteningobject_to_text: Moved fromtext_embeddings.pyfor object text conversionfilter_dict_keys: Moved fromcompression.pyfor dictionary filtering
- Modified
text_embeddings.pyto use the commonobject_to_textfunction - Updated
vector_store.pyto use the sharedcosine_similarityfunction - Revised
autoencoder.pyto use the centralizedflatten_dictfunction - Changed
compression.pyto use the commonfilter_dict_keysfunction - Enhanced
object_to_textfunction for better handling of empty values and nested structures - Refactored embeddings module by removing direct
TextEmbeddingEngineimport
- Added comprehensive module documentation in
__init__.pyfor LLM context - Added detailed documentation file
docs/Embeddings.mdexplaining the Embeddings module components and usage - Exposed utility functions in
__all__list for easy importing - Added comprehensive unit tests for the embeddings module:
- Tests for
AutoEncoderand dimensionality reduction capabilities - Tests for
NumericExtractorfunctionality - Tests for vector compression operations
- Tests for text embedding operations
- Tests for vector store functionality
- Tests for
- Added vector compression utilities:
quantize_vectoranddequantize_vectorfor bit-depth reductioncompress_vector_rpanddecompress_vector_rpfor random projection compressionCompressionConfigfor managing compression settings
- Added MockRedis implementation with pipeline and pubsub support for testing
- Enhanced MockRedis implementation for development and testing:
- Added configuration options in
RedisSTMConfigandRedisIMConfig - Created
RedisFactoryto streamline Redis client instantiation - Added comprehensive integration tests for Redis stores with MockRedis
- Updated README with MockRedis usage instructions
- Added configuration options in
- Eliminated code duplication across the embeddings module
- Enhanced maintainability by centralizing common functionality
- Improved consistency in utility functions across the codebase
- Enhanced RedisFactory to support MockRedis with custom ResilientRedisClient for better testing
- Improved MockRedis module documentation with usage examples and implementation details
- Updated random number generation in vector compression to use local
RandomStateinstead of global seed to prevent interference with other code