Skip to content

feat: improve local LLM stability with robust JSON parsing and memory…#4576

Open
karpizin wants to merge 1 commit intomem0ai:mainfrom
karpizin:feat/local-llm-optimizations
Open

feat: improve local LLM stability with robust JSON parsing and memory…#4576
karpizin wants to merge 1 commit intomem0ai:mainfrom
karpizin:feat/local-llm-optimizations

Conversation

@karpizin
Copy link
Copy Markdown

Linked Issue

Closes #

Description

  • Improves local LLM stability for memory extraction, especially with Ollama-hosted models that often return noisy or non-strict JSON output.
  • Makes JSON parsing more robust by extracting structured content from markdown code blocks, preamble text, and other imperfect local-model responses instead of assuming a clean raw JSON payload.
  • Adds support for memory granularity via entry_mode, allowing "fact" mode for atomic memories and "chunk" mode for larger semantic segments that preserve more document context.
  • Improves package import resilience by avoiding a hard failure when mem0ai package metadata is unavailable in local/dev environments.
  • Updates Ollama and memory-operation docs to explain local-model optimizations and when to use chunk-based extraction.

Type of Change

  • Bug fix (non-breaking change that fixes an issue)
  • New feature (non-breaking change that adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Refactor (no functional changes)
  • Documentation update

Breaking Changes

N/A

Test Coverage

  • I added/updated unit tests
  • I added/updated integration tests
  • I tested manually (describe below)
  • No tests needed (explain why)

Manual verification:

  • rebased the branch onto the latest main
  • verified conflict resolution in mem0/memory/utils.py
  • checked the new extract_json() behavior for:
    • fenced ```json code blocks
    • responses with leading/trailing text around JSON
    • raw JSON responses
  • reviewed the new entry_mode="chunk" flow in mem0/memory/main.py and prompt/config wiring
  • reviewed documentation updates for Ollama and memory add operations

Added/updated tests:

  • tests/memory/test_granularity.py
  • tests/memory/test_utils.py

Checklist

  • My code follows the project's style guidelines
  • I have performed a self-review of my code
  • I have added tests that prove my fix/feature works
  • New and existing tests pass locally
  • I have updated documentation if needed

@CLAassistant
Copy link
Copy Markdown

CLAassistant commented Mar 27, 2026

CLA assistant check
All committers have signed the CLA.

@karpizin karpizin force-pushed the feat/local-llm-optimizations branch from 8b9390d to 51e056f Compare March 27, 2026 22:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants