Add MCP server tester and update workspace configuration

- Introduced a new package `mcp_server_tester` in `Cargo.lock` with dependencies for testing MCP servers.
- Updated `Cargo.toml` to include `tests/mcp_server_tester` as a member of the workspace for better organization.
- Enhanced documentation in `TASKS.md` to outline the rewrite of MCP servers, detailing the hybrid approach and implementation phases.
- Removed deprecated MCP server binaries (`command-mcp`, `filesystem-mcp`, `memory-store-mcp`) to streamline the project structure.
- Improved logging and error handling in various modules to enhance maintainability and debugging capabilities.

Author: frostdev-ops

.cursor/rules/01-project-overview.mdc

@@ -16,5 +16,5 @@ See the main [README.md](mdc:README.md) for a full overview.
 *   **`gemini-mcp` ([mcp/README.md](mdc:mcp/README.md))**: Implements the Model Context Protocol (MCP) **host** side. Manages discovering, launching, and communicating with MCP **servers** (tools). Includes the `mcp-hostd` binary and built-in server implementations.
 *   **`gemini-memory` ([memory/README.md](mdc:memory/README.md))**: Implements persistent semantic memory using LanceDB. Relies on an MCP host (`McpHostInterface`) for embedding generation.
 *   **`gemini-cli` ([cli/README.md](mdc:cli/README.md))**: The main command-line interface. Integrates the other crates for user interaction, tool use, and memory.
-*   **`HAPPE` ([HAPPE/README.md](mdc:HAPPE/README.md))**: Host Application Environment daemon. Manages interactions between user, LLM, `IDA`, and MCP servers. Intended as the primary execution environment.
-*   **`IDA` ([IDA/README.md](mdc:IDA/README.md))**: Internal Dialogue App daemon. Manages persistent memory and background cognitive tasks, communicating with `HAPPE` via IPC.
+*   **`HAPPE` (@happe/README.md)**: Host Application Environment daemon. Manages interactions between user, LLM, `IDA`, and MCP servers. Intended as the primary execution environment.
+*   **`IDA` (@ida/README.md)**: Internal Dialogue App daemon. Manages persistent memory and background cognitive tasks, communicating with `HAPPE` via IPC.

.cursor/rules/05-daemons-ipc.mdc

@@ -7,8 +7,8 @@ alwaysApply: false
 
 For more complex, persistent scenarios, the system uses background daemons communicating via Inter-Process Communication (IPC).
 
-*   **`HAPPE` Daemon ([HAPPE/README.md](mdc:HAPPE/README.md))**: The Host Application Environment. Orchestrates user interactions, LLM calls, MCP tool execution, and communication with `IDA`.
-*   **`IDA` Daemon ([IDA/README.md](mdc:IDA/README.md))**: Internal Dialogue App. Manages persistent memory interactions (retrieval/storage via Memory MCP Server) and other background cognitive tasks.
+*   **`happe` Daemon (@happe/README.md)**: The Host Application Environment. Orchestrates user interactions, LLM calls, MCP tool execution, and communication with `IDA`.
+*   **`ida` Daemon (@ida/README.md)**: Internal Dialogue App. Manages persistent memory interactions (retrieval/storage via Memory MCP Server) and other background cognitive tasks.
 *   **`mcp-hostd` Daemon ([mcp/README.md](mdc:mcp/README.md))**: Standalone MCP host process. Manages connections to MCP tool servers.
 *   **IPC Definitions (`gemini-ipc`)**: The `gemini-ipc` crate ([ipc/README.md](mdc:ipc/README.md)) centralizes message structures for communication between these components.
     *   `internal_messages.rs` ([ipc/src/internal_messages.rs](mdc:ipc/src/internal_messages.rs)): Defines messages between `HAPPE` and `IDA`.

Cargo.lock

@@ -11,6 +11,7 @@ members = [
     "install",   # Installer crate for Gemini CLI Suite
     "daemon-manager", # CLI for managing gemini-suite daemons and MCP servers
     "tools",     # Tools for configuration, maintenance, etc.
+    "tests/mcp_server_tester", # Test binary for MCP Python servers
 ]
 
 [workspace.lints]

Cargo.toml

@@ -0,0 +1,129 @@
+# End-to-End Query Flow
+
+This document outlines the full lifecycle of a user query as it travels from the End User through the HAPPE daemon, internal communication with the IDA daemon, interactions with MCP servers, calls to the main LLM API, and back to the End User. It also covers the asynchronous memory storage process.
+
+---
+
+## Overview of Components
+
+- **End User**: The person or client application issuing a natural language query.
+- **HAPPE (Host Application Environment)**: Orchestrates user interactions, LLM calls, and MCP tool execution; communicates with IDA via IPC.
+- **IDA (Internal Dialogue App)**: Manages persistent memory: retrieves context and stores completed turns; communicates over IPC with HAPPE.
+- **Memory MCP Server**: Provides memory retrieval and storage via MCP protocol.
+- **Other MCP Servers**: Hosts additional MCP tools invoked during LLM generation.
+- **Main LLM API**: The external LLM endpoint that generates natural language responses.
+
+---
+
+## Sequence Diagram
+
+```mermaid
+sequenceDiagram
+    autonumber
+    participant User as End User
+    participant HAPPE as HAPPE Daemon
+    participant IDA   as IDA Daemon
+    participant MemMCP as Memory MCP Server
+    participant MCP   as Other MCP Server(s)
+    participant LLM   as Main LLM API
+
+    User->>HAPPE: Query Request (query, session_id)
+    activate HAPPE
+
+    HAPPE->>IDA: GetMemoriesRequest (query, context)
+    activate IDA
+
+    IDA->>MemMCP: retrieve_memories(query, ...)
+    activate MemMCP
+    note right of IDA: Semantic search + optional Broker LLM filtering
+    MemMCP-->>IDA: Vec<Memory>
+    deactivate MemMCP
+
+    IDA-->>HAPPE: GetMemoriesResponse { memories }
+    deactivate IDA
+
+    HAPPE->>HAPPE: Construct Prompt
+    activate LLM
+    HAPPE->>LLM: generate_response(contents, system_prompt, tools)
+
+    alt Function Call requested by LLM
+        LLM-->>HAPPE: function_calls: Vec<FunctionCall>
+        activate MCP
+        HAPPE->>MCP: execute_tool(server, tool, args)
+        MCP-->>HAPPE: result: JSON
+        deactivate MCP
+        HAPPE-->>LLM: Part::function_response(name, result)
+    end
+
+    LLM-->>HAPPE: Final Response (text, calls)
+    deactivate LLM
+
+    HAPPE-->>User: Query Response (response, session_id)
+    
+    note over HAPPE, IDA: Asynchronous Storage
+    HAPPE-)+IDA: StoreTurnRequest { turn_data }
+    activate IDA
+    IDA-)MemMCP: storage::handle_storage(...)
+    activate MemMCP
+    MemMCP--)IDA: Result<()>
+    deactivate MemMCP
+    deactivate IDA
+    deactivate HAPPE
+```
+
+---
+
+## Detailed Step-by-Step Flow
+
+1. **End User -> HAPPE**
+   - The end user or client application sends a raw text query to the HAPPE daemon over the chosen transport (e.g., HTTP, WebSocket, CLI).
+   - This is represented internally as `HappeQueryRequest { query, session_id }`.
+
+2. **HAPPE -> IDA (IPC)**
+   - HAPPE immediately forwards the raw query to the IDA daemon via IPC using `InternalMessage::GetMemoriesRequest { query, conversation_context }`.
+   - The IPC transport is configured in HAPPE's settings (Unix socket path).
+
+3. **IDA -> Memory MCP Server (MCP)**
+   - Upon receiving the query, IDA invokes the memory MCP server with `retrieve_memories(query, max_results, broker_llm)`.
+   - The memory store performs semantic search and optionally filters results using a broker LLM.
+
+4. **Memory MCP -> IDA**
+   - The memory service returns a list of relevant `Memory` objects (e.g., past messages, facts).
+   - IDA converts these to `MemoryItem` objects and prepares an `InternalMessage::GetMemoriesResponse` to send back.
+
+5. **IDA -> HAPPE (IPC Response)**
+   - IDA sends the retrieved memories back to HAPPE over the same IPC channel.
+
+6. **HAPPE Prompt Construction**
+   - HAPPE builds prompt parts that include memories and the user query.
+   - It prepares the prompt with a system message that includes MCP capabilities.
+   - Example: `"Relevant previous interactions: <memory_1>\n<memory_2>\n<original query>"`.
+
+7. **HAPPE -> Main LLM API Call**
+   - HAPPE calls `generate_response` with the constructed contents, system prompt, and available tools.
+   - Common parameters include conversation history, current query content, and tool declarations.
+
+8. **Tool Invocation (Optional)**
+   - If the LLM returns function calls, HAPPE processes each call:
+     - Extracts server name and tool name from the function call name
+     - Calls `mcp_client.execute_tool(server_name, tool_name, arguments)`
+     - Adds the tool result as a `Part::function_response` to the conversation
+     - Makes another LLM request with the updated history including the tool results
+
+9. **LLM -> HAPPE Response**
+   - The LLM returns the final `(response_text, function_calls)` tuple once all tool calls (if any) are resolved.
+
+10. **HAPPE -> End User**
+    - HAPPE returns `HappeQueryResponse { response, session_id }` to the user over the original transport.
+
+11. **Asynchronous Memory Storage**
+    - HAPPE sends an async `InternalMessage::StoreTurnRequest { turn_data }` to IDA containing:
+      - `user_query`: The original query string
+      - `retrieved_memories`: The memories used in the prompt
+      - `llm_response`: The final text response
+      - `turn_parts`: The full conversation turn parts
+    - IDA spawns a background task via `storage::handle_storage` to process and store the turn.
+
+---
+
+*End of document.*

QUERY_FLOW.md

@@ -324,3 +324,62 @@ This phase adds a stateful session management system to HAPPE, allowing it to ma
 - [ ] **Integration Tests:** Test IPC handler with session history.
 - [ ] **End-to-End Tests (`@cli` -> `happe-daemon`):** Verify context is maintained across multiple turns within a single CLI run.
 - [ ] **Refinement:** Assess performance, history pruning, error handling, security of session IDs.
+
+## Phase 11: Rewrite MCP Servers (Hybrid Approach - Python Facades, Rust Core Memory)
+
+This phase replaces `filesystem-mcp` and `command-mcp` with Python implementations. `memory-store-mcp` becomes a Python facade process, while its core logic (including vector store via `gemini-memory`) is embedded and executed directly within `mcp-hostd.rs`.
+
+### Sub-Phase 1: Investigation & Design Refinement
+
+- [x] **Analyze `mcp-hostd.rs` & `gemini_mcp` Crate:** (Config loading/launching confirmed).
+- [x] **Analyze `install` Crate:** (Binary/config handling understood).
+- [x] **Analyze `gemini-memory` Dependency:** (Will be embedded in `mcp-hostd`).
+- [x] **Refine Design:** Confirm hybrid strategy:
+    - [x] `mcp-hostd` embeds `gemini_memory::MemoryStore`.
+    - [x] `mcp-hostd` intercepts & handles `memory-store-mcp/*` tool calls internally.
+    - [x] Minimal `memory-store-mcp.py` facade launched via stdio for handshake.
+    - [x] Standard `filesystem-mcp.py` and `command-mcp.py` servers.
+
+### Sub-Phase 2: Implementation
+
+- [ ] **Modify `mcp-hostd.rs` (Initialization):**
+    - [ ] Ensure `gemini-memory` dependency available to `mcp-hostd`.
+    - [ ] Initialize `MemoryStore` instance during startup.
+    - [ ] Load necessary `MemoryStore` config (DB path, etc.) from main `config.toml`.
+    - [ ] Store `Arc<MemoryStore>` where `process_request` can access it.
+- [ ] **Modify `mcp-hostd.rs` (`process_request`):**
+    - [ ] Add logic to intercept `ExecuteTool` for `server == "memory-store-mcp"`.
+    - [ ] Implement internal calls to embedded `MemoryStore` for intercepted requests.
+    - [ ] Bypass standard `host.execute_tool` for these intercepted requests.
+- [ ] **Modify `mcp-hostd.rs` (Capabilities):**
+    - [ ] Manually add tool definitions for internal memory operations under the `memory-store-mcp` server name during capability aggregation.
+- [ ] **Implement `python_mcp/servers/base_server.py` (as before).**
+- [ ] **Implement `python_mcp/servers/filesystem_mcp.py` (as before).**
+- [ ] **Implement `python_mcp/servers/command_mcp.py` (as before).**
+- [ ] **Simplify `python_mcp/servers/memory_store_mcp.py`:**
+    - [ ] Remove `MemoryStore` class and tool handlers.
+    - [ ] `main` should only create `McpBaseServer`, register no tools, and call `run()`.
+- [ ] **Project Setup (`python_mcp`, `requirements.txt` - as before).**
+
+### Sub-Phase 3: Integration & Installation
+
+- [ ] **Modify `install/src/main.rs`:**
+    - [ ] Remove Rust MCP binary installation steps.
+    - [ ] Add step to copy `python_mcp` directory to installation target.
+    - [ ] Modify `install_unified_config`:
+        - [ ] Generate `mcp_servers.json` entries pointing all 3 servers (`memory-store-mcp`, `filesystem-mcp`, `command-mcp`) to their Python scripts.
+        - [ ] Ensure main `config.toml` generation includes necessary `[memory]` / `[ida]` sections for embedded `MemoryStore` config.
+
+### Sub-Phase 4: Testing & Deployment
+
+- [ ] Build & Install modified `install` crate.
+- [ ] Verify Python scripts and config files placement.
+- [ ] Install Python dependencies (`pip install -r requirements.txt`).
+- [ ] Start `mcp-hostd`. Check logs for embedded `MemoryStore` init & Python process handshakes.
+- [ ] IPC Client Testing:
+    - [ ] Test `GetCapabilities` (should show memory tools listed under `memory-store-mcp`).
+    - [ ] Test `filesystem-mcp` tools (check Python logs).
+    - [ ] Test `command-mcp` tools (check Python logs).
+    - [ ] Test `memory-store-mcp` tools (check `mcp-hostd` logs & DB state).
+- [ ] Shutdown Testing.
+- [ ] Deployment.