diff --git a/CHANGELOG.md b/CHANGELOG.md index 7ae89714..46922c11 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,16 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] +### Added + +#### Web App (app/) — Librarian (2026-05-04) + +AI chat panel for asking questions about your data using SQL lineage and uploaded PDF documentation. Supports OpenAI, Anthropic, and custom OpenAI-compatible endpoints; multilingual embeddings (`Xenova/multilingual-e5-small`, 100+ languages); per-project state isolation (RAM-only) so chat, PDFs, and embedded chunks don't leak between projects. + +Answers follow a structured **Summary / Data Lineage / Documentation** format. Schema identifiers are highlighted inline in assistant responses (case-insensitive, normalized to canonical schema casing). Clicking an assistant message reads the answer's Summary, highlights every referenced table and column in the Lineage view via the existing search pipeline (auto-enabling "show column edges" when a column is referenced), and gently pans + pulses on the source table containing the column. + +Schema view gains a search control (table-name or column-name substring, Prev/Next cycling). Librarian toggle lives in the analysis toolbar next to Schema (⌘L / Ctrl+L); a help popover in the panel header describes the assistant and usage. + ## [0.7.0] - 2026-04-23 ### Added diff --git a/README.md b/README.md index 7783cca4..b9d09d9a 100644 --- a/README.md +++ b/README.md @@ -32,6 +32,7 @@ Features: - Multi-file project support with schema DDL - dbt/Jinja template preprocessing for dbt models - Export to Mermaid, JSON, CSV, Excel, or HTML reports +- Librarian — AI chat panel that answers questions about your data based on lineage analysis and uploaded PDF docs - All processing happens in your browser — your SQL never leaves your machine ### Command-Line Interface @@ -114,6 +115,7 @@ See [CLI documentation](crates/flowscope-cli/README.md) for all options. - Structured diagnostics with spans for precise highlighting - Completion API for SQL authoring workflows - TypeScript API and optional React visualization components +- Librarian AI chat panel for natural-language Q&A over SQL lineage and uploaded PDF documentation (OpenAI, Anthropic, or custom endpoints) ## Components @@ -184,6 +186,7 @@ npm install @pondpilot/flowscope-react ## Documentation - [docs/README.md](docs/README.md) — documentation map and reference index +- [docs/librarian.md](docs/librarian.md) — Librarian AI chat panel user guide - [docs/guides/quickstart.md](docs/guides/quickstart.md) — TypeScript quickstart guide - [docs/guides/schema-metadata.md](docs/guides/schema-metadata.md) — schema metadata setup - [docs/dialect-coverage.md](docs/dialect-coverage.md) — dialect and statement coverage diff --git a/app/ARCHITECTURE.md b/app/ARCHITECTURE.md index 39da68a3..6eee399a 100644 --- a/app/ARCHITECTURE.md +++ b/app/ARCHITECTURE.md @@ -19,6 +19,8 @@ app/src/ │ ├── SchemaEditor.tsx # DDL schema editor │ ├── ShareDialog.tsx # Project export/sharing │ └── Workspace.tsx # Main two-panel layout +├── features/ # Feature modules (self-contained) +│ └── librarian/ # AI chat panel (Q&A over lineage + PDFs) ├── hooks/ # Custom React hooks │ ├── useAnalysis.ts # SQL analysis workflow │ ├── useFileNavigation.ts # Graph-to-editor navigation @@ -74,6 +76,23 @@ The app now supports "Schema-Aware" analysis. * Column validation * Precise column lineage +### 5. Feature Modules + +Self-contained feature folders under `app/src/features/` own all their code (components, hooks, services, workers, tests) and expose a public API via `index.ts`. + +#### Librarian (`features/librarian/`) + +AI-powered chat panel for SQL lineage Q&A. + +- `components/` — panel, chat messages, input, PDF upload, AI settings dialog +- `services/` — AI client (OpenAI / Anthropic / custom), context builder, lineage formatter, PDF processor, vector search, embedding service +- `workers/` — embedding Web Worker (local Xenova/transformers model) +- `hooks/use-librarian-chat.ts` — chat orchestrator +- `hooks/use-sync-active-project.ts` — mirrors `activeProjectId` from `useProject()` into the Librarian store and prunes buckets for deleted projects +- `store.ts` — Zustand store. Per-project buckets (`byProject` keyed by `activeProjectId`) hold messages, PDF files, and embedded chunks; `isLoading` and `hasConfig` are global. Selector hooks `useLibrarianMessages` / `useLibrarianPdfFiles` / `useLibrarianPdfChunks` return the active project's slice. `addMessageToProject(projectId, ...)` writes to an explicit bucket so an in-flight LLM response is routed back to the originating project even if the user switches mid-flight. + +State is Zustand (not React Context), UI is Radix + Tailwind. All AI calls hit the user's configured provider directly from the browser. See `docs/librarian.md` for the user guide. + ## Data Flow ### Analysis Loop @@ -87,12 +106,23 @@ The app now supports "Schema-Aware" analysis. 5. **Result**: The JSON result is dispatched to the Lineage Store. 6. **Rendering**: `AnalysisView` updates the Graph and Issues panel. +### Librarian Chat Flow + +1. User types a question in the Librarian panel. +2. `use-librarian-chat.ts` gathers: current lineage (from `useLineageState`), active SQL file content, last 10 chat messages, and vector-search results over uploaded PDF chunks. +3. `context-builder.ts` assembles a structured prompt with labeled data sources (Data Lineage / SQL Code / Documentation / Conversation History). +4. `ai-service.ts` sends the prompt via `fetch()` to the configured provider (OpenAI / Anthropic / custom endpoint). +5. Response is stored in the chat and rendered with markdown + identifier highlighting. + +PDF processing runs asynchronously: text extraction (pdfjs-dist) → 500-char chunking → embeddings (local `multilingual-e5-small` model in a Web Worker) → stored in the librarian store. + ## UI Architecture * **Layout**: `react-resizable-panels` provides the split-view. * **Styling**: Tailwind CSS with `shadcn/ui` (Radix Primitives) pattern. * **Icons**: Lucide React. * **Editor**: `CodeMirror` (via `@pondpilot/flowscope-react`). +* **Librarian icon**: Custom SVG (`/public/polly-icon.svg`) for the toolbar, chat avatar, and empty state. ## Configuration @@ -102,6 +132,7 @@ The app now supports "Schema-Aware" analysis. * `Cmd/Ctrl + P`: Switch Project * `Cmd/Ctrl + O`: Switch File * `Cmd/Ctrl + D`: Switch Dialect + * `Cmd/Ctrl + L`: Toggle Librarian panel ## Future Improvements diff --git a/app/package.json b/app/package.json index 962791a3..df49d8ca 100644 --- a/app/package.json +++ b/app/package.json @@ -11,7 +11,8 @@ "typecheck": "tsc --noEmit", "lint": "eslint src", "lint:fix": "eslint src --fix", - "test": "echo \"No tests defined for flowscope-app\"" + "test": "vitest run", + "test:watch": "vitest" }, "dependencies": { "@pondpilot/flowscope-core": "file:../packages/core", @@ -21,12 +22,14 @@ "@radix-ui/react-dialog": "^1.1.15", "@radix-ui/react-dropdown-menu": "^2.1.16", "@radix-ui/react-label": "^2.1.8", + "@radix-ui/react-popover": "^1.1.15", "@radix-ui/react-scroll-area": "^1.2.10", "@radix-ui/react-select": "^2.2.6", "@radix-ui/react-separator": "^1.1.8", "@radix-ui/react-slot": "^1.2.4", "@radix-ui/react-tabs": "^1.1.13", "@radix-ui/react-tooltip": "^1.2.8", + "@xenova/transformers": "^2.17.2", "class-variance-authority": "^0.7.1", "clsx": "^2.1.1", "fflate": "^0.8.2", @@ -34,6 +37,7 @@ "html-to-image": "^1.11.13", "lucide-react": "^0.562.0", "next-themes": "^0.4.6", + "pdfjs-dist": "4.8.69", "react": "^19.1.0", "react-dom": "^19.1.0", "react-resizable-panels": "^3.0.6", @@ -43,15 +47,20 @@ }, "devDependencies": { "@tailwindcss/postcss": "^4.1.0", + "@testing-library/dom": "^10.4.1", + "@testing-library/jest-dom": "^6.9.1", + "@testing-library/react": "^16.3.2", "@types/file-saver": "^2.0.7", "@types/react": "^19.1.0", "@types/react-dom": "^19.1.0", "@vitejs/plugin-react": "^4.4.0", + "jsdom": "^26.1.0", "postcss": "^8.5.6", "tailwindcss": "^4.1.0", "typescript": "^5.9.0", "vite": "^6.3.0", "vite-plugin-top-level-await": "^1.6.0", - "vite-plugin-wasm": "^3.5.0" + "vite-plugin-wasm": "^3.5.0", + "vitest": "^3.2.4" } } diff --git a/app/public/polly-icon.svg b/app/public/polly-icon.svg new file mode 100644 index 00000000..a2faa2d2 --- /dev/null +++ b/app/public/polly-icon.svg @@ -0,0 +1,66 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/app/src/components/AnalysisView.tsx b/app/src/components/AnalysisView.tsx index 165c511d..95efa16b 100644 --- a/app/src/components/AnalysisView.tsx +++ b/app/src/components/AnalysisView.tsx @@ -19,15 +19,18 @@ import { Tabs, TabsContent, TabsList, TabsTrigger } from '@/components/ui/tabs'; import { usePersistedLineageState } from '@/hooks/usePersistedLineageState'; import { usePersistedMatrixState } from '@/hooks/usePersistedMatrixState'; import { usePersistedSchemaState } from '@/hooks/usePersistedSchemaState'; +import { applyLineageNavigation } from '@/lib/lineage-navigation'; import { isValidTab, useNavigation } from '@/lib/navigation-context'; import { useViewStateStore, getNamespaceFilterStateWithDefaults } from '@/lib/view-state-store'; import { useProject } from '@/lib/project-store'; import { schemaMetadataToSQL } from '@/lib/schema-parser'; import { HierarchyView, type HierarchyViewRef } from './HierarchyView'; +import { LibrarianToggleButton } from './LibrarianToggleButton'; import { StatsPopover } from './StatsPopover'; import { NamespaceFilterBar } from './NamespaceFilterBar'; import { SchemaAwareIssuesPanel } from './SchemaAwareIssuesPanel'; import { SchemaEditor } from './SchemaEditor'; +import { SchemaSearchControl } from './SchemaSearchControl'; interface AnalysisViewProps { graphContainerRef?: React.RefObject; @@ -106,14 +109,14 @@ export function AnalysisView({ // Handle navigation target for GraphView - select and focus node/statement when navigating to lineage tab useEffect(() => { if (activeTab === 'lineage' && navigationTarget) { - if (navigationTarget.tableId) { - // Navigate to specific table node - actionsRef.current.selectNode(navigationTarget.tableId); - setLineageFocusNodeId(navigationTarget.tableId); - } else if (navigationTarget.fitView) { - // Trigger fitView to show all nodes (e.g., from Issues panel) - setFitViewTrigger((prev) => prev + 1); - } + applyLineageNavigation(navigationTarget, { + expandedTableIds: stateRef.current.expandedTableIds, + selectNode: actionsRef.current.selectNode, + toggleTableExpansion: actionsRef.current.toggleTableExpansion, + setFocusNodeId: setLineageFocusNodeId, + triggerFitView: () => setFitViewTrigger((prev) => prev + 1), + revealNodeInGraph: actionsRef.current.revealNodeInGraph, + }); clearNavigationTarget(); } }, [activeTab, navigationTarget, clearNavigationTarget]); @@ -336,24 +339,29 @@ export function AnalysisView({ if (!result || !summary) { return ( -
-
- {isAnalyzing ? ( - <> - -

Analyzing SQL

-

- Building lineage, schema, and issue details for the current analysis run. -

- - ) : ( - <> -

No Analysis Results

-

- Run analysis on your SQL script to see lineage and schema details here. -

- - )} +
+
+ +
+
+
+ {isAnalyzing ? ( + <> + +

Analyzing SQL

+

+ Building lineage, schema, and issue details for the current analysis run. +

+ + ) : ( + <> +

No Analysis Results

+

+ Run analysis on your SQL script to see lineage and schema details here. +

+ + )} +
); @@ -413,6 +421,7 @@ export function AnalysisView({ )} +
@@ -484,11 +493,20 @@ export function AnalysisView({ className="h-full mt-0 p-0 absolute inset-0 data-[state=inactive]:hidden" > {mountedTabs.has('schema') && ( - +
+ +
+ t.name)} + tables={schema} + onSelectTable={schemaState.setSelectedTableName} + /> +
+
)} diff --git a/app/src/components/GlobalDropZone.tsx b/app/src/components/GlobalDropZone.tsx index f7855fcd..063aef41 100644 --- a/app/src/components/GlobalDropZone.tsx +++ b/app/src/components/GlobalDropZone.tsx @@ -125,6 +125,10 @@ export function GlobalDropZone() { const handleDragEnter = useCallback( (e: DragEvent) => { + // Ignore drag events inside the Librarian PDF dropzone + const target = e.target as HTMLElement | null; + if (target?.closest?.('[data-librarian-dropzone]')) return; + e.preventDefault(); e.stopPropagation(); @@ -147,6 +151,10 @@ export function GlobalDropZone() { const handleDragLeave = useCallback( (e: DragEvent) => { + // Ignore drag events inside the Librarian PDF dropzone + const target = e.target as HTMLElement | null; + if (target?.closest?.('[data-librarian-dropzone]')) return; + e.preventDefault(); e.stopPropagation(); @@ -167,6 +175,13 @@ export function GlobalDropZone() { const handleDragOver = useCallback( (e: DragEvent) => { + const target = e.target as HTMLElement | null; + if (target?.closest?.('[data-librarian-dropzone]')) { + setIsDragOver(false); + dragCounter.current = 0; + return; + } + e.preventDefault(); e.stopPropagation(); @@ -179,8 +194,13 @@ export function GlobalDropZone() { const handleDrop = useCallback( async (e: DragEvent) => { + // Ignore drops inside the Librarian PDF dropzone + const target = e.target as HTMLElement | null; + if (target?.closest?.('[data-librarian-dropzone]')) return; + e.preventDefault(); e.stopPropagation(); + setIsDragOver(false); dragCounter.current = 0; setRejectedFiles([]); @@ -326,10 +346,10 @@ export function GlobalDropZone() { role="region" aria-label="File drop zone" className={cn( - 'fixed inset-0 z-50 flex items-center justify-center', + 'fixed inset-0 z-50 flex items-center justify-center pointer-events-none', 'bg-background/80 backdrop-blur-xs', 'transition-opacity duration-200', - isDragOver ? 'opacity-100' : 'opacity-0 pointer-events-none' + isDragOver ? 'opacity-100' : 'opacity-0' )} >
s.librarianOpen); + const toggleLibrarian = useViewStateStore((s) => s.toggleLibrarian); + const shortcut = getShortcutDisplay('toggle-librarian') ?? '⌘L'; + + return ( + + + + + + +

+ Toggle Librarian + + {shortcut} + +

+
+
+
+ ); +} diff --git a/app/src/components/SchemaSearchControl.tsx b/app/src/components/SchemaSearchControl.tsx new file mode 100644 index 00000000..cad88061 --- /dev/null +++ b/app/src/components/SchemaSearchControl.tsx @@ -0,0 +1,218 @@ +import { useCallback, useEffect, useMemo, useRef, useState } from 'react'; +import { ChevronLeft, ChevronRight, Search, X } from 'lucide-react'; + +import { Button } from '@/components/ui/button'; +import { Input } from '@/components/ui/input'; +import { cn } from '@/lib/utils'; + +interface TableWithColumns { + name: string; + columns?: { name: string }[]; +} + +interface SchemaSearchControlProps { + tableNames: string[]; + tables?: TableWithColumns[]; + onSelectTable: (tableName: string | undefined) => void; + className?: string; +} + +function findAllMatches( + tableNames: string[], + query: string, + tables?: TableWithColumns[] +): string[] { + if (!query) return []; + const q = query.toLowerCase(); + const matches = new Set(); + + // Match table names + for (const name of tableNames) { + if (name.toLowerCase().includes(q)) { + matches.add(name); + } + } + + // Match column names — add owning table + if (tables) { + for (const table of tables) { + if (table.columns?.some((col) => col.name.toLowerCase().includes(q))) { + matches.add(table.name); + } + } + } + + return Array.from(matches); +} + +export function SchemaSearchControl({ + tableNames, + tables, + onSelectTable, + className, +}: SchemaSearchControlProps) { + const [expanded, setExpanded] = useState(false); + const [value, setValue] = useState(''); + const [matchIndex, setMatchIndex] = useState(0); + const inputRef = useRef(null); + const hasInteractedRef = useRef(false); + + const matches = useMemo( + () => findAllMatches(tableNames, value.trim(), tables), + [tableNames, tables, value] + ); + const activeMatch = + matches.length > 0 ? matches[Math.min(matchIndex, matches.length - 1)] : undefined; + + useEffect(() => { + if (expanded) { + inputRef.current?.focus(); + } + }, [expanded]); + + useEffect(() => { + if (matches.length > 0 && matchIndex >= matches.length) { + setMatchIndex(0); + } + }, [matches.length, matchIndex]); + + // Update selection when an active search has matches. Do not clear an + // existing schema selection just because the control mounted with an empty + // query; only clear after the user has interacted with the search field. + // `hasInteractedRef` is scoped to a single search session — it is reset + // when the control collapses, so a fresh open behaves like a fresh mount + // and a parent re-render of the collapsed control does not clobber an + // externally-set selection. + useEffect(() => { + if (activeMatch !== undefined) { + onSelectTable(activeMatch); + } else if (hasInteractedRef.current) { + onSelectTable(undefined); + } + }, [activeMatch, matches.length, onSelectTable, value]); + + const handleChange = useCallback((e: React.ChangeEvent) => { + hasInteractedRef.current = true; + setValue(e.target.value); + setMatchIndex(0); + }, []); + + const goNext = useCallback(() => { + if (matches.length > 0) { + setMatchIndex((i) => (i + 1) % matches.length); + } + }, [matches.length]); + + const goPrev = useCallback(() => { + if (matches.length > 0) { + setMatchIndex((i) => (i - 1 + matches.length) % matches.length); + } + }, [matches.length]); + + const collapse = useCallback(() => { + hasInteractedRef.current = false; + setExpanded(false); + setValue(''); + setMatchIndex(0); + onSelectTable(undefined); + }, [onSelectTable]); + + const handleBlur = useCallback(() => { + if (!value) { + hasInteractedRef.current = false; + setExpanded(false); + } + }, [value]); + + const handleKeyDown = useCallback( + (e: React.KeyboardEvent) => { + if (e.key === 'Escape') { + e.preventDefault(); + collapse(); + } else if (e.key === 'Enter') { + e.preventDefault(); + if (e.shiftKey) { + goPrev(); + } else { + goNext(); + } + } + }, + [collapse, goNext, goPrev] + ); + + if (!expanded) { + return ( + + ); + } + + return ( +
+ + {matches.length > 0 && ( + <> + + {matchIndex + 1}/{matches.length} + + + + + )} + +
+ ); +} diff --git a/app/src/components/Workspace.tsx b/app/src/components/Workspace.tsx index e228b2b2..d069e786 100644 --- a/app/src/components/Workspace.tsx +++ b/app/src/components/Workspace.tsx @@ -17,7 +17,7 @@ import { KeyboardShortcutsDialog } from './KeyboardShortcutsDialog'; import { CommandPalette } from './CommandPalette'; import { Tooltip, TooltipContent, TooltipProvider, TooltipTrigger } from './ui/tooltip'; import { useProject } from '@/lib/project-store'; -import { NavigationProvider } from '@/lib/navigation-context'; +import { NavigationProvider, useNavigation } from '@/lib/navigation-context'; import { FocusRegistryProvider } from '@/lib/focus-registry'; import { useGlobalShortcuts, useAnalysis } from '@/hooks'; import type { GlobalShortcut } from '@/hooks'; @@ -25,6 +25,9 @@ import { useThemeStore, type Theme } from '@/lib/theme-store'; import { useViewStateStore } from '@/lib/view-state-store'; import { getShortcutDisplay } from '@/lib/shortcuts'; import { useBackend } from '@/lib/backend-context'; +import { LibrarianPanel, useSyncActiveProject } from '@/features/librarian'; +import type { ChatReference } from '@/features/librarian/utils/schema-identifiers'; +import { resolveLineageNodeIds } from '@/lib/lineage-node-resolver'; interface WorkspaceProps { backendReady: boolean; @@ -42,6 +45,7 @@ const EDITOR_PANEL_DEFAULT_SIZE = 33; export function Workspace({ backendReady, error, onRetry, isRetrying }: WorkspaceProps) { const { currentProject, selectFile, activeProjectId, isBackendMode } = useProject(); + useSyncActiveProject(); const { adapter } = useBackend(); const analysis = useAnalysis(backendReady, { adapter }); const lineageActions = useLineageActions(); @@ -71,6 +75,11 @@ export function Workspace({ backendReady, error, onRetry, isRetrying }: Workspac toast.success(`Theme: ${nextTheme.charAt(0).toUpperCase() + nextTheme.slice(1)}`); }, [theme, setTheme]); + // Librarian panel state + const librarianOpen = useViewStateStore((s) => s.librarianOpen); + const toggleLibrarian = useViewStateStore((s) => s.toggleLibrarian); + const setLibrarianOpen = useViewStateStore((s) => s.setLibrarianOpen); + const editorPanelRef = useRef(null); const graphContainerRef = useRef(null); @@ -206,8 +215,14 @@ export function Workspace({ backendReady, error, onRetry, isRetrying }: Workspac cmdOrCtrl: true, handler: () => setCommandPaletteOpen(true), }, + // Toggle Librarian panel + { + key: 'l', + cmdOrCtrl: true, + handler: toggleLibrarian, + }, ], - [toggleEditorPanel, currentProject, cycleTheme, isBackendMode] + [toggleEditorPanel, currentProject, cycleTheme, isBackendMode, toggleLibrarian] ); useGlobalShortcuts(shortcuts); @@ -237,6 +252,9 @@ export function Workspace({ backendReady, error, onRetry, isRetrying }: Workspac case 'toggle-editor': toggleEditorPanel(); break; + case 'toggle-librarian': + toggleLibrarian(); + break; // Tab switching case 'tab-lineage': @@ -295,6 +313,7 @@ export function Workspace({ backendReady, error, onRetry, isRetrying }: Workspac }, [ toggleEditorPanel, + toggleLibrarian, activeProjectId, setActiveTab, currentProject, @@ -468,6 +487,21 @@ export function Workspace({ backendReady, error, onRetry, isRetrying }: Workspac isAnalyzing={analysis.isAnalyzing} /> + + {/* Right: Librarian Panel */} + {librarianOpen && ( + <> + + + setLibrarianOpen(false)} /> + + + )}
@@ -475,3 +509,46 @@ export function Workspace({ backendReady, error, onRetry, isRetrying }: Workspac ); } + +function LibrarianPanelWithNavigation({ onClose }: { onClose: () => void }) { + const { navigateTo } = useNavigation(); + const { result, showColumnEdges } = useLineageState(); + const { toggleColumnEdges } = useLineageActions(); + const { activeProjectId } = useProject(); + const updateViewState = useViewStateStore((s) => s.updateViewState); + const handleNavigateToReferences = useCallback( + (refs: ChatReference[]) => { + if (refs.length === 0) return; + // Drive the lineage search pipeline first, regardless of whether the + // resolver can produce concrete node ids. If the answer references any + // column, prefer searching by the first column name (and force + // "show column edges" on so columns are actually matchable); otherwise + // fall back to the first table name. Writing the term to the persisted + // per-project view state propagates to GraphView's controlled + // searchTerm prop, so table cards highlight via `isNodeHighlighted` + // even when navigation is a no-op (e.g. column nodes absent from the + // current statement's lineage). + const firstColumn = refs.find((r) => r.columnName)?.columnName; + const firstTable = refs.find((r) => r.tableName)?.tableName; + const searchTerm = firstColumn ?? firstTable; + if (firstColumn && !showColumnEdges) { + toggleColumnEdges(); + } + if (searchTerm && activeProjectId) { + updateViewState(activeProjectId, 'lineage', { searchTerm }); + } + const { nodeIds, tablesToExpand, primaryFocusId } = resolveLineageNodeIds( + result ?? null, + refs + ); + if (nodeIds.length === 0) return; + navigateTo('lineage', { + highlightNodeIds: nodeIds, + tablesToExpand, + ...(primaryFocusId ? { primaryFocusId } : {}), + }); + }, + [navigateTo, result, showColumnEdges, toggleColumnEdges, activeProjectId, updateViewState] + ); + return ; +} diff --git a/app/src/components/ui/popover.tsx b/app/src/components/ui/popover.tsx new file mode 100644 index 00000000..a5222946 --- /dev/null +++ b/app/src/components/ui/popover.tsx @@ -0,0 +1,33 @@ +'use client'; + +import * as React from 'react'; +import * as PopoverPrimitive from '@radix-ui/react-popover'; + +import { cn } from '@/lib/utils'; + +const Popover = PopoverPrimitive.Root; + +const PopoverTrigger = PopoverPrimitive.Trigger; + +const PopoverAnchor = PopoverPrimitive.Anchor; + +const PopoverContent = React.forwardRef< + React.ElementRef, + React.ComponentPropsWithoutRef +>(({ className, align = 'center', sideOffset = 4, ...props }, ref) => ( + + + +)); +PopoverContent.displayName = PopoverPrimitive.Content.displayName; + +export { Popover, PopoverTrigger, PopoverContent, PopoverAnchor }; diff --git a/app/src/features/librarian/TEST_CASES.md b/app/src/features/librarian/TEST_CASES.md new file mode 100644 index 00000000..40440a59 --- /dev/null +++ b/app/src/features/librarian/TEST_CASES.md @@ -0,0 +1,162 @@ +# Librarian — Manual Test Cases + +Manual test cases for the Librarian feature. Automated tests live in `__tests__/`. + +## Prerequisites + +- FlowScope app running (`yarn dev` from `app/`) +- Valid AI provider configured (OpenAI, Anthropic, or custom endpoint) +- Sample PDFs available: + - `SAP_Invoice_Approval_Technical_Documentation.pdf` + - `SAP_Payment_Block_Reference.pdf` + - `large_test_file.pdf` (>10 MB, for size limit test) + - `test_document.docx` (for file type test) + +### Test SQL script + +Test cases assume the following SQL is loaded in the editor (SAP Accounts Payable: Invoice Processing Pipeline, standard SAP FI + MM tables): + +```sql +-- SAP Accounts Payable: Invoice Processing Pipeline +-- Standard SAP tables only (FI + MM modules) + +-- Invoice header with vendor and company details +SELECT + rbkp.BELNR AS invoice_doc_number, + rbkp.GJAHR AS fiscal_year, + rbkp.BUKRS AS company_code, + t001.BUTXT AS company_name, + rbkp.LIFNR AS vendor_number, + lfa1.NAME1 AS vendor_name, + lfa1.LAND1 AS vendor_country, + rbkp.RMWWR AS invoice_amount, + rbkp.WAERS AS currency, + rbkp.ZTERM AS payment_terms, + rbkp.BLDAT AS invoice_date, + rbkp.BUDAT AS posting_date, + rbkp.RBSTAT AS invoice_status, + rbkp.ZLSPR AS payment_block, + rbkp.WMWST1 AS tax_amount +FROM RBKP AS rbkp +JOIN T001 AS t001 + ON rbkp.MANDT = t001.MANDT + AND rbkp.BUKRS = t001.BUKRS +JOIN LFA1 AS lfa1 + ON rbkp.MANDT = lfa1.MANDT + AND rbkp.LIFNR = lfa1.LIFNR; + +-- Invoice line items linked to purchase orders +SELECT + rseg.BELNR AS invoice_doc_number, + rseg.GJAHR AS fiscal_year, + rseg.BUZEI AS line_item, + rseg.EBELN AS po_number, + rseg.EBELP AS po_item, + rseg.MATNR AS material_number, + rseg.WRBTR AS item_amount, + rseg.MENGE AS quantity, + rseg.BSTME AS unit, + rseg.KOSTL AS cost_center, + ekko.BSART AS po_doc_type, + ekko.BEDAT AS po_date, + ekko.LIFNR AS po_vendor, + ekpo.TXZ01 AS item_description, + ekpo.NETPR AS po_net_price +FROM RSEG AS rseg +JOIN EKKO AS ekko + ON rseg.MANDT = ekko.MANDT + AND rseg.EBELN = ekko.EBELN +JOIN EKPO AS ekpo + ON rseg.MANDT = ekpo.MANDT + AND rseg.EBELN = ekpo.EBELN + AND rseg.EBELP = ekpo.EBELP; + +-- Accounting document entries for posted invoices +SELECT + bkpf.BUKRS AS company_code, + bkpf.BELNR AS accounting_doc_number, + bkpf.GJAHR AS fiscal_year, + bkpf.BLDAT AS document_date, + bkpf.CPUDT AS entry_date, + bkpf.USNAM AS posted_by, + bkpf.BSTAT AS document_status, + bseg.BUZEI AS line_item, + bseg.KOART AS account_type, + bseg.KONTO AS account_number, + bseg.WRBTR AS amount, + bseg.MWSTS AS tax_amount, + bseg.KOSTL AS cost_center, + bseg.AUGBL AS clearing_document, + bseg.AUGDT AS clearing_date, + bseg.ZLSPR AS payment_block +FROM BKPF AS bkpf +JOIN BSEG AS bseg + ON bkpf.MANDT = bseg.MANDT + AND bkpf.BUKRS = bseg.BUKRS + AND bkpf.BELNR = bseg.BELNR + AND bkpf.GJAHR = bseg.GJAHR; +``` + +--- + +## 1. Functional + +| # | Test Case | Steps | Expected Result | +|---|-----------|-------|-----------------| +| 1 | Upload valid PDF via Upload panel | Click the drop zone and select `SAP_Invoice_Approval_Technical_Documentation.pdf` | File appears in the list with "ready" status | +| 2 | Upload valid PDF via drag-and-drop | Drag `SAP_Invoice_Approval_Technical_Documentation.pdf` onto the Librarian drop zone | File appears in the list (not intercepted by GlobalDropZone) | +| 3 | Upload PDF larger than 10 MB | Try to upload `large_test_file.pdf` | Error: "File exceeds 10 MB limit." | +| 4 | Upload non-PDF file | Try to upload `test_document.docx` | Error: "Only PDF files are supported." | +| 5 | Upload duplicate PDF | Upload `SAP_Invoice_Approval_Technical_Documentation.pdf`, then upload it again | Error: "A file with this name is already uploaded." | +| 6 | Upload multiple different PDFs | Upload `SAP_Payment_Block_Reference.pdf` + `SAP_Invoice_Approval_Technical_Documentation.pdf` | All files appear in the list | +| 7 | Delete PDF and verify | 1) Upload `SAP_Invoice_Approval_Technical_Documentation.pdf`; 2) Delete it; 3) Ask "Which SAP table contains details about invoice approvals?" | PDF removed from list; Documentation section in the answer has no PDF content | +| 8 | Chat scroll | Send 10+ messages | Auto-scrolls to the latest message; scrolling up works | +| 9 | Open/close panel | Toggle panel via the Librarian button (or ⌘L) | Panel opens and closes correctly | +| 10 | AI settings persistence | Configure AI, reload page | API key and model preserved (localStorage) | + +--- + +## 2. Embedding Model + +| # | Test Case | Steps | Expected Result | +|---|-----------|-------|-----------------| +| 11 | First load of embedding model | Clear browser cache, upload first PDF | Model downloads in background, UI remains responsive (not frozen) | +| 12 | Subsequent PDF upload | Upload second PDF after the first is processed | Model reused from cache, faster processing | + +--- + +## 3. LLM Quality — Data Lineage (from SQL) + +| # | Test Case | Question | Expected Answer | +|---|-----------|----------|-----------------| +| 13 | Off-topic question | "What is the capital of France?" | "I can only answer questions related to your data." | +| 14 | Chat context | 1) "What is BKPF?" → 2) "And what key fields does it have?" | Second answer refers to BKPF key fields without needing the full question | +| 15 | Table link | "How are BKPF and BSEG linked?" | Join on MANDT + BUKRS + BELNR + GJAHR (4-field key) | +| 16 | Invoice amount field | "What field stores the invoice amount?" | RBKP.RMWWR (aliased as `invoice_amount`) | +| 17 | Accounting document created date | "What column stores accounting document created date?" | BKPF.CPUDT (Entry Date) | +| 18 | Purchase order to invoice link | "What fields link purchase orders to invoices?" | RSEG.EBELN (PO number) + RSEG.EBELP (PO item), joined to EKKO and EKPO | +| 19 | Vendor country | "What column contains vendor country?" | LFA1.LAND1 (country) | +| 20 | Payment block | "Where is payment block stored?" | RBKP.ZLSPR (invoice level) and BSEG.ZLSPR (accounting level). Answer should appear in both Data Lineage and Documentation sections. | +| 21 | Tax amount | "Where is tax amount stored?" | RBKP.WMWST1 (header) and BSEG.MWSTS (line item) | +| 22 | RBKP vs RSEG | "Difference between RBKP and RSEG?" | RBKP = invoice header (1 record), RSEG = line items (many records) | + +--- + +## 4. LLM Quality — Documentation (from PDF) + +Requires `SAP_Invoice_Approval_Technical_Documentation.pdf` uploaded. + +| # | Test Case | Question | Expected Answer | +|---|-----------|----------|-----------------| +| 23 | Approval statuses | "Possible statuses in approval table?" | ZSTATUS: 01=Pending, 02=Approved, 03=Rejected | +| 24 | Approval table key | "Key for ZSAP_INV_APPROVAL?" | MANDT + ZINV_ID | +| 25 | Rejection reason storage | "Where is rejection reason stored?" | ZSAP_INV_APPROVAL.ZCOMMENT (CHAR 255) | + +--- + +## Notes + +- **Response format**: Every on-topic answer should include three sections: **Summary**, **Data Lineage**, **Documentation**. Each section either contains relevant info or "No information." (exactly). +- **Inline code formatting**: Table and column names should render with colored styling (accent color) in the chat. +- **Source attribution**: Documentation answers should cite the source PDF file name. +- **Off-topic refusal**: Only the refusal sentence — no Summary / Data Lineage / Documentation sections. diff --git a/app/src/features/librarian/__tests__/__mocks__/flowscope-core.ts b/app/src/features/librarian/__tests__/__mocks__/flowscope-core.ts new file mode 100644 index 00000000..e906876b --- /dev/null +++ b/app/src/features/librarian/__tests__/__mocks__/flowscope-core.ts @@ -0,0 +1,3 @@ +export function byteOffsetToCharOffset(_content: string, byteOffset: number): number { + return byteOffset; +} diff --git a/app/src/features/librarian/__tests__/__mocks__/flowscope-react.ts b/app/src/features/librarian/__tests__/__mocks__/flowscope-react.ts new file mode 100644 index 00000000..fcaac4ea --- /dev/null +++ b/app/src/features/librarian/__tests__/__mocks__/flowscope-react.ts @@ -0,0 +1,20 @@ +import { vi } from 'vitest'; + +export const useLineageState = vi.fn(() => ({ + result: null, + viewMode: 'table', + layoutAlgorithm: 'dagre', + hideCTEs: false, + highlightedSpan: null, +})); + +export const useLineageActions = vi.fn(() => ({ + highlightSpan: vi.fn(), + setViewMode: vi.fn(), + toggleColumnEdges: vi.fn(), + setAllNodesCollapsed: vi.fn(), + toggleShowScriptTables: vi.fn(), + setLayoutAlgorithm: vi.fn(), +})); + +export const SqlView = vi.fn(() => null); diff --git a/app/src/features/librarian/__tests__/ai-service.test.ts b/app/src/features/librarian/__tests__/ai-service.test.ts new file mode 100644 index 00000000..5ba1e6fd --- /dev/null +++ b/app/src/features/librarian/__tests__/ai-service.test.ts @@ -0,0 +1,458 @@ +import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'; + +import { + STORAGE_KEY_AI_API_KEY, + STORAGE_KEY_AI_ENDPOINT, + STORAGE_KEY_AI_MODEL, + STORAGE_KEY_AI_PROVIDER, + STORAGE_KEY_AI_SYSTEM_PROMPT, +} from '../constants'; +import { DEFAULT_LIBRARIAN_SYSTEM_PROMPT } from '../services/context-builder'; +import { + type AIConfig, + getDefaultModel, + loadAIConfig, + saveAIConfig, + sendChatMessage, +} from '../services/ai-service'; + +describe('ai-service', () => { + beforeEach(() => { + localStorage.clear(); + }); + + afterEach(() => { + vi.restoreAllMocks(); + }); + + describe('getDefaultModel', () => { + it('returns gpt-4o for openai', () => { + expect(getDefaultModel('openai')).toBe('gpt-4o'); + }); + + it('returns claude model for anthropic', () => { + expect(getDefaultModel('anthropic')).toBe('claude-sonnet-4-20250514'); + }); + }); + + describe('loadAIConfig', () => { + it('returns null when no config is stored', () => { + expect(loadAIConfig()).toBeNull(); + }); + + it('returns null when provider is missing', () => { + localStorage.setItem(STORAGE_KEY_AI_API_KEY, 'sk-test'); + expect(loadAIConfig()).toBeNull(); + }); + + it('returns null when api key is missing', () => { + localStorage.setItem(STORAGE_KEY_AI_PROVIDER, 'openai'); + expect(loadAIConfig()).toBeNull(); + }); + + it('loads config with default model when model is not stored', () => { + localStorage.setItem(STORAGE_KEY_AI_PROVIDER, 'openai'); + localStorage.setItem(STORAGE_KEY_AI_API_KEY, 'sk-test'); + + const config = loadAIConfig(); + expect(config).toEqual({ + provider: 'openai', + apiKey: 'sk-test', + model: 'gpt-4o', + }); + }); + + it('loads config with stored model', () => { + localStorage.setItem(STORAGE_KEY_AI_PROVIDER, 'anthropic'); + localStorage.setItem(STORAGE_KEY_AI_API_KEY, 'sk-ant-test'); + localStorage.setItem(STORAGE_KEY_AI_MODEL, 'claude-3-haiku'); + + const config = loadAIConfig(); + expect(config).toEqual({ + provider: 'anthropic', + apiKey: 'sk-ant-test', + model: 'claude-3-haiku', + }); + }); + + it('loads config with stored system prompt override', () => { + localStorage.setItem(STORAGE_KEY_AI_PROVIDER, 'openai'); + localStorage.setItem(STORAGE_KEY_AI_API_KEY, 'sk-test'); + localStorage.setItem(STORAGE_KEY_AI_SYSTEM_PROMPT, 'Custom prompt'); + + expect(loadAIConfig()).toEqual({ + provider: 'openai', + apiKey: 'sk-test', + model: 'gpt-4o', + systemPrompt: 'Custom prompt', + }); + }); + + it('returns null if localStorage throws', () => { + vi.spyOn(Storage.prototype, 'getItem').mockImplementation(() => { + throw new Error('localStorage disabled'); + }); + expect(loadAIConfig()).toBeNull(); + }); + }); + + describe('saveAIConfig', () => { + it('saves all config fields to localStorage', () => { + const config: AIConfig = { + provider: 'openai', + apiKey: 'sk-123', + model: 'gpt-4o-mini', + }; + + saveAIConfig(config); + + expect(localStorage.getItem(STORAGE_KEY_AI_PROVIDER)).toBe('openai'); + expect(localStorage.getItem(STORAGE_KEY_AI_API_KEY)).toBe('sk-123'); + expect(localStorage.getItem(STORAGE_KEY_AI_MODEL)).toBe('gpt-4o-mini'); + }); + + it('roundtrips through load', () => { + const config: AIConfig = { + provider: 'anthropic', + apiKey: 'sk-ant-456', + model: 'claude-sonnet-4-20250514', + }; + + saveAIConfig(config); + expect(loadAIConfig()).toEqual(config); + }); + + it('saves and loads custom provider config with apiEndpoint', () => { + const config: AIConfig = { + provider: 'custom', + apiKey: 'my-key', + model: 'my-model', + apiEndpoint: 'https://litellm.example.com', + }; + + saveAIConfig(config); + + expect(localStorage.getItem(STORAGE_KEY_AI_PROVIDER)).toBe('custom'); + expect(localStorage.getItem(STORAGE_KEY_AI_ENDPOINT)).toBe('https://litellm.example.com'); + + const loaded = loadAIConfig(); + expect(loaded).toEqual(config); + }); + + it('removes apiEndpoint from localStorage when not provided', () => { + localStorage.setItem(STORAGE_KEY_AI_ENDPOINT, 'https://old-endpoint.com'); + + saveAIConfig({ + provider: 'openai', + apiKey: 'sk-123', + model: 'gpt-4o', + }); + + expect(localStorage.getItem(STORAGE_KEY_AI_ENDPOINT)).toBeNull(); + }); + + it('saves a custom system prompt override', () => { + saveAIConfig({ + provider: 'openai', + apiKey: 'sk-123', + model: 'gpt-4o', + systemPrompt: 'Custom prompt', + }); + + expect(localStorage.getItem(STORAGE_KEY_AI_SYSTEM_PROMPT)).toBe('Custom prompt'); + expect(loadAIConfig()?.systemPrompt).toBe('Custom prompt'); + }); + + it('removes system prompt override when saving the default prompt', () => { + localStorage.setItem(STORAGE_KEY_AI_SYSTEM_PROMPT, 'Old prompt'); + + saveAIConfig({ + provider: 'openai', + apiKey: 'sk-123', + model: 'gpt-4o', + systemPrompt: DEFAULT_LIBRARIAN_SYSTEM_PROMPT, + }); + + expect(localStorage.getItem(STORAGE_KEY_AI_SYSTEM_PROMPT)).toBeNull(); + }); + + it('preserves trailing whitespace in the persisted system prompt', () => { + const promptWithWhitespace = 'Custom prompt\n\n'; + + saveAIConfig({ + provider: 'openai', + apiKey: 'sk-123', + model: 'gpt-4o', + systemPrompt: promptWithWhitespace, + }); + + expect(localStorage.getItem(STORAGE_KEY_AI_SYSTEM_PROMPT)).toBe(promptWithWhitespace); + expect(loadAIConfig()?.systemPrompt).toBe(promptWithWhitespace); + }); + + it('treats a whitespace-only system prompt as no override', () => { + localStorage.setItem(STORAGE_KEY_AI_SYSTEM_PROMPT, 'Old prompt'); + + saveAIConfig({ + provider: 'openai', + apiKey: 'sk-123', + model: 'gpt-4o', + systemPrompt: ' \n\t', + }); + + expect(localStorage.getItem(STORAGE_KEY_AI_SYSTEM_PROMPT)).toBeNull(); + }); + + it('does not persist the default prompt as an override when padded with whitespace', () => { + localStorage.setItem(STORAGE_KEY_AI_SYSTEM_PROMPT, 'Old prompt'); + + saveAIConfig({ + provider: 'openai', + apiKey: 'sk-123', + model: 'gpt-4o', + systemPrompt: ` ${DEFAULT_LIBRARIAN_SYSTEM_PROMPT}\n`, + }); + + expect(localStorage.getItem(STORAGE_KEY_AI_SYSTEM_PROMPT)).toBeNull(); + }); + }); + + describe('sendChatMessage', () => { + const openaiConfig: AIConfig = { + provider: 'openai', + apiKey: 'sk-test', + model: 'gpt-4o', + }; + + const anthropicConfig: AIConfig = { + provider: 'anthropic', + apiKey: 'sk-ant-test', + model: 'claude-sonnet-4-20250514', + }; + + it('sends correct request to OpenAI', async () => { + const mockFetch = vi.fn().mockResolvedValue({ + ok: true, + json: () => + Promise.resolve({ + choices: [{ message: { content: 'Hello from OpenAI' } }], + }), + }); + vi.stubGlobal('fetch', mockFetch); + + const result = await sendChatMessage(openaiConfig, 'system prompt', 'user message'); + + expect(result).toBe('Hello from OpenAI'); + expect(mockFetch).toHaveBeenCalledWith( + 'https://api.openai.com/v1/chat/completions', + expect.objectContaining({ + method: 'POST', + headers: expect.objectContaining({ + Authorization: 'Bearer sk-test', + }), + }) + ); + + const body = JSON.parse(mockFetch.mock.calls[0][1].body); + expect(body.model).toBe('gpt-4o'); + expect(body.messages).toEqual([ + { role: 'system', content: 'system prompt' }, + { role: 'user', content: 'user message' }, + ]); + }); + + it('sends correct request to Anthropic', async () => { + const mockFetch = vi.fn().mockResolvedValue({ + ok: true, + json: () => + Promise.resolve({ + content: [{ type: 'text', text: 'Hello from Anthropic' }], + }), + }); + vi.stubGlobal('fetch', mockFetch); + + const result = await sendChatMessage(anthropicConfig, 'system prompt', 'user message'); + + expect(result).toBe('Hello from Anthropic'); + expect(mockFetch).toHaveBeenCalledWith( + 'https://api.anthropic.com/v1/messages', + expect.objectContaining({ + method: 'POST', + headers: expect.objectContaining({ + 'x-api-key': 'sk-ant-test', + 'anthropic-version': '2023-06-01', + }), + }) + ); + + const body = JSON.parse(mockFetch.mock.calls[0][1].body); + expect(body.model).toBe('claude-sonnet-4-20250514'); + expect(body.system).toBe('system prompt'); + expect(body.messages).toEqual([{ role: 'user', content: 'user message' }]); + }); + + it('throws on OpenAI API error', async () => { + vi.stubGlobal( + 'fetch', + vi.fn().mockResolvedValue({ + ok: false, + status: 401, + text: () => Promise.resolve('Invalid API key'), + }) + ); + + await expect(sendChatMessage(openaiConfig, 'sys', 'msg')).rejects.toThrow( + 'OpenAI API error (401): Invalid API key' + ); + }); + + it('throws on Anthropic API error', async () => { + vi.stubGlobal( + 'fetch', + vi.fn().mockResolvedValue({ + ok: false, + status: 429, + text: () => Promise.resolve('Rate limited'), + }) + ); + + await expect(sendChatMessage(anthropicConfig, 'sys', 'msg')).rejects.toThrow( + 'Anthropic API error (429): Rate limited' + ); + }); + + it('passes abort signal to fetch', async () => { + const controller = new AbortController(); + const mockFetch = vi.fn().mockResolvedValue({ + ok: true, + json: () => + Promise.resolve({ + choices: [{ message: { content: 'ok' } }], + }), + }); + vi.stubGlobal('fetch', mockFetch); + + await sendChatMessage(openaiConfig, 'sys', 'msg', controller.signal); + + expect(mockFetch.mock.calls[0][1].signal).toBe(controller.signal); + }); + + it('throws when OpenAI response has no content', async () => { + vi.stubGlobal( + 'fetch', + vi.fn().mockResolvedValue({ + ok: true, + json: () => Promise.resolve({ choices: [{ message: {} }] }), + }) + ); + + await expect(sendChatMessage(openaiConfig, 'sys', 'msg')).rejects.toThrow( + 'OpenAI API returned no text content.' + ); + }); + + it('throws when Anthropic response has no text block', async () => { + vi.stubGlobal( + 'fetch', + vi.fn().mockResolvedValue({ + ok: true, + json: () => Promise.resolve({ content: [] }), + }) + ); + + await expect(sendChatMessage(anthropicConfig, 'sys', 'msg')).rejects.toThrow( + 'Anthropic API returned no text content.' + ); + }); + + it('routes custom provider to OpenAI format with custom endpoint URL', async () => { + const customConfig: AIConfig = { + provider: 'custom', + apiKey: 'custom-key', + model: 'my-local-model', + apiEndpoint: 'https://litellm.example.com', + }; + + const mockFetch = vi.fn().mockResolvedValue({ + ok: true, + json: () => + Promise.resolve({ + choices: [{ message: { content: 'Hello from custom' } }], + }), + }); + vi.stubGlobal('fetch', mockFetch); + + const result = await sendChatMessage(customConfig, 'system prompt', 'user message'); + + expect(result).toBe('Hello from custom'); + expect(mockFetch).toHaveBeenCalledWith( + 'https://litellm.example.com/v1/chat/completions', + expect.objectContaining({ + method: 'POST', + headers: expect.objectContaining({ + Authorization: 'Bearer custom-key', + }), + }) + ); + + const body = JSON.parse(mockFetch.mock.calls[0][1].body); + expect(body.model).toBe('my-local-model'); + }); + + it('strips trailing slashes from custom endpoint URL', async () => { + const customConfig: AIConfig = { + provider: 'custom', + apiKey: 'key', + model: 'model', + apiEndpoint: 'https://example.com/', + }; + + const mockFetch = vi.fn().mockResolvedValue({ + ok: true, + json: () => + Promise.resolve({ + choices: [{ message: { content: 'ok' } }], + }), + }); + vi.stubGlobal('fetch', mockFetch); + + await sendChatMessage(customConfig, 'sys', 'msg'); + + expect(mockFetch.mock.calls[0][0]).toBe('https://example.com/v1/chat/completions'); + }); + + it('throws custom endpoint error label on failure', async () => { + const customConfig: AIConfig = { + provider: 'custom', + apiKey: 'key', + model: 'model', + apiEndpoint: 'https://bad.example.com', + }; + + vi.stubGlobal( + 'fetch', + vi.fn().mockResolvedValue({ + ok: false, + status: 500, + text: () => Promise.resolve('Internal error'), + }) + ); + + await expect(sendChatMessage(customConfig, 'sys', 'msg')).rejects.toThrow( + 'Custom endpoint API error (500): Internal error' + ); + }); + + it('throws when custom provider has no apiEndpoint', async () => { + const configWithoutEndpoint: AIConfig = { + provider: 'custom', + apiKey: 'sk-custom', + model: 'my-model', + }; + + await expect(sendChatMessage(configWithoutEndpoint, 'sys', 'msg')).rejects.toThrow( + 'Custom provider requires a base URL' + ); + }); + }); +}); diff --git a/app/src/features/librarian/__tests__/components.test.tsx b/app/src/features/librarian/__tests__/components.test.tsx new file mode 100644 index 00000000..7cd85820 --- /dev/null +++ b/app/src/features/librarian/__tests__/components.test.tsx @@ -0,0 +1,885 @@ +/// +import { cleanup, fireEvent, render, screen, waitFor } from '@testing-library/react'; +import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'; + +import { MAX_MESSAGE_LENGTH, MAX_PDF_SIZE_MB } from '../constants'; +import { useLibrarianStore } from '../store'; +import type { ChatMessage, PdfFile } from '../types'; + +// ---------- Mocks ---------- + +vi.mock('../services/ai-service', () => ({ + loadAIConfig: vi.fn(() => ({ + provider: 'openai', + apiKey: 'sk-test', + model: 'gpt-4o', + })), + saveAIConfig: vi.fn(), + sendChatMessage: vi.fn(() => Promise.resolve('ok')), + getDefaultModel: vi.fn((p: string) => (p === 'openai' ? 'gpt-4o' : 'claude-sonnet-4-20250514')), +})); + +import { + getDefaultModel, + loadAIConfig, + saveAIConfig, + sendChatMessage, +} from '../services/ai-service'; +import { DEFAULT_LIBRARIAN_SYSTEM_PROMPT } from '../services/context-builder'; + +// ---------- Helpers ---------- + +function makeMessage(overrides: Partial = {}): ChatMessage { + return { + id: crypto.randomUUID(), + role: 'user', + content: 'hello', + timestamp: Date.now(), + ...overrides, + }; +} + +function makePdfFile(overrides: Partial = {}): PdfFile { + return { + id: 'file-1', + name: 'test.pdf', + size: 1024, + status: 'ready', + uploadedAt: Date.now(), + ...overrides, + }; +} + +// ---------- Components under test ---------- + +import { AISettingsDialog } from '../components/ai-settings-dialog'; +import { ChatInput } from '../components/chat-input'; +import { ChatMessages } from '../components/chat-messages'; +import { PdfUpload } from '../components/pdf-upload'; +import type { SchemaIdentifiers } from '../utils/schema-identifiers'; + +function makeSchema(tables: string[], columns: string[]): SchemaIdentifiers { + return { + tables: new Set(tables), + columns: new Set(columns), + columnOwners: new Map(), + }; +} + +// ---------- Shared setup ---------- + +beforeEach(() => { + localStorage.clear(); + useLibrarianStore.setState({ + byProject: { 'proj-1': { messages: [], pdfFiles: [], pdfChunks: [] } }, + activeProjectId: 'proj-1', + isLoading: false, + hasConfig: true, + }); + vi.clearAllMocks(); + // Restore default mock return values after clearAllMocks + vi.mocked(loadAIConfig).mockReturnValue({ + provider: 'openai', + apiKey: 'sk-test', + model: 'gpt-4o', + }); + vi.mocked(sendChatMessage).mockResolvedValue('ok'); + vi.mocked(getDefaultModel).mockImplementation((p: string) => + p === 'openai' ? 'gpt-4o' : 'claude-sonnet-4-20250514' + ); +}); + +afterEach(() => { + cleanup(); +}); + +// ============================================================================ +// AISettingsDialog +// ============================================================================ + +describe('AISettingsDialog', () => { + it('renders when open', () => { + render(); + expect(screen.getByText('AI Settings')).toBeInTheDocument(); + }); + + it('does not render when closed', () => { + render(); + expect(screen.queryByText('AI Settings')).not.toBeInTheDocument(); + }); + + it('shows provider selector', () => { + render(); + expect(screen.getByText('Provider')).toBeInTheDocument(); + }); + + it('shows API key input', () => { + render(); + expect(screen.getByLabelText('API Key')).toBeInTheDocument(); + }); + + it('shows model input', () => { + render(); + expect(screen.getByLabelText('Model')).toBeInTheDocument(); + }); + + it('shows editable system prompt with size', () => { + render(); + const promptInput = screen.getByTestId('system-prompt-textarea') as HTMLTextAreaElement; + expect(promptInput.value).toContain('expert on SQL lineage and data flow'); + expect(screen.getByTestId('prompt-size')).toHaveTextContent(/chars/); + }); + + it('loads existing config on open', () => { + render(); + const apiKeyInput = screen.getByLabelText('API Key') as HTMLInputElement; + expect(apiKeyInput.value).toBe('sk-test'); + }); + + it('calls saveAIConfig on save', () => { + render(); + fireEvent.click(screen.getByText('Save')); + expect(saveAIConfig).toHaveBeenCalled(); + }); + + it('saves a custom system prompt override', () => { + render(); + fireEvent.change(screen.getByTestId('system-prompt-textarea'), { + target: { value: 'Custom Librarian instructions' }, + }); + fireEvent.click(screen.getByText('Save')); + expect(saveAIConfig).toHaveBeenCalledWith( + expect.objectContaining({ systemPrompt: 'Custom Librarian instructions' }) + ); + }); + + it('resets the system prompt to the default', () => { + render(); + const promptInput = screen.getByTestId('system-prompt-textarea') as HTMLTextAreaElement; + fireEvent.change(promptInput, { target: { value: 'Custom Librarian instructions' } }); + fireEvent.click(screen.getByText('Reset to default')); + expect(promptInput.value).toBe(DEFAULT_LIBRARIAN_SYSTEM_PROMPT); + }); + + it('calls refreshConfig after save to update store', () => { + vi.mocked(loadAIConfig).mockReturnValue({ + provider: 'openai', + apiKey: 'sk-new-key', + model: 'gpt-4o', + }); + useLibrarianStore.setState({ hasConfig: false }); + render(); + fireEvent.click(screen.getByText('Save')); + expect(useLibrarianStore.getState().hasConfig).toBe(true); + }); + + it('disables save when API key is empty', () => { + vi.mocked(loadAIConfig).mockReturnValue(null); + render(); + expect(screen.getByText('Save')).toBeDisabled(); + }); + + it('tests connection', async () => { + render(); + // Wait for useEffect to load config and enable button + await waitFor(() => { + expect(screen.getByText('Test Connection')).not.toBeDisabled(); + }); + fireEvent.click(screen.getByText('Test Connection')); + await waitFor(() => { + expect(screen.getByTestId('test-result')).toHaveTextContent('Connection successful'); + }); + }); + + it('shows error on failed connection test', async () => { + vi.mocked(sendChatMessage).mockRejectedValueOnce(new Error('Invalid API key')); + render(); + await waitFor(() => { + expect(screen.getByText('Test Connection')).not.toBeDisabled(); + }); + fireEvent.click(screen.getByText('Test Connection')); + await waitFor(() => { + expect(screen.getByTestId('test-result')).toHaveTextContent('Invalid API key'); + }); + }); +}); + +// ============================================================================ +// ChatMessages +// ============================================================================ + +describe('ChatMessages', () => { + it('renders empty state when no messages', () => { + render(); + expect(screen.getByTestId('empty-state')).toBeInTheDocument(); + expect(screen.getByText(/Ask questions about your data/)).toBeInTheDocument(); + }); + + it('does not show empty state when loading', () => { + render(); + expect(screen.queryByTestId('empty-state')).not.toBeInTheDocument(); + }); + + it('renders user messages', () => { + const messages = [makeMessage({ role: 'user', content: 'Hello there' })]; + render(); + expect(screen.getByText('Hello there')).toBeInTheDocument(); + expect(screen.getByTestId('message-user')).toBeInTheDocument(); + }); + + it('renders assistant messages', () => { + const messages = [makeMessage({ role: 'assistant', content: 'Hi back' })]; + render(); + expect(screen.getByText('Hi back')).toBeInTheDocument(); + expect(screen.getByTestId('message-assistant')).toBeInTheDocument(); + }); + + it('renders loading indicator', () => { + render(); + expect(screen.getByTestId('loading-indicator')).toBeInTheDocument(); + }); + + it('renders code blocks', () => { + const messages = [ + makeMessage({ + role: 'assistant', + content: 'Here is code:\n```sql\nSELECT * FROM t\n```', + }), + ]; + render(); + expect(screen.getByText('SELECT * FROM t')).toBeInTheDocument(); + }); + + it('renders multiple messages in order', () => { + const messages = [ + makeMessage({ id: '1', role: 'user', content: 'Question' }), + makeMessage({ id: '2', role: 'assistant', content: 'Answer' }), + ]; + render(); + expect(screen.getByText('Question')).toBeInTheDocument(); + expect(screen.getByText('Answer')).toBeInTheDocument(); + }); + + it('wraps schema identifiers in assistant messages with the identifier class', () => { + const schema = makeSchema([], ['MANDT']); + const messages = [ + makeMessage({ role: 'assistant', content: 'The client column is MANDT in this table.' }), + ]; + const { container } = render( + + ); + + const idSpan = container.querySelector('[data-identifier="MANDT"]'); + expect(idSpan).not.toBeNull(); + expect(idSpan).toHaveTextContent('MANDT'); + expect(idSpan?.className).toContain('font-mono'); + expect(idSpan?.className).toContain('text-primary'); + expect(idSpan?.className).toContain('font-medium'); + }); + + it('does not style surrounding text as an identifier', () => { + const schema = makeSchema([], ['MANDT']); + const messages = [makeMessage({ role: 'assistant', content: 'The client column is MANDT.' })]; + const { container } = render( + + ); + + const all = container.querySelectorAll('[data-identifier]'); + expect(all).toHaveLength(1); + expect(all[0]).toHaveTextContent('MANDT'); + }); + + it('does not style identifiers in user messages', () => { + const schema = makeSchema([], ['MANDT']); + const messages = [makeMessage({ role: 'user', content: 'What is MANDT used for?' })]; + const { container } = render( + + ); + + expect(container.querySelector('[data-identifier]')).toBeNull(); + }); + + it('makes assistant messages clickable when they reference a table and fires callback', () => { + const schema = makeSchema(['MARA'], []); + const onNavigateToReferences = vi.fn(); + const messages = [makeMessage({ role: 'assistant', content: 'Check MARA for this.' })]; + render( + + ); + + const bubble = screen.getByTestId('message-assistant').querySelector('[role="button"]'); + expect(bubble).not.toBeNull(); + expect(bubble).toHaveAttribute('data-reference-table', 'MARA'); + expect(bubble).toHaveAttribute('data-reference-count', '1'); + expect(bubble?.className).toContain('cursor-pointer'); + + fireEvent.click(bubble!); + expect(onNavigateToReferences).toHaveBeenCalledWith([{ tableName: 'MARA' }]); + }); + + it('passes bare column references through unchanged for the host to resolve', () => { + const schema: SchemaIdentifiers = { + tables: new Set(['MARA']), + columns: new Set(['MANDT']), + columnOwners: new Map([['MANDT', ['MARA']]]), + }; + const onNavigateToReferences = vi.fn(); + const messages = [makeMessage({ role: 'assistant', content: 'The MANDT column exists.' })]; + render( + + ); + + const bubble = screen.getByTestId('message-assistant').querySelector('[role="button"]'); + expect(bubble).toHaveAttribute('data-reference-column', 'MANDT'); + expect(bubble).not.toHaveAttribute('data-reference-table'); + expect(bubble).toHaveAttribute('aria-label', 'Open highlighted nodes in lineage view'); + + fireEvent.click(bubble!); + expect(onNavigateToReferences).toHaveBeenCalledWith([ + { columnName: 'MANDT', bareColumn: true }, + ]); + }); + + it('passes every parsed reference for a multi-identifier message', () => { + const schema: SchemaIdentifiers = { + tables: new Set(['BKPF', 'BSEG']), + columns: new Set(['MANDT', 'BUKRS']), + columnOwners: new Map([ + ['MANDT', ['BKPF', 'BSEG']], + ['BUKRS', ['BKPF']], + ]), + }; + const onNavigateToReferences = vi.fn(); + const messages = [ + makeMessage({ + role: 'assistant', + content: 'BKPF.MANDT links to BSEG via BUKRS.', + }), + ]; + render( + + ); + + const bubble = screen.getByTestId('message-assistant').querySelector('[role="button"]')!; + expect(bubble).toHaveAttribute('data-reference-count', '3'); + + fireEvent.click(bubble); + expect(onNavigateToReferences).toHaveBeenCalledWith([ + { tableName: 'BKPF', columnName: 'MANDT' }, + { tableName: 'BSEG' }, + { columnName: 'BUKRS', bareColumn: true }, + ]); + }); + + it('does not make assistant messages clickable when there is no resolvable reference', () => { + const schema = makeSchema(['MARA'], []); + const onNavigateToReferences = vi.fn(); + const messages = [ + makeMessage({ role: 'assistant', content: 'Just a plain answer with no references.' }), + ]; + render( + + ); + + const bubble = screen.getByTestId('message-assistant').querySelector('[role="button"]'); + expect(bubble).toBeNull(); + }); + + it('does not make assistant messages clickable when no callback is provided', () => { + const schema = makeSchema(['MARA'], []); + const messages = [makeMessage({ role: 'assistant', content: 'Check MARA.' })]; + render(); + + const bubble = screen.getByTestId('message-assistant').querySelector('[role="button"]'); + expect(bubble).toBeNull(); + }); + + it('activates the callback via keyboard (Enter)', () => { + const schema = makeSchema(['MARA'], []); + const onNavigateToReferences = vi.fn(); + const messages = [makeMessage({ role: 'assistant', content: 'MARA row.' })]; + render( + + ); + + const bubble = screen.getByTestId('message-assistant').querySelector('[role="button"]'); + fireEvent.keyDown(bubble!, { key: 'Enter' }); + expect(onNavigateToReferences).toHaveBeenCalledWith([{ tableName: 'MARA' }]); + }); + + it('keyboard activation ignores stale page-wide text selection', () => { + // Regression: the selection guard previously fired for both mouse clicks + // and keyboard activation. If the user had any text selected anywhere on + // the page (e.g., in another message) and pressed Enter on a focused + // bubble, navigation was silently aborted. + const schema = makeSchema(['MARA'], []); + const onNavigateToReferences = vi.fn(); + const messages = [makeMessage({ role: 'assistant', content: 'Check MARA for this.' })]; + render( + + ); + + const bubble = screen.getByTestId('message-assistant').querySelector('[role="button"]')!; + const originalGetSelection = window.getSelection; + window.getSelection = vi.fn( + () => ({ toString: () => 'stale page-wide selection' }) as unknown as Selection + ); + try { + fireEvent.keyDown(bubble, { key: 'Enter' }); + expect(onNavigateToReferences).toHaveBeenCalledWith([{ tableName: 'MARA' }]); + } finally { + window.getSelection = originalGetSelection; + } + }); + + it('does not navigate when text is selected inside the bubble (preserves copy/select)', () => { + const schema = makeSchema(['MARA'], []); + const onNavigateToReferences = vi.fn(); + const messages = [makeMessage({ role: 'assistant', content: 'Check MARA for this.' })]; + render( + + ); + + const bubble = screen.getByTestId('message-assistant').querySelector('[role="button"]')!; + const originalGetSelection = window.getSelection; + window.getSelection = vi.fn( + () => ({ toString: () => 'some selected text' }) as unknown as Selection + ); + try { + fireEvent.click(bubble); + expect(onNavigateToReferences).not.toHaveBeenCalled(); + } finally { + window.getSelection = originalGetSelection; + } + }); + + it('does not navigate when clicking inside a code block (allows code copy)', () => { + const schema = makeSchema(['MARA'], []); + const onNavigateToReferences = vi.fn(); + const messages = [ + makeMessage({ + role: 'assistant', + content: 'See MARA below.\n```sql\nSELECT * FROM MARA;\n```', + }), + ]; + const { container } = render( + + ); + + const pre = container.querySelector('pre')!; + expect(pre).not.toBeNull(); + fireEvent.click(pre); + expect(onNavigateToReferences).not.toHaveBeenCalled(); + }); + + it('does not make user messages clickable even when they mention identifiers', () => { + const schema = makeSchema(['MARA'], []); + const onNavigateToReferences = vi.fn(); + const messages = [makeMessage({ role: 'user', content: 'Tell me about MARA.' })]; + render( + + ); + + const bubble = screen.getByTestId('message-user').querySelector('[role="button"]'); + expect(bubble).toBeNull(); + }); +}); + +// ============================================================================ +// ChatInput +// ============================================================================ + +describe('ChatInput', () => { + it('renders textarea', () => { + render(); + expect(screen.getByTestId('chat-textarea')).toBeInTheDocument(); + }); + + it('renders send button', () => { + render(); + expect(screen.getByLabelText('Send message')).toBeInTheDocument(); + }); + + it('calls onSend when clicking send button', () => { + const onSend = vi.fn(); + render(); + const textarea = screen.getByTestId('chat-textarea'); + fireEvent.change(textarea, { target: { value: 'test message' } }); + fireEvent.click(screen.getByLabelText('Send message')); + expect(onSend).toHaveBeenCalledWith('test message'); + }); + + it('calls onSend on Enter key', () => { + const onSend = vi.fn(); + render(); + const textarea = screen.getByTestId('chat-textarea'); + fireEvent.change(textarea, { target: { value: 'enter test' } }); + fireEvent.keyDown(textarea, { key: 'Enter', shiftKey: false }); + expect(onSend).toHaveBeenCalledWith('enter test'); + }); + + it('does not send on Shift+Enter', () => { + const onSend = vi.fn(); + render(); + const textarea = screen.getByTestId('chat-textarea'); + fireEvent.change(textarea, { target: { value: 'shift enter' } }); + fireEvent.keyDown(textarea, { key: 'Enter', shiftKey: true }); + expect(onSend).not.toHaveBeenCalled(); + }); + + it('clears input after sending', () => { + const onSend = vi.fn(); + render(); + const textarea = screen.getByTestId('chat-textarea') as HTMLTextAreaElement; + fireEvent.change(textarea, { target: { value: 'clear me' } }); + fireEvent.click(screen.getByLabelText('Send message')); + expect(textarea.value).toBe(''); + }); + + it('does not send empty messages', () => { + const onSend = vi.fn(); + render(); + fireEvent.click(screen.getByLabelText('Send message')); + expect(onSend).not.toHaveBeenCalled(); + }); + + it('does not send whitespace-only messages', () => { + const onSend = vi.fn(); + render(); + const textarea = screen.getByTestId('chat-textarea'); + fireEvent.change(textarea, { target: { value: ' ' } }); + fireEvent.click(screen.getByLabelText('Send message')); + expect(onSend).not.toHaveBeenCalled(); + }); + + it('disables textarea when disabled prop is true', () => { + render(); + expect(screen.getByTestId('chat-textarea')).toBeDisabled(); + }); + + it('shows hint when AI is not configured', () => { + useLibrarianStore.setState({ hasConfig: false }); + render(); + expect(screen.getByTestId('config-hint')).toBeInTheDocument(); + }); + + it('enables send when hasConfig becomes true in store', () => { + useLibrarianStore.setState({ hasConfig: false }); + const { rerender } = render(); + expect(screen.getByTestId('chat-textarea')).toBeDisabled(); + + useLibrarianStore.setState({ hasConfig: true }); + rerender(); + expect(screen.getByTestId('chat-textarea')).not.toBeDisabled(); + }); + + it('shows no-project hint and disables input when noActiveProject is true', () => { + render(); + expect(screen.getByTestId('no-project-hint')).toHaveTextContent( + /Open or create a project to use Librarian/ + ); + expect(screen.getByTestId('chat-textarea')).toBeDisabled(); + expect(screen.queryByTestId('config-hint')).not.toBeInTheDocument(); + }); + + it('does not call onSend when noActiveProject is true', () => { + const onSend = vi.fn(); + render(); + const textarea = screen.getByTestId('chat-textarea'); + fireEvent.change(textarea, { target: { value: 'test' } }); + fireEvent.keyDown(textarea, { key: 'Enter', shiftKey: false }); + expect(onSend).not.toHaveBeenCalled(); + }); + + it('truncates input to MAX_MESSAGE_LENGTH', () => { + render(); + const textarea = screen.getByTestId('chat-textarea') as HTMLTextAreaElement; + const longText = 'a'.repeat(MAX_MESSAGE_LENGTH + 100); + fireEvent.change(textarea, { target: { value: longText } }); + expect(textarea.value.length).toBe(MAX_MESSAGE_LENGTH); + }); +}); + +// ============================================================================ +// PdfUpload +// ============================================================================ + +describe('PdfUpload', () => { + it('renders drop zone', () => { + render(); + expect(screen.getByTestId('drop-zone')).toBeInTheDocument(); + expect(screen.getByText(/Drop a PDF/)).toBeInTheDocument(); + }); + + it('calls onUpload for valid PDF file', () => { + const onUpload = vi.fn(); + render(); + const input = screen.getByTestId('file-input'); + const file = new File(['content'], 'test.pdf', { type: 'application/pdf' }); + fireEvent.change(input, { target: { files: [file] } }); + expect(onUpload).toHaveBeenCalledWith(file); + }); + + it('rejects non-PDF files', () => { + const onUpload = vi.fn(); + render(); + const input = screen.getByTestId('file-input'); + const file = new File(['content'], 'test.txt', { type: 'text/plain' }); + fireEvent.change(input, { target: { files: [file] } }); + expect(onUpload).not.toHaveBeenCalled(); + expect(screen.getByTestId('upload-error')).toHaveTextContent('Only PDF files'); + }); + + it('rejects files exceeding size limit', () => { + const onUpload = vi.fn(); + render(); + const input = screen.getByTestId('file-input'); + const bigContent = new ArrayBuffer(MAX_PDF_SIZE_MB * 1024 * 1024 + 1); + const file = new File([bigContent], 'big.pdf', { type: 'application/pdf' }); + fireEvent.change(input, { target: { files: [file] } }); + expect(onUpload).not.toHaveBeenCalled(); + expect(screen.getByTestId('upload-error')).toHaveTextContent(`${MAX_PDF_SIZE_MB} MB`); + }); + + it('allows uploading many files (no file count limit)', () => { + const files = Array.from({ length: 20 }, (_, i) => + makePdfFile({ id: `f-${i}`, name: `file-${i}.pdf` }) + ); + useLibrarianStore.setState({ + byProject: { 'proj-1': { messages: [], pdfFiles: files, pdfChunks: [] } }, + activeProjectId: 'proj-1', + }); + + const onUpload = vi.fn(); + render(); + const input = screen.getByTestId('file-input'); + const file = new File(['content'], 'extra.pdf', { type: 'application/pdf' }); + fireEvent.change(input, { target: { files: [file] } }); + expect(onUpload).toHaveBeenCalledWith(file); + }); + + it('rejects duplicate file names', () => { + useLibrarianStore.setState({ + byProject: { + 'proj-1': { + messages: [], + pdfFiles: [makePdfFile({ name: 'dup.pdf' })], + pdfChunks: [], + }, + }, + activeProjectId: 'proj-1', + }); + + const onUpload = vi.fn(); + render(); + const input = screen.getByTestId('file-input'); + const file = new File(['content'], 'dup.pdf', { type: 'application/pdf' }); + fireEvent.change(input, { target: { files: [file] } }); + expect(onUpload).not.toHaveBeenCalled(); + expect(screen.getByTestId('upload-error')).toHaveTextContent('already uploaded'); + }); + + it('renders file list with status', () => { + useLibrarianStore.setState({ + byProject: { + 'proj-1': { + messages: [], + pdfFiles: [ + makePdfFile({ id: 'f1', name: 'ready.pdf', status: 'ready' }), + makePdfFile({ id: 'f2', name: 'processing.pdf', status: 'processing' }), + ], + pdfChunks: [], + }, + }, + activeProjectId: 'proj-1', + }); + + render(); + const items = screen.getAllByTestId('pdf-file-item'); + expect(items).toHaveLength(2); + expect(screen.getByText('ready.pdf')).toBeInTheDocument(); + expect(screen.getByText('processing.pdf')).toBeInTheDocument(); + }); + + it('removes file when clicking remove button', () => { + useLibrarianStore.setState({ + byProject: { + 'proj-1': { + messages: [], + pdfFiles: [makePdfFile({ id: 'f1', name: 'remove-me.pdf' })], + pdfChunks: [], + }, + }, + activeProjectId: 'proj-1', + }); + + render(); + fireEvent.click(screen.getByLabelText('Remove remove-me.pdf')); + expect(useLibrarianStore.getState().byProject['proj-1'].pdfFiles).toHaveLength(0); + }); + + it('only lists PDFs from the active project', () => { + useLibrarianStore.setState({ + byProject: { + 'proj-1': { + messages: [], + pdfFiles: [makePdfFile({ id: 'a1', name: 'project-a.pdf' })], + pdfChunks: [], + }, + 'proj-2': { + messages: [], + pdfFiles: [makePdfFile({ id: 'b1', name: 'project-b.pdf' })], + pdfChunks: [], + }, + }, + activeProjectId: 'proj-1', + }); + + render(); + expect(screen.getByText('project-a.pdf')).toBeInTheDocument(); + expect(screen.queryByText('project-b.pdf')).not.toBeInTheDocument(); + }); + + it('handles drag and drop', () => { + const onUpload = vi.fn(); + render(); + const dropZone = screen.getByTestId('drop-zone'); + + const file = new File(['content'], 'dropped.pdf', { type: 'application/pdf' }); + const dataTransfer = { files: [file] }; + + fireEvent.dragOver(dropZone, { dataTransfer }); + fireEvent.drop(dropZone, { dataTransfer }); + + expect(onUpload).toHaveBeenCalledWith(file); + }); + + it('renders ScrollArea with max-h-[64px] when many files are uploaded', () => { + const files = Array.from({ length: 6 }, (_, i) => + makePdfFile({ id: `scroll-${i}`, name: `doc-${i}.pdf` }) + ); + useLibrarianStore.setState({ + byProject: { 'proj-1': { messages: [], pdfFiles: files, pdfChunks: [] } }, + activeProjectId: 'proj-1', + }); + + const { container } = render(); + const items = screen.getAllByTestId('pdf-file-item'); + expect(items).toHaveLength(6); + + const scrollArea = container.querySelector('.max-h-\\[64px\\]'); + expect(scrollArea).toBeInTheDocument(); + }); + + it('keeps size and delete button visible while truncating long file names', () => { + const longName = + 'a-very-long-pdf-file-name-that-should-be-truncated-with-ellipsis-when-the-panel-is-narrow.pdf'; + useLibrarianStore.setState({ + byProject: { + 'proj-1': { + messages: [], + pdfFiles: [makePdfFile({ id: 'long-1', name: longName, size: 2_500_000 })], + pdfChunks: [], + }, + }, + activeProjectId: 'proj-1', + }); + + render(); + + const item = screen.getByTestId('pdf-file-item'); + const nameSpan = item.querySelector('span.truncate'); + expect(nameSpan).not.toBeNull(); + expect(nameSpan).toHaveTextContent(longName); + expect(nameSpan?.className).toContain('min-w-0'); + expect(nameSpan?.className).toContain('flex-1'); + expect(nameSpan?.className).toContain('truncate'); + + const sizeSpan = screen.getByText('2.4 MB'); + expect(sizeSpan.className).toContain('shrink-0'); + expect(sizeSpan.className).toContain('whitespace-nowrap'); + + const removeButton = screen.getByLabelText(`Remove ${longName}`); + expect(removeButton.className).toContain('shrink-0'); + }); + + it('has data-librarian-dropzone attribute on drop zone', () => { + render(); + const dropZone = screen.getByTestId('drop-zone'); + expect(dropZone).toHaveAttribute('data-librarian-dropzone'); + }); + + it('calls stopPropagation on drop to prevent global handler', () => { + const onUpload = vi.fn(); + render(); + const dropZone = screen.getByTestId('drop-zone'); + + const file = new File(['content'], 'stopped.pdf', { type: 'application/pdf' }); + const stopPropagation = vi.fn(); + const dropEvent = new Event('drop', { bubbles: true }); + Object.defineProperty(dropEvent, 'dataTransfer', { value: { files: [file] } }); + Object.defineProperty(dropEvent, 'stopPropagation', { value: stopPropagation }); + Object.defineProperty(dropEvent, 'preventDefault', { value: vi.fn() }); + + fireEvent(dropZone, dropEvent); + expect(stopPropagation).toHaveBeenCalled(); + }); + + it('calls stopPropagation on dragOver to prevent global handler', () => { + render(); + const dropZone = screen.getByTestId('drop-zone'); + + const stopPropagation = vi.fn(); + const dragOverEvent = new Event('dragover', { bubbles: true }); + Object.defineProperty(dragOverEvent, 'stopPropagation', { value: stopPropagation }); + Object.defineProperty(dragOverEvent, 'preventDefault', { value: vi.fn() }); + + fireEvent(dropZone, dragOverEvent); + expect(stopPropagation).toHaveBeenCalled(); + }); +}); diff --git a/app/src/features/librarian/__tests__/constants.test.ts b/app/src/features/librarian/__tests__/constants.test.ts new file mode 100644 index 00000000..952f1c10 --- /dev/null +++ b/app/src/features/librarian/__tests__/constants.test.ts @@ -0,0 +1,47 @@ +import { describe, expect, it } from 'vitest'; + +import { + CHAT_HISTORY_LIMIT, + EMBEDDING_MODEL, + MAX_MESSAGE_LENGTH, + MAX_PDF_SIZE_BYTES, + MAX_PDF_SIZE_MB, + PDF_CHUNK_OVERLAP, + PDF_CHUNK_SIZE, + STORAGE_KEY_AI_API_KEY, + STORAGE_KEY_AI_MODEL, + STORAGE_KEY_AI_PROVIDER, + VECTOR_SEARCH_TOP_K, +} from '../constants'; + +describe('constants', () => { + it('CHAT_HISTORY_LIMIT is 10', () => { + expect(CHAT_HISTORY_LIMIT).toBe(10); + }); + + it('MAX_MESSAGE_LENGTH is 4000', () => { + expect(MAX_MESSAGE_LENGTH).toBe(4000); + }); + + it('MAX_PDF_SIZE_BYTES equals MAX_PDF_SIZE_MB * 1024 * 1024', () => { + expect(MAX_PDF_SIZE_BYTES).toBe(MAX_PDF_SIZE_MB * 1024 * 1024); + }); + + it('PDF_CHUNK_OVERLAP is less than PDF_CHUNK_SIZE', () => { + expect(PDF_CHUNK_OVERLAP).toBeLessThan(PDF_CHUNK_SIZE); + }); + + it('VECTOR_SEARCH_TOP_K is positive', () => { + expect(VECTOR_SEARCH_TOP_K).toBeGreaterThan(0); + }); + + it('EMBEDDING_MODEL is a valid model identifier', () => { + expect(EMBEDDING_MODEL).toBe('Xenova/multilingual-e5-small'); + }); + + it('storage keys are unique strings', () => { + const keys = [STORAGE_KEY_AI_PROVIDER, STORAGE_KEY_AI_API_KEY, STORAGE_KEY_AI_MODEL]; + expect(new Set(keys).size).toBe(keys.length); + keys.forEach((key) => expect(typeof key).toBe('string')); + }); +}); diff --git a/app/src/features/librarian/__tests__/context-builder.test.ts b/app/src/features/librarian/__tests__/context-builder.test.ts new file mode 100644 index 00000000..ee93bbca --- /dev/null +++ b/app/src/features/librarian/__tests__/context-builder.test.ts @@ -0,0 +1,243 @@ +import { describe, expect, it } from 'vitest'; + +import type { ChatMessage } from '../types'; +import { + DEFAULT_LIBRARIAN_SYSTEM_PROMPT, + buildContext, + buildPrompt, + getPromptStats, +} from '../services/context-builder'; + +function makeMessage(role: 'user' | 'assistant', content: string): ChatMessage { + return { id: `msg-${Date.now()}`, role, content, timestamp: Date.now() }; +} + +describe('buildContext', () => { + it('passes through lineage and pdfCitations unchanged', () => { + const ctx = buildContext({ + lineage: 'some lineage', + pdfCitations: 'citation text', + chatHistory: [], + sqlSnippet: '', + }); + expect(ctx.lineage).toBe('some lineage'); + expect(ctx.pdfCitations).toBe('citation text'); + }); + + it('formats chat history as role: content lines', () => { + const ctx = buildContext({ + lineage: '', + pdfCitations: '', + chatHistory: [makeMessage('user', 'Hello'), makeMessage('assistant', 'Hi there')], + sqlSnippet: '', + }); + expect(ctx.chatHistory).toContain('user: Hello'); + expect(ctx.chatHistory).toContain('assistant: Hi there'); + }); + + it('returns empty chatHistory for no messages', () => { + const ctx = buildContext({ + lineage: '', + pdfCitations: '', + chatHistory: [], + sqlSnippet: '', + }); + expect(ctx.chatHistory).toBe(''); + }); + + it('truncates SQL beyond 3000 characters', () => { + const longSql = 'SELECT ' + 'x'.repeat(3000); + expect(longSql.length).toBeGreaterThan(3000); + const ctx = buildContext({ + lineage: '', + pdfCitations: '', + chatHistory: [], + sqlSnippet: longSql, + }); + expect(ctx.sqlSnippet).toContain('... (truncated)'); + // First 3000 chars are kept, then the suffix is appended + expect(ctx.sqlSnippet.startsWith(longSql.slice(0, 3000))).toBe(true); + expect(ctx.sqlSnippet.endsWith('... (truncated)')).toBe(true); + }); + + it('does not truncate SQL within 3000 characters', () => { + const shortSql = 'SELECT 1'; + const ctx = buildContext({ + lineage: '', + pdfCitations: '', + chatHistory: [], + sqlSnippet: shortSql, + }); + expect(ctx.sqlSnippet).toBe(shortSql); + }); +}); + +describe('buildPrompt', () => { + it('includes system instruction', () => { + const prompt = buildPrompt({ + lineage: '', + pdfCitations: '', + chatHistory: '', + sqlSnippet: '', + }); + expect(prompt).toContain('expert on SQL lineage and data flow'); + }); + + it('uses a custom system prompt when provided', () => { + const prompt = buildPrompt( + { + lineage: 'lineage data', + pdfCitations: '', + chatHistory: '', + sqlSnippet: '', + }, + { systemPrompt: 'Custom instructions' } + ); + expect(prompt).toContain('Custom instructions'); + expect(prompt).not.toContain(DEFAULT_LIBRARIAN_SYSTEM_PROMPT); + expect(prompt).toContain('lineage data'); + }); + + it('falls back to the default prompt when the custom prompt is blank', () => { + const prompt = buildPrompt( + { + lineage: '', + pdfCitations: '', + chatHistory: '', + sqlSnippet: '', + }, + { systemPrompt: ' ' } + ); + expect(prompt).toContain('expert on SQL lineage and data flow'); + }); + + it('includes lineage section when present', () => { + const prompt = buildPrompt({ + lineage: 'table: users', + pdfCitations: '', + chatHistory: '', + sqlSnippet: '', + }); + expect(prompt).toContain('## DATA SOURCE: Data Lineage (from SQL analysis)'); + expect(prompt).toContain('table: users'); + }); + + it('includes SQL section when present', () => { + const prompt = buildPrompt({ + lineage: '', + pdfCitations: '', + chatHistory: '', + sqlSnippet: 'SELECT 1', + }); + expect(prompt).toContain('## DATA SOURCE: SQL Code (from SQL analysis)'); + expect(prompt).toContain('SELECT 1'); + }); + + it('includes documentation section when present', () => { + const prompt = buildPrompt({ + lineage: '', + pdfCitations: 'From doc.pdf page 3: ...', + chatHistory: '', + sqlSnippet: '', + }); + expect(prompt).toContain('## DATA SOURCE: Documentation (from uploaded PDFs)'); + expect(prompt).toContain('From doc.pdf page 3'); + }); + + it('includes conversation history when present', () => { + const prompt = buildPrompt({ + lineage: '', + pdfCitations: '', + chatHistory: 'user: hi\nassistant: hello', + sqlSnippet: '', + }); + expect(prompt).toContain('## Conversation History'); + expect(prompt).toContain('user: hi'); + }); + + it('omits empty sections', () => { + const prompt = buildPrompt({ + lineage: '', + pdfCitations: '', + chatHistory: '', + sqlSnippet: '', + }); + expect(prompt).not.toContain('## DATA SOURCE: Data Lineage'); + expect(prompt).not.toContain('## DATA SOURCE: SQL Code'); + expect(prompt).not.toContain('## DATA SOURCE: Documentation'); + expect(prompt).not.toContain('## Conversation History'); + }); + + it('includes all sections when all present', () => { + const prompt = buildPrompt({ + lineage: 'lineage data', + pdfCitations: 'pdf data', + chatHistory: 'chat data', + sqlSnippet: 'sql data', + }); + expect(prompt).toContain('## DATA SOURCE: Data Lineage (from SQL analysis)'); + expect(prompt).toContain('## DATA SOURCE: SQL Code (from SQL analysis)'); + expect(prompt).toContain('## DATA SOURCE: Documentation (from uploaded PDFs)'); + expect(prompt).toContain('## Conversation History'); + }); + + it('includes source attribution instructions in system prompt', () => { + const prompt = buildPrompt({ + lineage: 'lineage data', + pdfCitations: 'pdf data', + chatHistory: '', + sqlSnippet: '', + }); + expect(prompt).toContain('answer from Data Lineage and SQL Code sources ONLY'); + expect(prompt).toContain('answer from Documentation ONLY'); + expect(prompt).toContain('Never mix sources between sections'); + }); + + it('includes the off-topic refusal rule with the exact canned response', () => { + const prompt = buildPrompt({ + lineage: '', + pdfCitations: '', + chatHistory: '', + sqlSnippet: '', + }); + expect(prompt).toContain('Politely decline off-topic questions'); + expect(prompt).toContain('I can only answer questions related to your data.'); + }); + + it('keeps the "no information" data fallback distinct from the off-topic refusal', () => { + const prompt = buildPrompt({ + lineage: '', + pdfCitations: '', + chatHistory: '', + sqlSnippet: '', + }); + expect(prompt).toContain('If a source has no relevant information, write "No information"'); + }); + + it('includes Summary format guidance with technical names in parentheses', () => { + const prompt = buildPrompt({ + lineage: '', + pdfCitations: '', + chatHistory: '', + sqlSnippet: '', + }); + expect(prompt).toContain('document number (BELNR)'); + }); + + it('instructs the model to render identifiers as inline code', () => { + const prompt = buildPrompt({ + lineage: '', + pdfCitations: '', + chatHistory: '', + sqlSnippet: '', + }); + expect(prompt).toContain('Write table and column names as inline code'); + }); +}); + +describe('getPromptStats', () => { + it('returns character and byte counts', () => { + expect(getPromptStats('abc')).toEqual({ characters: 3, bytes: 3 }); + expect(getPromptStats('é')).toEqual({ characters: 1, bytes: 2 }); + }); +}); diff --git a/app/src/features/librarian/__tests__/embedding-worker.test.ts b/app/src/features/librarian/__tests__/embedding-worker.test.ts new file mode 100644 index 00000000..c568702b --- /dev/null +++ b/app/src/features/librarian/__tests__/embedding-worker.test.ts @@ -0,0 +1,78 @@ +import { beforeEach, describe, expect, it, vi } from 'vitest'; + +const transformersMock = vi.hoisted(() => { + const model = vi.fn(async () => ({ data: new Float32Array([1, 2, 3]) })); + const pipeline = vi.fn(async () => model); + const env = { + allowLocalModels: true, + allowRemoteModels: false, + useBrowserCache: false, + backends: { + onnx: { + wasm: { + wasmPaths: '', + }, + }, + }, + }; + + return { env, model, pipeline }; +}); + +vi.mock('@xenova/transformers', () => transformersMock); + +describe('embedding worker', () => { + beforeEach(() => { + vi.resetModules(); + transformersMock.model.mockClear(); + transformersMock.pipeline.mockReset(); + transformersMock.pipeline.mockResolvedValue(transformersMock.model); + }); + + it('retries model loading after a transient failure', async () => { + const responses: unknown[] = []; + const originalPostMessage = globalThis.postMessage; + const originalOnMessage = globalThis.onmessage; + + Object.defineProperty(globalThis, 'postMessage', { + configurable: true, + value: vi.fn((message: unknown) => responses.push(message)), + }); + + transformersMock.pipeline + .mockRejectedValueOnce(new Error('network interrupted')) + .mockResolvedValue(transformersMock.model); + + try { + await import('../workers/embedding-worker'); + + const sendMessage = globalThis.onmessage as (event: MessageEvent) => Promise; + await sendMessage({ + data: { type: 'embed', id: 1, texts: ['first'], mode: 'passage' }, + } as MessageEvent); + await sendMessage({ + data: { type: 'embed', id: 2, texts: ['second'], mode: 'passage' }, + } as MessageEvent); + } finally { + Object.defineProperty(globalThis, 'postMessage', { + configurable: true, + value: originalPostMessage, + }); + globalThis.onmessage = originalOnMessage; + } + + expect(transformersMock.pipeline).toHaveBeenCalledTimes(2); + expect(responses).toEqual([ + { + id: 1, + success: false, + error: 'network interrupted', + }, + { + id: 2, + success: true, + result: [[1, 2, 3]], + }, + ]); + }); +}); diff --git a/app/src/features/librarian/__tests__/global-dropzone.test.tsx b/app/src/features/librarian/__tests__/global-dropzone.test.tsx new file mode 100644 index 00000000..a34b7666 --- /dev/null +++ b/app/src/features/librarian/__tests__/global-dropzone.test.tsx @@ -0,0 +1,175 @@ +/// +import { act, cleanup, render } from '@testing-library/react'; +import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'; + +// ---------- Mocks ---------- + +const mockImportFiles = vi.fn(); + +vi.mock('@/lib/project-store', () => ({ + useProject: () => ({ + importFiles: mockImportFiles, + currentProject: { id: 'test-project' }, + isReadOnly: false, + }), +})); + +vi.mock('@/lib/constants', () => ({ + ACCEPTED_FILE_TYPES_ARRAY: ['.sql'], + FILE_LIMITS: { MAX_SIZE: 10 * 1024 * 1024, MAX_COUNT: 1000 }, +})); + +import { GlobalDropZone } from '@/components/GlobalDropZone'; + +beforeEach(() => { + vi.clearAllMocks(); +}); + +afterEach(() => { + cleanup(); +}); + +describe('GlobalDropZone', () => { + it('ignores drops inside a librarian dropzone element', async () => { + // Render GlobalDropZone and a librarian dropzone together + const { container } = render( +
+ +
+ Drop here +
+
+ ); + + // Simulate dragenter to activate the overlay + const dragEnterEvent = new Event('dragenter', { bubbles: true }); + Object.defineProperty(dragEnterEvent, 'dataTransfer', { + value: { types: ['Files'], dropEffect: 'copy' }, + }); + Object.defineProperty(dragEnterEvent, 'preventDefault', { value: vi.fn() }); + Object.defineProperty(dragEnterEvent, 'stopPropagation', { value: vi.fn() }); + window.dispatchEvent(dragEnterEvent); + + // Create a drop event whose target is inside the librarian dropzone + const innerTarget = container.querySelector('[data-testid="inner-target"]')!; + const dropEvent = new Event('drop', { bubbles: true }); + Object.defineProperty(dropEvent, 'target', { value: innerTarget }); + Object.defineProperty(dropEvent, 'dataTransfer', { + value: { + items: [{ kind: 'file', type: 'application/pdf' }], + files: [new File(['pdf'], 'test.pdf', { type: 'application/pdf' })], + }, + }); + Object.defineProperty(dropEvent, 'preventDefault', { value: vi.fn() }); + Object.defineProperty(dropEvent, 'stopPropagation', { value: vi.fn() }); + + window.dispatchEvent(dropEvent); + + // importFiles should NOT have been called because drop was inside librarian zone + expect(mockImportFiles).not.toHaveBeenCalled(); + }); + + it('hides overlay when dragging over librarian dropzone and reappears when leaving', () => { + const { container } = render( +
+ +
+ Drop here +
+
+ ); + + // Simulate dragenter on window to activate overlay + const dragEnterEvent = new Event('dragenter', { bubbles: true }); + Object.defineProperty(dragEnterEvent, 'dataTransfer', { + value: { types: ['Files'], dropEffect: 'copy' }, + }); + Object.defineProperty(dragEnterEvent, 'preventDefault', { value: vi.fn() }); + Object.defineProperty(dragEnterEvent, 'stopPropagation', { value: vi.fn() }); + act(() => { + window.dispatchEvent(dragEnterEvent); + }); + + // Overlay should be visible + let overlay = container.querySelector('[aria-label="File drop zone"]'); + expect(overlay).toBeInTheDocument(); + + // Simulate dragover with target inside librarian zone — overlay should hide + const innerTarget = container.querySelector('[data-testid="inner-target"]')!; + const dragOverEvent = new Event('dragover', { bubbles: true }); + Object.defineProperty(dragOverEvent, 'target', { value: innerTarget }); + Object.defineProperty(dragOverEvent, 'dataTransfer', { + value: { types: ['Files'], dropEffect: 'copy' }, + }); + Object.defineProperty(dragOverEvent, 'preventDefault', { value: vi.fn() }); + Object.defineProperty(dragOverEvent, 'stopPropagation', { value: vi.fn() }); + act(() => { + window.dispatchEvent(dragOverEvent); + }); + + // Overlay should be gone + overlay = container.querySelector('[aria-label="File drop zone"]'); + expect(overlay).not.toBeInTheDocument(); + + // Simulate dragenter again outside librarian area — overlay should reappear + const reEnterEvent = new Event('dragenter', { bubbles: true }); + Object.defineProperty(reEnterEvent, 'dataTransfer', { + value: { types: ['Files'], dropEffect: 'copy' }, + }); + Object.defineProperty(reEnterEvent, 'preventDefault', { value: vi.fn() }); + Object.defineProperty(reEnterEvent, 'stopPropagation', { value: vi.fn() }); + act(() => { + window.dispatchEvent(reEnterEvent); + }); + + overlay = container.querySelector('[aria-label="File drop zone"]'); + expect(overlay).toBeInTheDocument(); + }); + + it('processes drops outside librarian dropzone normally', async () => { + render( +
+ +
+ Regular area +
+
+ ); + + // Simulate dragenter + const dragEnterEvent = new Event('dragenter', { bubbles: true }); + Object.defineProperty(dragEnterEvent, 'dataTransfer', { + value: { types: ['Files'], dropEffect: 'copy' }, + }); + Object.defineProperty(dragEnterEvent, 'preventDefault', { value: vi.fn() }); + Object.defineProperty(dragEnterEvent, 'stopPropagation', { value: vi.fn() }); + window.dispatchEvent(dragEnterEvent); + + // Create a drop event with a regular target (no librarian dropzone ancestor) + const regularTarget = document.createElement('div'); + document.body.appendChild(regularTarget); + + const sqlFile = new File(['SELECT 1'], 'query.sql', { type: 'text/plain' }); + Object.defineProperty(sqlFile, 'name', { value: 'query.sql' }); + + const dropEvent = new Event('drop', { bubbles: true }); + Object.defineProperty(dropEvent, 'target', { value: regularTarget }); + Object.defineProperty(dropEvent, 'dataTransfer', { + value: { + items: [{ kind: 'file', type: 'text/plain' }], + files: [sqlFile], + }, + }); + Object.defineProperty(dropEvent, 'preventDefault', { value: vi.fn() }); + Object.defineProperty(dropEvent, 'stopPropagation', { value: vi.fn() }); + + window.dispatchEvent(dropEvent); + + // Wait for async processing + await vi.waitFor(() => { + expect(mockImportFiles).toHaveBeenCalled(); + }); + + document.body.removeChild(regularTarget); + }); +}); diff --git a/app/src/features/librarian/__tests__/librarian-panel.test.tsx b/app/src/features/librarian/__tests__/librarian-panel.test.tsx new file mode 100644 index 00000000..873fc8f7 --- /dev/null +++ b/app/src/features/librarian/__tests__/librarian-panel.test.tsx @@ -0,0 +1,247 @@ +/// +import { cleanup, fireEvent, render, screen } from '@testing-library/react'; +import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'; + +import { useLibrarianStore } from '../store'; + +// ---------- Mocks ---------- + +const mockSendMessage = vi.fn(); +const mockCancel = vi.fn(); +vi.mock('../hooks/use-librarian-chat', () => ({ + useLibrarianChat: () => ({ + sendMessage: mockSendMessage, + cancel: mockCancel, + }), +})); + +vi.mock('../services/ai-service', () => ({ + loadAIConfig: vi.fn(() => ({ + provider: 'openai', + apiKey: 'sk-test', + model: 'gpt-4o', + })), + saveAIConfig: vi.fn(), + sendChatMessage: vi.fn(() => Promise.resolve('ok')), + getDefaultModel: vi.fn((p: string) => (p === 'openai' ? 'gpt-4o' : 'claude-sonnet-4-20250514')), +})); + +vi.mock('../services/pdf-processor', () => ({ + processPdf: vi.fn(() => Promise.resolve([])), +})); + +vi.mock('../services/embedding-service', () => ({ + embedTexts: vi.fn(() => Promise.resolve([[0.1, 0.2]])), +})); + +vi.mock('@pondpilot/flowscope-react', () => ({ + useLineageState: () => ({ result: null }), +})); + +vi.mock('@/lib/project-store', () => ({ + useProject: () => ({ currentProject: null }), +})); + +import { LibrarianPanel } from '../components/librarian-panel'; + +// ---------- Setup ---------- + +beforeEach(() => { + useLibrarianStore.setState({ + byProject: { 'proj-1': { messages: [], pdfFiles: [], pdfChunks: [] } }, + activeProjectId: 'proj-1', + isLoading: false, + }); + vi.clearAllMocks(); +}); + +afterEach(() => { + cleanup(); +}); + +// ---------- Tests ---------- + +describe('LibrarianPanel', () => { + it('renders the panel with header', () => { + render(); + expect(screen.getByTestId('librarian-panel')).toBeInTheDocument(); + expect(screen.getByText('Librarian')).toBeInTheDocument(); + }); + + it('renders settings button', () => { + render(); + expect(screen.getByTestId('settings-button')).toBeInTheDocument(); + }); + + it('renders close button', () => { + render(); + expect(screen.getByTestId('close-button')).toBeInTheDocument(); + }); + + it('calls onClose when close button is clicked', () => { + const onClose = vi.fn(); + render(); + fireEvent.click(screen.getByTestId('close-button')); + expect(onClose).toHaveBeenCalledOnce(); + }); + + it('opens settings dialog when settings button is clicked', () => { + render(); + fireEvent.click(screen.getByTestId('settings-button')); + expect(screen.getByText('AI Settings')).toBeInTheDocument(); + }); + + it('renders empty chat state initially', () => { + render(); + expect(screen.getByTestId('empty-state')).toBeInTheDocument(); + }); + + it('renders chat input', () => { + render(); + expect(screen.getByTestId('chat-textarea')).toBeInTheDocument(); + }); + + it('renders the last prompt size when available', () => { + useLibrarianStore.setState({ + byProject: { + 'proj-1': { + messages: [], + pdfFiles: [], + pdfChunks: [], + lastPromptStats: { characters: 12430, bytes: 13100 }, + }, + }, + activeProjectId: 'proj-1', + }); + + render(); + expect(screen.getByTestId('last-prompt-size')).toHaveTextContent( + 'Last prompt: 12,430 chars / 12.8 KB' + ); + }); + + it('renders documentation toggle', () => { + render(); + expect(screen.getByTestId('docs-toggle')).toBeInTheDocument(); + expect(screen.getByText('Documentation')).toBeInTheDocument(); + }); + + it('expands documentation section when toggle is clicked', () => { + render(); + expect(screen.queryByTestId('docs-section')).not.toBeInTheDocument(); + + fireEvent.click(screen.getByTestId('docs-toggle')); + expect(screen.getByTestId('docs-section')).toBeInTheDocument(); + expect(screen.getByTestId('drop-zone')).toBeInTheDocument(); + }); + + it('collapses documentation section when toggle is clicked again', () => { + render(); + + fireEvent.click(screen.getByTestId('docs-toggle')); + expect(screen.getByTestId('docs-section')).toBeInTheDocument(); + + fireEvent.click(screen.getByTestId('docs-toggle')); + expect(screen.queryByTestId('docs-section')).not.toBeInTheDocument(); + }); + + it('renders messages from active project bucket', () => { + useLibrarianStore.setState({ + byProject: { + 'proj-1': { + messages: [ + { + id: '1', + role: 'user', + content: 'What tables exist?', + timestamp: Date.now(), + }, + { + id: '2', + role: 'assistant', + content: 'There are 3 tables.', + timestamp: Date.now(), + }, + ], + pdfFiles: [], + pdfChunks: [], + }, + }, + activeProjectId: 'proj-1', + }); + + render(); + expect(screen.getByText('What tables exist?')).toBeInTheDocument(); + expect(screen.getByText('There are 3 tables.')).toBeInTheDocument(); + }); + + it('does not render messages from a non-active project bucket', () => { + useLibrarianStore.setState({ + byProject: { + 'proj-1': { messages: [], pdfFiles: [], pdfChunks: [] }, + 'proj-2': { + messages: [ + { + id: '1', + role: 'user', + content: 'Hidden in project 2', + timestamp: Date.now(), + }, + ], + pdfFiles: [], + pdfChunks: [], + }, + }, + activeProjectId: 'proj-1', + }); + + render(); + expect(screen.queryByText('Hidden in project 2')).not.toBeInTheDocument(); + expect(screen.getByTestId('empty-state')).toBeInTheDocument(); + }); + + it('disables chat input and shows hint when no active project', () => { + useLibrarianStore.setState({ + byProject: {}, + activeProjectId: null, + }); + + render(); + expect(screen.getByTestId('no-project-hint')).toHaveTextContent( + /Open or create a project to use Librarian/ + ); + expect(screen.getByTestId('chat-textarea')).toBeDisabled(); + }); + + it('renders loading indicator when loading', () => { + useLibrarianStore.setState({ isLoading: true }); + render(); + expect(screen.getByTestId('loading-indicator')).toBeInTheDocument(); + }); + + it('renders help button in header', () => { + render(); + const helpButton = screen.getByTestId('help-button'); + expect(helpButton).toBeInTheDocument(); + expect(helpButton).toHaveAttribute('aria-label', 'About Librarian'); + }); + + it('opens help popover with full help text when help button is clicked', () => { + render(); + + expect(screen.queryByTestId('help-popover')).not.toBeInTheDocument(); + + fireEvent.click(screen.getByTestId('help-button')); + + const popover = screen.getByTestId('help-popover'); + expect(popover).toBeInTheDocument(); + expect(popover).toHaveTextContent(/Hi, I'm Librarian!/); + expect(popover).toHaveTextContent( + /I answer questions about your data structure using your database schema and uploaded technical documentation\./ + ); + expect(popover).toHaveTextContent(/How to use:/); + expect(popover).toHaveTextContent(/Configure your AI provider in Settings/); + expect(popover).toHaveTextContent(/Upload relevant PDF docs \(optional\)/); + expect(popover).toHaveTextContent(/Ask questions about your data/); + }); +}); diff --git a/app/src/features/librarian/__tests__/librarian-toggle-button.test.tsx b/app/src/features/librarian/__tests__/librarian-toggle-button.test.tsx new file mode 100644 index 00000000..8b46d6bc --- /dev/null +++ b/app/src/features/librarian/__tests__/librarian-toggle-button.test.tsx @@ -0,0 +1,50 @@ +/// +import { cleanup, fireEvent, render, screen } from '@testing-library/react'; +import { afterEach, beforeEach, describe, expect, it } from 'vitest'; + +import { LibrarianToggleButton } from '@/components/LibrarianToggleButton'; +import { useViewStateStore } from '@/lib/view-state-store'; + +beforeEach(() => { + useViewStateStore.setState({ librarianOpen: false }); +}); + +afterEach(() => { + cleanup(); +}); + +describe('LibrarianToggleButton', () => { + it('renders the Librarian label and Polly icon', () => { + render(); + const button = screen.getByTestId('librarian-toggle-button'); + expect(button).toBeInTheDocument(); + expect(button).toHaveTextContent('Librarian'); + const icon = button.querySelector('img'); + expect(icon).not.toBeNull(); + expect(icon?.getAttribute('src')).toBe('/polly-icon.svg'); + }); + + it('reflects the closed state with aria-pressed=false', () => { + render(); + const button = screen.getByTestId('librarian-toggle-button'); + expect(button).toHaveAttribute('aria-pressed', 'false'); + }); + + it('toggles the store state when clicked', () => { + render(); + const button = screen.getByTestId('librarian-toggle-button'); + + fireEvent.click(button); + expect(useViewStateStore.getState().librarianOpen).toBe(true); + + fireEvent.click(button); + expect(useViewStateStore.getState().librarianOpen).toBe(false); + }); + + it('reflects the open state with aria-pressed=true when the panel is open', () => { + useViewStateStore.setState({ librarianOpen: true }); + render(); + const button = screen.getByTestId('librarian-toggle-button'); + expect(button).toHaveAttribute('aria-pressed', 'true'); + }); +}); diff --git a/app/src/features/librarian/__tests__/lineage-formatter.test.ts b/app/src/features/librarian/__tests__/lineage-formatter.test.ts new file mode 100644 index 00000000..63a5d10c --- /dev/null +++ b/app/src/features/librarian/__tests__/lineage-formatter.test.ts @@ -0,0 +1,178 @@ +import { describe, expect, it } from 'vitest'; +import type { AnalyzeResult } from '@pondpilot/flowscope-core'; + +import { formatLineage } from '../services/lineage-formatter'; + +function makeResult(overrides: Partial = {}): AnalyzeResult { + return { + statements: [], + nodes: [], + edges: [], + issues: [], + summary: { + tableCount: 0, + columnCount: 0, + statementCount: 0, + joinCount: 0, + complexityScore: 0, + issueCount: { errors: 0, warnings: 0, infos: 0 }, + hasErrors: false, + }, + ...overrides, + }; +} + +describe('formatLineage', () => { + it('returns empty string for null input', () => { + expect(formatLineage(null)).toBe(''); + }); + + it('returns empty string for empty lineage and no schema', () => { + expect(formatLineage(makeResult())).toBe(''); + }); + + it('formats resolved schema tables with columns', () => { + const result = makeResult({ + resolvedSchema: { + tables: [ + { + name: 'users', + schema: 'public', + columns: [ + { name: 'id', dataType: 'integer', isPrimaryKey: true }, + { name: 'email', dataType: 'varchar' }, + { + name: 'org_id', + dataType: 'integer', + foreignKey: { table: 'orgs', column: 'id' }, + }, + ], + origin: 'imported', + updatedAt: '2026-01-01T00:00:00Z', + }, + ], + }, + }); + + const output = formatLineage(result); + expect(output).toContain('public.users'); + expect(output).toContain('id | integer | PK'); + expect(output).toContain('email | varchar'); + expect(output).toContain('org_id | integer | FK -> orgs.id'); + }); + + it('falls back to global lineage nodes when no resolved schema', () => { + const result = makeResult({ + nodes: [ + { + id: 'n1', + type: 'table', + label: 'orders', + canonicalName: { name: 'orders' }, + statementIds: [0], + }, + { + id: 'n2', + type: 'view', + label: 'order_summary', + canonicalName: { name: 'order_summary' }, + statementIds: [0], + }, + ], + edges: [], + }); + + const output = formatLineage(result); + expect(output).toContain('- orders'); + expect(output).toContain('- order_summary'); + }); + + it('formats relationships from edges', () => { + const result = makeResult({ + nodes: [ + { + id: 'n1', + type: 'table', + label: 'users', + canonicalName: { name: 'users' }, + statementIds: [0], + }, + { + id: 'n2', + type: 'column', + label: 'users.id', + canonicalName: { name: 'id', column: 'id' }, + statementIds: [0], + }, + ], + edges: [ + { + id: 'e1', + from: 'n1', + to: 'n2', + type: 'ownership', + statementIds: [0], + }, + ], + }); + + const output = formatLineage(result); + expect(output).toContain('users --[ownership]--> users.id'); + }); + + it('uses edge IDs as fallback when node labels not found', () => { + const result = makeResult({ + nodes: [], + edges: [ + { + id: 'e1', + from: 'unknown1', + to: 'unknown2', + type: 'data_flow', + statementIds: [0], + }, + ], + }); + + const output = formatLineage(result); + expect(output).toContain('unknown1 --[data_flow]--> unknown2'); + }); + + it('includes both schema and relationships when both present', () => { + const result = makeResult({ + resolvedSchema: { + tables: [ + { + name: 'a', + columns: [{ name: 'col1' }], + origin: 'imported', + updatedAt: '2026-01-01T00:00:00Z', + }, + ], + }, + nodes: [ + { + id: 'n1', + type: 'table', + label: 'a', + canonicalName: { name: 'a' }, + statementIds: [0], + }, + { + id: 'n2', + type: 'table', + label: 'b', + canonicalName: { name: 'b' }, + statementIds: [1], + }, + ], + edges: [{ id: 'e1', from: 'n1', to: 'n2', type: 'cross_statement', statementIds: [0, 1] }], + }); + + const output = formatLineage(result); + expect(output).toContain('Tables:'); + expect(output).toContain('Relationships:'); + // Should use resolved schema for tables, not fall back to nodes + expect(output).toContain(' - col1'); + }); +}); diff --git a/app/src/features/librarian/__tests__/pdf-processor.test.ts b/app/src/features/librarian/__tests__/pdf-processor.test.ts new file mode 100644 index 00000000..c6b63785 --- /dev/null +++ b/app/src/features/librarian/__tests__/pdf-processor.test.ts @@ -0,0 +1,168 @@ +import { describe, expect, it, vi } from 'vitest'; + +import { splitIntoChunks, processPdf } from '../services/pdf-processor'; +import type { PageText } from '../services/pdf-processor'; + +// Mock pdfjs-dist so extractTextFromPdf doesn't need a real PDF runtime +vi.mock('pdfjs-dist', () => ({ + GlobalWorkerOptions: { workerSrc: '' }, + getDocument: vi.fn(), +})); + +function makePdfFile(name: string): File { + const file = new File(['fake-pdf-content'], name, { type: 'application/pdf' }); + Object.defineProperty(file, 'arrayBuffer', { + value: vi.fn().mockResolvedValue(new ArrayBuffer(8)), + }); + return file; +} + +describe('splitIntoChunks', () => { + it('returns empty array for empty pages', () => { + expect(splitIntoChunks([])).toEqual([]); + }); + + it('returns empty array for pages with empty text', () => { + const pages: PageText[] = [{ pageNumber: 1, text: '' }]; + expect(splitIntoChunks(pages)).toEqual([]); + }); + + it('returns single chunk for text shorter than chunkSize', () => { + const pages: PageText[] = [{ pageNumber: 1, text: 'Hello world' }]; + const chunks = splitIntoChunks(pages, 500, 50); + expect(chunks).toHaveLength(1); + expect(chunks[0]).toEqual({ pageNumber: 1, text: 'Hello world' }); + }); + + it('returns single chunk for text equal to chunkSize', () => { + const text = 'a'.repeat(100); + const pages: PageText[] = [{ pageNumber: 1, text }]; + const chunks = splitIntoChunks(pages, 100, 10); + expect(chunks).toHaveLength(1); + expect(chunks[0].text).toBe(text); + }); + + it('splits long text into overlapping chunks', () => { + const text = 'a'.repeat(250); + const pages: PageText[] = [{ pageNumber: 1, text }]; + // chunkSize=100, overlap=20 => step=80 + // chunks: 0-100, 80-180, 160-250 + const chunks = splitIntoChunks(pages, 100, 20); + expect(chunks).toHaveLength(3); + expect(chunks[0].text).toBe('a'.repeat(100)); + expect(chunks[1].text).toBe('a'.repeat(100)); + expect(chunks[2].text).toBe('a'.repeat(90)); // 160-250 + // All retain the page number + expect(chunks.every((c) => c.pageNumber === 1)).toBe(true); + }); + + it('handles multiple pages independently', () => { + const pages: PageText[] = [ + { pageNumber: 1, text: 'Short text' }, + { pageNumber: 2, text: 'b'.repeat(200) }, + ]; + const chunks = splitIntoChunks(pages, 100, 10); + expect(chunks[0]).toEqual({ pageNumber: 1, text: 'Short text' }); + // Page 2 should be split + const page2Chunks = chunks.filter((c) => c.pageNumber === 2); + expect(page2Chunks.length).toBeGreaterThan(1); + }); + + it('handles overlap larger than chunkSize gracefully', () => { + const text = 'a'.repeat(200); + const pages: PageText[] = [{ pageNumber: 1, text }]; + // overlap=150, chunkSize=100 => Math.min(150, 99)=99 => step=1 + const chunks = splitIntoChunks(pages, 100, 150); + // Should still produce chunks without infinite loop + expect(chunks.length).toBeGreaterThan(0); + expect(chunks[0].text.length).toBe(100); + }); + + it('handles zero overlap', () => { + const text = 'a'.repeat(200); + const pages: PageText[] = [{ pageNumber: 1, text }]; + const chunks = splitIntoChunks(pages, 100, 0); + expect(chunks).toHaveLength(2); + }); +}); + +describe('processPdf', () => { + it('returns empty array when no text extracted', async () => { + // We need to mock extractTextFromPdf indirectly via pdfjs-dist + const pdfjs = await import('pdfjs-dist'); + const mockGetDocument = vi.mocked(pdfjs.getDocument); + + mockGetDocument.mockReturnValue({ + promise: Promise.resolve({ + numPages: 1, + getPage: vi.fn().mockResolvedValue({ + getTextContent: vi.fn().mockResolvedValue({ items: [] }), + }), + destroy: vi.fn(), + }), + } as unknown as ReturnType); + + const file = makePdfFile('test.pdf'); + const embedFn = vi.fn().mockResolvedValue([]); + + const result = await processPdf(file, 'file-1', embedFn); + expect(result).toEqual([]); + expect(embedFn).not.toHaveBeenCalled(); + }); + + it('processes a PDF with text and returns embedded chunks', async () => { + const pdfjs = await import('pdfjs-dist'); + const mockGetDocument = vi.mocked(pdfjs.getDocument); + + const textItems = [{ str: 'Hello ' }, { str: 'World' }]; + + mockGetDocument.mockReturnValue({ + promise: Promise.resolve({ + numPages: 1, + getPage: vi.fn().mockResolvedValue({ + getTextContent: vi.fn().mockResolvedValue({ items: textItems }), + }), + destroy: vi.fn(), + }), + } as unknown as ReturnType); + + const file = makePdfFile('doc.pdf'); + const embedFn = vi.fn().mockResolvedValue([[0.1, 0.2, 0.3]]); + + const result = await processPdf(file, 'f1', embedFn); + + expect(result).toHaveLength(1); + expect(result[0]).toEqual({ + id: 'f1-chunk-0', + fileId: 'f1', + fileName: 'doc.pdf', + text: 'Hello World', + pageNumber: 1, + embedding: [0.1, 0.2, 0.3], + }); + expect(embedFn).toHaveBeenCalledWith(['Hello World']); + }); + + it('throws on embedding count mismatch', async () => { + const pdfjs = await import('pdfjs-dist'); + const mockGetDocument = vi.mocked(pdfjs.getDocument); + + mockGetDocument.mockReturnValue({ + promise: Promise.resolve({ + numPages: 1, + getPage: vi.fn().mockResolvedValue({ + getTextContent: vi.fn().mockResolvedValue({ + items: [{ str: 'Some text' }], + }), + }), + destroy: vi.fn(), + }), + } as unknown as ReturnType); + + const file = makePdfFile('test.pdf'); + // Return wrong number of embeddings + const embedFn = vi.fn().mockResolvedValue([[0.1], [0.2]]); + + await expect(processPdf(file, 'f1', embedFn)).rejects.toThrow('Embedding count mismatch'); + }); +}); diff --git a/app/src/features/librarian/__tests__/schema-identifiers.test.ts b/app/src/features/librarian/__tests__/schema-identifiers.test.ts new file mode 100644 index 00000000..63376e10 --- /dev/null +++ b/app/src/features/librarian/__tests__/schema-identifiers.test.ts @@ -0,0 +1,366 @@ +import { describe, expect, it } from 'vitest'; + +import { + buildSchemaIdentifiers, + detectIdentifiers, + extractSummary, + resolveAllReferences, + type SchemaIdentifiers, +} from '../utils/schema-identifiers'; + +function makeSchema( + tables: string[], + columns: string[], + columnOwners: Record = {} +): SchemaIdentifiers { + return { + tables: new Set(tables), + columns: new Set(columns), + columnOwners: new Map(Object.entries(columnOwners)), + }; +} + +describe('detectIdentifiers', () => { + it('returns a single text segment when schema is empty', () => { + const schema = makeSchema([], []); + const segments = detectIdentifiers('Hello MANDT world', schema); + expect(segments).toEqual([{ type: 'text', value: 'Hello MANDT world' }]); + }); + + it('returns empty array for empty text', () => { + const schema = makeSchema(['MANDT'], []); + expect(detectIdentifiers('', schema)).toEqual([]); + }); + + it('matches a column name (case-insensitive, word-bounded)', () => { + const schema = makeSchema([], ['MANDT']); + const segments = detectIdentifiers('Client is MANDT.', schema); + expect(segments).toEqual([ + { type: 'text', value: 'Client is ' }, + { type: 'identifier', value: 'MANDT', kind: 'column' }, + { type: 'text', value: '.' }, + ]); + }); + + it('matches a table name exactly', () => { + const schema = makeSchema(['ekko'], []); + const segments = detectIdentifiers('See ekko for details.', schema); + expect(segments[1]).toEqual({ type: 'identifier', value: 'ekko', kind: 'table' }); + }); + + it('matches lower-case variants and normalizes to canonical casing', () => { + const schema = makeSchema([], ['MANDT']); + const segments = detectIdentifiers('This mandt is lowercase.', schema); + expect(segments).toEqual([ + { type: 'text', value: 'This ' }, + { type: 'identifier', value: 'MANDT', kind: 'column' }, + { type: 'text', value: ' is lowercase.' }, + ]); + }); + + it('matches mixed-case variants and normalizes to canonical casing', () => { + const schema = makeSchema(['BKPF'], ['MANDT']); + const segments = detectIdentifiers('See Bkpf.MaNdT now.', schema); + const ids = segments.filter((s) => s.type === 'identifier'); + expect(ids).toEqual([ + { type: 'identifier', value: 'BKPF', kind: 'table' }, + { type: 'identifier', value: 'MANDT', kind: 'column' }, + ]); + }); + + it('does not match a non-identifier word that contains an identifier as a prefix', () => { + const schema = makeSchema([], ['MANDT']); + const segments = detectIdentifiers('The mandate is renewed.', schema); + expect(segments.every((s) => s.type === 'text')).toBe(true); + }); + + it('does not match embedded substrings (word boundary)', () => { + const schema = makeSchema([], ['MANDT']); + const segments = detectIdentifiers('MANDT_X and xMANDT and MANDTy', schema); + expect(segments.every((s) => s.type === 'text')).toBe(true); + }); + + it('matches multiple identifiers on a single line', () => { + const schema = makeSchema(['ekko', 'ekpo'], ['EBELN', 'EBELP']); + const segments = detectIdentifiers('Join ekko.EBELN = ekpo.EBELP today.', schema); + const identifiers = segments.filter((s) => s.type === 'identifier').map((s) => s.value); + expect(identifiers).toEqual(['ekko', 'EBELN', 'ekpo', 'EBELP']); + }); + + it('marks identifiers present in both tables and columns as tables', () => { + const schema = makeSchema(['shared'], ['shared']); + const segments = detectIdentifiers('the shared name', schema); + const id = segments.find((s) => s.type === 'identifier'); + expect(id?.kind).toBe('table'); + }); + + it('handles identifiers at start and end of text', () => { + const schema = makeSchema([], ['MANDT', 'BUKRS']); + const segments = detectIdentifiers('MANDT is here BUKRS', schema); + expect(segments[0]).toEqual({ type: 'identifier', value: 'MANDT', kind: 'column' }); + expect(segments[segments.length - 1]).toEqual({ + type: 'identifier', + value: 'BUKRS', + kind: 'column', + }); + }); + + it('matches identifiers surrounded by punctuation (backticks, parens)', () => { + const schema = makeSchema([], ['MANDT']); + const segments = detectIdentifiers('Use `MANDT` and (MANDT).', schema); + const matches = segments.filter((s) => s.type === 'identifier'); + expect(matches).toHaveLength(2); + expect(matches.every((m) => m.value === 'MANDT')).toBe(true); + }); +}); + +describe('buildSchemaIdentifiers', () => { + it('returns an empty set when result is null/undefined', () => { + const fromNull = buildSchemaIdentifiers(null); + const fromUndef = buildSchemaIdentifiers(undefined); + expect(fromNull.tables.size).toBe(0); + expect(fromNull.columns.size).toBe(0); + expect(fromUndef.tables.size).toBe(0); + }); + + it('collects table and column names from resolvedSchema', () => { + const result = { + resolvedSchema: { + tables: [ + { + name: 'ekko', + columns: [{ name: 'EBELN' }, { name: 'MANDT' }], + }, + { + name: 'ekpo', + columns: [{ name: 'EBELN' }, { name: 'EBELP' }], + }, + ], + }, + } as unknown as Parameters[0]; + + const ids = buildSchemaIdentifiers(result); + expect([...ids.tables].sort()).toEqual(['ekko', 'ekpo']); + expect([...ids.columns].sort()).toEqual(['EBELN', 'EBELP', 'MANDT']); + expect(ids.columnOwners.get('EBELN')?.sort()).toEqual(['ekko', 'ekpo']); + expect(ids.columnOwners.get('MANDT')).toEqual(['ekko']); + }); + + it('handles missing resolvedSchema and nodes gracefully', () => { + const result = {} as unknown as Parameters[0]; + const ids = buildSchemaIdentifiers(result); + expect(ids.tables.size).toBe(0); + expect(ids.columns.size).toBe(0); + }); + + it('falls back to lineage nodes when resolvedSchema is absent', () => { + const result = { + nodes: [ + { + id: 'table-1', + type: 'table', + label: 'orders_alias', + canonicalName: { schema: 'public', name: 'orders' }, + }, + { + id: 'view-1', + type: 'view', + label: 'invoice_view', + canonicalName: { name: 'invoice_view' }, + }, + { + id: 'column-1', + type: 'column', + label: 'order_id', + canonicalName: { schema: 'public', name: 'orders', column: 'ORDER_ID' }, + }, + ], + } as unknown as Parameters[0]; + + const ids = buildSchemaIdentifiers(result); + + expect([...ids.tables].sort()).toEqual(['invoice_view', 'orders']); + expect([...ids.columns]).toEqual(['ORDER_ID']); + expect(ids.columnOwners.get('ORDER_ID')).toEqual(['orders']); + }); + + it('maps column owners correctly for analyzer-shaped canonical names', () => { + // Matches what `parse_canonical_name` in flowscope-core actually emits: + // 2-part `orders.total_amount` becomes `{ schema: 'orders', name: 'total_amount' }`, + // with no `column` field. Without the schema-as-owner fallback this + // assertion would record `total_amount -> ['total_amount']`. + const result = { + nodes: [ + { + id: 'col-1', + type: 'column', + label: 'total_amount', + canonicalName: { schema: 'orders', name: 'total_amount' }, + }, + { + id: 'col-2', + type: 'column', + label: 'customer_id', + canonicalName: { catalog: 'main', schema: 'orders', name: 'customer_id' }, + }, + ], + } as unknown as Parameters[0]; + + const ids = buildSchemaIdentifiers(result); + + expect([...ids.columns].sort()).toEqual(['customer_id', 'total_amount']); + expect(ids.columnOwners.get('total_amount')).toEqual(['orders']); + expect(ids.columnOwners.get('customer_id')).toEqual(['orders']); + }); +}); + +describe('resolveAllReferences', () => { + it('returns an empty list when text has no identifiers', () => { + const schema = makeSchema(['MARA'], ['MANDT']); + expect(resolveAllReferences('Nothing to resolve here.', schema)).toEqual([]); + }); + + it('returns an empty list for unknown identifiers (skipped silently)', () => { + const schema = makeSchema(['MARA'], ['MANDT']); + expect(resolveAllReferences('UNKNOWN_TABLE and OTHER_COL', schema)).toEqual([]); + }); + + it('resolves a dotted column as a qualified reference', () => { + const schema = makeSchema(['BKPF'], ['MANDT']); + expect(resolveAllReferences('Look at BKPF.MANDT now.', schema)).toEqual([ + { tableName: 'BKPF', columnName: 'MANDT' }, + ]); + }); + + it('resolves a space-separated table+column as a qualified reference', () => { + const schema = makeSchema(['BKPF'], ['MANDT']); + expect(resolveAllReferences('Look at BKPF MANDT now.', schema)).toEqual([ + { tableName: 'BKPF', columnName: 'MANDT' }, + ]); + }); + + it('resolves a standalone table as a table reference', () => { + const schema = makeSchema(['BKPF'], ['MANDT']); + expect(resolveAllReferences('Look at BKPF now.', schema)).toEqual([{ tableName: 'BKPF' }]); + }); + + it('resolves a standalone column as a bare-column reference', () => { + const schema = makeSchema(['BKPF'], ['MANDT']); + expect(resolveAllReferences('The MANDT column appears everywhere.', schema)).toEqual([ + { columnName: 'MANDT', bareColumn: true }, + ]); + }); + + it('does not qualify a column when the preceding text is more than a separator', () => { + const schema = makeSchema(['BKPF'], ['MANDT']); + expect(resolveAllReferences('BKPF and then MANDT.', schema)).toEqual([ + { tableName: 'BKPF' }, + { columnName: 'MANDT', bareColumn: true }, + ]); + }); + + it('does not qualify a column when the preceding identifier is another column', () => { + const schema = makeSchema([], ['MANDT', 'BUKRS']); + expect(resolveAllReferences('MANDT BUKRS pair', schema)).toEqual([ + { columnName: 'MANDT', bareColumn: true }, + { columnName: 'BUKRS', bareColumn: true }, + ]); + }); + + it('returns mixed references in order, deduplicated', () => { + const schema = makeSchema(['BKPF', 'BSEG'], ['MANDT', 'BUKRS']); + const refs = resolveAllReferences( + 'BKPF.MANDT joins BSEG.MANDT; mention MANDT alone, then BKPF.MANDT again, and BSEG too.', + schema + ); + expect(refs).toEqual([ + { tableName: 'BKPF', columnName: 'MANDT' }, + { tableName: 'BSEG', columnName: 'MANDT' }, + { columnName: 'MANDT', bareColumn: true }, + { tableName: 'BSEG' }, + ]); + }); + + it('treats names that are both tables and columns as table references', () => { + const schema = makeSchema(['shared'], ['shared']); + expect(resolveAllReferences('the shared name', schema)).toEqual([{ tableName: 'shared' }]); + }); + + it('does not qualify across a newline gap', () => { + const schema = makeSchema(['BKPF'], ['MANDT']); + expect(resolveAllReferences('Tables: BKPF.\n\nKey columns: MANDT.', schema)).toEqual([ + { tableName: 'BKPF' }, + { columnName: 'MANDT', bareColumn: true }, + ]); + }); + + it('does not qualify when the gap contains more than one dot', () => { + const schema = makeSchema(['BKPF'], ['MANDT']); + expect(resolveAllReferences('BKPF..MANDT', schema)).toEqual([ + { tableName: 'BKPF' }, + { columnName: 'MANDT', bareColumn: true }, + ]); + }); + + it('qualifies through a single dot surrounded by horizontal whitespace', () => { + const schema = makeSchema(['BKPF'], ['MANDT']); + expect(resolveAllReferences('BKPF . MANDT is the key', schema)).toEqual([ + { tableName: 'BKPF', columnName: 'MANDT' }, + ]); + }); + + it('resolves all four case variants of a qualified reference to the same canonical ref', () => { + const schema = makeSchema(['BKPF'], ['MANDT']); + const expected = [{ tableName: 'BKPF', columnName: 'MANDT' }]; + expect(resolveAllReferences('BKPF.MANDT', schema)).toEqual(expected); + expect(resolveAllReferences('bkpf.MANDT', schema)).toEqual(expected); + expect(resolveAllReferences('BKPF.mandt', schema)).toEqual(expected); + expect(resolveAllReferences('bkpf.mandt', schema)).toEqual(expected); + }); + + it('resolves a bare lowercase column to a bare-column reference with canonical casing', () => { + const schema = makeSchema(['BKPF'], ['MANDT']); + expect(resolveAllReferences('the mandt column appears everywhere.', schema)).toEqual([ + { columnName: 'MANDT', bareColumn: true }, + ]); + }); +}); + +describe('extractSummary', () => { + it('returns empty string for empty input', () => { + expect(extractSummary('')).toBe(''); + }); + + it('returns the original text when no Summary marker is present', () => { + const text = 'Just a plain answer with no sections.'; + expect(extractSummary(text)).toBe(text); + }); + + it('extracts content between Summary: and Data Lineage:', () => { + const text = + 'Summary: MANDT is a technical key in BKPF and BSEG.\n' + + 'Data Lineage: bkpf.MANDT = bseg.MANDT.\n' + + 'Documentation: No information.'; + expect(extractSummary(text)).toBe('MANDT is a technical key in BKPF and BSEG.'); + }); + + it('extracts content between Summary: and Documentation: when Data Lineage is absent', () => { + const text = 'Summary: Vendor country lives in LFA1.LAND1.\nDocumentation: see PDF foo.pdf.'; + expect(extractSummary(text)).toBe('Vendor country lives in LFA1.LAND1.'); + }); + + it('handles markdown-bold Summary header', () => { + const text = '**Summary:** Payment block is in RBKP.ZLSPR.\n**Data Lineage:** rbkp.ZLSPR.'; + expect(extractSummary(text)).toBe('Payment block is in RBKP.ZLSPR.'); + }); + + it('extracts everything after Summary when no terminating section follows', () => { + const text = 'Summary: Just one section here, no other markers.'; + expect(extractSummary(text)).toBe('Just one section here, no other markers.'); + }); + + it('is case-insensitive on the Summary marker', () => { + const text = 'summary: lowercase header.\nData Lineage: details.'; + expect(extractSummary(text)).toBe('lowercase header.'); + }); +}); diff --git a/app/src/features/librarian/__tests__/schema-search-control.test.tsx b/app/src/features/librarian/__tests__/schema-search-control.test.tsx new file mode 100644 index 00000000..d78d4e67 --- /dev/null +++ b/app/src/features/librarian/__tests__/schema-search-control.test.tsx @@ -0,0 +1,170 @@ +/// +import { cleanup, fireEvent, render, screen } from '@testing-library/react'; +import { afterEach, describe, expect, it, vi } from 'vitest'; + +import { SchemaSearchControl } from '@/components/SchemaSearchControl'; + +afterEach(() => { + cleanup(); +}); + +describe('SchemaSearchControl', () => { + it('renders as an icon-only button initially', () => { + render(); + expect(screen.getByTestId('schema-search-toggle')).toBeInTheDocument(); + expect(screen.queryByTestId('schema-search-input')).not.toBeInTheDocument(); + }); + + it('expands into an input on click', () => { + render(); + fireEvent.click(screen.getByTestId('schema-search-toggle')); + expect(screen.getByTestId('schema-search-input')).toBeInTheDocument(); + expect(screen.queryByTestId('schema-search-toggle')).not.toBeInTheDocument(); + }); + + it('selects a matching table on keystroke (case-insensitive prefix)', () => { + const onSelectTable = vi.fn(); + render( + + ); + fireEvent.click(screen.getByTestId('schema-search-toggle')); + const input = screen.getByTestId('schema-search-input'); + + fireEvent.change(input, { target: { value: 'bk' } }); + expect(onSelectTable).toHaveBeenLastCalledWith('BKPF'); + + fireEvent.change(input, { target: { value: 'MA' } }); + expect(onSelectTable).toHaveBeenLastCalledWith('MARA'); + }); + + it('passes undefined to onSelectTable when no table matches', () => { + const onSelectTable = vi.fn(); + render(); + fireEvent.click(screen.getByTestId('schema-search-toggle')); + const input = screen.getByTestId('schema-search-input'); + + fireEvent.change(input, { target: { value: 'zzz' } }); + expect(onSelectTable).toHaveBeenLastCalledWith(undefined); + }); + + it('clears selection when the input is emptied', () => { + const onSelectTable = vi.fn(); + render(); + fireEvent.click(screen.getByTestId('schema-search-toggle')); + const input = screen.getByTestId('schema-search-input'); + + fireEvent.change(input, { target: { value: 'MA' } }); + expect(onSelectTable).toHaveBeenLastCalledWith('MARA'); + + fireEvent.change(input, { target: { value: '' } }); + expect(onSelectTable).toHaveBeenLastCalledWith(undefined); + }); + + it('collapses back to icon when close button is clicked', () => { + const onSelectTable = vi.fn(); + render(); + fireEvent.click(screen.getByTestId('schema-search-toggle')); + fireEvent.change(screen.getByTestId('schema-search-input'), { + target: { value: 'ma' }, + }); + + fireEvent.click(screen.getByTestId('schema-search-close')); + expect(screen.queryByTestId('schema-search-input')).not.toBeInTheDocument(); + expect(screen.getByTestId('schema-search-toggle')).toBeInTheDocument(); + // Selection cleared on collapse + expect(onSelectTable).toHaveBeenLastCalledWith(undefined); + }); + + it('collapses on blur when the input is empty', () => { + render(); + fireEvent.click(screen.getByTestId('schema-search-toggle')); + const input = screen.getByTestId('schema-search-input'); + + fireEvent.blur(input); + expect(screen.queryByTestId('schema-search-input')).not.toBeInTheDocument(); + expect(screen.getByTestId('schema-search-toggle')).toBeInTheDocument(); + }); + + it('does not collapse on blur when there is text in the field', () => { + render(); + fireEvent.click(screen.getByTestId('schema-search-toggle')); + const input = screen.getByTestId('schema-search-input'); + + fireEvent.change(input, { target: { value: 'ma' } }); + fireEvent.blur(input); + expect(screen.getByTestId('schema-search-input')).toBeInTheDocument(); + }); + + it('collapses and clears selection on Escape', () => { + const onSelectTable = vi.fn(); + render(); + fireEvent.click(screen.getByTestId('schema-search-toggle')); + const input = screen.getByTestId('schema-search-input'); + + fireEvent.change(input, { target: { value: 'ma' } }); + fireEvent.keyDown(input, { key: 'Escape' }); + + expect(screen.queryByTestId('schema-search-input')).not.toBeInTheDocument(); + expect(screen.getByTestId('schema-search-toggle')).toBeInTheDocument(); + expect(onSelectTable).toHaveBeenLastCalledWith(undefined); + }); + + it('does not re-clear selection on parent re-render after collapse', () => { + // Regression: hasInteractedRef leaked across search sessions. After the + // user typed once and then closed the control, a subsequent parent + // re-render would fire the selection effect with empty matches and an + // already-true ref, calling onSelectTable(undefined) again — which would + // clobber any selection set externally between collapse and re-render. + const onSelectTable = vi.fn(); + const { rerender } = render( + + ); + fireEvent.click(screen.getByTestId('schema-search-toggle')); + fireEvent.change(screen.getByTestId('schema-search-input'), { target: { value: 'MA' } }); + fireEvent.click(screen.getByTestId('schema-search-close')); + + expect(onSelectTable).toHaveBeenLastCalledWith(undefined); + const callsAfterCollapse = onSelectTable.mock.calls.length; + + rerender( + + ); + + expect(onSelectTable.mock.calls.length).toBe(callsAfterCollapse); + }); + + it('clamps the active match when the match list shrinks', () => { + const onSelectTable = vi.fn(); + const { rerender } = render( + + ); + fireEvent.click(screen.getByTestId('schema-search-toggle')); + const input = screen.getByTestId('schema-search-input'); + + fireEvent.change(input, { target: { value: 'ma' } }); + fireEvent.click(screen.getByTestId('schema-search-next')); + fireEvent.click(screen.getByTestId('schema-search-next')); + expect(screen.getByText('3/3')).toBeInTheDocument(); + + rerender(); + + expect(screen.getByText('1/1')).toBeInTheDocument(); + expect(onSelectTable).toHaveBeenLastCalledWith('MARA'); + }); + + it('does not reselect the same match when parent props get new array identities', () => { + const onSelectTable = vi.fn(); + const { rerender } = render( + + ); + fireEvent.click(screen.getByTestId('schema-search-toggle')); + fireEvent.change(screen.getByTestId('schema-search-input'), { target: { value: 'MA' } }); + + expect(onSelectTable).toHaveBeenCalledTimes(1); + expect(onSelectTable).toHaveBeenLastCalledWith('MARA'); + + rerender(); + + expect(onSelectTable).toHaveBeenCalledTimes(1); + }); +}); diff --git a/app/src/features/librarian/__tests__/setup.ts b/app/src/features/librarian/__tests__/setup.ts new file mode 100644 index 00000000..b28a0e26 --- /dev/null +++ b/app/src/features/librarian/__tests__/setup.ts @@ -0,0 +1,49 @@ +import { expect } from 'vitest'; +import * as matchers from '@testing-library/jest-dom/matchers'; + +expect.extend(matchers); + +// Node 22+ exposes an experimental global localStorage object in some +// configurations. It is not the jsdom Storage implementation and may be +// missing clear/setItem, which breaks tests and Zustand persist. Install a +// deterministic in-memory Storage-compatible shim before modules import stores. +class MemoryStorage implements Storage { + private values = new Map(); + + get length(): number { + return this.values.size; + } + + clear(): void { + this.values.clear(); + } + + getItem(key: string): string | null { + return this.values.has(key) ? (this.values.get(key) ?? null) : null; + } + + key(index: number): string | null { + return Array.from(this.values.keys())[index] ?? null; + } + + removeItem(key: string): void { + this.values.delete(key); + } + + setItem(key: string, value: string): void { + this.values.set(key, String(value)); + } +} + +const localStorageShim = new MemoryStorage(); +Object.defineProperty(globalThis, 'localStorage', { + value: localStorageShim, + configurable: true, +}); +Object.defineProperty(window, 'localStorage', { + value: localStorageShim, + configurable: true, +}); + +// jsdom doesn't implement scrollIntoView +Element.prototype.scrollIntoView = () => {}; diff --git a/app/src/features/librarian/__tests__/store.test.ts b/app/src/features/librarian/__tests__/store.test.ts new file mode 100644 index 00000000..ae1f2ff1 --- /dev/null +++ b/app/src/features/librarian/__tests__/store.test.ts @@ -0,0 +1,549 @@ +import { describe, it, expect, beforeEach, vi } from 'vitest'; + +vi.mock('../services/ai-service', () => ({ + loadAIConfig: vi.fn(() => null), +})); + +import { loadAIConfig } from '../services/ai-service'; + +import { CHAT_HISTORY_LIMIT } from '../constants'; +import { + useLibrarianStore, + useLibrarianMessages, + useLibrarianPdfFiles, + useLibrarianPdfChunks, +} from '../store'; +import type { PdfChunk, PdfFile } from '../types'; + +const PROJECT_A = 'proj-a'; +const PROJECT_B = 'proj-b'; + +function makePdfFile(overrides: Partial = {}): PdfFile { + return { + id: 'file-1', + name: 'test.pdf', + size: 1024, + status: 'processing', + uploadedAt: Date.now(), + ...overrides, + }; +} + +function makePdfChunk(overrides: Partial = {}): PdfChunk { + return { + id: 'chunk-1', + fileId: 'file-1', + fileName: 'test.pdf', + text: 'chunk text', + pageNumber: 1, + embedding: [0.1, 0.2], + ...overrides, + }; +} + +function resetStore(activeProjectId: string | null = PROJECT_A) { + useLibrarianStore.setState({ + byProject: {}, + activeProjectId, + isLoading: false, + hasConfig: false, + messages: [], + pdfFiles: [], + pdfChunks: [], + }); +} + +describe('useLibrarianStore', () => { + beforeEach(() => { + vi.mocked(loadAIConfig).mockReturnValue(null); + resetStore(PROJECT_A); + }); + + // ---------- messages ---------- + + describe('addMessage', () => { + it('adds a user message', () => { + useLibrarianStore.getState().addMessage('user', 'hello'); + const { messages } = useLibrarianStore.getState(); + expect(messages).toHaveLength(1); + expect(messages[0].role).toBe('user'); + expect(messages[0].content).toBe('hello'); + expect(messages[0].id).toBeTruthy(); + expect(messages[0].timestamp).toBeGreaterThan(0); + }); + + it('adds an assistant message', () => { + useLibrarianStore.getState().addMessage('assistant', 'hi there'); + const { messages } = useLibrarianStore.getState(); + expect(messages).toHaveLength(1); + expect(messages[0].role).toBe('assistant'); + }); + + it('keeps all messages without truncating', () => { + const store = useLibrarianStore.getState(); + const total = CHAT_HISTORY_LIMIT + 5; + for (let i = 0; i < total; i++) { + store.addMessage('user', `msg-${i}`); + } + const { messages } = useLibrarianStore.getState(); + expect(messages).toHaveLength(total); + expect(messages[0].content).toBe('msg-0'); + expect(messages[messages.length - 1].content).toBe(`msg-${total - 1}`); + }); + + it('no-ops when activeProjectId is null', () => { + resetStore(null); + useLibrarianStore.getState().addMessage('user', 'orphan'); + const state = useLibrarianStore.getState(); + expect(state.messages).toHaveLength(0); + expect(state.byProject).toEqual({}); + }); + + it('lazily initializes the bucket on first write', () => { + // No bucket exists yet; addMessage should create one for the active project. + expect(useLibrarianStore.getState().byProject[PROJECT_A]).toBeUndefined(); + useLibrarianStore.getState().addMessage('user', 'hi'); + const bucket = useLibrarianStore.getState().byProject[PROJECT_A]; + expect(bucket).toBeDefined(); + expect(bucket.messages).toHaveLength(1); + expect(bucket.pdfFiles).toEqual([]); + expect(bucket.pdfChunks).toEqual([]); + }); + }); + + describe('clearMessages', () => { + it('removes all messages for the active project', () => { + const store = useLibrarianStore.getState(); + store.addMessage('user', 'a'); + store.addMessage('assistant', 'b'); + store.clearMessages(); + expect(useLibrarianStore.getState().messages).toHaveLength(0); + expect(useLibrarianStore.getState().byProject[PROJECT_A].messages).toHaveLength(0); + }); + + it('does not touch other projects', () => { + useLibrarianStore.getState().addMessage('user', 'a-msg'); + useLibrarianStore.getState().setActiveProjectId(PROJECT_B); + useLibrarianStore.getState().addMessage('user', 'b-msg'); + useLibrarianStore.getState().clearMessages(); + const { byProject } = useLibrarianStore.getState(); + expect(byProject[PROJECT_A].messages).toHaveLength(1); + expect(byProject[PROJECT_B].messages).toHaveLength(0); + }); + + it('no-ops when activeProjectId is null', () => { + useLibrarianStore.getState().addMessage('user', 'a-msg'); + resetStore(null); + useLibrarianStore.setState({ + byProject: { [PROJECT_A]: { messages: [], pdfFiles: [], pdfChunks: [] } }, + }); + // No active id; clearMessages should not throw and not modify state. + expect(() => useLibrarianStore.getState().clearMessages()).not.toThrow(); + }); + }); + + // ---------- loading ---------- + + describe('setLoading', () => { + it('sets isLoading to true', () => { + useLibrarianStore.getState().setLoading(true); + expect(useLibrarianStore.getState().isLoading).toBe(true); + }); + + it('sets isLoading to false', () => { + useLibrarianStore.getState().setLoading(true); + useLibrarianStore.getState().setLoading(false); + expect(useLibrarianStore.getState().isLoading).toBe(false); + }); + + it('isLoading is global (not per project)', () => { + useLibrarianStore.getState().setLoading(true); + useLibrarianStore.getState().setActiveProjectId(PROJECT_B); + expect(useLibrarianStore.getState().isLoading).toBe(true); + }); + }); + + // ---------- PDF files ---------- + + describe('addPdfFile', () => { + it('adds a PDF file', () => { + useLibrarianStore.getState().addPdfFile(makePdfFile()); + expect(useLibrarianStore.getState().pdfFiles).toHaveLength(1); + expect(useLibrarianStore.getState().pdfFiles[0].name).toBe('test.pdf'); + }); + + it('adds multiple PDF files', () => { + const store = useLibrarianStore.getState(); + store.addPdfFile(makePdfFile({ id: 'f1', name: 'a.pdf' })); + store.addPdfFile(makePdfFile({ id: 'f2', name: 'b.pdf' })); + expect(useLibrarianStore.getState().pdfFiles).toHaveLength(2); + }); + + it('no-ops when activeProjectId is null', () => { + resetStore(null); + useLibrarianStore.getState().addPdfFile(makePdfFile()); + expect(useLibrarianStore.getState().pdfFiles).toHaveLength(0); + }); + }); + + describe('addPdfChunks', () => { + it('adds chunks to the store', () => { + useLibrarianStore.getState().addPdfFile(makePdfFile({ id: 'file-1' })); + useLibrarianStore + .getState() + .addPdfChunks([makePdfChunk({ id: 'c1' }), makePdfChunk({ id: 'c2' })]); + expect(useLibrarianStore.getState().pdfChunks).toHaveLength(2); + }); + + it('appends to existing chunks', () => { + const store = useLibrarianStore.getState(); + store.addPdfFile(makePdfFile({ id: 'file-1' })); + store.addPdfChunks([makePdfChunk({ id: 'c1' })]); + store.addPdfChunks([makePdfChunk({ id: 'c2' })]); + expect(useLibrarianStore.getState().pdfChunks).toHaveLength(2); + }); + + it('drops chunks for files that no longer exist', () => { + const store = useLibrarianStore.getState(); + store.addPdfFile(makePdfFile({ id: 'f1' })); + store.removePdf('f1'); + store.addPdfChunks([makePdfChunk({ id: 'c1', fileId: 'f1' })]); + expect(useLibrarianStore.getState().pdfChunks).toEqual([]); + }); + + it('logs a debug trace when chunks are dropped for missing files', () => { + const debugSpy = vi.spyOn(console, 'debug').mockImplementation(() => {}); + const store = useLibrarianStore.getState(); + store.addPdfFile(makePdfFile({ id: 'f1' })); + store.addPdfChunks([ + makePdfChunk({ id: 'c1', fileId: 'f1' }), + makePdfChunk({ id: 'c2', fileId: 'ghost' }), + ]); + expect(debugSpy).toHaveBeenCalledWith(expect.stringContaining('dropped 1 chunk(s)')); + debugSpy.mockRestore(); + }); + }); + + describe('removePdf', () => { + it('removes the file and its associated chunks', () => { + const store = useLibrarianStore.getState(); + store.addPdfFile(makePdfFile({ id: 'f1' })); + store.addPdfFile(makePdfFile({ id: 'f2', name: 'other.pdf' })); + store.addPdfChunks([ + makePdfChunk({ id: 'c1', fileId: 'f1' }), + makePdfChunk({ id: 'c2', fileId: 'f1' }), + makePdfChunk({ id: 'c3', fileId: 'f2' }), + ]); + + useLibrarianStore.getState().removePdf('f1'); + + const state = useLibrarianStore.getState(); + expect(state.pdfFiles).toHaveLength(1); + expect(state.pdfFiles[0].id).toBe('f2'); + expect(state.pdfChunks).toHaveLength(1); + expect(state.pdfChunks[0].id).toBe('c3'); + }); + + it('does nothing when fileId does not exist', () => { + const store = useLibrarianStore.getState(); + store.addPdfFile(makePdfFile({ id: 'f1' })); + store.removePdf('nonexistent'); + expect(useLibrarianStore.getState().pdfFiles).toHaveLength(1); + }); + }); + + describe('setPdfStatus', () => { + it('updates the status of a PDF file', () => { + const store = useLibrarianStore.getState(); + store.addPdfFile(makePdfFile({ id: 'f1', status: 'processing' })); + store.setPdfStatus('f1', 'ready'); + expect(useLibrarianStore.getState().pdfFiles[0].status).toBe('ready'); + }); + + it('sets an error message', () => { + const store = useLibrarianStore.getState(); + store.addPdfFile(makePdfFile({ id: 'f1' })); + store.setPdfStatus('f1', 'error', 'something went wrong'); + const file = useLibrarianStore.getState().pdfFiles[0]; + expect(file.status).toBe('error'); + expect(file.error).toBe('something went wrong'); + }); + + it('does not affect other files', () => { + const store = useLibrarianStore.getState(); + store.addPdfFile(makePdfFile({ id: 'f1', status: 'processing' })); + store.addPdfFile(makePdfFile({ id: 'f2', name: 'b.pdf', status: 'processing' })); + store.setPdfStatus('f1', 'ready'); + expect(useLibrarianStore.getState().pdfFiles[1].status).toBe('processing'); + }); + }); + + describe('hasPdfFile', () => { + it('returns true when file exists in the active project', () => { + useLibrarianStore.getState().addPdfFile(makePdfFile({ name: 'test.pdf' })); + expect(useLibrarianStore.getState().hasPdfFile('test.pdf')).toBe(true); + }); + + it('returns false when file does not exist', () => { + expect(useLibrarianStore.getState().hasPdfFile('nope.pdf')).toBe(false); + }); + + it('matches by name, not id', () => { + useLibrarianStore.getState().addPdfFile(makePdfFile({ id: 'f1', name: 'report.pdf' })); + expect(useLibrarianStore.getState().hasPdfFile('report.pdf')).toBe(true); + expect(useLibrarianStore.getState().hasPdfFile('f1')).toBe(false); + }); + + it('matches names case-insensitively', () => { + useLibrarianStore.getState().addPdfFile(makePdfFile({ id: 'f1', name: 'Report.pdf' })); + expect(useLibrarianStore.getState().hasPdfFile('report.pdf')).toBe(true); + expect(useLibrarianStore.getState().hasPdfFile('REPORT.PDF')).toBe(true); + }); + + it('is scoped to the active project', () => { + // Upload report.pdf to project A + useLibrarianStore.getState().addPdfFile(makePdfFile({ id: 'f1', name: 'report.pdf' })); + expect(useLibrarianStore.getState().hasPdfFile('report.pdf')).toBe(true); + + // Switch to project B — A's file must not be visible + useLibrarianStore.getState().setActiveProjectId(PROJECT_B); + expect(useLibrarianStore.getState().hasPdfFile('report.pdf')).toBe(false); + }); + + it('returns false when activeProjectId is null', () => { + resetStore(null); + expect(useLibrarianStore.getState().hasPdfFile('anything.pdf')).toBe(false); + }); + }); + + // ---------- per-project isolation ---------- + + describe('per-project isolation', () => { + it('isolates messages and PDFs across two project ids', () => { + const store = useLibrarianStore.getState(); + // Project A + store.addMessage('user', 'hello from A'); + store.addPdfFile(makePdfFile({ id: 'a1', name: 'a.pdf' })); + store.addPdfChunks([makePdfChunk({ id: 'ac1', fileId: 'a1', fileName: 'a.pdf' })]); + + // Switch to Project B and add different content + useLibrarianStore.getState().setActiveProjectId(PROJECT_B); + const stateAfterSwitch = useLibrarianStore.getState(); + expect(stateAfterSwitch.messages).toEqual([]); + expect(stateAfterSwitch.pdfFiles).toEqual([]); + expect(stateAfterSwitch.pdfChunks).toEqual([]); + + useLibrarianStore.getState().addMessage('user', 'hello from B'); + useLibrarianStore.getState().addPdfFile(makePdfFile({ id: 'b1', name: 'b.pdf' })); + useLibrarianStore + .getState() + .addPdfChunks([makePdfChunk({ id: 'bc1', fileId: 'b1', fileName: 'b.pdf' })]); + + const { byProject } = useLibrarianStore.getState(); + expect(byProject[PROJECT_A].messages.map((m) => m.content)).toEqual(['hello from A']); + expect(byProject[PROJECT_A].pdfFiles[0].name).toBe('a.pdf'); + expect(byProject[PROJECT_A].pdfChunks).toHaveLength(1); + expect(byProject[PROJECT_B].messages.map((m) => m.content)).toEqual(['hello from B']); + expect(byProject[PROJECT_B].pdfFiles[0].name).toBe('b.pdf'); + expect(byProject[PROJECT_B].pdfChunks).toHaveLength(1); + }); + + it('switching back to a previous project restores its data', () => { + const store = useLibrarianStore.getState(); + store.addMessage('user', 'A says hi'); + store.addPdfFile(makePdfFile({ id: 'a1', name: 'a.pdf' })); + + useLibrarianStore.getState().setActiveProjectId(PROJECT_B); + useLibrarianStore.getState().addMessage('user', 'B says hi'); + + useLibrarianStore.getState().setActiveProjectId(PROJECT_A); + const state = useLibrarianStore.getState(); + expect(state.messages.map((m) => m.content)).toEqual(['A says hi']); + expect(state.pdfFiles[0].name).toBe('a.pdf'); + }); + + it('setActiveProjectId(null) blanks the flat mirror without dropping buckets', () => { + const store = useLibrarianStore.getState(); + store.addMessage('user', 'A says hi'); + useLibrarianStore.getState().setActiveProjectId(null); + const state = useLibrarianStore.getState(); + expect(state.messages).toEqual([]); + expect(state.pdfFiles).toEqual([]); + expect(state.pdfChunks).toEqual([]); + expect(state.byProject[PROJECT_A].messages).toHaveLength(1); + }); + }); + + // ---------- explicit-bucket writes ---------- + + describe('addMessageToProject', () => { + it('writes to a non-active bucket that already exists', () => { + // Seed B's bucket while it's active, then switch back to A. + useLibrarianStore.getState().setActiveProjectId(PROJECT_B); + useLibrarianStore.getState().addMessage('user', 'B existing'); + useLibrarianStore.getState().setActiveProjectId(PROJECT_A); + + // Now write to B from A's context — bucket exists, write succeeds. + useLibrarianStore.getState().addMessageToProject(PROJECT_B, 'assistant', 'for B'); + const state = useLibrarianStore.getState(); + expect(state.byProject[PROJECT_B].messages.map((m) => m.content)).toEqual([ + 'B existing', + 'for B', + ]); + // A is untouched; flat mirror reflects the active project (A), which is empty. + expect(state.byProject[PROJECT_A]).toBeUndefined(); + expect(state.messages).toEqual([]); + }); + + it('updates the flat mirror when writing to the active project', () => { + useLibrarianStore.getState().addMessageToProject(PROJECT_A, 'assistant', 'reply'); + const state = useLibrarianStore.getState(); + expect(state.byProject[PROJECT_A].messages).toHaveLength(1); + expect(state.messages.map((m) => m.content)).toEqual(['reply']); + }); + + it('preserves cross-project routing after a mid-flight switch', () => { + // Simulates use-librarian-chat: capture the project id, switch active, + // then write the assistant response back to the captured id. + useLibrarianStore.getState().addMessageToProject(PROJECT_A, 'user', 'A asked'); + useLibrarianStore.getState().setActiveProjectId(PROJECT_B); + useLibrarianStore.getState().addMessageToProject(PROJECT_A, 'assistant', 'A answer'); + + const state = useLibrarianStore.getState(); + expect(state.byProject[PROJECT_A].messages.map((m) => m.content)).toEqual([ + 'A asked', + 'A answer', + ]); + // B's bucket is untouched (no leakage). + expect(state.byProject[PROJECT_B]?.messages ?? []).toEqual([]); + }); + + it('drops the write when the bucket was pruned (project deleted mid-flight)', () => { + // Seed A's bucket as active, then simulate deletion: switch away and + // prune. A late-arriving response must not resurrect a zombie bucket. + useLibrarianStore.getState().addMessage('user', 'A asked'); + useLibrarianStore.getState().setActiveProjectId(PROJECT_B); + useLibrarianStore.getState().pruneProjectBuckets(new Set([PROJECT_B])); + expect(useLibrarianStore.getState().byProject[PROJECT_A]).toBeUndefined(); + + useLibrarianStore.getState().addMessageToProject(PROJECT_A, 'assistant', 'late reply'); + const state = useLibrarianStore.getState(); + expect(state.byProject[PROJECT_A]).toBeUndefined(); + }); + }); + + describe('addPdfFileToProject / addPdfChunksToProject / setPdfStatusForProject', () => { + it('routes the full PDF lifecycle to the originating project after a mid-flight switch', () => { + // Simulates handlePdfUpload capturing projectId at upload time: + // user uploads in A, switches to B mid-process, chunks/status arrive late. + useLibrarianStore + .getState() + .addPdfFileToProject(PROJECT_A, makePdfFile({ id: 'f1', status: 'processing' })); + useLibrarianStore.getState().setActiveProjectId(PROJECT_B); + + useLibrarianStore + .getState() + .addPdfChunksToProject(PROJECT_A, [makePdfChunk({ id: 'c1', fileId: 'f1' })]); + useLibrarianStore.getState().setPdfStatusForProject(PROJECT_A, 'f1', 'ready'); + + const state = useLibrarianStore.getState(); + expect(state.byProject[PROJECT_A].pdfFiles[0].status).toBe('ready'); + expect(state.byProject[PROJECT_A].pdfChunks).toHaveLength(1); + // B's bucket is untouched. + expect(state.byProject[PROJECT_B]?.pdfFiles ?? []).toEqual([]); + expect(state.byProject[PROJECT_B]?.pdfChunks ?? []).toEqual([]); + // Flat mirror tracks active project B. + expect(state.pdfFiles).toEqual([]); + expect(state.pdfChunks).toEqual([]); + }); + + it('drops PDF writes when the bucket was pruned', () => { + useLibrarianStore.getState().addPdfFile(makePdfFile({ id: 'f1' })); + useLibrarianStore.getState().setActiveProjectId(PROJECT_B); + useLibrarianStore.getState().pruneProjectBuckets(new Set([PROJECT_B])); + + useLibrarianStore + .getState() + .addPdfChunksToProject(PROJECT_A, [makePdfChunk({ id: 'c1', fileId: 'f1' })]); + useLibrarianStore.getState().setPdfStatusForProject(PROJECT_A, 'f1', 'ready'); + + expect(useLibrarianStore.getState().byProject[PROJECT_A]).toBeUndefined(); + }); + }); + + describe('pruneProjectBuckets', () => { + it('drops buckets whose id is not in the valid set', () => { + useLibrarianStore.getState().addMessage('user', 'A msg'); + useLibrarianStore.getState().setActiveProjectId(PROJECT_B); + useLibrarianStore.getState().addMessage('user', 'B msg'); + + // Only PROJECT_A is valid — B's bucket should be dropped. + useLibrarianStore.getState().pruneProjectBuckets(new Set([PROJECT_A])); + const state = useLibrarianStore.getState(); + expect(state.byProject[PROJECT_A]).toBeDefined(); + expect(state.byProject[PROJECT_B]).toBeUndefined(); + }); + + it('is a no-op when all buckets are valid', () => { + useLibrarianStore.getState().addMessage('user', 'A msg'); + const before = useLibrarianStore.getState().byProject; + useLibrarianStore.getState().pruneProjectBuckets(new Set([PROJECT_A, PROJECT_B])); + // Reference equality: state was returned unchanged. + expect(useLibrarianStore.getState().byProject).toBe(before); + }); + + it('does not modify the flat mirror when the active project survives', () => { + useLibrarianStore.getState().addMessage('user', 'A msg'); + useLibrarianStore.getState().pruneProjectBuckets(new Set([PROJECT_A])); + const state = useLibrarianStore.getState(); + expect(state.activeProjectId).toBe(PROJECT_A); + expect(state.messages.map((m) => m.content)).toEqual(['A msg']); + }); + }); + + // ---------- selector hooks ---------- + + describe('selector hooks', () => { + it('useLibrarianMessages returns the active project bucket', () => { + useLibrarianStore.getState().addMessage('user', 'a'); + // Hooks return state by reading the store synchronously when not inside a component. + // Here we just verify the selector logic against getState(). + const id = useLibrarianStore.getState().activeProjectId!; + expect(useLibrarianStore.getState().byProject[id].messages).toHaveLength(1); + // Indirect: stable empty array when project absent + useLibrarianStore.getState().setActiveProjectId('unknown-id'); + const state = useLibrarianStore.getState(); + expect(state.byProject['unknown-id']).toBeUndefined(); + // The selector hooks themselves are exercised via component tests; here we + // confirm they're exported and typed correctly. + expect(useLibrarianMessages).toBeTypeOf('function'); + expect(useLibrarianPdfFiles).toBeTypeOf('function'); + expect(useLibrarianPdfChunks).toBeTypeOf('function'); + }); + }); + + // ---------- hasConfig / refreshConfig ---------- + + describe('hasConfig', () => { + it('defaults to false when no config exists', () => { + expect(useLibrarianStore.getState().hasConfig).toBe(false); + }); + + it('updates to true after refreshConfig when config exists', () => { + vi.mocked(loadAIConfig).mockReturnValue({ + provider: 'openai', + apiKey: 'sk-test', + model: 'gpt-4o', + }); + useLibrarianStore.getState().refreshConfig(); + expect(useLibrarianStore.getState().hasConfig).toBe(true); + }); + + it('updates to false after refreshConfig when config is removed', () => { + useLibrarianStore.setState({ hasConfig: true }); + vi.mocked(loadAIConfig).mockReturnValue(null); + useLibrarianStore.getState().refreshConfig(); + expect(useLibrarianStore.getState().hasConfig).toBe(false); + }); + }); +}); diff --git a/app/src/features/librarian/__tests__/use-librarian-chat.test.ts b/app/src/features/librarian/__tests__/use-librarian-chat.test.ts new file mode 100644 index 00000000..cf77fba0 --- /dev/null +++ b/app/src/features/librarian/__tests__/use-librarian-chat.test.ts @@ -0,0 +1,507 @@ +/* eslint-disable @typescript-eslint/no-explicit-any */ +import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'; +import { renderHook, act } from '@testing-library/react'; + +import { useLibrarianStore } from '../store'; +import type { PdfChunk } from '../types'; + +// ---------- Mocks ---------- + +vi.mock('../services/ai-service', () => ({ + loadAIConfig: vi.fn(), + sendChatMessage: vi.fn(), +})); + +vi.mock('../services/lineage-formatter', () => ({ + formatLineage: vi.fn(), +})); + +vi.mock('../services/context-builder', () => ({ + buildContext: vi.fn(), + buildPrompt: vi.fn(), + getPromptStats: vi.fn(), +})); + +vi.mock('../services/embedding-service', () => ({ + embedTexts: vi.fn(), +})); + +vi.mock('../services/vector-search', () => ({ + searchChunks: vi.fn(), +})); + +// Mock lineage state +const mockResult = { globalLineage: { nodes: [], edges: [] } }; +vi.mock('@pondpilot/flowscope-react', () => ({ + useLineageState: () => ({ result: mockResult }), +})); + +// Mock project store +const mockCurrentProject = { + id: 'proj-1', + name: 'Test Project', + files: [ + { id: 'file-1', name: 'query.sql', path: 'query.sql', content: 'SELECT 1', language: 'sql' }, + ], + activeFileId: 'file-1', + dialect: 'generic', + runMode: 'all', + selectedFileIds: [], + schemaSQL: '', + templateMode: 'raw', +}; +vi.mock('@/lib/project-store', () => ({ + useProject: () => ({ currentProject: mockCurrentProject, activeProjectId: 'proj-1' }), +})); + +// Import mocked modules after vi.mock +import { loadAIConfig, sendChatMessage } from '../services/ai-service'; +import { formatLineage } from '../services/lineage-formatter'; +import { buildContext, buildPrompt, getPromptStats } from '../services/context-builder'; +import { embedTexts } from '../services/embedding-service'; +import { searchChunks } from '../services/vector-search'; +import { useLibrarianChat } from '../hooks/use-librarian-chat'; + +// Typed mocks +const mockedLoadAIConfig = vi.mocked(loadAIConfig); +const mockedSendChatMessage = vi.mocked(sendChatMessage); +const mockedFormatLineage = vi.mocked(formatLineage); +const mockedBuildContext = vi.mocked(buildContext); +const mockedBuildPrompt = vi.mocked(buildPrompt); +const mockedGetPromptStats = vi.mocked(getPromptStats); +const mockedEmbedTexts = vi.mocked(embedTexts); +const mockedSearchChunks = vi.mocked(searchChunks); + +// ---------- Setup ---------- + +beforeEach(() => { + useLibrarianStore.setState({ + byProject: { 'proj-1': { messages: [], pdfFiles: [], pdfChunks: [] } }, + activeProjectId: 'proj-1', + messages: [], + isLoading: false, + pdfFiles: [], + pdfChunks: [], + }); + vi.clearAllMocks(); + + // Restore default implementations + mockedLoadAIConfig.mockReturnValue({ + provider: 'openai', + apiKey: 'sk-test', + model: 'gpt-4o', + }); + mockedSendChatMessage.mockResolvedValue('AI response'); + mockedFormatLineage.mockReturnValue('formatted lineage'); + mockedBuildContext.mockReturnValue({ + lineage: 'lineage', + pdfCitations: '', + chatHistory: '', + sqlSnippet: '', + }); + mockedBuildPrompt.mockReturnValue('system prompt'); + mockedGetPromptStats.mockReturnValue({ characters: 13, bytes: 13 }); + mockedEmbedTexts.mockResolvedValue([[0.1, 0.2, 0.3]]); + mockedSearchChunks.mockReturnValue([]); +}); + +afterEach(() => { + vi.restoreAllMocks(); +}); + +// ---------- Tests ---------- + +describe('useLibrarianChat', () => { + it('adds user and assistant messages on successful send', async () => { + const { result } = renderHook(() => useLibrarianChat()); + + await act(async () => { + await result.current.sendMessage('Hello'); + }); + + const state = useLibrarianStore.getState(); + expect(state.messages).toHaveLength(2); + expect(state.messages[0].role).toBe('user'); + expect(state.messages[0].content).toBe('Hello'); + expect(state.messages[1].role).toBe('assistant'); + expect(state.messages[1].content).toBe('AI response'); + }); + + it('shows config message when AI is not configured', async () => { + mockedLoadAIConfig.mockReturnValue(null); + const { result } = renderHook(() => useLibrarianChat()); + + await act(async () => { + await result.current.sendMessage('Hello'); + }); + + const state = useLibrarianStore.getState(); + expect(state.messages).toHaveLength(1); + expect(state.messages[0].role).toBe('assistant'); + expect(state.messages[0].content).toContain('configure'); + }); + + it('calls formatLineage with the analysis result', async () => { + const { result } = renderHook(() => useLibrarianChat()); + + await act(async () => { + await result.current.sendMessage('test'); + }); + + expect(mockedFormatLineage).toHaveBeenCalledWith(mockResult); + }); + + it('passes SQL from active file to context builder', async () => { + const { result } = renderHook(() => useLibrarianChat()); + + await act(async () => { + await result.current.sendMessage('test'); + }); + + expect(mockedBuildContext).toHaveBeenCalledWith( + expect.objectContaining({ sqlSnippet: 'SELECT 1' }) + ); + }); + + it('sends prompt and user message to AI service', async () => { + const { result } = renderHook(() => useLibrarianChat()); + + await act(async () => { + await result.current.sendMessage('my question'); + }); + + expect(mockedSendChatMessage).toHaveBeenCalledWith( + expect.objectContaining({ provider: 'openai' }), + 'system prompt', + 'my question', + expect.any(AbortSignal) + ); + }); + + it('passes configured prompt override to the prompt builder', async () => { + mockedLoadAIConfig.mockReturnValue({ + provider: 'openai', + apiKey: 'sk-test', + model: 'gpt-4o', + systemPrompt: 'Custom prompt', + }); + const { result } = renderHook(() => useLibrarianChat()); + + await act(async () => { + await result.current.sendMessage('my question'); + }); + + expect(mockedBuildPrompt).toHaveBeenCalledWith(expect.any(Object), { + systemPrompt: 'Custom prompt', + }); + }); + + it('stores the final prompt size for the originating project', async () => { + const { result } = renderHook(() => useLibrarianChat()); + + await act(async () => { + await result.current.sendMessage('my question'); + }); + + expect(mockedGetPromptStats).toHaveBeenCalledWith('system prompt'); + expect(useLibrarianStore.getState().byProject['proj-1'].lastPromptStats).toEqual({ + characters: 13, + bytes: 13, + }); + }); + + it('sets loading state during request', async () => { + let loadingDuringRequest = false; + mockedSendChatMessage.mockImplementation(async () => { + loadingDuringRequest = useLibrarianStore.getState().isLoading; + return 'response'; + }); + + const { result } = renderHook(() => useLibrarianChat()); + + await act(async () => { + await result.current.sendMessage('test'); + }); + + expect(loadingDuringRequest).toBe(true); + expect(useLibrarianStore.getState().isLoading).toBe(false); + }); + + it('handles AI service errors gracefully', async () => { + mockedSendChatMessage.mockRejectedValue(new Error('API limit reached')); + const { result } = renderHook(() => useLibrarianChat()); + + await act(async () => { + await result.current.sendMessage('test'); + }); + + const state = useLibrarianStore.getState(); + expect(state.messages).toHaveLength(2); + expect(state.messages[1].role).toBe('assistant'); + expect(state.messages[1].content).toContain('API limit reached'); + }); + + it('handles non-Error exceptions', async () => { + mockedSendChatMessage.mockRejectedValue('string error'); + const { result } = renderHook(() => useLibrarianChat()); + + await act(async () => { + await result.current.sendMessage('test'); + }); + + const state = useLibrarianStore.getState(); + expect(state.messages[1].content).toContain('unexpected error'); + }); + + it('handles abort/cancellation', async () => { + mockedSendChatMessage.mockImplementation(async () => { + const error = new Error('Aborted'); + error.name = 'AbortError'; + throw error; + }); + + const { result } = renderHook(() => useLibrarianChat()); + + await act(async () => { + await result.current.sendMessage('test'); + }); + + const state = useLibrarianStore.getState(); + expect(state.messages[1].content).toContain('cancelled'); + expect(state.isLoading).toBe(false); + }); + + it('cancel aborts the current request', async () => { + let capturedSignal: AbortSignal | undefined; + mockedSendChatMessage.mockImplementation( + async (_config: any, _prompt: any, _msg: any, signal?: AbortSignal) => { + capturedSignal = signal; + await new Promise((resolve) => setTimeout(resolve, 100)); + if (signal?.aborted) { + const err = new Error('Aborted'); + err.name = 'AbortError'; + throw err; + } + return 'response'; + } + ); + + const { result } = renderHook(() => useLibrarianChat()); + + let sendPromise: Promise; + act(() => { + sendPromise = result.current.sendMessage('test'); + }); + + act(() => { + result.current.cancel(); + }); + + await act(async () => { + await sendPromise!; + }); + + expect(capturedSignal?.aborted).toBe(true); + }); + + describe('PDF vector search', () => { + const pdfChunks: PdfChunk[] = [ + { + id: 'c1', + fileId: 'f1', + fileName: 'doc.pdf', + text: 'relevant content', + pageNumber: 3, + embedding: [0.5, 0.5, 0.5], + }, + ]; + + it('searches PDF chunks when available', async () => { + useLibrarianStore.setState({ + byProject: { 'proj-1': { messages: [], pdfFiles: [], pdfChunks } }, + }); + mockedSearchChunks.mockReturnValue(pdfChunks); + + const { result } = renderHook(() => useLibrarianChat()); + + await act(async () => { + await result.current.sendMessage('question'); + }); + + expect(mockedEmbedTexts).toHaveBeenCalledWith(['question'], 'query'); + expect(mockedSearchChunks).toHaveBeenCalled(); + expect(mockedBuildContext).toHaveBeenCalledWith( + expect.objectContaining({ + pdfCitations: expect.stringContaining('doc.pdf'), + }) + ); + }); + + it('skips vector search when no PDF chunks', async () => { + const { result } = renderHook(() => useLibrarianChat()); + + await act(async () => { + await result.current.sendMessage('question'); + }); + + expect(mockedEmbedTexts).not.toHaveBeenCalled(); + expect(mockedSearchChunks).not.toHaveBeenCalled(); + }); + + it('reports an error if PDF embedding fails', async () => { + useLibrarianStore.setState({ + byProject: { 'proj-1': { messages: [], pdfFiles: [], pdfChunks } }, + }); + mockedEmbedTexts.mockRejectedValue(new Error('embedding error')); + + const { result } = renderHook(() => useLibrarianChat()); + + await act(async () => { + await result.current.sendMessage('question'); + }); + + expect(mockedSendChatMessage).not.toHaveBeenCalled(); + expect(mockedBuildContext).not.toHaveBeenCalled(); + expect(useLibrarianStore.getState().messages[1].content).toContain( + 'Failed to search uploaded PDFs: embedding error' + ); + }); + + it('does not search project A PDF chunks while project B is active', async () => { + // Seed project A with chunks; project B has none. Active project is A. + useLibrarianStore.setState({ + activeProjectId: 'proj-a', + byProject: { + 'proj-a': { messages: [], pdfFiles: [], pdfChunks }, + 'proj-b': { messages: [], pdfFiles: [], pdfChunks: [] }, + }, + }); + + // Switch active project to B before sending. + useLibrarianStore.setState({ activeProjectId: 'proj-b' }); + + const { result } = renderHook(() => useLibrarianChat()); + + await act(async () => { + await result.current.sendMessage('question'); + }); + + expect(mockedEmbedTexts).not.toHaveBeenCalled(); + expect(mockedSearchChunks).not.toHaveBeenCalled(); + expect(mockedBuildContext).toHaveBeenCalledWith( + expect.objectContaining({ pdfCitations: '' }) + ); + }); + }); + + describe('mid-flight project switch', () => { + it('routes the assistant response to the originating project, not the live active one', async () => { + // Hold the network call open until we explicitly resolve it. + let resolveSend: (value: string) => void = () => {}; + mockedSendChatMessage.mockImplementation( + () => + new Promise((resolve) => { + resolveSend = resolve; + }) + ); + + useLibrarianStore.setState({ + activeProjectId: 'proj-1', + byProject: { 'proj-1': { messages: [], pdfFiles: [], pdfChunks: [] } }, + }); + + const { result } = renderHook(() => useLibrarianChat()); + + let sendPromise!: Promise; + act(() => { + sendPromise = result.current.sendMessage('question for proj-1'); + }); + + // User switches to a different project before the response arrives. + // Seed the destination bucket so it's a realistic switch. + act(() => { + useLibrarianStore.setState({ + activeProjectId: 'proj-2', + byProject: { + ...useLibrarianStore.getState().byProject, + 'proj-2': { messages: [], pdfFiles: [], pdfChunks: [] }, + }, + }); + }); + + await act(async () => { + resolveSend('answer for proj-1'); + await sendPromise; + }); + + const state = useLibrarianStore.getState(); + const proj1Messages = state.byProject['proj-1'].messages.map((m) => m.content); + const proj2Messages = state.byProject['proj-2']?.messages.map((m) => m.content) ?? []; + expect(proj1Messages).toEqual(['question for proj-1', 'answer for proj-1']); + expect(proj2Messages).toEqual([]); + }); + + it('routes errors to the originating project too', async () => { + let rejectSend: (reason: Error) => void = () => {}; + mockedSendChatMessage.mockImplementation( + () => + new Promise((_resolve, reject) => { + rejectSend = reject; + }) + ); + + useLibrarianStore.setState({ + activeProjectId: 'proj-1', + byProject: { 'proj-1': { messages: [], pdfFiles: [], pdfChunks: [] } }, + }); + + const { result } = renderHook(() => useLibrarianChat()); + + let sendPromise!: Promise; + act(() => { + sendPromise = result.current.sendMessage('question'); + }); + + act(() => { + useLibrarianStore.setState({ + activeProjectId: 'proj-2', + byProject: { + ...useLibrarianStore.getState().byProject, + 'proj-2': { messages: [], pdfFiles: [], pdfChunks: [] }, + }, + }); + }); + + await act(async () => { + rejectSend(new Error('network down')); + await sendPromise; + }); + + const state = useLibrarianStore.getState(); + const proj1Messages = state.byProject['proj-1'].messages.map((m) => m.content); + const proj2Messages = state.byProject['proj-2']?.messages.map((m) => m.content) ?? []; + expect(proj1Messages[0]).toBe('question'); + expect(proj1Messages[1]).toContain('network down'); + expect(proj2Messages).toEqual([]); + }); + }); + + describe('no active project', () => { + it('bails silently when activeProjectId is null', async () => { + useLibrarianStore.setState({ activeProjectId: null, byProject: {} }); + + const { result } = renderHook(() => useLibrarianChat()); + + await act(async () => { + await result.current.sendMessage('question'); + }); + + // No bucket exists to write to — the chat input UI shows the + // "Open or create a project" hint via its `noActiveProject` prop, + // so the hook just bails without producing a message. + expect(mockedSendChatMessage).not.toHaveBeenCalled(); + expect(mockedBuildContext).not.toHaveBeenCalled(); + expect(useLibrarianStore.getState().byProject).toEqual({}); + }); + }); +}); diff --git a/app/src/features/librarian/__tests__/use-sync-active-project.test.tsx b/app/src/features/librarian/__tests__/use-sync-active-project.test.tsx new file mode 100644 index 00000000..b09ad6c4 --- /dev/null +++ b/app/src/features/librarian/__tests__/use-sync-active-project.test.tsx @@ -0,0 +1,102 @@ +import { beforeEach, describe, expect, it, vi } from 'vitest'; +import { renderHook } from '@testing-library/react'; + +vi.mock('../services/ai-service', () => ({ + loadAIConfig: vi.fn(() => null), +})); + +let mockActiveProjectId: string | null = null; +let mockProjects: { id: string }[] = []; +vi.mock('@/lib/project-store', () => ({ + useProject: () => ({ activeProjectId: mockActiveProjectId, projects: mockProjects }), +})); + +import { useLibrarianStore } from '../store'; +import { useSyncActiveProject } from '../hooks/use-sync-active-project'; + +const PROJECT_A = 'proj-a'; +const PROJECT_B = 'proj-b'; + +beforeEach(() => { + mockActiveProjectId = null; + mockProjects = []; + useLibrarianStore.setState({ + byProject: {}, + activeProjectId: null, + isLoading: false, + hasConfig: false, + messages: [], + pdfFiles: [], + pdfChunks: [], + }); +}); + +describe('useSyncActiveProject', () => { + it('pushes the initial activeProjectId from useProject() into the store', () => { + mockActiveProjectId = PROJECT_A; + renderHook(() => useSyncActiveProject()); + expect(useLibrarianStore.getState().activeProjectId).toBe(PROJECT_A); + }); + + it('updates the store when activeProjectId changes between renders', () => { + mockActiveProjectId = PROJECT_A; + const { rerender } = renderHook(() => useSyncActiveProject()); + expect(useLibrarianStore.getState().activeProjectId).toBe(PROJECT_A); + + mockActiveProjectId = PROJECT_B; + rerender(); + expect(useLibrarianStore.getState().activeProjectId).toBe(PROJECT_B); + }); + + it('syncs null when no project is active', () => { + mockActiveProjectId = PROJECT_A; + const { rerender } = renderHook(() => useSyncActiveProject()); + expect(useLibrarianStore.getState().activeProjectId).toBe(PROJECT_A); + + mockActiveProjectId = null; + rerender(); + expect(useLibrarianStore.getState().activeProjectId).toBeNull(); + }); + + it('switching back re-points the flat mirror to the original bucket', () => { + // Start on project A and seed it via the store mutators + mockActiveProjectId = PROJECT_A; + const { rerender } = renderHook(() => useSyncActiveProject()); + useLibrarianStore.getState().addMessage('user', 'A says hi'); + expect(useLibrarianStore.getState().messages.map((m) => m.content)).toEqual(['A says hi']); + + // Switch to project B — the flat mirror should be empty + mockActiveProjectId = PROJECT_B; + rerender(); + expect(useLibrarianStore.getState().activeProjectId).toBe(PROJECT_B); + expect(useLibrarianStore.getState().messages).toEqual([]); + useLibrarianStore.getState().addMessage('user', 'B says hi'); + + // Switch back to A — the original bucket data must be restored + mockActiveProjectId = PROJECT_A; + rerender(); + expect(useLibrarianStore.getState().activeProjectId).toBe(PROJECT_A); + expect(useLibrarianStore.getState().messages.map((m) => m.content)).toEqual(['A says hi']); + }); + + it('drops Librarian buckets for projects removed from the project list', () => { + // Both projects exist; seed both buckets. + mockActiveProjectId = PROJECT_A; + mockProjects = [{ id: PROJECT_A }, { id: PROJECT_B }]; + const { rerender } = renderHook(() => useSyncActiveProject()); + useLibrarianStore.getState().addMessage('user', 'A msg'); + useLibrarianStore.getState().setActiveProjectId(PROJECT_B); + useLibrarianStore.getState().addMessage('user', 'B msg'); + useLibrarianStore.getState().setActiveProjectId(PROJECT_A); + + expect(Object.keys(useLibrarianStore.getState().byProject).sort()).toEqual([ + PROJECT_A, + PROJECT_B, + ]); + + // Project B is deleted from the project list — its bucket must be dropped. + mockProjects = [{ id: PROJECT_A }]; + rerender(); + expect(Object.keys(useLibrarianStore.getState().byProject)).toEqual([PROJECT_A]); + }); +}); diff --git a/app/src/features/librarian/__tests__/vector-search.test.ts b/app/src/features/librarian/__tests__/vector-search.test.ts new file mode 100644 index 00000000..e601aebf --- /dev/null +++ b/app/src/features/librarian/__tests__/vector-search.test.ts @@ -0,0 +1,83 @@ +import { describe, expect, it } from 'vitest'; + +import type { PdfChunk } from '../types'; +import { cosineSimilarity, searchChunks } from '../services/vector-search'; + +function makeChunk(id: string, embedding: number[]): PdfChunk { + return { + id, + fileId: 'file-1', + fileName: 'test.pdf', + text: `chunk ${id}`, + pageNumber: 1, + embedding, + }; +} + +describe('cosineSimilarity', () => { + it('returns 1 for identical unit vectors', () => { + const v = [1, 0, 0]; + expect(cosineSimilarity(v, v)).toBeCloseTo(1); + }); + + it('returns 0 for orthogonal vectors', () => { + expect(cosineSimilarity([1, 0], [0, 1])).toBeCloseTo(0); + }); + + it('returns -1 for opposite vectors', () => { + expect(cosineSimilarity([1, 0], [-1, 0])).toBeCloseTo(-1); + }); + + it('returns 0 for empty vectors', () => { + expect(cosineSimilarity([], [])).toBe(0); + }); + + it('returns 0 for mismatched lengths', () => { + expect(cosineSimilarity([1, 2], [1, 2, 3])).toBe(0); + }); + + it('returns 0 for zero vectors', () => { + expect(cosineSimilarity([0, 0, 0], [1, 2, 3])).toBe(0); + }); + + it('computes correct similarity for non-unit vectors', () => { + // [3, 4] and [4, 3]: dot=24, |a|=5, |b|=5 => 24/25 = 0.96 + expect(cosineSimilarity([3, 4], [4, 3])).toBeCloseTo(0.96); + }); +}); + +describe('searchChunks', () => { + const chunks = [ + makeChunk('a', [1, 0, 0]), + makeChunk('b', [0, 1, 0]), + makeChunk('c', [0.9, 0.1, 0]), + makeChunk('d', [0.5, 0.5, 0]), + ]; + + it('returns empty array for empty chunks', () => { + expect(searchChunks([1, 0, 0], [], 5)).toEqual([]); + }); + + it('returns empty array for topK <= 0', () => { + expect(searchChunks([1, 0, 0], chunks, 0)).toEqual([]); + expect(searchChunks([1, 0, 0], chunks, -1)).toEqual([]); + }); + + it('returns top-K chunks sorted by similarity descending', () => { + const result = searchChunks([1, 0, 0], chunks, 2); + expect(result).toHaveLength(2); + expect(result[0].id).toBe('a'); // exact match + expect(result[1].id).toBe('c'); // close to [1,0,0] + }); + + it('returns all chunks when topK exceeds chunk count', () => { + const result = searchChunks([1, 0, 0], chunks, 100); + expect(result).toHaveLength(4); + expect(result[0].id).toBe('a'); + }); + + it('ranks by similarity correctly for different query', () => { + const result = searchChunks([0, 1, 0], chunks, 2); + expect(result[0].id).toBe('b'); // exact match to [0,1,0] + }); +}); diff --git a/app/src/features/librarian/__tests__/view-state-store.test.ts b/app/src/features/librarian/__tests__/view-state-store.test.ts new file mode 100644 index 00000000..f27963aa --- /dev/null +++ b/app/src/features/librarian/__tests__/view-state-store.test.ts @@ -0,0 +1,56 @@ +import { describe, it, expect, beforeEach } from 'vitest'; +import { useViewStateStore } from '@/lib/view-state-store'; + +describe('view-state-store - librarian', () => { + beforeEach(() => { + // Reset the store state between tests + useViewStateStore.setState({ librarianOpen: false }); + }); + + it('defaults librarianOpen to false', () => { + expect(useViewStateStore.getState().librarianOpen).toBe(false); + }); + + it('toggleLibrarian toggles the value', () => { + const { toggleLibrarian } = useViewStateStore.getState(); + + toggleLibrarian(); + expect(useViewStateStore.getState().librarianOpen).toBe(true); + + toggleLibrarian(); + expect(useViewStateStore.getState().librarianOpen).toBe(false); + }); + + it('setLibrarianOpen sets the value directly', () => { + const { setLibrarianOpen } = useViewStateStore.getState(); + + setLibrarianOpen(true); + expect(useViewStateStore.getState().librarianOpen).toBe(true); + + setLibrarianOpen(false); + expect(useViewStateStore.getState().librarianOpen).toBe(false); + }); + + it('librarianOpen is included in persisted state', () => { + useViewStateStore.getState().setLibrarianOpen(true); + + const persisted = localStorage.getItem('flowscope-view-states'); + expect(persisted).not.toBeNull(); + const parsed = JSON.parse(persisted as string); + expect(parsed.state).toHaveProperty('librarianOpen', true); + + useViewStateStore.getState().setLibrarianOpen(false); + const persistedAfter = JSON.parse(localStorage.getItem('flowscope-view-states') as string); + expect(persistedAfter.state).toHaveProperty('librarianOpen', false); + }); + + it('toggleLibrarian does not affect other store state', () => { + const { setActiveTab, toggleLibrarian } = useViewStateStore.getState(); + setActiveTab('test-project', 'hierarchy'); + + toggleLibrarian(); + + expect(useViewStateStore.getState().librarianOpen).toBe(true); + expect(useViewStateStore.getState().getActiveTab('test-project')).toBe('hierarchy'); + }); +}); diff --git a/app/src/features/librarian/components/ai-settings-dialog.tsx b/app/src/features/librarian/components/ai-settings-dialog.tsx new file mode 100644 index 00000000..384e0134 --- /dev/null +++ b/app/src/features/librarian/components/ai-settings-dialog.tsx @@ -0,0 +1,231 @@ +import { useCallback, useEffect, useMemo, useState } from 'react'; + +import { Button } from '@/components/ui/button'; +import { + Dialog, + DialogContent, + DialogDescription, + DialogFooter, + DialogHeader, + DialogTitle, +} from '@/components/ui/dialog'; +import { Input } from '@/components/ui/input'; +import { Label } from '@/components/ui/label'; +import { + Select, + SelectContent, + SelectItem, + SelectTrigger, + SelectValue, +} from '@/components/ui/select'; + +import { + type AIConfig, + type AIProvider, + getDefaultModel, + loadAIConfig, + saveAIConfig, + sendChatMessage, +} from '../services/ai-service'; +import { DEFAULT_LIBRARIAN_SYSTEM_PROMPT, getPromptStats } from '../services/context-builder'; +import { useLibrarianStore } from '../store'; + +interface AISettingsDialogProps { + open: boolean; + onOpenChange: (open: boolean) => void; +} + +export function AISettingsDialog({ open, onOpenChange }: AISettingsDialogProps) { + const [provider, setProvider] = useState('openai'); + const [apiKey, setApiKey] = useState(''); + const [model, setModel] = useState(''); + const [apiEndpoint, setApiEndpoint] = useState(''); + const [systemPrompt, setSystemPrompt] = useState(DEFAULT_LIBRARIAN_SYSTEM_PROMPT); + const [testing, setTesting] = useState(false); + const [testResult, setTestResult] = useState<{ ok: boolean; message: string } | null>(null); + + useEffect(() => { + if (open) { + const config = loadAIConfig(); + if (config) { + setProvider(config.provider); + setApiKey(config.apiKey); + setModel(config.model); + setApiEndpoint(config.apiEndpoint ?? ''); + setSystemPrompt(config.systemPrompt ?? DEFAULT_LIBRARIAN_SYSTEM_PROMPT); + } else { + setProvider('openai'); + setApiKey(''); + setModel(getDefaultModel('openai')); + setApiEndpoint(''); + setSystemPrompt(DEFAULT_LIBRARIAN_SYSTEM_PROMPT); + } + setTestResult(null); + } + }, [open]); + + const handleProviderChange = useCallback((value: string) => { + const newProvider = value as AIProvider; + setProvider(newProvider); + if (newProvider === 'custom') { + setModel(''); + setApiEndpoint(''); + } else { + setModel(getDefaultModel(newProvider)); + } + }, []); + + const refreshConfig = useLibrarianStore((s) => s.refreshConfig); + + const handleSave = useCallback(() => { + const config: AIConfig = { + provider, + apiKey, + model: provider === 'custom' ? model : model || getDefaultModel(provider), + ...(provider === 'custom' && apiEndpoint ? { apiEndpoint } : {}), + systemPrompt, + }; + saveAIConfig(config); + refreshConfig(); + onOpenChange(false); + }, [provider, apiKey, model, apiEndpoint, systemPrompt, onOpenChange, refreshConfig]); + + const handleTestConnection = useCallback(async () => { + setTesting(true); + setTestResult(null); + try { + const config: AIConfig = { + provider, + apiKey, + model: provider === 'custom' ? model : model || getDefaultModel(provider), + ...(provider === 'custom' && apiEndpoint ? { apiEndpoint } : {}), + }; + await sendChatMessage(config, 'You are a test.', 'Say "ok" and nothing else.'); + setTestResult({ ok: true, message: 'Connection successful' }); + } catch (err) { + setTestResult({ + ok: false, + message: err instanceof Error ? err.message : 'Connection failed', + }); + } finally { + setTesting(false); + } + }, [provider, apiKey, model, apiEndpoint]); + + const promptStats = useMemo(() => getPromptStats(systemPrompt), [systemPrompt]); + + const canSave = + apiKey.trim().length > 0 && + (provider !== 'custom' || (apiEndpoint.trim().length > 0 && model.trim().length > 0)); + + return ( + + + + AI Settings + Configure your AI provider for the Librarian. + + +
+
+ + +
+ +
+ + setApiKey(e.target.value)} + placeholder={ + provider === 'openai' + ? 'sk-...' + : provider === 'anthropic' + ? 'sk-ant-...' + : 'API key' + } + /> +
+ + {provider === 'custom' && ( +
+ + setApiEndpoint(e.target.value)} + placeholder="https://your-server.com" + /> +
+ )} + +
+ + setModel(e.target.value)} + placeholder={provider === 'custom' ? 'model-name' : getDefaultModel(provider)} + /> +
+ +
+
+ + + {promptStats.characters.toLocaleString()} chars /{' '} + {promptStats.bytes.toLocaleString()} bytes + +
+