Skip to content

Commit 7de31ec

Browse files
Alexandre Oliveiraclaude
Alexandre Oliveira
and
claude
committed
docs(turing): pre-render Mermaid diagrams, add Integration page, replace Tab with Section
Mermaid pre-rendering: - Install @mermaid-js/mermaid-cli as dev dependency - Extract 7 large diagrams (14+ lines) to diagrams/*.mmd source files - Add npm run diagrams script; prebuild hook runs it automatically before build - Replace mermaid code blocks with static SVG image references in: architecture-overview, sn-concepts, ai-agents, genai-llm, chat - Generate placeholder SVGs in static/img/diagrams/ for dev mode - Add puppeteer-config.json with --no-sandbox flags for CI - Convert remaining graph LR (core-concepts) to graph TB for vertical layout - Add CSS for pre-rendered SVGs: centered, max-width 900px, dark mode invert New page — Integration: - Document the full Integration screen (/admin/integration/instance) - Covers: Settings, Sources (AEM config with Delta Tracking, Locales, Author/Publish), Indexing Rules, Indexing Manager (4 operations + Recursive), Monitoring (11 statuses, auto-refresh), Indexing Stats, Double Check (Missing/Extra), System Information - Architecture section: proxy model + SSRF protection - Add integration to Enterprise Search category in sidebars-turing.ts Terminology cleanup: - Replace Tab with Section across admin docs (semantic-navigation, administration-guide) - Update chat.md: Tab 1/2/3 headings removed, prose updated to sections/views - Fix broken anchor genai-llm -> semantic-navigation#generative-ai Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
1 parent de6e7c4 commit 7de31ec

28 files changed

Lines changed: 2763 additions & 236 deletions

‎diagrams/turing-agent-flow.mmd‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
%%{init: {'theme': 'base', 'themeVariables': {'fontSize': '14px'}}}%%
2+
sequenceDiagram
3+
participant User
4+
participant Turing as Turing ES
5+
participant LLM as LLM Instance
6+
participant Tool as Tool / MCP Server
7+
8+
User->>Turing: Send message
9+
Turing->>LLM: User message + system prompt + tool definitions
10+
loop Reasoning chain
11+
LLM->>Turing: Request tool call (tool name + arguments)
12+
Turing->>Tool: Execute tool
13+
Tool-->>Turing: Tool result
14+
Turing->>LLM: Tool result
15+
end
16+
LLM-->>Turing: Final response
17+
Turing-->>User: Streamed response (SSE)

‎diagrams/turing-architecture.mmd‎

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,44 @@
1+
%%{init: {'theme': 'base', 'themeVariables': {'fontSize': '14px'}}}%%
2+
graph TB
3+
subgraph DumontDEP ["Viglet Dumont DEP (External)"]
4+
CON["Connectors\nWebCrawler · Database · FileSystem\nAEM/WEM · WordPress"]
5+
end
6+
7+
subgraph TuringApp ["Turing ES Application"]
8+
API[REST & GraphQL API]
9+
Q[Apache Artemis\nMessage Queue]
10+
IDX[Indexing Pipeline\n+ Merge Provider]
11+
SN[Semantic Navigation\nSearch Engine]
12+
GENAI[GenAI / RAG Engine\n+ Tool Calling + AI Agents]
13+
ADMIN[Admin Console]
14+
end
15+
16+
subgraph Backends ["Backends & Storage"]
17+
SOLR[Apache Solr + Zookeeper]
18+
EMB[Embedding Stores\nChromaDB · PgVector · Milvus]
19+
DB[(Database\nMariaDB / MySQL / PostgreSQL)]
20+
MONGO[MongoDB\nApplication Logs]
21+
MINIO[MinIO\nAsset Store]
22+
end
23+
24+
LLM["LLM Providers\nClaude · OpenAI · Azure OpenAI\nGemini · Ollama"]
25+
KC[Keycloak\nOAuth2 / OIDC]
26+
CLIENTS["Clients\nJS SDK · Java SDK · Custom Apps"]
27+
28+
CON -->|POST via REST API| API
29+
API -->|Enqueue indexing job| Q
30+
Q --> IDX
31+
IDX --> SOLR
32+
IDX --> EMB
33+
IDX --> DB
34+
MINIO -->|RAG assets| IDX
35+
SN --> SOLR
36+
GENAI --> EMB
37+
GENAI --> LLM
38+
API --> SN
39+
API --> GENAI
40+
ADMIN --> API
41+
ADMIN -->|File/folder UI| MINIO
42+
IDX -.->|Application logs| MONGO
43+
KC -->|OAuth2 / OIDC| API
44+
CLIENTS --> API

‎diagrams/turing-chat-layout.mmd‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
%%{init: {'theme': 'base', 'themeVariables': {'fontSize': '14px'}}}%%
2+
graph TB
3+
A[Header] --> B[Tab navigation — Chat · Semantic Nav · AI Agents]
4+
A --> C[LLM model selector]
5+
A --> D[New Chat button]
6+
A --> E[Dark mode toggle]
7+
A --> F[Session history button]
8+
G[Message area] --> H[User messages with avatar]
9+
G --> I[Assistant messages with avatar + copy button]
10+
G --> J[Rich content — Markdown, code, D2 diagrams, HTML sandbox]
11+
K[Input area] --> L[Multiline textarea]
12+
K --> M[File drag-and-drop]
13+
K --> N[Send button — Enter to submit]
14+
O[Context Bar] --> P[Token counter]
15+
O --> Q[Context window % bar]
16+
O --> R[Compact button]

‎diagrams/turing-indexing-flow.mmd‎

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
%%{init: {'theme': 'base', 'themeVariables': {'fontSize': '13px'}}}%%
2+
sequenceDiagram
3+
participant Dumont as Viglet Dumont DEP<br/>(Connector)
4+
participant API as Turing ES REST API
5+
participant Artemis as Apache Artemis Queue
6+
participant Pipeline as Indexing Pipeline
7+
participant SNSite as SN Site Configuration
8+
participant Merge as Merge Provider Engine
9+
participant Solr as Apache Solr
10+
participant DB as Database
11+
participant Emb as Embedding Store<br/>(Chroma / PgVector / Milvus)
12+
13+
Dumont->>API: POST document to SN Site
14+
API->>SNSite: Load site configuration<br/>(fields, facets, search behavior, spotlights)
15+
SNSite-->>API: Site config
16+
API->>Artemis: Enqueue indexing job (async)
17+
Artemis->>Pipeline: Deliver job
18+
19+
Pipeline->>Merge: Check Merge Provider rules<br/>for this site and document
20+
alt Merge rule matched
21+
Merge->>Solr: Fetch existing document by join key
22+
Solr-->>Merge: Existing document fields
23+
Merge-->>Pipeline: Merged document<br/>(overwrite configured fields)
24+
else No merge rule
25+
Merge-->>Pipeline: Document unchanged
26+
end
27+
28+
Pipeline->>Solr: Index document with mapped fields and facets
29+
alt Document matches a Spotlight
30+
Pipeline->>DB: Persist spotlight content<br/>(title, description, URL, position)
31+
end
32+
Pipeline->>Emb: Store vector embedding<br/>(if GenAI enabled for this site)

‎diagrams/turing-rag-flow.mmd‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
%%{init: {'theme': 'base', 'themeVariables': {'fontSize': '14px'}}}%%
2+
sequenceDiagram
3+
participant User
4+
participant Chat as Chat / AI Agent
5+
participant LLM as LLM Instance
6+
participant Emb as Embedding Store
7+
participant RAGTool as RAG Tool Calling
8+
participant SN as Semantic Navigation
9+
10+
User->>Chat: Question or request
11+
Chat->>LLM: Send message with context
12+
LLM->>RAGTool: Call search_knowledge_base(query)
13+
RAGTool->>Emb: Vector similarity search<br/>(top-10, threshold 0.7)
14+
Emb-->>RAGTool: Relevant document chunks
15+
RAGTool-->>LLM: Retrieved context
16+
LLM->>SN: Call search_site(site, query) [optional]
17+
SN-->>LLM: Search results
18+
LLM-->>Chat: Generated response grounded in retrieved content
19+
Chat-->>User: Final response

‎diagrams/turing-search-flow.mmd‎

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
%%{init: {'theme': 'base', 'themeVariables': {'fontSize': '13px'}}}%%
2+
sequenceDiagram
3+
participant Client
4+
participant API as REST API
5+
participant SNProcess as TurSNSearchProcess
6+
participant Plugin as Search Engine Plugin
7+
participant Solr as Apache Solr
8+
participant Spotlight as Spotlight Engine
9+
participant TR as Targeting Rules Engine
10+
participant Metrics as Metrics Logger
11+
12+
Client->>API: GET /api/sn/{siteName}/search?q=...&profile attributes
13+
API->>SNProcess: search(context)
14+
15+
SNProcess->>SNProcess: Validate site configuration
16+
SNProcess->>TR: Translate profile attributes<br/>into Solr filter queries
17+
TR-->>SNProcess: Additional filter queries
18+
19+
SNProcess->>Plugin: Select plugin (Solr / ES / Lucene)
20+
Plugin->>Solr: Execute query<br/>+ facets + targeting filter queries
21+
Solr-->>Plugin: Filtered result set + facet counts
22+
Plugin-->>SNProcess: Structured results
23+
24+
SNProcess->>SNProcess: Enrich facets<br/>Map fields<br/>Apply highlighting
25+
26+
SNProcess->>Spotlight: Check for spotlight term match
27+
Spotlight-->>SNProcess: Inject curated documents<br/>at configured positions in response
28+
29+
SNProcess->>Metrics: Log query + result count (async)
30+
31+
SNProcess-->>API: Assembled response<br/>(results + facets + spotlights + locales + spellCheck + similar)
32+
API-->>Client: JSON response

‎diagrams/turing-targeting-rules-flow.mmd‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
%%{init: {'theme': 'base', 'themeVariables': {'fontSize': '14px'}}}%%
2+
sequenceDiagram
3+
participant Client
4+
participant API as TurSNSearchProcess
5+
participant Builder as TurSolrQueryBuilder
6+
participant Rules as TurSNTargetingRules
7+
participant Solr
8+
9+
Client->>API: POST /api/sn/{siteName}/search<br/>{ targetingRules / targetingRulesWithConditionAND / OR }
10+
API->>Builder: Format values (add quotes if needed)
11+
Builder->>Rules: Build filter query from rule type
12+
Rules-->>Builder: Solr fq clause (AND or OR logic)
13+
Builder->>Solr: Main query + fq (targeting filter)
14+
Solr-->>Client: Filtered results
15+
API->>API: Record applied rules in sn_site_metric_access_trs

‎docs-turing/administration-guide.md‎

Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -94,9 +94,9 @@ Lists all local user accounts in the system.
9494
| Email | User's email address |
9595
| Password | Account password |
9696

97-
**Tabs:**
97+
**Sections:**
9898

99-
| Tab | Description |
99+
| Section | Description |
100100
|---|---|
101101
| **Groups** | Add or remove the user from groups |
102102
| **Roles** | Displays roles inherited from the user's groups (read-only) |
@@ -114,9 +114,9 @@ Organises users into groups for role-based access control.
114114
| Name | Group name |
115115
| Description | Purpose or scope of the group |
116116

117-
**Tabs:**
117+
**Sections:**
118118

119-
| Tab | Description |
119+
| Section | Description |
120120
|---|---|
121121
| **Users** | Add or remove members of this group |
122122
| **Roles** | Assign or remove roles granted to this group |
@@ -215,9 +215,9 @@ Used by Turing ES to send notifications and test email connectivity.
215215

216216
#### System Information
217217

218-
A diagnostic panel to monitor the health of the Turing ES instance. Divided into two tabs:
218+
A diagnostic panel to monitor the health of the Turing ES instance. Divided into two sections:
219219

220-
##### Overview Tab
220+
##### Overview
221221

222222
| Item | Description |
223223
|---|---|
@@ -231,7 +231,7 @@ A diagnostic panel to monitor the health of the Turing ES instance. Divided into
231231
| MongoDB Status | Connected / disconnected (shown only if MongoDB is enabled) |
232232
| MinIO Status | Connected / disconnected (shown only if MinIO is enabled) |
233233

234-
##### System Variables Tab
234+
##### System Variables
235235

236236
A searchable table of all JVM properties and environment variables active at runtime. Useful for verifying configuration at deployment.
237237

‎docs-turing/ai-agents.md‎

Lines changed: 1 addition & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -67,24 +67,7 @@ A local agent for internal IT queries — runs fully on-premise using a local LL
6767

6868
When a user sends a message to an AI Agent, the following loop runs:
6969

70-
```mermaid
71-
sequenceDiagram
72-
participant User
73-
participant Turing as Turing ES
74-
participant LLM as LLM Instance
75-
participant Tool as Tool / MCP Server
76-
77-
User->>Turing: Send message
78-
Turing->>LLM: User message + system prompt + tool definitions
79-
loop Reasoning chain
80-
LLM->>Turing: Request tool call (tool name + arguments)
81-
Turing->>Tool: Execute tool
82-
Tool-->>Turing: Tool result
83-
Turing->>LLM: Tool result
84-
end
85-
LLM-->>Turing: Final response
86-
Turing-->>User: Streamed response (SSE)
87-
```
70+
![AI Agent — Execution Flow](/img/diagrams/turing-agent-flow.svg)
8871

8972
1. Turing ES sends the user message to the LLM along with a system prompt describing the agent and the definitions of all enabled tools.
9073
2. The LLM decides which tools (if any) to call, based on the message content and tool descriptions.

‎docs-turing/architecture-overview.md‎

Lines changed: 3 additions & 111 deletions
Original file line numberDiff line numberDiff line change
@@ -18,51 +18,7 @@ This document describes the system's components, internal modules, and the two c
1818

1919
## High-Level Component Diagram
2020

21-
```mermaid
22-
graph TB
23-
subgraph DumontDEP ["Viglet Dumont DEP (External)"]
24-
CON["Connectors\nWebCrawler · Database · FileSystem\nAEM/WEM · WordPress"]
25-
end
26-
27-
subgraph TuringApp ["Turing ES Application"]
28-
API[REST & GraphQL API]
29-
Q[Apache Artemis\nMessage Queue]
30-
IDX[Indexing Pipeline\n+ Merge Provider]
31-
SN[Semantic Navigation\nSearch Engine]
32-
GENAI[GenAI / RAG Engine\n+ Tool Calling + AI Agents]
33-
ADMIN[Admin Console]
34-
end
35-
36-
subgraph Backends ["Backends & Storage"]
37-
SOLR[Apache Solr + Zookeeper]
38-
EMB[Embedding Stores\nChromaDB · PgVector · Milvus]
39-
DB[(Database\nMariaDB / MySQL / PostgreSQL)]
40-
MONGO[MongoDB\nApplication Logs]
41-
MINIO[MinIO\nAsset Store]
42-
end
43-
44-
LLM["LLM Providers\nClaude · OpenAI · Azure OpenAI\nGemini · Ollama"]
45-
KC[Keycloak\nOAuth2 / OIDC]
46-
CLIENTS["Clients\nJS SDK · Java SDK · Custom Apps"]
47-
48-
CON -->|POST via REST API| API
49-
API -->|Enqueue indexing job| Q
50-
Q --> IDX
51-
IDX --> SOLR
52-
IDX --> EMB
53-
IDX --> DB
54-
MINIO -->|RAG assets| IDX
55-
SN --> SOLR
56-
GENAI --> EMB
57-
GENAI --> LLM
58-
API --> SN
59-
API --> GENAI
60-
ADMIN --> API
61-
ADMIN -->|File/folder UI| MINIO
62-
IDX -.->|Application logs| MONGO
63-
KC -->|OAuth2 / OIDC| API
64-
CLIENTS --> API
65-
```
21+
![Turing ES — High-Level Architecture](/img/diagrams/turing-architecture.svg)
6622

6723
---
6824

@@ -94,39 +50,7 @@ Content ingestion is handled externally by **Viglet Dumont DEP**. Each connector
9450

9551
The **Semantic Navigation Site** is the central configuration artifact that drives the entire indexing behavior: it defines which Solr instance to use, which fields the documents carry, how those fields are mapped and used (title, text, URL, date, image, facets, etc.), how search will behave, and which spotlights are configured. The indexing pipeline reads this configuration to know exactly what to do with each incoming document.
9652

97-
```mermaid
98-
sequenceDiagram
99-
participant Dumont as Viglet Dumont DEP\n(Connector)
100-
participant API as Turing ES REST API
101-
participant Artemis as Apache Artemis Queue
102-
participant Pipeline as Indexing Pipeline
103-
participant SNSite as SN Site Configuration
104-
participant Merge as Merge Provider Engine
105-
participant Solr as Apache Solr
106-
participant DB as Database
107-
participant Emb as Embedding Store\n(Chroma / PgVector / Milvus)
108-
109-
Dumont->>API: POST document to SN Site
110-
API->>SNSite: Load site configuration\n(fields, facets, search behavior, spotlights)
111-
SNSite-->>API: Site config
112-
API->>Artemis: Enqueue indexing job (async)
113-
Artemis->>Pipeline: Deliver job
114-
115-
Pipeline->>Merge: Check Merge Provider rules\nfor this site and document
116-
alt Merge rule matched
117-
Merge->>Solr: Fetch existing document by join key
118-
Solr-->>Merge: Existing document fields
119-
Merge-->>Pipeline: Merged document\n(overwrite configured fields)
120-
else No merge rule
121-
Merge-->>Pipeline: Document unchanged
122-
end
123-
124-
Pipeline->>Solr: Index document with mapped fields and facets
125-
alt Document matches a Spotlight
126-
Pipeline->>DB: Persist spotlight content\n(title, description, URL, position)
127-
end
128-
Pipeline->>Emb: Store vector embedding\n(if GenAI enabled for this site)
129-
```
53+
![Turing ES — Indexing Flow](/img/diagrams/turing-indexing-flow.svg)
13054

13155
### Key indexing concepts
13256

@@ -152,39 +76,7 @@ sequenceDiagram
15276

15377
The search flow is synchronous and request-driven. Every request goes through a structured pipeline inside `TurSNSearchProcess` before a response is returned to the client.
15478

155-
```mermaid
156-
sequenceDiagram
157-
participant Client
158-
participant API as REST API
159-
participant SNProcess as TurSNSearchProcess
160-
participant Plugin as Search Engine Plugin
161-
participant Solr as Apache Solr
162-
participant Spotlight as Spotlight Engine
163-
participant TR as Targeting Rules Engine
164-
participant Metrics as Metrics Logger
165-
166-
Client->>API: GET /api/sn/{siteName}/search?q=...&profile attributes
167-
API->>SNProcess: search(context)
168-
169-
SNProcess->>SNProcess: Validate site configuration
170-
SNProcess->>TR: Translate profile attributes\ninto Solr filter queries
171-
TR-->>SNProcess: Additional filter queries
172-
173-
SNProcess->>Plugin: Select plugin (Solr / ES / Lucene)
174-
Plugin->>Solr: Execute query\n+ facets + targeting filter queries
175-
Solr-->>Plugin: Filtered result set + facet counts
176-
Plugin-->>SNProcess: Structured results
177-
178-
SNProcess->>SNProcess: Enrich facets\nMap fields\nApply highlighting
179-
180-
SNProcess->>Spotlight: Check for spotlight term match
181-
Spotlight-->>SNProcess: Inject curated documents\nat configured positions in response
182-
183-
SNProcess->>Metrics: Log query + result count (async)
184-
185-
SNProcess-->>API: Assembled response\n(results + facets + spotlights + locales + spellCheck + similar)
186-
API-->>Client: JSON response
187-
```
79+
![Turing ES — Search Flow](/img/diagrams/turing-search-flow.svg)
18880

18981
### Key search concepts
19082

0 commit comments

Comments
 (0)