Skip to content

Commit 0b046c6

Browse files
shantanu patilclaude
authored and
shantanu patil
committed
feat: Phase 2-3 completion — hook extraction, diagram overhaul, Mermaid fixes, and cleanup
Phase 2 (Core Polish): - Extract page.tsx (2,623→844 lines) into 4 hooks: useWikiGeneration, useRepoStructure, useWikiCache, useWikiExport - Migrate hardcoded colors in DependencyGraph to Tailwind theme classes - Add Playwright E2E tests (landing, wiki-viewer, routing) - Add Vitest unit tests for utility functions Phase 3 Batch 1 (Differentiators): - Embed widget route (/embed/[owner]/[repo]/[pageId]) with iframe support - Per-page regeneration via POST /api/wiki/regenerate_page - Wiki template system (comprehensive, architecture, api-docs, onboarding, security) Cleanup: - Remove slides and workshop features entirely - Simplify navigation to Wiki + Explorer tabs only - Fix wiki generation failure (empty provider → fetch defaultProvider) - Add shared layout with tab navigation Diagram improvements: - Fix Mermaid unicode arrow rendering (—→, →, ⟶ etc. from LLM output) - Add Mermaid error fallback with source display and retry button - Fix explorer dark mode: rgba() node backgrounds, darkStroke edges, MutationObserver-based theme detection - Smoothstep edge routing for cleaner architectural connections - Connected-node highlighting (dim unrelated nodes on selection) - File count badges and description tooltips on explorer nodes - Improved layout spacing across all three explorer views Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
1 parent e7fc86e commit 0b046c6

56 files changed

Lines changed: 11104 additions & 5153 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎api/api.py‎

Lines changed: 497 additions & 9 deletions
Large diffs are not rendered by default.

‎api/config/wiki_templates.json‎

Lines changed: 55 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,55 @@
1+
{
2+
"templates": {
3+
"comprehensive": {
4+
"id": "comprehensive",
5+
"name": "Comprehensive",
6+
"description": "Full wiki with architecture, data flow, API docs, and diagrams",
7+
"icon": "BookOpen",
8+
"prompt_guidance": "",
9+
"structure_hint": "Create a structured wiki with sections covering: Overview, System Architecture, Core Features, Data Management/Flow, Frontend Components, Backend Systems, Deployment/Infrastructure, and Extensibility.",
10+
"page_count": "8-12",
11+
"focus_areas": []
12+
},
13+
"architecture": {
14+
"id": "architecture",
15+
"name": "Architecture Guide",
16+
"description": "System design, component relationships, design patterns, and trade-offs",
17+
"icon": "Layers",
18+
"prompt_guidance": "Focus on system architecture, component relationships, design patterns, and technical trade-offs. Prioritize diagrams showing data flow and component interactions. Emphasize how modules connect, what protocols they use, and the reasoning behind architectural decisions.",
19+
"structure_hint": "Create a wiki focused on architecture with sections: High-Level Architecture, Component Breakdown, Design Patterns Used, Data Flow & Communication, Dependency Graph, and Trade-offs & Decisions.",
20+
"page_count": "6-10",
21+
"focus_areas": ["architecture", "design patterns", "system design", "component relationships", "data flow"]
22+
},
23+
"api-docs": {
24+
"id": "api-docs",
25+
"name": "API Documentation",
26+
"description": "API endpoints, request/response formats, authentication, and error handling",
27+
"icon": "Code",
28+
"prompt_guidance": "Focus on API documentation: endpoints, request/response schemas, authentication mechanisms, error codes, and rate limiting. Document every public interface, including REST endpoints, WebSocket protocols, GraphQL schemas, and RPC methods. Include example requests and responses.",
29+
"structure_hint": "Create a wiki focused on API documentation with sections: API Overview & Base URLs, Authentication & Authorization, REST Endpoints (grouped by resource), WebSocket/Real-time APIs, Error Handling & Status Codes, and SDK/Client Usage.",
30+
"page_count": "6-10",
31+
"focus_areas": ["api", "endpoints", "routes", "handlers", "controllers", "middleware", "schemas", "serializers"]
32+
},
33+
"onboarding": {
34+
"id": "onboarding",
35+
"name": "Onboarding Guide",
36+
"description": "Getting started, dev environment setup, key concepts, and contribution guide",
37+
"icon": "Rocket",
38+
"prompt_guidance": "Write for a developer joining this project for the first time. Focus on getting started quickly, development environment setup, key concepts they need to understand, common development tasks, and how to contribute. Use a friendly, instructional tone. Include step-by-step instructions where possible.",
39+
"structure_hint": "Create a wiki for new developers with sections: Quick Start & Prerequisites, Development Environment Setup, Project Structure Walkthrough, Key Concepts & Terminology, Common Development Tasks, Testing & Debugging, and Contribution Guide.",
40+
"page_count": "5-8",
41+
"focus_areas": ["readme", "setup", "configuration", "package.json", "docker", "makefile", "contributing", "tests"]
42+
},
43+
"security": {
44+
"id": "security",
45+
"name": "Security Review",
46+
"description": "Threat model, auth flows, data handling, dependency risks, and OWASP considerations",
47+
"icon": "Shield",
48+
"prompt_guidance": "Analyze this codebase from a security perspective. Cover authentication flows, authorization patterns, data handling and sanitization, input validation, dependency risks, secrets management, and potential vulnerabilities. Reference OWASP Top 10 where applicable. Flag areas that need security attention.",
49+
"structure_hint": "Create a security-focused wiki with sections: Security Overview & Threat Model, Authentication & Session Management, Authorization & Access Control, Input Validation & Data Sanitization, Secrets & Configuration Management, Dependency Security, and Security Recommendations.",
50+
"page_count": "5-8",
51+
"focus_areas": ["auth", "security", "middleware", "validation", "sanitize", "crypto", "password", "token", "session", "cors", "csrf"]
52+
}
53+
},
54+
"default_template": "comprehensive"
55+
}

‎api/openrouter_client.py‎

Lines changed: 23 additions & 135 deletions
Original file line numberDiff line numberDiff line change
@@ -173,144 +173,32 @@ async def content_generator():
173173
content = choice["message"]["content"]
174174
log.info("Successfully retrieved response")
175175

176-
# Check if the content is XML and ensure it's properly formatted
177-
if content.strip().startswith("<") and ">" in content:
178-
# It's likely XML, let's make sure it's properly formatted
176+
# Check if content looks like a wiki_structure response
177+
# (XML or JSON). Use the robust parser to normalize
178+
# it and re-emit clean XML for the frontend.
179+
if "<wiki_structure>" in content or (
180+
'"pages"' in content and '"title"' in content
181+
):
179182
try:
180-
# Extract the XML content
181-
xml_content = content
182-
183-
# Check if it's a wiki_structure XML
184-
if "<wiki_structure>" in xml_content:
185-
log.info("Found wiki_structure XML, ensuring proper format")
186-
187-
# Extract just the wiki_structure XML
188-
import re
189-
wiki_match = re.search(r'<wiki_structure>[\s\S]*?<\/wiki_structure>', xml_content)
190-
if wiki_match:
191-
# Get the raw XML
192-
raw_xml = wiki_match.group(0)
193-
194-
# Clean the XML by removing any leading/trailing whitespace
195-
# and ensuring it's properly formatted
196-
clean_xml = raw_xml.strip()
197-
198-
# Try to fix common XML issues
199-
try:
200-
# Replace problematic characters in XML
201-
fixed_xml = clean_xml
202-
203-
# Replace & with &amp; if not already part of an entity
204-
fixed_xml = re.sub(r'&(?!amp;|lt;|gt;|apos;|quot;)', '&amp;', fixed_xml)
205-
206-
# Fix other common XML issues
207-
fixed_xml = fixed_xml.replace('</', '</').replace(' >', '>')
208-
209-
# Try to parse the fixed XML
210-
from xml.dom.minidom import parseString
211-
dom = parseString(fixed_xml)
212-
213-
# Get the pretty-printed XML with proper indentation
214-
pretty_xml = dom.toprettyxml()
215-
216-
# Remove XML declaration
217-
if pretty_xml.startswith('<?xml'):
218-
pretty_xml = pretty_xml[pretty_xml.find('?>')+2:].strip()
219-
220-
log.info(f"Extracted and validated XML: {pretty_xml[:100]}...")
221-
yield pretty_xml
222-
except Exception as xml_parse_error:
223-
log.warning(f"XML validation failed: {str(xml_parse_error)}, using raw XML")
224-
225-
# If XML validation fails, try a more aggressive approach
226-
try:
227-
# Use regex to extract just the structure without any problematic characters
228-
import re
229-
230-
# Extract the basic structure
231-
structure_match = re.search(r'<wiki_structure>(.*?)</wiki_structure>', clean_xml, re.DOTALL)
232-
if structure_match:
233-
structure = structure_match.group(1).strip()
234-
235-
# Rebuild a clean XML structure
236-
clean_structure = "<wiki_structure>\n"
237-
238-
# Extract title
239-
title_match = re.search(r'<title>(.*?)</title>', structure, re.DOTALL)
240-
if title_match:
241-
title = title_match.group(1).strip()
242-
clean_structure += f" <title>{title}</title>\n"
243-
244-
# Extract description
245-
desc_match = re.search(r'<description>(.*?)</description>', structure, re.DOTALL)
246-
if desc_match:
247-
desc = desc_match.group(1).strip()
248-
clean_structure += f" <description>{desc}</description>\n"
249-
250-
# Add pages section
251-
clean_structure += " <pages>\n"
252-
253-
# Extract pages
254-
pages = re.findall(r'<page id="(.*?)">(.*?)</page>', structure, re.DOTALL)
255-
for page_id, page_content in pages:
256-
clean_structure += f' <page id="{page_id}">\n'
257-
258-
# Extract page title
259-
page_title_match = re.search(r'<title>(.*?)</title>', page_content, re.DOTALL)
260-
if page_title_match:
261-
page_title = page_title_match.group(1).strip()
262-
clean_structure += f" <title>{page_title}</title>\n"
263-
264-
# Extract page description
265-
page_desc_match = re.search(r'<description>(.*?)</description>', page_content, re.DOTALL)
266-
if page_desc_match:
267-
page_desc = page_desc_match.group(1).strip()
268-
clean_structure += f" <description>{page_desc}</description>\n"
269-
270-
# Extract importance
271-
importance_match = re.search(r'<importance>(.*?)</importance>', page_content, re.DOTALL)
272-
if importance_match:
273-
importance = importance_match.group(1).strip()
274-
clean_structure += f" <importance>{importance}</importance>\n"
275-
276-
# Extract relevant files
277-
clean_structure += " <relevant_files>\n"
278-
file_paths = re.findall(r'<file_path>(.*?)</file_path>', page_content, re.DOTALL)
279-
for file_path in file_paths:
280-
clean_structure += f" <file_path>{file_path.strip()}</file_path>\n"
281-
clean_structure += " </relevant_files>\n"
282-
283-
# Extract related pages
284-
clean_structure += " <related_pages>\n"
285-
related_pages = re.findall(r'<related>(.*?)</related>', page_content, re.DOTALL)
286-
for related in related_pages:
287-
clean_structure += f" <related>{related.strip()}</related>\n"
288-
clean_structure += " </related_pages>\n"
289-
290-
clean_structure += " </page>\n"
291-
292-
clean_structure += " </pages>\n</wiki_structure>"
293-
294-
log.info("Successfully rebuilt clean XML structure")
295-
yield clean_structure
296-
else:
297-
log.warning("Could not extract wiki structure, using raw XML")
298-
yield clean_xml
299-
except Exception as rebuild_error:
300-
log.warning(f"Failed to rebuild XML: {str(rebuild_error)}, using raw XML")
301-
yield clean_xml
302-
else:
303-
# If we can't extract it, just yield the original content
304-
log.warning("Could not extract wiki_structure XML, yielding original content")
305-
yield xml_content
306-
else:
307-
# For other XML content, just yield it as is
308-
yield content
309-
except Exception as xml_error:
310-
log.error(f"Error processing XML content: {str(xml_error)}")
183+
from api.wiki_structure_parser import (
184+
parse_wiki_structure,
185+
convert_json_to_xml,
186+
)
187+
188+
structure = parse_wiki_structure(content)
189+
clean_xml = convert_json_to_xml(structure)
190+
log.info(
191+
f"Parsed and normalized wiki structure: {clean_xml[:100]}..."
192+
)
193+
yield clean_xml
194+
except Exception as parse_error:
195+
log.warning(
196+
f"Wiki structure parsing failed ({parse_error}), "
197+
"yielding original content"
198+
)
311199
yield content
312200
else:
313-
# Not XML, just yield the content
201+
# Not a wiki structure response, yield as-is
314202
yield content
315203
else:
316204
log.error(f"Unexpected response format: {data}")

0 commit comments

Comments
 (0)