Skip to content

Commit 9f6aee6

Browse files
Copilotlmangani
Copilot
and
lmangani
committed
Add comprehensive Copilot custom instructions
Co-authored-by: lmangani <1423657+lmangani@users.noreply.github.com>
1 parent 5d70793 commit 9f6aee6

1 file changed

Lines changed: 393 additions & 0 deletions

File tree

‎.github/copilot-instructions.md‎

Lines changed: 393 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,393 @@
1+
# GitHub Copilot Custom Instructions for AceForge
2+
3+
## Project Overview
4+
5+
**AceForge** is a local-first AI music workstation for macOS Silicon that provides a comprehensive interface for AI-powered music generation, training, and audio processing. The project integrates multiple AI models including ACE-Step (text-to-music), Demucs (stem separation), XTTS v2 (voice cloning), and basic-pitch (MIDI generation).
6+
7+
### Key Characteristics
8+
- **Platform**: macOS-first with Apple Silicon (M1/M2/M3/M4) optimization
9+
- **Architecture**: Flask backend + React/TypeScript frontend
10+
- **AI/ML Focus**: Heavy use of PyTorch with MPS (Metal Performance Shaders) acceleration
11+
- **Deployment**: Both source-based and PyInstaller-bundled macOS app
12+
- **Privacy**: 100% local processing - no cloud dependencies after model downloads
13+
14+
## Technology Stack
15+
16+
### Backend (Python)
17+
- **Framework**: Flask with Blueprint-based architecture
18+
- **ML/AI**: PyTorch (with MPS support), Transformers, ACE-Step, Demucs
19+
- **Audio Processing**: pydub, audioread, soundfile, librosa
20+
- **Dependencies**: See `requirements_ace_macos.txt`
21+
- **Python Version**: 3.10+
22+
23+
### Frontend (TypeScript/React)
24+
- **Framework**: React 19 with TypeScript
25+
- **Build Tool**: Vite
26+
- **UI Library**: Custom components with Lucide icons
27+
- **State Management**: React hooks and context
28+
- **Package Manager**: Bun (preferred) or npm
29+
30+
### Build & Distribution
31+
- **Bundling**: PyInstaller for macOS app bundles
32+
- **Code Signing**: Ad-hoc signing with optional Developer ID support
33+
- **CI/CD**: GitHub Actions for automated builds and tests
34+
35+
## Code Style and Conventions
36+
37+
### Python Backend
38+
39+
#### File Organization
40+
- Prefix module names with `cdmf_` for core functionality (e.g., `cdmf_generation.py`, `cdmf_training.py`)
41+
- Use Blueprint pattern for Flask routes (e.g., `create_generation_blueprint()`)
42+
- Keep business logic separate from route handlers
43+
44+
#### Type Hints
45+
- Use `from __future__ import annotations` for forward references
46+
- Prefer typed function signatures:
47+
```python
48+
def create_blueprint(
49+
html_template: str,
50+
ui_defaults: Dict[str, Any],
51+
callback: Callable[..., Dict[str, Any]],
52+
) -> Blueprint:
53+
```
54+
55+
#### Error Handling
56+
- Use try-except blocks with specific exception types
57+
- Log errors with context using `print(..., flush=True)` for immediate console output
58+
- Return structured error responses in API endpoints:
59+
```python
60+
return jsonify({"error": "Description", "details": str(e)}), 500
61+
```
62+
63+
#### Documentation
64+
- Use docstrings for functions explaining purpose, parameters, and return values
65+
- Add inline comments for complex logic, especially ML/audio processing steps
66+
- Document environment variables and configuration options
67+
68+
### TypeScript/React Frontend
69+
70+
#### Component Structure
71+
- Use functional components with hooks
72+
- Prefer named exports for components
73+
- Co-locate types when component-specific, otherwise use `types.ts`
74+
75+
#### Type Safety
76+
- Define interfaces for props and state
77+
- Use discriminated unions for view states
78+
- Avoid `any` type - use `unknown` when type is truly dynamic
79+
80+
#### State Management
81+
- Use React Context for global state (auth, theme, responsive)
82+
- Keep component state local when possible
83+
- Use refs for imperative DOM access and intervals
84+
85+
#### API Calls
86+
- Centralize API calls in `services/api.ts`
87+
- Use async/await with proper error handling
88+
- Show user-friendly error messages in UI
89+
90+
## Architecture Patterns
91+
92+
### Backend Architecture
93+
94+
#### Blueprint Pattern
95+
Each major feature has its own Blueprint:
96+
- `cdmf_generation.py` - Music generation
97+
- `cdmf_training.py` - LoRA training
98+
- `cdmf_stem_splitting.py` - Audio stem separation
99+
- `cdmf_voice_cloning.py` - TTS voice cloning
100+
- `cdmf_midi_generation.py` - Audio-to-MIDI transcription
101+
102+
#### State Management
103+
- Global state in `cdmf_state.py` for model loading and training status
104+
- Track library management in `cdmf_tracks.py`
105+
- Path handling centralized in `cdmf_paths.py`
106+
107+
#### Model Loading
108+
- Lazy loading: Models loaded on first use, not at startup
109+
- Singleton pattern: One instance per model type
110+
- MPS optimization: Automatic device selection (`mps`, `cuda`, `cpu`)
111+
112+
### Frontend Architecture
113+
114+
#### Component Hierarchy
115+
```
116+
App.tsx (main container)
117+
├── Sidebar (navigation)
118+
├── CreatePanel (music generation UI)
119+
├── LibraryView (track browser)
120+
├── TrainingPanel (LoRA training UI)
121+
├── StemSplittingPanel (Demucs UI)
122+
├── VoiceCloningPanel (XTTS UI)
123+
├── MidiPanel (basic-pitch UI)
124+
├── Player (audio playback)
125+
└── RightSidebar (queue/info)
126+
```
127+
128+
#### View Management
129+
- Single `currentView` state determines which panel is visible
130+
- Views: `'create'`, `'library'`, `'training'`, `'stem-splitting'`, `'voice-cloning'`, `'midi'`
131+
- Responsive design with mobile/desktop layouts
132+
133+
## AI/ML Specific Guidelines
134+
135+
### Model Integration
136+
- **Check for bundled execution**: Set `TORCH_JIT=0` to disable JIT in frozen apps
137+
- **MPS optimization**: Use `torch.device("mps")` for Apple Silicon
138+
- **Memory management**: Implement cleanup with `torch.mps.empty_cache()`
139+
- **Model paths**: Use `cdmf_paths.py` functions for consistent path resolution
140+
141+
### Audio Processing
142+
- **Format handling**: Support MP3, WAV, FLAC via pydub/librosa
143+
- **Normalization**: Apply consistent audio normalization across pipelines
144+
- **Sample rates**: Be aware of model-specific sample rate requirements (e.g., ACE-Step uses 24kHz)
145+
- **Stem separation**: Use audio-separator or Demucs depending on use case
146+
147+
### Training
148+
- **LoRA training**: Focus on lightweight adapter training, not full model fine-tuning
149+
- **Dataset structure**: Require `_prompt.txt` and `_lyrics.txt` files alongside audio
150+
- **Checkpointing**: Save periodic checkpoints to prevent data loss
151+
- **PyTorch Lightning**: Use Lightning for training orchestration
152+
153+
## Testing Guidelines
154+
155+
### Backend Testing
156+
- Test files in `tests/` directory
157+
- Use pytest for test execution
158+
- Test API endpoints with sample requests
159+
- Mock heavy ML operations for unit tests
160+
- Integration tests should use small, fast models when possible
161+
162+
### Shell Script Tests
163+
- Prefix test scripts with `test_` (e.g., `test_generation_simple.sh`)
164+
- Test bundled app functionality separately (`test_bundled_app.sh`)
165+
- Include cleanup in tests (remove generated files)
166+
167+
### CI/CD Testing
168+
- GitHub Actions workflows in `.github/workflows/`
169+
- Test on macOS runners only (Apple Silicon specific)
170+
- Workflows:
171+
- `build-release.yml` - Build and release macOS app
172+
- `macos-tests.yml` - Run test suite
173+
- `test-ace-generation.yml` - Test music generation
174+
- `installation-test.yml` - Verify installation process
175+
176+
## Security Considerations
177+
178+
### API Security
179+
- Validate user inputs before processing
180+
- Sanitize file paths using `secure_filename()` or path validation
181+
- Limit file upload sizes for audio/model files
182+
- Validate audio file formats before processing
183+
184+
### Model Security
185+
- Verify model checksums when downloading (if implemented)
186+
- Sandbox model execution when possible
187+
- Limit resource usage (memory, disk) for user operations
188+
189+
### Local-First Privacy
190+
- No telemetry or analytics
191+
- No external API calls except for model downloads
192+
- All processing happens locally
193+
- Clear documentation of data handling
194+
195+
## Performance Optimization
196+
197+
### Apple Silicon (MPS) Optimization
198+
- Use MPS backend for PyTorch when available
199+
- Set `PYTORCH_MPS_HIGH_WATERMARK_RATIO=0.0` to prevent OOM
200+
- Monitor unified memory usage
201+
- Implement progressive processing for long audio files
202+
203+
### Memory Management
204+
- Clear GPU/unified memory after heavy operations
205+
- Use streaming for large audio files when possible
206+
- Implement max clip duration limits in training
207+
- Provide user controls for memory-intensive operations
208+
209+
### UI Performance
210+
- Lazy load components and data
211+
- Debounce API calls for search/filter operations
212+
- Use virtual scrolling for large track lists
213+
- Optimize audio player with proper buffering
214+
215+
## Platform-Specific Guidelines
216+
217+
### macOS Specific
218+
- **MPS Support**: Leverage Metal Performance Shaders for GPU acceleration
219+
- **Code Signing**: Support both ad-hoc and Developer ID signing
220+
- **App Bundle**: Handle PyInstaller frozen app considerations
221+
- **Gatekeeper**: Document workarounds for non-notarized apps
222+
- **File Paths**: Use `pathlib.Path` for cross-platform compatibility
223+
224+
### Frozen App Considerations
225+
- **Resource Access**: Use `sys._MEIPASS` for bundled resources
226+
- **TorchScript**: Disable JIT compilation (`TORCH_JIT=0`)
227+
- **TTS Models**: Handle TOS agreements programmatically
228+
- **Module Inspection**: Avoid `inspect.getsource()` in frozen apps
229+
230+
## Documentation Standards
231+
232+
### Code Comments
233+
- Explain **why**, not **what** (code should be self-documenting for "what")
234+
- Document ML/audio processing algorithms with references
235+
- Add TODO comments with issue numbers when applicable
236+
- Mark experimental features clearly
237+
238+
### User Documentation
239+
- **README.md**: High-level overview and quick start
240+
- **USAGE.md**: Detailed user guide with examples
241+
- **UIDEV.md**: UI development guide
242+
- **Build docs**: In `build/macos/README.md`
243+
244+
### API Documentation
245+
- Document Flask endpoints with expected inputs/outputs
246+
- Include example requests and responses
247+
- Document error codes and messages
248+
- Keep API versioning in mind for future compatibility
249+
250+
## Common Pitfalls to Avoid
251+
252+
### Backend
253+
- ❌ Don't load all models at startup (lazy loading only)
254+
- ❌ Don't process very long audio without chunking
255+
- ❌ Don't assume `mps` backend is always available
256+
- ❌ Don't hard-code paths (use `cdmf_paths.py`)
257+
- ❌ Don't use blocking operations in Flask routes (use background threads)
258+
259+
### Frontend
260+
- ❌ Don't store large data in component state
261+
- ❌ Don't make API calls in render functions
262+
- ❌ Don't forget to clean up intervals/timeouts
263+
- ❌ Don't hardcode backend URL (use relative paths or config)
264+
- ❌ Don't ignore mobile/responsive design
265+
266+
### General
267+
- ❌ Don't commit model files to git (use .gitignore)
268+
- ❌ Don't include secrets or API keys
269+
- ❌ Don't ignore cross-platform path handling
270+
- ❌ Don't skip error handling in ML/audio pipelines
271+
272+
## Contribution Workflow
273+
274+
When suggesting code changes:
275+
276+
1. **Understand context**: Review related files and recent changes
277+
2. **Minimal changes**: Prefer small, focused modifications
278+
3. **Test implications**: Consider what tests might be affected
279+
4. **Documentation**: Update docs if changing user-facing behavior
280+
5. **Platform compatibility**: Ensure changes work on macOS with Apple Silicon
281+
6. **Performance**: Consider memory and compute implications for ML operations
282+
283+
## Review Guidelines
284+
285+
When reviewing code:
286+
287+
1. **Functionality**: Does it solve the intended problem?
288+
2. **Performance**: Is it efficient for the target hardware (Apple Silicon)?
289+
3. **Memory**: Does it handle large models/audio files safely?
290+
4. **Error handling**: Are edge cases handled gracefully?
291+
5. **Testing**: Can this be tested? Are tests included?
292+
6. **Documentation**: Is it clear how to use this feature?
293+
7. **Security**: Are inputs validated and sanitized?
294+
8. **Maintainability**: Is the code readable and well-structured?
295+
296+
## Example Code Patterns
297+
298+
### Backend: Creating a new feature blueprint
299+
300+
```python
301+
from __future__ import annotations
302+
from typing import Dict, Any
303+
from flask import Blueprint, request, jsonify
304+
305+
def create_feature_blueprint() -> Blueprint:
306+
"""Create blueprint for new feature."""
307+
bp = Blueprint("feature_name", __name__)
308+
309+
@bp.route("/api/feature", methods=["POST"])
310+
def handle_feature():
311+
try:
312+
# Validate input
313+
data = request.get_json()
314+
if not data:
315+
return jsonify({"error": "No data provided"}), 400
316+
317+
# Process
318+
result = process_feature(data)
319+
320+
# Return success
321+
return jsonify({"status": "success", "result": result}), 200
322+
323+
except Exception as e:
324+
print(f"[Feature] Error: {e}", flush=True)
325+
return jsonify({"error": str(e)}), 500
326+
327+
return bp
328+
```
329+
330+
### Frontend: Creating a new panel component
331+
332+
```typescript
333+
import React, { useState, useEffect } from 'react';
334+
import { featureApi } from '../services/api';
335+
336+
interface FeaturePanelProps {
337+
onComplete?: (result: any) => void;
338+
}
339+
340+
export function FeaturePanel({ onComplete }: FeaturePanelProps) {
341+
const [isProcessing, setIsProcessing] = useState(false);
342+
const [error, setError] = useState<string | null>(null);
343+
344+
const handleSubmit = async () => {
345+
setIsProcessing(true);
346+
setError(null);
347+
348+
try {
349+
const result = await featureApi.process({
350+
// ... parameters
351+
});
352+
onComplete?.(result);
353+
} catch (err) {
354+
setError(err instanceof Error ? err.message : 'Unknown error');
355+
} finally {
356+
setIsProcessing(false);
357+
}
358+
};
359+
360+
return (
361+
<div className="feature-panel">
362+
{/* UI elements */}
363+
{error && <div className="error">{error}</div>}
364+
</div>
365+
);
366+
}
367+
```
368+
369+
## Quick Reference
370+
371+
### Important Files
372+
- `music_forge_ui.py` - Main Flask application entry point
373+
- `aceforge_app.py` - PyInstaller bundled app launcher
374+
- `cdmf_paths.py` - Path management and configuration
375+
- `cdmf_state.py` - Global state for models and training
376+
- `ui/App.tsx` - Main React application
377+
- `ui/services/api.ts` - API client functions
378+
379+
### Key Commands
380+
- `./run_local.sh` - Run from source (development)
381+
- `./build_local.sh` - Build macOS app bundle
382+
- `./build_and_test_local.sh` - Build and run tests
383+
- `cd ui && bun run build` - Build frontend only
384+
385+
### Environment Variables
386+
- `TORCH_JIT=0` - Disable JIT (required for frozen apps)
387+
- `PYTORCH_MPS_HIGH_WATERMARK_RATIO=0.0` - MPS memory management
388+
- `COQUI_TOS_AGREED=1` - Skip TTS TOS prompt
389+
- `HF_HUB_DISABLE_SYMLINKS=1` - Disable symlinks for models
390+
391+
---
392+
393+
**Remember**: AceForge is designed for musicians and audio professionals who value local processing and privacy. Keep the user experience smooth, the code maintainable, and the documentation clear.

0 commit comments

Comments
 (0)