Elite Quant Motif Discovery System

Overview

This is a production-grade, GPU-accelerated motif discovery system for stock market data, optimized for discovering profitable "explosive upward moves" across thousands of stocks. It uses advanced pre-filtering to focus only on subsequences that lead to significant gains (+35% to +300%) in the next 30 days, reducing data by 95%+ while ensuring all discovered motifs have proven predictive power.

Detailed Process Workflow

The system operates in a multi-stage pipeline designed for both historical discovery and live signal generation:

1. Data Acquisition & Preparation

• Symbol Selection: Filters for symbols with Market Cap > $300M to ensure liquidity and avoid penny stock noise.
• Price Loading: Fetches daily adjusted close prices for the last 15+ years.
• Log-Scale Transformation: All prices are converted to natural log values to focus on percentage moves rather than absolute dollar changes.

2. Subsequence Extraction ("The Sliding Window")

• Multi-Scale Windowing: The system "unfolds" the price series into subsequences using multiple window sizes (e.g., 30, 45, 60, 75, 90 trading days).
• Geometric Normalization:
  • Detrending: Removes linear trends using polynomial fitting so "flat" base patterns can be matched even if they occur on different slopes.
  • Z-Normalization: Standardizes the mean and standard deviation to match the shape of the move regardless of volatility levels.
  • Resampling: All subsequences (from 30d to 90d) are resampled to a common length of 100 points using linear interpolation for direct comparison.

3. The Golden Filter (Profitability Filtering)

• Forward Returns: For every historical subsequence, the system calculates the future 30-day return.
• Thresholding: Only subsequences that preceded a gain of +35% to +300% are kept as "Candidate Motifs".
• Live Patterns: The most recent patterns (those ending within the last 20 days) are always kept, regardless of future return (which is unknown), to act as live queries against historical winners.

4. GPU-Accelerated Matching

• Massive Euclidean Search: Uses PyTorch to compute the distance between every "Live/Candidate" pattern and the entire historical database of winners.
• Exclusion Zones: Prevents "trivial matches" by masking patterns that overlap in time on the same symbol (at least 50% window separation required).
• Distance Thresholding: Matches are only qualified if their Euclidean distance is < 0.31, ensuring high geometric similarity.

5. Memory & Batching Logic

• Market Chunking: Processes the market in chunks of ~600 stocks (in Validation mode) or ~5000 stocks (in Discovery mode) to prevent System RAM exhaustion.
• VRAM Batching: GPU computations are performed in chunks of ~2,000,000 candidate comparisons at a time to stay within GPU memory limits (typically ~800MB per chunk for 100-len float32).

6. Phase-Based Workflow

• PHASE 1: DISCOVERY: Performs a global search to find every "Winner vs. Winner" pair in history. These are saved to motifs.pkl.
• PHASE 2: VALIDATION: Takes the discovered winners and tests them against the entire market (all stocks, all dates) to calculate the "True Win Rate" and "Average Return" for that specific pattern type.

7. Signal Generation & Reporting

• Qualified Motifs: Filters for patterns with a Win Rate > 50% and Avg Return > 5%.
• Live Breakouts: Scans the most recent 20 days of market data for any stock currently tracing a shape similar to a historical high-win-rate motif.

Key Features

• Pre-Filtering for Profitability: Focuses only on subsequences that lead to explosive gains, eliminating noise.
• Cross-Stock & Cross-Scale Discovery: Finds similar patterns across different stocks and time windows.
• Distributed GPU Acceleration: Uses PyTorch and multi-processing for high-speed pattern matching.
• Scale Invariance: Z-normalized log-price subsequences ensure geometric similarity detection.

Architecture

• Data Source: PostgreSQL with daily stock prices.
• Processing: Python, PyTorch (CUDA), NumPy, Pandas.
• Persistence: explosive_motifs table in PostgreSQL.

Deployment

This project uses automated deployment via GitHub Actions. Simply push changes to the main branch to trigger deployment.

Manual Setup:

python3.10 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

Running the Engine:

# Discovery Mode (Find Winners)
python motif_pattern_recognition.py

# Validation Mode (Calculate Win Rates)
python motif_pattern_recognition.py --validate --motif_file motifs.pkl

# Target Specific Date
python motif_pattern_recognition.py --process_date 2024-01-01

Results Example

On 2,500+ stocks: 1.2M filtered subs, 10k+ motifs, top gains +193% (e.g., ASTS↔INSM).

This system delivers quant-grade motif discovery for real trading signals! 🚀