REGIME TERMINAL

Machine Learning Training Status

Our trading infrastructure has 3 ML systems. Here's where each one stands:

1. HMM Regime Terminal — OPERATIONAL

Status: FULLY TRAINED — This is the only system that can train immediately because it learns from price data (17,000+ observations), not trade outcomes.

• Scans 36 markets every 30 minutes
• Classifies each into 1 of 7 regimes
• Uses EM algorithm with 5 random restarts
• Feeds regime data to KIMI and Alpha Engine via Regime Bridge
• Last scan: -

2. Alpha Engine Random Forest — TRAINING IN PROGRESS

Status: HEURISTIC MODE — Needs 50 closed trade outcomes to begin experimental training. Currently using rule-based scoring (this is the professionally correct cold-start approach).

• 8 closed picks so far (need 50 for experimental, 150+ for reliable ML)
• 20 active picks being tracked
• Turbo Mode: ACTIVE — accelerating pick closure
• 18 features (will reduce to 5-6 for first training to prevent overfitting)
• Est. 50 picks (~10-15 days) → experimental | 150 picks (~2-3 months) → useful | 300 picks (~5 months) → production
• Runs every 15 minutes via GitHub Actions

3. KIMI Random Forest — WAITING FOR DATA

Status: HEURISTIC MODE — Zero closed picks. The KIMI tournament system generates picks but doesn't yet close and record them for ML training.

• 81 algorithms compete in a tournament bracket
• Uses heuristic scoring until enough data accumulates
• Now receiving HMM regime data via the bridge
• Runs every 15 minutes via GitHub Actions

Realistic ML Readiness Timeline

Based on academic research (Lopez de Prado 2018, BMC Medical Informatics 2021) and current pick accumulation rates. This is an honest assessment — not hype.

Why so long? Random Forest with 14-18 features needs at minimum 10-20 observations per feature (140-360 picks) to avoid overfitting. Our 50-pick threshold is for experimental training only — not production-quality predictions. The HMM bypasses this entirely by learning from price data (17,000+ observations), which is why it's operational now.

Automated Scan Schedule

What Is a Hidden Markov Model (HMM)?

Imagine the stock market has different "moods" — sometimes it's excited and prices go up (bull), sometimes it's scared and prices crash (bear), sometimes it's confused and goes sideways (neutral). These moods are hidden — you can't directly see them. You can only see the results: price changes, volume, and volatility.

An HMM is an AI that works backwards from the results to figure out what mood the market is probably in. It's like being a detective: you can't see the criminal, but you can see the evidence and deduce what happened.

How Does It Learn?

The learning process has 3 steps:

• Step 1 — Gather Evidence: For each asset, we download the last year of daily data: price returns, volume changes, volatility, and momentum. These are the "clues."
• Step 2 — Train the Model (EM Algorithm): The AI assumes there are 7 hidden states. It starts with random guesses, then repeatedly: (a) estimates which state each day was probably in, and (b) adjusts its Gaussian parameters to better fit the data. After ~200 rounds, it converges.
• Step 3 — Classify Today: Using the trained model, it calculates the posterior probability of being in each of the 7 states right now.

The 5 Features (Clues the AI Uses)

• Log Return: Today's price change (e.g., +1.2%)
• Volatility: How much the price swings (high minus low)
• Volume Change: Is trading activity above or below normal?
• 5-Day Momentum: Short-term trend direction
• 20-Day Momentum: Medium-term trend direction

The 7 Hidden States (Regimes)

After training, the AI sorts the 7 states by their average return:

• Crash — Extreme negative returns, very high volatility. Rare but dangerous.
• Strong Bear — Significantly negative returns, elevated volatility.
• Mild Bear — Slightly negative returns, moderate volatility.
• Chop/Neutral — Near-zero returns, medium volatility. Market going sideways.
• Accumulation — Slightly positive returns but with specific volume patterns suggesting buildup.
• Mild Bull — Moderately positive returns, lower volatility.
• Strong Bull — Strongly positive returns, trending momentum.

8-Point Confirmation System

Even when the HMM says "this is a bull market," we don't blindly trust it. We require at least 7 of 8 technical indicators to agree before generating a signal:

• RSI — Not overbought/oversold
• MACD — Trend direction confirms
• EMA Cross — Short-term above long-term
• Volume — Above-average trading activity
• Momentum — Price direction matches regime
• Bollinger Bands — Position within bands
• ATR — Volatility appropriate for regime
• Trend Alignment — 50d and 200d moving averages agree

Why Is This Better Than Traditional Approaches?

Most trading algorithms use fixed rules like "if RSI < 30, buy." The problem? Markets change. A rule that works in a calm market fails in a crash.

The HMM adapts because it first figures out what type of market we're in, then applies the right strategy. It's like a doctor who first diagnoses the disease before prescribing medicine, instead of giving everyone the same pill.

This approach was pioneered by Renaissance Technologies, the most successful hedge fund in history (66% annual returns for 30 years).

Pipeline Health Dashboard

Real-time status of all automated systems. Green = healthy, orange = needs attention, red = broken.

Recent Pipeline Log