Methodology: How Our Forecasting Works
Deep dive into our machine learning methodology, model architecture, and validation framework that powers our S&P 500 predictions
Machine Learning Explained
What is AI-powered forecasting?
Our system employs a multi-horizon ensemble approach using CatBoost gradient boosting with sophisticated feature engineering and rigorous validation methodologies. The system operates on three distinct prediction horizons (3M, 6M, 12M) to capture different market dynamics and provide actionable signals for various investment strategies.
Unlike traditional investment strategies that rely on fixed rules or human judgment, our machine learning system adapts to changing market conditions through quarterly retraining. We analyze over 50 engineered indicators derived from market data, macroeconomic fundamentals, and valuation metrics simultaneously—far more than any human could track—to identify opportunities and risks across multiple time horizons.
The system doesn't "predict the future" in a magical sense. Instead, it calculates probabilities based on historical patterns using purged time-series cross-validation with embargo periods. When market conditions look similar to past scenarios that led to positive returns, the models flag these as opportunities. When conditions resemble periods before market declines, they signal caution.
Built on robust foundations
Our forecasting system is designed with institutional-grade rigor used by professional hedge funds and quantitative investment firms. We combine cutting-edge CatBoost gradient boosting with time-tested investment principles and rigorous validation methodologies.
Every component—from data quality assurance to feature engineering to model training to signal generation—follows strict validation procedures including purged time-series cross-validation with embargo periods and regime-aware sample weighting. This ensures our predictions are based on genuine patterns, not statistical flukes or overfitting to past data.
- Machine Learning Models
- 3
- Engineered Features
- 50+
- Years of Historical Data
- 50+
- Prediction Horizons
- 3
The Architecture
From data to decisions
Our forecasting pipeline operates in three distinct stages, each designed to extract maximum insight from market data while avoiding common pitfalls.
- Data Collection
- We gather market prices, economic indicators, and valuation metrics from trusted financial data providers including Yahoo Finance, FRED, and Multpl.com, updated daily.
- Feature Engineering
- Raw data is transformed into meaningful patterns using statistical transformations, momentum indicators, and market regime classification across 50+ engineered features.
- AI Prediction
- Three specialized CatBoost gradient boosting models analyze patterns to forecast returns over 3, 6, and 12-month horizons using purged time-series cross-validation.
Market Data
Prices, volatility, valuations
Feature Engineering
50+ engineered indicators
AI Models
3, 6, 12-month forecasts
Trading Signal
LONG or CASH
Data Architecture & Processing
Our forecasting system relies on a comprehensive data architecture that combines market data, economic indicators, and valuation metrics. The system processes data from Yahoo Finance, FRED (Federal Reserve Economic Data), and Multpl.com to create a robust foundation for machine learning models. Each data category serves a specific purpose in understanding market dynamics across different economic cycles.
Market Data
The foundation of our analysis consists of real-time market data that captures the immediate sentiment and movements of financial markets. This includes price action, volatility measures, and trading volume data that reflect investor behavior and market stress levels.
- • S&P 500 (^GSPC) daily price data with dividend adjustments for accurate returns calculation
- • VIX (^VIX) volatility index measuring market fear and uncertainty levels
- • Historical price data spanning over 50 years (from 1970) for robust pattern recognition
- • Volume and realized volatility metrics for market stress and liquidity analysis
Economic Indicators (FRED)
Macroeconomic data from the Federal Reserve provides the fundamental backdrop against which market movements occur. These indicators capture the health of the economy, monetary policy conditions, and broader economic trends that influence long-term market direction.
- • 10-Year Real Treasury Rate reflecting real cost of money and economic conditions
- • M2 Money Supply growth tracking liquidity conditions in the economy
- • GDP for overall economic health assessment
- • Yield curves (10Y-2Y, 10Y-3M) with inversion indicators for recession forecasting
- • BAA credit spreads measuring corporate bond risk premiums
- • Federal Funds Rate for monetary policy stance
- • Initial Jobless Claims for labor market health
Valuation Metrics (Multpl.com)
Valuation indicators help determine whether markets are trading at reasonable levels relative to historical norms. These metrics provide crucial context for understanding whether current market levels are sustainable or potentially overextended.
- • S&P 500 Price-to-Earnings ratio for current valuation context
- • Shiller CAPE (Cyclically-Adjusted P/E) for long-term valuation assessment
- • S&P 500 Dividend Yield for income-focused valuation perspectives
- • Buffett Indicator (Market Cap / GDP) for broad market valuation assessment
Data Quality & Processing Pipeline
All data undergoes rigorous processing and quality assurance procedures to ensure reliability and consistency across our models:
- • Daily frequency standardization to align all data sources on a consistent timeline
- • Forward-fill procedures for lower-frequency data (monthly economic indicators, weekly claims data)
- • Missing value threshold filtering (rows with >50% missing values excluded)
- • Automated data validation checks and cross-reference validation against alternative sources
- • Expanding window statistical calculations to prevent look-ahead bias
Feature Engineering Framework
Raw market data—like prices or interest rates—doesn't tell the full story. We engineer over 50 specialized indicators that capture market momentum, valuation extremes, and economic relationships through sophisticated statistical transformations.
Statistical Transformations
We apply rigorous statistical methods to normalize data and make it comparable across different time periods and market conditions. All transformations use expanding windows to avoid look-ahead bias.
# Z-Score Normalization (Expanding Window)
z_score = (value - expanding_mean) / expanding_std
# Percentile Ranking (Expanding Window)
percentile = expanding_rank(value) / total_observations
Applied to returns, volatility, valuation ratios, yield curves, credit spreads, and economic indicators to create normalized features that work across different market regimes.
1Momentum Features
Tracking whether markets are trending up, down, or sideways across multiple timeframes:
- • Rolling sum returns: 21d, 63d, 126d
- • RSI (Relative Strength Index): 21d, 63d, 126d periods
- • Valuation momentum: 252d (12M) and 504d (24M)
- • Mean reversion strength via linear regression slopes (63d, 126d)
2Overbought / Oversold Features
Tactical mean-reversion signals detecting when markets have moved too far, too fast:
- • Drawdown from all-time high (key buy-the-dip signal)
- • Drawdown Z-score (normalized depth vs. historical distribution)
- • Distance from MA50 and MA200 (short/long-term overbought gauges)
- • Bollinger Band %B (20d MA +/- 2 std deviations)
3Macroeconomic & Monetary Policy Features
Advanced economic indicators capturing policy, growth, and labor market dynamics:
- • Yield curve slopes and inversion indicators (10Y-2Y, 10Y-3M)
- • M2 Money Supply growth rates (annualized)
- • BAA credit spread Z-scores and rate of change (63d, 126d)
- • Federal Funds Rate Z-score and change indicators
- • Initial Jobless Claims Z-score, 4-week MA, and YoY change
4Volatility & Composite Features
Sophisticated indicators combining multiple data sources for market context:
- • VIX level, change, moving averages, and deviation from MA
- • VIX Z-score and realized volatility Z-score (expanding)
- • VIX minus realized volatility spread (implied vs. actual fear)
- • Buffett Indicator (Market Cap / GDP) Z-score
- • Market regime classification (Oversold / Neutral / Overbought)
Machine Learning Architecture
CatBoost Implementation
We use CatBoost, an advanced gradient boosting algorithm developed by Yandex, specifically chosen for its superior handling of financial data. CatBoost excels at managing categorical features, preventing overfitting through ordered boosting, and providing robust performance with mixed data types.
The algorithm works by building a series of decision trees, where each subsequent tree corrects the errors of the previous ones. This ensemble approach allows the model to capture complex, non-linear relationships in financial data that would be impossible to model with traditional statistical methods. The "ordered boosting" feature is particularly valuable for time series data, as it prevents the model from using future information to predict past events.
Our configuration uses a conservative learning rate of 0.015, which means each new tree makes only small adjustments to the overall prediction. This slow, steady approach helps prevent overfitting and creates more stable, reliable predictions. The Bayesian bootstrap method adds randomness to the training process, further improving the model's generalization ability.
# CatBoost Common Configuration
catboost_common = {
'learning_rate': 0.015,
'colsample_bylevel': 0.9,
'bootstrap_type': 'Bayesian',
'bagging_temperature': 1.0,
'od_type': 'Iter', # Overfitting detector
'od_wait': 30,
'eval_metric': 'AUC',
'loss_function': 'Logloss',
'auto_class_weights': 'Balanced',
'border_count': 32
}
# Horizon-Specific Parameters
horizon_configs = {
# 3M: shallow trees — short horizon is noisy, avoid overfitting
'3M': {
'iterations': 800,
'depth': 3,
'l2_leaf_reg': 80,
'min_data_in_leaf': 80
},
# 6M & 12M: deeper trees — longer horizons have clearer patterns
'6M': {
'iterations': 800,
'depth': 5,
'l2_leaf_reg': 50,
'min_data_in_leaf': 80
},
'12M': {
'iterations': 800,
'depth': 5,
'l2_leaf_reg': 50,
'min_data_in_leaf': 80
}
}
3M Horizon
Depth: 3 (shallow)
800 iterations, L2=80 — conservative to handle noisy short-term signals
6M Horizon
Depth: 5 (deeper)
800 iterations, L2=50 — captures clearer medium-term patterns
12M Horizon
Depth: 5 (deeper)
800 iterations, L2=50 — leverages strong long-term fundamental signals
Regime-Aware Sample Weighting
Not all historical data is equally relevant for predicting the future. Our sample weighting system combines two components to ensure the model learns from the most relevant historical periods while never forgetting the lessons of major market crises.
# Regime-Aware Temporal Weighting
sample_weighting = {
# Component 1: Exponential decay
'half_life_years': 10.0, # Recent data weighted higher
# Component 2: Crisis-period floor
'crisis_floor': 0.8, # Crashes never downweighted
'crisis_periods': [
['2000-03', '2003-03'], # Dot-com crash
['2007-10', '2009-06'], # Global Financial Crisis
['2020-02', '2020-06'], # COVID crash
['2022-01', '2022-10'] # 2022 bear market
]
}
Temporal Decay
Exponential decay with a 10-year half-life ensures recent market dynamics are weighted more heavily, while still leveraging decades of historical patterns.
Crisis Floor Protection
Major market crises (dot-com, GFC, COVID, 2022) maintain a minimum weight of 0.8, ensuring the model never forgets how to recognize dangerous market conditions.
Two-Stage Feature Selection
Feature selection is crucial because including too many irrelevant features can hurt model performance, while too few might miss important patterns. We employ a sophisticated two-stage selection process using permutation importance and cross-fold stability analysis:
# Two-Stage Feature Selection
# Stage 1: Permutation importance across all CV folds
for fold in cv_folds:
importance = permutation_importance(model, X_val, y_val)
selected_features[fold] = top_features(importance)
# Stage 2: Cross-fold stability check
stable_features = features_selected_in(
folds >= stability_threshold # >= 60% of folds
).head(max_features) # Max 12 per horizon
Stability Threshold:60% of CV folds
Max Features Per Horizon:12
Selection Method:Permutation Importance
This ensures features are not only predictive but also stable across different time periods. A feature that appears important in only one fold is likely an artifact; only features consistently chosen across at least 60% of cross-validation folds are retained for the final model.
Target Variable Construction
Our binary classification framework creates targets based on the sign of forward log returns (direction-only). The threshold is 0.0%, meaning LONG when the forward return is positive:
# Binary Classification Framework
def create_targets(df, horizon_days, signal_threshold=0.0):
# Forward log return
future_returns = log(df['GSPC'].shift(-horizon_days) / df['GSPC'])
targets = (future_returns > signal_threshold).astype(int)
return targets
Purged Time-Series Cross-Validation
To ensure our models will work in real markets (not just on historical data), we use strict purged walk-forward validation with embargo periods:
- Purged CV: Overlapping samples between train and test sets are removed to prevent leakage
- 5-fold time-series splits: Each fold uses ~63 trading days (~3 months) for validation
- 10-day embargo: A gap between training and test data prevents information bleed at fold boundaries
- Early stopping: Training halts after 30 rounds without improvement to prevent overfitting
# Purged Time-Series CV Configuration
cv_config = {
'n_splits': 5,
'test_size_days': 63, # ~3 months validation window
'embargo_days': 10 # Gap between train/test
}
Signal Generation & Portfolio Construction
Multi-Model Consensus Mechanism
Rather than relying on a single model, we use a "wisdom of the crowd" approach. Three independent models—each trained on different time horizons—vote on market direction. The consensus threshold requires at least 2 out of 3 models to agree for a LONG signal, creating a natural filter against false positives.
# Multi-Model Voting System
def generate_consensus_signal(model_predictions):
long_votes = sum(1 for prob in model_predictions.values()
if prob > 0.5)
position_sizes = {0: 0.0, 1: 0.0, 2: 0.67, 3: 1.0}
return {
'consensus_signal': 1 if long_votes >= 2 else 0,
'position_size': position_sizes[long_votes]
}
3/3 models bullish:100% invested (STRONG LONG)
2/3 models bullish:67% invested (MODERATE LONG)
1/3 models bullish:0% invested (CASH)
0/3 models bullish:0% invested (CASH)
This consensus mechanism reduces the impact of any single model being wrong and naturally adjusts position size based on conviction level, creating dynamic risk management through conviction-based allocation.
Risk Management Framework
Our portfolio construction follows institutional-grade risk management principles with comprehensive metrics and systematic position sizing:
Portfolio Construction Rules
- • Maximum position size: 100% equity exposure
- • Minimum position size: 0% (cash position)
- • No leverage or short selling
- • Consensus threshold: 2/3 models required for LONG
Risk Metrics
- • Maximum Drawdown: Peak-to-trough decline
- • Sharpe Ratio: Risk-adjusted returns
- • Sortino Ratio: Downside deviation-adjusted
- • Win Rate: Percentage of profitable trades
Daily Signal Process
Every trading day, the system follows a structured workflow to generate actionable signals:
- 1. Data Collection: Latest market data collected from Yahoo Finance, FRED, and Multpl.com
- 2. Feature Engineering: All 50+ indicators computed from fresh data using statistical transformations
- 3. Model Predictions: Each horizon model outputs probability scores for positive returns
- 4. Signal Conversion: Probabilities above 50% become LONG signals
- 5. Consensus Aggregation: Multi-model voting determines overall signal and position size
- 6. Database Update: Predictions and signals pushed to Supabase for frontend display
- 7. Execution Timing: Signals delayed by one day for realistic trading
Validation Framework & Model Quality
Rigorous validation is the cornerstone of reliable machine learning models. Our validation framework ensures that our models will perform well in real-world trading conditions, not just on historical data. We use multiple validation techniques and track comprehensive metrics.
Model Performance Metrics
We track multiple metrics to evaluate model performance, each providing different insights into how well our models work. These metrics are stored per model, per horizon, and are updated with every quarterly retraining cycle.
# Comprehensive Performance Metrics
metrics = {
'auc': roc_auc_score(y_true, y_proba),
'accuracy': accuracy_score(y_true, y_pred),
'precision': precision_score(y_true, y_pred),
'recall': recall_score(y_true, y_pred),
'f1': f1_score(y_true, y_pred),
'log_loss': log_loss(y_true, y_proba),
'brier_score': brier_score_loss(y_true, y_proba),
'pr_auc': average_precision_score(y_true, y_proba)
}
Stored Model Artifacts
Every training run stores comprehensive artifacts in our database for full transparency and reproducibility:
Feature Importances
Top 20 features per horizon model stored with their importance scores, enabling analysis of which market signals drive predictions.
Confusion Matrices
Full classification matrices per model showing true positives, false positives, true negatives, and false negatives.
CV Fold Results
Train and validation AUC scores for every cross-validation fold, enabling overfitting detection and model stability assessment.
Feature Importance Analysis
Understanding which features are most important helps us interpret our models and ensure they're making decisions based on sensible market signals. Our analysis reveals that different time horizons rely on different types of information:
3M Model (Short-term)
Tends to rely on momentum, overbought/oversold conditions, and recent volatility — capturing short-term mean reversion and market stress.
6M Model (Medium-term)
Balances momentum with valuation metrics and yield curve signals — bridging tactical and fundamental perspectives.
12M Model (Long-term)
Emphasizes valuation, monetary policy, and macroeconomic fundamentals — capturing structural drivers of long-term returns.
Feature importances are dynamically determined during each quarterly retraining cycle. The specific top features may shift as market regimes evolve, which is a feature of the system, not a bug — it ensures the model adapts to changing market dynamics.
Risk Considerations & Limitations
While our system employs institutional-grade methodologies and rigorous validation, it's crucial to understand both the capabilities and limitations of any forecasting system:
Model Risk Factors
Overfitting Prevention
- • Purged cross-validation with strict temporal ordering
- • Two-stage feature selection with stability requirements
- • Regime-aware sample weighting preserves crisis lessons
- • Conservative hyperparameters (high regularization, shallow trees)
- • Ensemble approach reduces single-model risk
Data Risk Management
- • Multiple data source validation (Yahoo, FRED, Multpl)
- • Forward-fill for lower-frequency data with proper alignment
- • Missing value threshold filtering (>50% threshold)
- • Expanding window calculations prevent look-ahead bias
Known Limitations
Predictive Limitations: Cannot predict black swan events or unprecedented market conditions. Performance degrades during regime changes and is limited to S&P 500 index (not individual stocks).
Technical Limitations: Dependency on data quality and timeliness from third-party providers (Yahoo Finance, FRED, Multpl.com). Model performance varies by market regime.
Market Regime Risk: Models trained across multiple market cycles (1970-present) with quarterly retraining to adapt to changing conditions. Regime-aware sample weighting helps, but structural market changes may still impact performance.
Regulatory Considerations: Not registered as investment advice. Performance claims based on historical backtesting. No guarantee of future results. Users responsible for their own investment decisions.
System Architecture & Maintenance
Our system includes comprehensive monitoring and maintenance procedures:
- • Quarterly Retraining: Models retrained every 3 months with latest data and feature selection
- • Daily Predictions: Automated via GitHub Actions using persisted model artifacts
- • Model Persistence: Trained models stored as GitHub Actions artifacts for reliable daily inference
- • Performance Monitoring: CV fold results and classification metrics stored per training run
- • Version Control: Full model metadata, features, and hyperparameters tracked with each version
Technical Methodology Disclosure: This comprehensive methodology documentation is provided for institutional investors and quantitative analysts. While our models use advanced CatBoost machine learning techniques, purged time-series cross-validation with embargo, and rigorous validation methods, no forecasting system can predict the future with certainty.
Past performance does not guarantee future results. This methodology is provided for transparency and educational purposes only. Always consult with a qualified financial advisor before making investment decisions. The system serves as a foundation for systematic investment strategies while maintaining rigorous risk management standards expected by institutional investors.