FinceptTerminal is a modern finance application offering advanced market analytics, investment research, and economic data tools, designed for interactive exploration and data-driven decision-making in a user-friendly environment.
C++ ai-agentsalgorithmic-tradingbloomberg-terminalcppfinancefinancial-marketsfintechgood-first-issueinvestmentinvestment-researchmachine-learningopensourcepythonqtquantitative-financestock-markettrading
python update
tilakpatel22e5030c2482a1
diff --git a/fincept-qt/resources/requirements-numpy2.txt b/fincept-qt/resources/requirements-numpy2.txtindex 34ffb6aa4..efdde1b17 100644--- a/fincept-qt/resources/requirements-numpy2.txt+++ b/fincept-qt/resources/requirements-numpy2.txt@@ -9,7 +9,12 @@ numpy>=2.2.3,<3.0pandas>=2.3.3,<3.0# Trading platform (requires NumPy 2.2+)-vnpy==4.3.0+# Excluded on Linux ARM64: vnpy pulls pyside6==6.8.2.1, which publishes no+# aarch64 Linux wheel. Without the marker the ENTIRE numpy2 set is unresolvable+# there and setup dies at "Install Analytics Libraries", taking every other+# analytics library with it. Linux ARM64 is a source-build target only — CI+# ships Linux x64, macOS arm64 and Windows x64.+vnpy==4.3.0; platform_machine != "aarch64" or sys_platform != "linux"# Data fetchingccxt>=4.5.44,<5.0@@ -127,7 +132,11 @@ fortitudo.tech==1.2quantstats>=0.0.62# Quant libraries-pyqlib==0.9.7+# Excluded on Linux ARM64 for the same reason as vnpy above: pyqlib 0.9.7 ships+# wheels for manylinux_2_17_x86_64 / manylinux2014_x86_64 / macosx_10_9_universal2+# / win_amd64 only, and has no sdist to fall back to. Costs the qlib features+# (scripts/ai_quant_lab/qlib_*.py, scripts/agents/rdagents/cli.py) on that arch.+pyqlib==0.9.7; platform_machine != "aarch64" or sys_platform != "linux"rdagent==0.8.0deepagents==0.4.12gs-quant==1.4.67@@ -169,7 +178,16 @@ pandarallel>=1.6.0gymnasium>=1.0.0# RL libraries-stable-baselines3>=2.7.0+# Excluded on macOS Intel: no SB3 release can satisfy this file there. PyTorch+# dropped x86_64 macOS wheels after 2.2.2 (see the torch pin above), and SB3+# splits cleanly either side of that line — >=2.6.0 requires torch>=2.3, while+# <=2.5.0 requires numpy<2.0 or gymnasium<0.30, both of which this file rules+# out. Without the marker the ENTIRE numpy2 set is unresolvable on+# x86_64-apple-darwin and setup dies at "Install Analytics Libraries", taking+# every other analytics library down with it. RL trading is therefore+# unavailable on Intel Macs; the shipped DMG is arm64-only, so this only+# affects x86_64 source builds.+stable-baselines3>=2.7.0; platform_machine != "x86_64" or sys_platform != "darwin"# MLflow# `>=2.0.0` already resolved to 3.x (venv has 3.12.0) across MLflow's breakingdiff --git a/fincept-qt/scripts/Analytics/finrl/__init__.py b/fincept-qt/scripts/Analytics/finrl/__init__.pydeleted file mode 100644index 6e7402ee3..000000000--- a/fincept-qt/scripts/Analytics/finrl/__init__.py+++ /dev/null@@ -1,48 +0,0 @@-"""-FinRL Wrapper Module for Fincept Terminal-=========================================--Complete wrapper around the FinRL (Financial Reinforcement Learning) library.-Provides CLI-accessible Python scripts for all FinRL capabilities.--Modules:----------finrl_config - Configuration, ticker universes, hyperparameters, date ranges-finrl_data - Data downloading, preprocessing, feature engineering, splitting-finrl_environments - Trading environments (stock, crypto, portfolio, stop-loss, etc.)-finrl_agents - DRL agent training (A2C, PPO, DDPG, SAC, TD3)-finrl_backtest - Backtesting, performance metrics, algorithm comparison-finrl_portfolio - Portfolio allocation & optimization with DRL-finrl_trading - Paper trading (Alpaca), live trading, crypto trading-finrl_plot - Visualization, equity curves, drawdowns, comparison charts-finrl_ensemble - Ensemble strategies, validation-based selection, turbulence switching--Library Coverage:-------------------finrl.config -> finrl_config.py-finrl.config_tickers -> finrl_config.py-finrl.config_private -> finrl_config.py-finrl.train -> finrl_agents.py-finrl.test -> finrl_backtest.py-finrl.trade -> finrl_trading.py-finrl.plot -> finrl_plot.py (custom, pyfolio-free)-finrl.main -> finrl_config.py--finrl.meta.data_processor -> finrl_data.py-finrl.meta.data_processors.* -> finrl_data.py-finrl.meta.preprocessor.* -> finrl_data.py--finrl.meta.env_stock_trading.* -> finrl_environments.py-finrl.meta.env_cryptocurrency_trading.*-> finrl_environments.py-finrl.meta.env_portfolio_allocation.* -> finrl_environments.py-finrl.meta.env_portfolio_optimization.*-> finrl_environments.py--finrl.agents.stablebaselines3.* -> finrl_agents.py, finrl_ensemble.py-finrl.agents.portfolio_optimization.* -> finrl_portfolio.py--finrl.applications.stock_trading.* -> finrl_ensemble.py-"""--__version__ = "1.0.0"-__library__ = "finrl"-__library_version__ = "0.3.7"diff --git a/fincept-qt/scripts/Analytics/finrl/finrl_agents.py b/fincept-qt/scripts/Analytics/finrl/finrl_agents.pydeleted file mode 100644index 65581fd5d..000000000--- a/fincept-qt/scripts/Analytics/finrl/finrl_agents.py+++ /dev/null@@ -1,450 +0,0 @@-"""-FinRL Agent Training & Model Management-Wrapper around finrl.agents.stablebaselines3, finrl.train, and model management.--Covers:-- DRLAgent: Single-agent training (A2C, PPO, DDPG, SAC, TD3)-- DRLEnsembleAgent: Ensemble training with validation-based model selection-- Model saving/loading-- Hyperparameter configuration-- TensorBoard logging integration-- Training from the high-level finrl.train API--Usage (CLI):- python finrl_agents.py --action list_algorithms- python finrl_agents.py --action train --algorithm ppo --data_file train.csv --tickers AAPL MSFT --timesteps 50000- python finrl_agents.py --action train --algorithm a2c --data_file train.csv --tickers AAPL MSFT --timesteps 100000 --tensorboard- python finrl_agents.py --action train_highlevel --algorithm ppo --tickers AAPL MSFT --train_start 2018-01-01 --train_end 2022-12-31 --source yahoofinance- python finrl_agents.py --action load_predict --algorithm ppo --model_path trained_models/ppo.zip --data_file test.csv --tickers AAPL MSFT-"""--import json-import sys-import os-import argparse-import numpy as np-import pandas as pd--from finrl.agents.stablebaselines3.models import DRLAgent, DRLEnsembleAgent, MODELS, MODEL_KWARGS-from finrl.config import (- INDICATORS, TRAINED_MODEL_DIR, TENSORBOARD_LOG_DIR,- A2C_PARAMS, PPO_PARAMS, DDPG_PARAMS, SAC_PARAMS, TD3_PARAMS,-)-from finrl.train import train as finrl_train--from stable_baselines3 import A2C, PPO, DDPG, SAC, TD3-from stable_baselines3.common.vec_env import DummyVecEnv--from finrl_environments import create_stock_trading_env---# ============================================================================-# Algorithm Registry-# ============================================================================--ALGORITHM_INFO = {- "a2c": {- "name": "Advantage Actor-Critic (A2C)",- "description": "Synchronous version of A3C. Good baseline, fast training.",- "best_for": "Simple strategies, quick prototyping",- "default_params": A2C_PARAMS,- "sb3_class": "A2C",- },- "ppo": {- "name": "Proximal Policy Optimization (PPO)",- "description": "Most stable and widely used algorithm. Clips policy updates.",- "best_for": "General purpose, most reliable choice",- "default_params": PPO_PARAMS,- "sb3_class": "PPO",- },- "ddpg": {- "name": "Deep Deterministic Policy Gradient (DDPG)",- "description": "Off-policy, continuous action spaces. Uses replay buffer.",- "best_for": "Continuous action spaces, deterministic policies",- "default_params": DDPG_PARAMS,- "sb3_class": "DDPG",- },- "sac": {- "name": "Soft Actor-Critic (SAC)",- "description": "Entropy-regularized off-policy. Explores efficiently.",- "best_for": "Complex strategies needing exploration, robust performance",- "default_params": SAC_PARAMS,- "sb3_class": "SAC",- },- "td3": {- "name": "Twin Delayed DDPG (TD3)",- "description": "Improved DDPG with twin critics and delayed updates.",- "best_for": "When DDPG is unstable, continuous actions",- "default_params": TD3_PARAMS,- "sb3_class": "TD3",- },-}---# ============================================================================-# Training Functions-# ============================================================================--def train_agent(- df: pd.DataFrame,- algorithm: str = "ppo",- stock_dim: int = None,- hmax: int = 100,- initial_amount: int = 1_000_000,- buy_cost_pct: float = 0.001,- sell_cost_pct: float = 0.001,- reward_scaling: float = 1e-4,- tech_indicator_list: list = None,- total_timesteps: int = 50_000,- policy: str = "MlpPolicy",- model_kwargs: dict = None,- policy_kwargs: dict = None,- verbose: int = 1,- seed: int = None,- tensorboard_log: str = None,- save_path: str = None,- turbulence_threshold: float = None,-) -> dict:- """- Train a single DRL agent on stock trading environment.-- Returns dict with model info and training results.- """- if tech_indicator_list is None:- tech_indicator_list = list(INDICATORS)-- if stock_dim is None:- if "tic" in df.columns:- stock_dim = df["tic"].nunique()- else:- stock_dim = 1-- # Create environment- env = create_stock_trading_env(- df=df,- stock_dim=stock_dim,- hmax=hmax,- initial_amount=initial_amount,- buy_cost_pct=[buy_cost_pct] * stock_dim,- sell_cost_pct=[sell_cost_pct] * stock_dim,- reward_scaling=reward_scaling,- tech_indicator_list=tech_indicator_list,- turbulence_threshold=turbulence_threshold,- )-- # Create agent- agent = DRLAgent(env=env)-- # Get model with hyperparameters- if model_kwargs is None:- algo_key = algorithm.lower()- if algo_key in MODEL_KWARGS:- model_kwargs = MODEL_KWARGS[algo_key]- else:- model_kwargs = {}-- tb_log = tensorboard_log or TENSORBOARD_LOG_DIR if tensorboard_log is not None else None-- model = agent.get_model(- model_name=algorithm.lower(),- policy=policy,- policy_kwargs=policy_kwargs,- model_kwargs=model_kwargs,- verbose=verbose,- seed=seed,- tensorboard_log=tb_log,- )-- # Train model- trained_model = agent.train_model(- model=model,- tb_log_name=algorithm.lower(),- total_timesteps=total_timesteps,- )-- # Save model- if save_path is None:- os.makedirs(TRAINED_MODEL_DIR, exist_ok=True)- save_path = os.path.join(TRAINED_MODEL_DIR, f"{algorithm.lower()}_agent")-- trained_model.save(save_path)-- return {- "status": "success",- "algorithm": algorithm,- "total_timesteps": total_timesteps,- "stock_dim": stock_dim,- "state_space": 1 + 2 * stock_dim + len(tech_indicator_list) * stock_dim,- "action_space": stock_dim,- "model_saved_to": save_path,- "policy": policy,- "model_kwargs": {k: str(v) for k, v in (model_kwargs or {}).items()},- }---def train_highlevel(- algorithm: str = "ppo",- tickers: list = None,- start_date: str = "2018-01-01",- end_date: str = "2022-12-31",- data_source: str = "yahoofinance",- time_interval: str = "1D",- indicators: list = None,- drl_lib: str = "stable_baselines3",- if_vix: bool = True,- **kwargs,-) -> dict:- """- Train using FinRL's high-level train() API.- Handles data download, preprocessing, and training in one call.- """- if tickers is None:- tickers = ["AAPL", "MSFT", "GOOGL"]- if indicators is None:- indicators = list(INDICATORS)-- from finrl.meta.env_stock_trading.env_stocktrading import StockTradingEnv-- finrl_train(- start_date=start_date,- end_date=end_date,- ticker_list=tickers,- data_source=data_source,- time_interval=time_interval,- technical_indicator_list=indicators,- drl_lib=drl_lib,- env=StockTradingEnv,- model_name=algorithm.lower(),- if_vix=if_vix,- **kwargs,- )-- return {- "status": "success",- "algorithm": algorithm,- "tickers": tickers,- "date_range": f"{start_date} to {end_date}",- "data_source": data_source,- "drl_lib": drl_lib,- }---# ============================================================================-# Prediction / Inference Functions-# ============================================================================--def predict_with_model(- model_path: str,- df: pd.DataFrame,- algorithm: str = "ppo",- stock_dim: int = None,- hmax: int = 100,- initial_amount: int = 1_000_000,- buy_cost_pct: float = 0.001,- sell_cost_pct: float = 0.001,- reward_scaling: float = 1e-4,- tech_indicator_list: list = None,- deterministic: bool = True,-) -> dict:- """Load a trained model and run predictions on test data."""- if tech_indicator_list is None:- tech_indicator_list = list(INDICATORS)-- if stock_dim is None:- if "tic" in df.columns:- stock_dim = df["tic"].nunique()- else:- stock_dim = 1-- # Create test environment- env = create_stock_trading_env(- df=df,- stock_dim=stock_dim,- hmax=hmax,- initial_amount=initial_amount,- buy_cost_pct=[buy_cost_pct] * stock_dim,- sell_cost_pct=[sell_cost_pct] * stock_dim,- reward_scaling=reward_scaling,- tech_indicator_list=tech_indicator_list,- make_plots=False,- initial=True,- )-- # Load and predict- account_value, actions = DRLAgent.DRL_prediction_load_from_file(- model_name=algorithm.lower(),- environment=env,- cwd=os.path.dirname(model_path),- deterministic=deterministic,- )-- # Calculate metrics- if isinstance(account_value, pd.DataFrame) and "account_value" in account_value.columns:- values = account_value["account_value"].values- elif isinstance(account_value, list):- values = np.array(account_value)- else:- values = np.array([initial_amount])-- total_return = (values[-1] / values[0] - 1) * 100 if len(values) > 1 else 0- daily_returns = np.diff(values) / values[:-1] if len(values) > 1 else np.array([0])- sharpe = np.mean(daily_returns) / (np.std(daily_returns) + 1e-8) * np.sqrt(252)- max_dd = np.min(values / np.maximum.accumulate(values) - 1) * 100 if len(values) > 1 else 0-- return {- "status": "success",- "algorithm": algorithm,- "model_path": model_path,- "initial_value": float(values[0]),- "final_value": float(values[-1]),- "total_return_pct": float(total_return),- "sharpe_ratio": float(sharpe),- "max_drawdown_pct": float(max_dd),- "total_steps": len(values),- "num_trades": int(np.sum(np.abs(np.array(actions)) > 0)) if actions is not None else 0,- }---# ============================================================================-# Model Management-# ============================================================================--def list_saved_models(model_dir: str = None) -> dict:- """List all saved models in the trained models directory."""- if model_dir is None:- model_dir = TRAINED_MODEL_DIR-- if not os.path.exists(model_dir):- return {"models": [], "directory": model_dir, "count": 0}-- models = []- for f in os.listdir(model_dir):- fpath = os.path.join(model_dir, f)- if os.path.isfile(fpath) and (f.endswith(".zip") or f.endswith(".pth") or f.endswith(".pt")):- models.append({- "filename": f,- "path": fpath,- "size_mb": round(os.path.getsize(fpath) / 1024 / 1024, 2),- "algorithm": f.split("_")[0] if "_" in f else f.split(".")[0],- })-- return {"models": models, "directory": model_dir, "count": len(models)}---def get_model_summary(algorithm: str) -> dict:- """Get summary info about an algorithm."""- key = algorithm.lower()- if key not in ALGORITHM_INFO:- return {"error": f"Unknown algorithm '{algorithm}'. Available: {list(ALGORITHM_INFO.keys())}"}- return ALGORITHM_INFO[key]---# ============================================================================-# CLI Entry Point-# ============================================================================--def main():- parser = argparse.ArgumentParser(description="FinRL Agent Training Manager")- parser.add_argument("--action", required=True,- choices=["list_algorithms", "train", "train_highlevel",- "load_predict", "list_models", "algorithm_info"])- parser.add_argument("--algorithm", default="ppo",- choices=["a2c", "ppo", "ddpg", "sac", "td3"])- parser.add_argument("--data_file", help="CSV training data file")- parser.add_argument("--tickers", nargs="*", default=["AAPL", "MSFT", "GOOGL"])- parser.add_argument("--stock_dim", type=int)- parser.add_argument("--timesteps", type=int, default=50_000)- parser.add_argument("--initial_amount", type=float, default=1_000_000)- parser.add_argument("--hmax", type=int, default=100)- parser.add_argument("--buy_cost", type=float, default=0.001)- parser.add_argument("--sell_cost", type=float, default=0.001)- parser.add_argument("--reward_scaling", type=float, default=1e-4)- parser.add_argument("--policy", default="MlpPolicy")- parser.add_argument("--seed", type=int)- parser.add_argument("--tensorboard", action="store_true")- parser.add_argument("--save_path")- parser.add_argument("--model_path", help="Path to saved model for prediction")- parser.add_argument("--model_dir", default=TRAINED_MODEL_DIR)- parser.add_argument("--train_start", default="2018-01-01")- parser.add_argument("--train_end", default="2022-12-31")- parser.add_argument("--source", default="yahoofinance")- parser.add_argument("--indicators", nargs="*")-- args = parser.parse_args()-- if args.action == "list_algorithms":- result = {name: {k: v for k, v in info.items() if k != "default_params"}- for name, info in ALGORITHM_INFO.items()}- result["_note"] = "Use --action algorithm_info --algorithm <name> for full details with params"-- elif args.action == "algorithm_info":- result = get_model_summary(args.algorithm)-- elif args.action == "list_models":- result = list_saved_models(args.model_dir)-- elif args.action == "train":- if not args.data_file or not os.path.exists(args.data_file):- result = {"error": "Provide valid --data_file CSV path for training data"}- print(json.dumps(result, indent=2))- return-- df = pd.read_csv(args.data_file)- tb_log = TENSORBOARD_LOG_DIR if args.tensorboard else None-- result = train_agent(- df=df,- algorithm=args.algorithm,- stock_dim=args.stock_dim,- hmax=args.hmax,- initial_amount=int(args.initial_amount),- buy_cost_pct=args.buy_cost,- sell_cost_pct=args.sell_cost,- reward_scaling=args.reward_scaling,- tech_indicator_list=args.indicators,- total_timesteps=args.timesteps,- policy=args.policy,- seed=args.seed,- tensorboard_log=tb_log,- save_path=args.save_path,- )-- elif args.action == "train_highlevel":- result = train_highlevel(- algorithm=args.algorithm,- tickers=args.tickers,- start_date=args.train_start,- end_date=args.train_end,- data_source=args.source,- indicators=args.indicators,- )-- elif args.action == "load_predict":- if not args.model_path:- result = {"error": "Provide --model_path to saved model"}- print(json.dumps(result, indent=2))- return- if not args.data_file or not os.path.exists(args.data_file):- result = {"error": "Provide valid --data_file CSV for test data"}- print(json.dumps(result, indent=2))- return-- df = pd.read_csv(args.data_file)- result = predict_with_model(- model_path=args.model_path,- df=df,- algorithm=args.algorithm,- stock_dim=args.stock_dim,- hmax=args.hmax,- initial_amount=int(args.initial_amount),- tech_indicator_list=args.indicators,- )-- else:- result = {"error": f"Unknown action: {args.action}"}-- print(json.dumps(result, indent=2, default=str))---if __name__ == "__main__":- main()diff --git a/fincept-qt/scripts/Analytics/finrl/finrl_backtest.py b/fincept-qt/scripts/Analytics/finrl/finrl_backtest.pydeleted file mode 100644index 7a5120651..000000000--- a/fincept-qt/scripts/Analytics/finrl/finrl_backtest.py+++ /dev/null@@ -1,462 +0,0 @@-"""-FinRL Backtesting & Evaluation-Wrapper around finrl.test and performance evaluation utilities.--Covers:-- Backtest trained RL agents on test data-- Performance metrics (Sharpe, Sortino, Calmar, max drawdown)-- Account value tracking-- Action/trade analysis-- Multi-algorithm comparison-- High-level finrl.test API--Usage (CLI):- python finrl_backtest.py --action backtest --algorithm ppo --model_path trained_models/ppo_agent.zip --data_file test.csv --tickers AAPL MSFT- python finrl_backtest.py --action backtest_highlevel --algorithm ppo --tickers AAPL MSFT --test_start 2023-01-01 --test_end 2023-12-31- python finrl_backtest.py --action compare --model_dir trained_models --data_file test.csv --algorithms a2c ppo sac- python finrl_backtest.py --action metrics --account_values_file results/account_values.csv-"""--import json-import sys-import os-import argparse-import numpy as np-import pandas as pd--from finrl.agents.stablebaselines3.models import DRLAgent-from finrl.test import test as finrl_test-from finrl.config import (- INDICATORS, TRAINED_MODEL_DIR, RESULTS_DIR,- TEST_START_DATE, TEST_END_DATE,-)--from finrl_environments import create_stock_trading_env---# ============================================================================-# Performance Metrics-# ============================================================================--def calculate_metrics(account_values: np.ndarray, risk_free_rate: float = 0.02) -> dict:- """Calculate comprehensive performance metrics from account value series."""- if len(account_values) < 2:- return {"error": "Need at least 2 data points"}-- values = np.array(account_values, dtype=float)- daily_returns = np.diff(values) / values[:-1]-- # Basic returns- total_return = (values[-1] / values[0] - 1) * 100- annualized_return = ((values[-1] / values[0]) ** (252 / len(daily_returns)) - 1) * 100-- # Risk metrics- daily_vol = np.std(daily_returns) * 100- annual_vol = daily_vol * np.sqrt(252)-- # Sharpe ratio (annualized)- excess_return = np.mean(daily_returns) - risk_free_rate / 252- sharpe = excess_return / (np.std(daily_returns) + 1e-8) * np.sqrt(252)-- # Sortino ratio- downside_returns = daily_returns[daily_returns < 0]- downside_std = np.std(downside_returns) if len(downside_returns) > 0 else 1e-8- sortino = excess_return / (downside_std + 1e-8) * np.sqrt(252)-- # Maximum drawdown- running_max = np.maximum.accumulate(values)- drawdowns = (values - running_max) / running_max- max_drawdown = np.min(drawdowns) * 100- max_drawdown_duration = 0- current_dd_duration = 0- for i in range(len(values)):- if values[i] < running_max[i]:- current_dd_duration += 1- max_drawdown_duration = max(max_drawdown_duration, current_dd_duration)- else:- current_dd_duration = 0-- # Calmar ratio- calmar = annualized_return / (abs(max_drawdown) + 1e-8) if max_drawdown != 0 else 0-- # Win rate- winning_days = np.sum(daily_returns > 0)- losing_days = np.sum(daily_returns < 0)- win_rate = winning_days / (winning_days + losing_days) * 100 if (winning_days + losing_days) > 0 else 0-- # Profit factor- gross_profit = np.sum(daily_returns[daily_returns > 0])- gross_loss = abs(np.sum(daily_returns[daily_returns < 0]))- profit_factor = gross_profit / (gross_loss + 1e-8)-- # VaR (95%) and CVaR- var_95 = np.percentile(daily_returns, 5) * 100- cvar_95 = np.mean(daily_returns[daily_returns <= np.percentile(daily_returns, 5)]) * 100-- return {- "total_return_pct": round(float(total_return), 4),- "annualized_return_pct": round(float(annualized_return), 4),- "sharpe_ratio": round(float(sharpe), 4),- "sortino_ratio": round(float(sortino), 4),- "calmar_ratio": round(float(calmar), 4),- "max_drawdown_pct": round(float(max_drawdown), 4),- "max_drawdown_duration_days": int(max_drawdown_duration),- "daily_volatility_pct": round(float(daily_vol), 4),- "annual_volatility_pct": round(float(annual_vol), 4),- "win_rate_pct": round(float(win_rate), 2),- "profit_factor": round(float(profit_factor), 4),- "var_95_pct": round(float(var_95), 4),- "cvar_95_pct": round(float(cvar_95), 4),- "total_trading_days": int(len(daily_returns)),- "winning_days": int(winning_days),- "losing_days": int(losing_days),- "initial_value": round(float(values[0]), 2),- "final_value": round(float(values[-1]), 2),- "peak_value": round(float(np.max(values)), 2),- "trough_value": round(float(np.min(values)), 2),- }---def analyze_actions(actions: np.ndarray, tickers: list = None) -> dict:- """Analyze trading actions from backtest."""- actions = np.array(actions)- if actions.ndim == 1:- actions = actions.reshape(-1, 1)-- num_steps, num_stocks = actions.shape- total_buys = int(np.sum(actions > 0))- total_sells = int(np.sum(actions < 0))- total_holds = int(np.sum(actions == 0))-- per_stock = []- for i in range(num_stocks):- stock_actions = actions[:, i]- ticker_name = tickers[i] if tickers and i < len(tickers) else f"Stock_{i}"- per_stock.append({- "ticker": ticker_name,- "buys": int(np.sum(stock_actions > 0)),- "sells": int(np.sum(stock_actions < 0)),- "holds": int(np.sum(stock_actions == 0)),- "avg_buy_size": round(float(np.mean(stock_actions[stock_actions > 0])), 2) if np.any(stock_actions > 0) else 0,- "avg_sell_size": round(float(np.mean(stock_actions[stock_actions < 0])), 2) if np.any(stock_actions < 0) else 0,- "max_position_change": round(float(np.max(np.abs(stock_actions))), 2),- })-- return {- "total_steps": num_steps,- "num_stocks": num_stocks,- "total_buys": total_buys,- "total_sells": total_sells,- "total_holds": total_holds,- "activity_rate_pct": round((total_buys + total_sells) / (num_steps * num_stocks) * 100, 2),- "per_stock_analysis": per_stock,- }---# ============================================================================-# Backtesting Functions-# ============================================================================--def backtest_agent(- model_path: str,- df: pd.DataFrame,- algorithm: str = "ppo",- stock_dim: int = None,- hmax: int = 100,- initial_amount: int = 1_000_000,- buy_cost_pct: float = 0.001,- sell_cost_pct: float = 0.001,- reward_scaling: float = 1e-4,- tech_indicator_list: list = None,- deterministic: bool = True,- save_results: bool = True,- results_dir: str = None,- tickers: list = None,-) -> dict:- """- Backtest a trained agent on test data.- Returns comprehensive performance metrics.- """- if tech_indicator_list is None:- tech_indicator_list = list(INDICATORS)- if results_dir is None:- results_dir = RESULTS_DIR- if stock_dim is None:- stock_dim = df["tic"].nunique() if "tic" in df.columns else 1-- # Create test environment- env = create_stock_trading_env(- df=df,- stock_dim=stock_dim,- hmax=hmax,- initial_amount=initial_amount,- buy_cost_pct=[buy_cost_pct] * stock_dim,- sell_cost_pct=[sell_cost_pct] * stock_dim,- reward_scaling=reward_scaling,- tech_indicator_list=tech_indicator_list,- make_plots=False,- initial=True,- )-- # Load model and predict- account_memory = DRLAgent.DRL_prediction_load_from_file(- model_name=algorithm.lower(),- environment=env,- cwd=os.path.dirname(model_path) or ".",- deterministic=deterministic,- )-- # Extract account values- if isinstance(account_memory, tuple):- df_account, df_actions = account_memory- else:- df_account = account_memory- df_actions = None-- if isinstance(df_account, pd.DataFrame) and "account_value" in df_account.columns:- values = df_account["account_value"].values- elif isinstance(df_account, list):- values = np.array(df_account)- else:- values = np.array([initial_amount])-- # Calculate metrics- metrics = calculate_metrics(values)- metrics["algorithm"] = algorithm- metrics["model_path"] = model_path-- # Action analysis- if df_actions is not None:- actions_array = df_actions.values if isinstance(df_actions, pd.DataFrame) else np.array(df_actions)- metrics["action_analysis"] = analyze_actions(actions_array, tickers)-- # Save results- if save_results:- os.makedirs(results_dir, exist_ok=True)- if isinstance(df_account, pd.DataFrame):- df_account.to_csv(os.path.join(results_dir, f"{algorithm}_account_values.csv"), index=False)- if df_actions is not None and isinstance(df_actions, pd.DataFrame):- df_actions.to_csv(os.path.join(results_dir, f"{algorithm}_actions.csv"), index=False)- metrics["results_saved_to"] = results_dir-- return metrics---def backtest_highlevel(- algorithm: str = "ppo",- tickers: list = None,- test_start: str = None,- test_end: str = None,- data_source: str = "yahoofinance",- time_interval: str = "1D",- indicators: list = None,- if_vix: bool = True,- **kwargs,-) -> dict:- """Backtest using FinRL's high-level test() API."""- if tickers is None:- tickers = ["AAPL", "MSFT", "GOOGL"]- if indicators is None:- indicators = list(INDICATORS)- if test_start is None:- test_start = TEST_START_DATE- if test_end is None:- test_end = TEST_END_DATE-- from finrl.meta.env_stock_trading.env_stocktrading import StockTradingEnv-- result = finrl_test(- start_date=test_start,- end_date=test_end,- ticker_list=tickers,- data_source=data_source,- time_interval=time_interval,- technical_indicator_list=indicators,- drl_lib="stable_baselines3",- env=StockTradingEnv,- model_name=algorithm.lower(),- if_vix=if_vix,- **kwargs,- )-- return {- "status": "success",- "algorithm": algorithm,- "test_period": f"{test_start} to {test_end}",- "tickers": tickers,- "result": str(result)[:500] if result else "No result returned",- }---def compare_algorithms(- df: pd.DataFrame,- algorithms: list = None,- model_dir: str = None,- stock_dim: int = None,- hmax: int = 100,- initial_amount: int = 1_000_000,- tech_indicator_list: list = None,- tickers: list = None,-) -> dict:- """Compare multiple trained algorithms on the same test data."""- if algorithms is None:- algorithms = ["a2c", "ppo", "sac"]- if model_dir is None:- model_dir = TRAINED_MODEL_DIR- if tech_indicator_list is None:- tech_indicator_list = list(INDICATORS)- if stock_dim is None:- stock_dim = df["tic"].nunique() if "tic" in df.columns else 1-- results = {}- for algo in algorithms:- # Try multiple path patterns- possible_paths = [- os.path.join(model_dir, f"{algo}_agent.zip"),- os.path.join(model_dir, f"{algo}_agent"),- os.path.join(model_dir, f"{algo}.zip"),- os.path.join(model_dir, algo),- ]-- model_path = None- for p in possible_paths:- if os.path.exists(p):- model_path = p- break-- if model_path is None:- results[algo] = {"error": f"Model not found. Tried: {possible_paths}"}- continue-- try:- metrics = backtest_agent(- model_path=model_path,- df=df,- algorithm=algo,- stock_dim=stock_dim,- hmax=hmax,- initial_amount=initial_amount,- tech_indicator_list=tech_indicator_list,- save_results=False,- tickers=tickers,- )- results[algo] = metrics- except Exception as e:- results[algo] = {"error": str(e)}-- # Ranking- valid = {k: v for k, v in results.items() if "sharpe_ratio" in v}- if valid:- ranking = sorted(valid.items(), key=lambda x: x[1]["sharpe_ratio"], reverse=True)- return {- "comparison": results,- "ranking_by_sharpe": [{"rank": i + 1, "algorithm": k, "sharpe": v["sharpe_ratio"],- "return": v["total_return_pct"]}- for i, (k, v) in enumerate(ranking)],- "best_algorithm": ranking[0][0],- }-- return {"comparison": results}---# ============================================================================-# CLI Entry Point-# ============================================================================--def main():- parser = argparse.ArgumentParser(description="FinRL Backtesting & Evaluation")- parser.add_argument("--action", required=True,- choices=["backtest", "backtest_highlevel", "compare",- "metrics", "list_metrics"])- parser.add_argument("--algorithm", default="ppo")- parser.add_argument("--algorithms", nargs="*", default=["a2c", "ppo", "sac"])- parser.add_argument("--model_path", help="Path to trained model")- parser.add_argument("--model_dir", default=TRAINED_MODEL_DIR)- parser.add_argument("--data_file", help="CSV test data file")- parser.add_argument("--account_values_file", help="CSV with account values")- parser.add_argument("--tickers", nargs="*")- parser.add_argument("--stock_dim", type=int)- parser.add_argument("--initial_amount", type=float, default=1_000_000)- parser.add_argument("--hmax", type=int, default=100)- parser.add_argument("--test_start", default=TEST_START_DATE)- parser.add_argument("--test_end", default=TEST_END_DATE)- parser.add_argument("--source", default="yahoofinance")- parser.add_argument("--results_dir", default=RESULTS_DIR)- parser.add_argument("--indicators", nargs="*")-- args = parser.parse_args()-- if args.action == "list_metrics":- result = {- "available_metrics": [- "total_return_pct", "annualized_return_pct",- "sharpe_ratio", "sortino_ratio", "calmar_ratio",- "max_drawdown_pct", "max_drawdown_duration_days",- "daily_volatility_pct", "annual_volatility_pct",- "win_rate_pct", "profit_factor",- "var_95_pct", "cvar_95_pct",- "total_trading_days", "winning_days", "losing_days",- "initial_value", "final_value", "peak_value", "trough_value",- ]- }-- elif args.action == "backtest":- if not args.model_path or not args.data_file:- result = {"error": "Provide --model_path and --data_file"}- elif not os.path.exists(args.data_file):- result = {"error": f"Data file not found: {args.data_file}"}- else:- df = pd.read_csv(args.data_file)- result = backtest_agent(- model_path=args.model_path,- df=df,- algorithm=args.algorithm,- stock_dim=args.stock_dim,- initial_amount=int(args.initial_amount),- hmax=args.hmax,- tech_indicator_list=args.indicators,- results_dir=args.results_dir,- tickers=args.tickers,- )-- elif args.action == "backtest_highlevel":- result = backtest_highlevel(- algorithm=args.algorithm,- tickers=args.tickers,- test_start=args.test_start,- test_end=args.test_end,- data_source=args.source,- indicators=args.indicators,- )-- elif args.action == "compare":- if not args.data_file or not os.path.exists(args.data_file):- result = {"error": "Provide valid --data_file for comparison"}- else:- df = pd.read_csv(args.data_file)- result = compare_algorithms(- df=df,- algorithms=args.algorithms,- model_dir=args.model_dir,- stock_dim=args.stock_dim,- initial_amount=int(args.initial_amount),- tech_indicator_list=args.indicators,- tickers=args.tickers,- )-- elif args.action == "metrics":- if not args.account_values_file or not os.path.exists(args.account_values_file):- result = {"error": "Provide valid --account_values_file"}- else:- df = pd.read_csv(args.account_values_file)- col = "account_value" if "account_value" in df.columns else df.columns[-1]- result = calculate_metrics(df[col].values)-- else:- result = {"error": f"Unknown action: {args.action}"}-- print(json.dumps(result, indent=2, default=str))---if __name__ == "__main__":- main()diff --git a/fincept-qt/scripts/Analytics/finrl/finrl_config.py b/fincept-qt/scripts/Analytics/finrl/finrl_config.pydeleted file mode 100644index 5027b464f..000000000--- a/fincept-qt/scripts/Analytics/finrl/finrl_config.py+++ /dev/null@@ -1,258 +0,0 @@-"""-FinRL Configuration & Ticker Constants-Wrapper around finrl.config, finrl.config_tickers, finrl.meta.meta_config--Provides all default hyperparameters, date ranges, indicator lists,-and pre-built ticker universes for major global indices.--Usage (CLI):- python finrl_config.py --action list_tickers --universe DOW_30- python finrl_config.py --action list_indicators- python finrl_config.py --action get_hyperparams --algorithm PPO- python finrl_config.py --action get_dates- python finrl_config.py --action custom --train_start 2018-01-01 --train_end 2022-12-31 --test_start 2023-01-01 --test_end 2023-12-31-"""--import json-import sys-import argparse--from finrl.config import (- INDICATORS,- TRAIN_START_DATE, TRAIN_END_DATE,- TEST_START_DATE, TEST_END_DATE,- TRADE_START_DATE, TRADE_END_DATE,- A2C_PARAMS, PPO_PARAMS, DDPG_PARAMS, SAC_PARAMS, TD3_PARAMS,- ERL_PARAMS, RLlib_PARAMS,- DATA_SAVE_DIR, TRAINED_MODEL_DIR, TENSORBOARD_LOG_DIR, RESULTS_DIR,-)-from finrl.config_tickers import (- DOW_30_TICKER, NAS_100_TICKER, SP_500_TICKER,- HSI_50_TICKER, SSE_50_TICKER, CSI_300_TICKER,- DAX_30_TICKER, CAC_40_TICKER, TECDAX_TICKER,- MDAX_50_TICKER, SDAX_50_TICKER,- LQ45_TICKER, SRI_KEHATI_TICKER,- SINGLE_TICKER, FX_TICKER,-)--# ============================================================================-# Ticker Universes-# ============================================================================--TICKER_UNIVERSES = {- "SINGLE": SINGLE_TICKER,- "DOW_30": DOW_30_TICKER,- "NAS_100": NAS_100_TICKER,- "SP_500": SP_500_TICKER,- "HSI_50": HSI_50_TICKER,- "SSE_50": SSE_50_TICKER,- "CSI_300": CSI_300_TICKER,- "DAX_30": DAX_30_TICKER,- "CAC_40": CAC_40_TICKER,- "TECDAX": TECDAX_TICKER,- "MDAX_50": MDAX_50_TICKER,- "SDAX_50": SDAX_50_TICKER,- "LQ45": LQ45_TICKER,- "SRI_KEHATI": SRI_KEHATI_TICKER,- "FX": FX_TICKER,-}--# ============================================================================-# Algorithm Hyperparameters-# ============================================================================--ALGORITHM_PARAMS = {- "a2c": A2C_PARAMS,- "ppo": PPO_PARAMS,- "ddpg": DDPG_PARAMS,- "sac": SAC_PARAMS,- "td3": TD3_PARAMS,- "erl": ERL_PARAMS,- "rllib": RLlib_PARAMS,-}--# ============================================================================-# Default Date Ranges-# ============================================================================--DEFAULT_DATES = {- "train_start": TRAIN_START_DATE,- "train_end": TRAIN_END_DATE,- "test_start": TEST_START_DATE,- "test_end": TEST_END_DATE,- "trade_start": TRADE_START_DATE,- "trade_end": TRADE_END_DATE,-}--# ============================================================================-# Directory Configuration-# ============================================================================--DIRECTORIES = {- "data_save_dir": DATA_SAVE_DIR,- "trained_model_dir": TRAINED_MODEL_DIR,- "tensorboard_log_dir": TENSORBOARD_LOG_DIR,- "results_dir": RESULTS_DIR,-}---# ============================================================================-# Public API Functions-# ============================================================================--def get_ticker_list(universe: str) -> list:- """Get ticker list by universe name."""- key = universe.upper().replace(" ", "_")- if key not in TICKER_UNIVERSES:- raise ValueError(f"Unknown universe '{universe}'. Available: {list(TICKER_UNIVERSES.keys())}")- return TICKER_UNIVERSES[key]---def get_hyperparams(algorithm: str) -> dict:- """Get default hyperparameters for an algorithm."""- key = algorithm.lower()- if key not in ALGORITHM_PARAMS:- raise ValueError(f"Unknown algorithm '{algorithm}'. Available: {list(ALGORITHM_PARAMS.keys())}")- return ALGORITHM_PARAMS[key]---def get_indicators() -> list:- """Get default technical indicator list."""- return list(INDICATORS)---def get_all_available_indicators() -> list:- """Get extended list of all supported indicators."""- return [- "macd", "boll_ub", "boll_lb", "rsi_30", "cci_30", "dx_30",- "close_30_sma", "close_60_sma", "rsi_14", "rsi_7",- "cci_14", "dx_14", "adx", "atr", "obv", "mfi",- "willr", "stoch_k", "stoch_d", "ema_10", "ema_20", "ema_50",- "close_5_sma", "close_10_sma", "close_20_sma", "close_120_sma",- "vwap", "kdjk", "kdjd", "kdjj", "trix", "tema",- ]---def get_default_dates() -> dict:- """Get default date configuration."""- return dict(DEFAULT_DATES)---def build_custom_config(- tickers: list = None,- universe: str = None,- indicators: list = None,- algorithm: str = "ppo",- train_start: str = None,- train_end: str = None,- test_start: str = None,- test_end: str = None,- trade_start: str = None,- trade_end: str = None,- initial_amount: float = 1_000_000,- hmax: int = 100,- buy_cost_pct: float = 0.001,- sell_cost_pct: float = 0.001,- reward_scaling: float = 1e-4,- data_source: str = "yahoofinance",- time_interval: str = "1D",-) -> dict:- """Build a complete configuration dict for FinRL pipeline."""- if tickers is None:- if universe:- tickers = get_ticker_list(universe)- else:- tickers = DOW_30_TICKER-- if indicators is None:- indicators = get_indicators()-- algo_params = get_hyperparams(algorithm)-- return {- "tickers": tickers,- "indicators": indicators,- "algorithm": algorithm,- "algorithm_params": algo_params,- "dates": {- "train_start": train_start or TRAIN_START_DATE,- "train_end": train_end or TRAIN_END_DATE,- "test_start": test_start or TEST_START_DATE,- "test_end": test_end or TEST_END_DATE,- "trade_start": trade_start or TRADE_START_DATE,- "trade_end": trade_end or TRADE_END_DATE,- },- "environment": {- "initial_amount": initial_amount,- "hmax": hmax,- "buy_cost_pct": buy_cost_pct,- "sell_cost_pct": sell_cost_pct,- "reward_scaling": reward_scaling,- },- "data_source": data_source,- "time_interval": time_interval,- "directories": DIRECTORIES,- "stock_dim": len(tickers),- "state_space": 1 + 2 * len(tickers) + len(indicators) * len(tickers),- "action_space": len(tickers),- }---# ============================================================================-# CLI Entry Point-# ============================================================================--def main():- parser = argparse.ArgumentParser(description="FinRL Configuration Manager")- parser.add_argument("--action", required=True,- choices=["list_tickers", "list_indicators", "list_universes",- "get_hyperparams", "get_dates", "custom", "full_config"],- help="Action to perform")- parser.add_argument("--universe", default="DOW_30", help="Ticker universe name")- parser.add_argument("--algorithm", default="ppo", help="RL algorithm name")- parser.add_argument("--tickers", nargs="*", help="Custom tickers list")- parser.add_argument("--train_start", help="Training start date")- parser.add_argument("--train_end", help="Training end date")- parser.add_argument("--test_start", help="Test start date")- parser.add_argument("--test_end", help="Test end date")- parser.add_argument("--initial_amount", type=float, default=1_000_000)- parser.add_argument("--data_source", default="yahoofinance")- parser.add_argument("--time_interval", default="1D")-- args = parser.parse_args()-- if args.action == "list_tickers":- result = {"universe": args.universe, "tickers": get_ticker_list(args.universe),- "count": len(get_ticker_list(args.universe))}-- elif args.action == "list_indicators":- result = {"default_indicators": get_indicators(),- "all_available": get_all_available_indicators()}-- elif args.action == "list_universes":- result = {k: {"count": len(v), "sample": v[:5]} for k, v in TICKER_UNIVERSES.items()}-- elif args.action == "get_hyperparams":- result = {"algorithm": args.algorithm, "params": get_hyperparams(args.algorithm)}-- elif args.action == "get_dates":- result = get_default_dates()-- elif args.action in ("custom", "full_config"):- result = build_custom_config(- tickers=args.tickers, universe=args.universe,- algorithm=args.algorithm,- train_start=args.train_start, train_end=args.train_end,- test_start=args.test_start, test_end=args.test_end,- initial_amount=args.initial_amount,- data_source=args.data_source,- time_interval=args.time_interval,- )- else:- result = {"error": f"Unknown action: {args.action}"}-- print(json.dumps(result, indent=2, default=str))---if __name__ == "__main__":- main()diff --git a/fincept-qt/scripts/Analytics/finrl/finrl_data.py b/fincept-qt/scripts/Analytics/finrl/finrl_data.pydeleted file mode 100644index 8526f1ba6..000000000--- a/fincept-qt/scripts/Analytics/finrl/finrl_data.py+++ /dev/null@@ -1,458 +0,0 @@-"""-FinRL Data Downloading & Preprocessing-Wrapper around finrl.meta.data_processor, finrl.meta.preprocessor,-and all data processor backends (YahooFinance, Alpaca, CCXT, WRDS).--Covers:-- Multi-source data downloading (Yahoo Finance, Alpaca, CCXT crypto, WRDS)-- Data cleaning & gap-filling-- Technical indicator calculation (20+ indicators)-- Turbulence index computation-- VIX integration-- Feature engineering pipeline-- Train/test/trade data splitting-- Rolling window date generation--Usage (CLI):- python finrl_data.py --action download --source yahoofinance --tickers AAPL MSFT GOOGL --start 2020-01-01 --end 2023-12-31 --interval 1D- python finrl_data.py --action preprocess --source yahoofinance --tickers AAPL MSFT --start 2020-01-01 --end 2023-12-31- python finrl_data.py --action add_indicators --input_file data.csv --indicators macd rsi_30 boll_ub boll_lb- python finrl_data.py --action split --input_file data.csv --train_start 2020-01-01 --train_end 2022-12-31 --test_start 2023-01-01 --test_end 2023-12-31- python finrl_data.py --action full_pipeline --source yahoofinance --tickers AAPL MSFT --start 2018-01-01 --end 2023-12-31 --train_end 2022-12-31 --test_start 2023-01-01- python finrl_data.py --action rolling_dates --train_start 2018-01-01 --train_end 2020-12-31 --trade_start 2021-01-01 --trade_end 2023-12-31 --window 63-"""--import json-import sys-import os-import argparse-import numpy as np-import pandas as pd--from finrl.meta.data_processor import DataProcessor-from finrl.meta.preprocessor.preprocessors import FeatureEngineer, data_split-from finrl.meta.preprocessor.yahoodownloader import YahooDownloader-from finrl.meta.data_processors.func import (- calc_train_trade_starts_ends_if_rolling,- str2date, date2str,-)-from finrl.config import INDICATORS, DATA_SAVE_DIR---# ============================================================================-# Data Download Functions-# ============================================================================--def download_data(- source: str = "yahoofinance",- tickers: list = None,- start_date: str = "2020-01-01",- end_date: str = "2023-12-31",- time_interval: str = "1D",- save_path: str = None,- **kwargs-) -> pd.DataFrame:- """- Download market data from various sources.-- Supported sources: yahoofinance, alpaca, ccxt, wrds- """- processor = DataProcessor(data_source=source, **kwargs)- df = processor.download_data(- ticker_list=tickers,- start_date=start_date,- end_date=end_date,- time_interval=time_interval,- )- df = processor.clean_data(df)-- if save_path:- os.makedirs(os.path.dirname(save_path) or ".", exist_ok=True)- df.to_csv(save_path, index=False)-- return df---def download_yahoo(- tickers: list,- start_date: str,- end_date: str,- save_path: str = None,-) -> pd.DataFrame:- """Download data via YahooDownloader (legacy interface)."""- downloader = YahooDownloader(- start_date=start_date,- end_date=end_date,- ticker_list=tickers,- )- df = downloader.fetch_data()-- if save_path:- os.makedirs(os.path.dirname(save_path) or ".", exist_ok=True)- df.to_csv(save_path, index=False)-- return df---# ============================================================================-# Feature Engineering Functions-# ============================================================================--def add_technical_indicators(- df: pd.DataFrame,- indicators: list = None,- source: str = "yahoofinance",-) -> pd.DataFrame:- """Add technical indicators to dataframe via DataProcessor."""- if indicators is None:- indicators = list(INDICATORS)- processor = DataProcessor(data_source=source)- return processor.add_technical_indicator(df, indicators)---def add_turbulence(df: pd.DataFrame, source: str = "yahoofinance") -> pd.DataFrame:- """Add turbulence index to dataframe."""- processor = DataProcessor(data_source=source)- return processor.add_turbulence(df)---def add_vix(df: pd.DataFrame, source: str = "yahoofinance") -> pd.DataFrame:- """Add VIX data to dataframe."""- processor = DataProcessor(data_source=source)- return processor.add_vix(df)---def feature_engineer(- df: pd.DataFrame,- use_technical_indicator: bool = True,- tech_indicator_list: list = None,- use_vix: bool = False,- use_turbulence: bool = False,- user_defined_feature: bool = False,-) -> pd.DataFrame:- """- Full feature engineering pipeline using FeatureEngineer.- Cleans data, adds indicators, VIX, turbulence.- """- if tech_indicator_list is None:- tech_indicator_list = list(INDICATORS)-- fe = FeatureEngineer(- use_technical_indicator=use_technical_indicator,- tech_indicator_list=tech_indicator_list,- use_vix=use_vix,- use_turbulence=use_turbulence,- user_defined_feature=user_defined_feature,- )- return fe.preprocess_data(df)---# ============================================================================-# Data Splitting Functions-# ============================================================================--def split_data(- df: pd.DataFrame,- train_start: str,- train_end: str,- test_start: str = None,- test_end: str = None,- date_col: str = "date",-) -> dict:- """Split dataframe into train and test sets."""- train_df = data_split(df, train_start, train_end, target_date_col=date_col)- result = {"train": train_df}-- if test_start and test_end:- test_df = data_split(df, test_start, test_end, target_date_col=date_col)- result["test"] = test_df-- return result---def generate_rolling_dates(- train_start: str,- train_end: str,- trade_start: str,- trade_end: str,- rolling_window: int = 63,-) -> dict:- """Generate rolling window train/trade date pairs."""- import datetime- train_dates = []- d = str2date(train_start)- end = str2date(train_end)- delta = datetime.timedelta(days=1)- while d <= end:- train_dates.append(date2str(d))- d += delta-- trade_dates = []- d = str2date(trade_start)- end = str2date(trade_end)- while d <= end:- trade_dates.append(date2str(d))- d += delta-- train_starts, train_ends, trade_starts, trade_ends = \- calc_train_trade_starts_ends_if_rolling(- train_dates, trade_dates, rolling_window- )-- return {- "windows": [- {- "window": i,- "train_start": train_starts[i],- "train_end": train_ends[i],- "trade_start": trade_starts[i],- "trade_end": trade_ends[i],- }- for i in range(len(train_starts))- ],- "total_windows": len(train_starts),- }---# ============================================================================-# Array Conversion (for Environment)-# ============================================================================--def df_to_arrays(- df: pd.DataFrame,- indicators: list = None,- if_vix: bool = True,- source: str = "yahoofinance",-) -> dict:- """Convert preprocessed dataframe to arrays for environment."""- if indicators is None:- indicators = list(INDICATORS)- processor = DataProcessor(data_source=source)- arrays = processor.df_to_array(df, if_vix)- return {- "price_array_shape": list(arrays[0].shape) if len(arrays) > 0 else [],- "tech_array_shape": list(arrays[1].shape) if len(arrays) > 1 else [],- "turbulence_array_shape": list(arrays[2].shape) if len(arrays) > 2 else [],- "num_arrays": len(arrays),- }---# ============================================================================-# Full Pipeline-# ============================================================================--def full_data_pipeline(- source: str = "yahoofinance",- tickers: list = None,- start_date: str = "2018-01-01",- end_date: str = "2023-12-31",- time_interval: str = "1D",- indicators: list = None,- use_vix: bool = False,- use_turbulence: bool = False,- train_start: str = None,- train_end: str = None,- test_start: str = None,- test_end: str = None,- save_dir: str = None,- **kwargs,-) -> dict:- """- Execute the full data pipeline:- 1. Download data- 2. Clean data- 3. Add technical indicators- 4. Optionally add VIX and turbulence- 5. Split into train/test- """- if tickers is None:- tickers = ["AAPL", "MSFT", "GOOGL"]- if indicators is None:- indicators = list(INDICATORS)- if train_start is None:- train_start = start_date- if train_end is None:- train_end = end_date-- # Step 1: Download- processor = DataProcessor(data_source=source, **kwargs)- df = processor.download_data(tickers, start_date, end_date, time_interval)-- # Step 2: Clean- df = processor.clean_data(df)-- # Step 3: Feature engineer- fe = FeatureEngineer(- use_technical_indicator=True,- tech_indicator_list=indicators,- use_vix=use_vix,- use_turbulence=use_turbulence,- )- df = fe.preprocess_data(df)-- # Step 4: Split- splits = split_data(df, train_start, train_end, test_start, test_end)-- # Step 5: Save- if save_dir:- os.makedirs(save_dir, exist_ok=True)- df.to_csv(os.path.join(save_dir, "full_data.csv"), index=False)- splits["train"].to_csv(os.path.join(save_dir, "train_data.csv"), index=False)- if "test" in splits:- splits["test"].to_csv(os.path.join(save_dir, "test_data.csv"), index=False)-- result = {- "source": source,- "tickers": tickers,- "date_range": f"{start_date} to {end_date}",- "indicators": indicators,- "total_rows": len(df),- "columns": list(df.columns),- "train_rows": len(splits["train"]),- "test_rows": len(splits.get("test", [])),- "unique_dates": int(df["date"].nunique()) if "date" in df.columns else 0,- "unique_tickers": int(df["tic"].nunique()) if "tic" in df.columns else len(tickers),- }-- if save_dir:- result["saved_to"] = save_dir-- return result---# ============================================================================-# CLI Entry Point-# ============================================================================--def main():- parser = argparse.ArgumentParser(description="FinRL Data Manager")- parser.add_argument("--action", required=True,- choices=["download", "preprocess", "add_indicators",- "split", "full_pipeline", "rolling_dates",- "list_sources"])- parser.add_argument("--source", default="yahoofinance")- parser.add_argument("--tickers", nargs="*", default=["AAPL", "MSFT", "GOOGL"])- parser.add_argument("--start", default="2020-01-01")- parser.add_argument("--end", default="2023-12-31")- parser.add_argument("--interval", default="1D")- parser.add_argument("--indicators", nargs="*")- parser.add_argument("--use_vix", action="store_true")- parser.add_argument("--use_turbulence", action="store_true")- parser.add_argument("--train_start")- parser.add_argument("--train_end")- parser.add_argument("--test_start")- parser.add_argument("--test_end")- parser.add_argument("--trade_start")- parser.add_argument("--trade_end")- parser.add_argument("--window", type=int, default=63)- parser.add_argument("--input_file")- parser.add_argument("--save_dir", default=DATA_SAVE_DIR)-- args = parser.parse_args()-- if args.action == "list_sources":- result = {- "supported_sources": [- {"name": "yahoofinance", "description": "Yahoo Finance - Free, global equities/ETFs/indices"},- {"name": "alpaca", "description": "Alpaca Markets - US equities, requires API key"},- {"name": "ccxt", "description": "CCXT - 100+ crypto exchanges"},- {"name": "wrds", "description": "WRDS - Academic financial database"},- {"name": "joinquant", "description": "JoinQuant - Chinese market data"},- {"name": "quantconnect", "description": "QuantConnect - Multi-asset data"},- ]- }-- elif args.action == "download":- df = download_data(- source=args.source, tickers=args.tickers,- start_date=args.start, end_date=args.end,- time_interval=args.interval,- save_path=os.path.join(args.save_dir, "downloaded_data.csv"),- )- result = {- "status": "success",- "rows": len(df),- "columns": list(df.columns),- "tickers": args.tickers,- "date_range": f"{args.start} to {args.end}",- "saved_to": os.path.join(args.save_dir, "downloaded_data.csv"),- }-- elif args.action == "preprocess":- result = full_data_pipeline(- source=args.source, tickers=args.tickers,- start_date=args.start, end_date=args.end,- time_interval=args.interval,- indicators=args.indicators,- use_vix=args.use_vix,- use_turbulence=args.use_turbulence,- train_start=args.train_start, train_end=args.train_end,- test_start=args.test_start, test_end=args.test_end,- save_dir=args.save_dir,- )-- elif args.action == "add_indicators":- if args.input_file and os.path.exists(args.input_file):- df = pd.read_csv(args.input_file)- else:- result = {"error": "Provide --input_file with valid CSV path"}- print(json.dumps(result, indent=2))- return-- df = add_technical_indicators(df, args.indicators, args.source)- save_path = os.path.join(args.save_dir, "data_with_indicators.csv")- df.to_csv(save_path, index=False)- result = {- "status": "success",- "indicators_added": args.indicators or list(INDICATORS),- "rows": len(df),- "columns": list(df.columns),- "saved_to": save_path,- }-- elif args.action == "split":- if args.input_file and os.path.exists(args.input_file):- df = pd.read_csv(args.input_file)- else:- result = {"error": "Provide --input_file with valid CSV path"}- print(json.dumps(result, indent=2))- return-- splits = split_data(df, args.train_start, args.train_end,- args.test_start, args.test_end)- result = {- "train_rows": len(splits["train"]),- "test_rows": len(splits.get("test", [])),- }-- elif args.action == "full_pipeline":- result = full_data_pipeline(- source=args.source, tickers=args.tickers,- start_date=args.start, end_date=args.end,- time_interval=args.interval,- indicators=args.indicators,- use_vix=args.use_vix,- use_turbulence=args.use_turbulence,- train_start=args.train_start or args.start,- train_end=args.train_end or args.end,- test_start=args.test_start, test_end=args.test_end,- save_dir=args.save_dir,- )-- elif args.action == "rolling_dates":- result = generate_rolling_dates(- args.train_start or "2018-01-01",- args.train_end or "2020-12-31",- args.trade_start or "2021-01-01",- args.trade_end or "2023-12-31",- args.window,- )-- else:- result = {"error": f"Unknown action: {args.action}"}-- print(json.dumps(result, indent=2, default=str))---if __name__ == "__main__":- main()diff --git a/fincept-qt/scripts/Analytics/finrl/finrl_ensemble.py b/fincept-qt/scripts/Analytics/finrl/finrl_ensemble.pydeleted file mode 100644index 98f0872b4..000000000--- a/fincept-qt/scripts/Analytics/finrl/finrl_ensemble.py+++ /dev/null@@ -1,430 +0,0 @@-"""-FinRL Ensemble Strategies-Wrapper around finrl.agents.stablebaselines3.models.DRLEnsembleAgent-and finrl.applications.stock_trading.ensemble_stock_trading.--Covers:-- Ensemble strategy training (A2C + PPO + DDPG + SAC + TD3)-- Validation-window based model selection-- Rolling-window ensemble with rebalancing-- Turbulence-based strategy switching-- Sharpe-ratio-based algorithm selection--Usage (CLI):- python finrl_ensemble.py --action list_strategies- python finrl_ensemble.py --action train --data_file data.csv --tickers AAPL MSFT GOOGL --rebalance_window 63 --validation_window 63- python finrl_ensemble.py --action backtest --data_file test.csv --model_dir trained_models --tickers AAPL MSFT GOOGL- python finrl_ensemble.py --action compare_single_vs_ensemble --data_file test.csv --model_dir trained_models --tickers AAPL MSFT-"""--import json-import sys-import os-import argparse-import numpy as np-import pandas as pd--from finrl.agents.stablebaselines3.models import DRLEnsembleAgent, DRLAgent-from finrl.config import (- INDICATORS, TRAINED_MODEL_DIR, RESULTS_DIR,- A2C_PARAMS, PPO_PARAMS, DDPG_PARAMS, SAC_PARAMS, TD3_PARAMS,-)-from finrl.meta.preprocessor.preprocessors import data_split---# ============================================================================-# Ensemble Strategy Registry-# ============================================================================--ENSEMBLE_STRATEGIES = {- "validation_sharpe": {- "description": "Select best algorithm per rebalance window based on validation Sharpe ratio",- "algorithms_used": ["A2C", "PPO", "DDPG", "SAC", "TD3"],- "method": "Train all 5 algorithms, validate on recent window, pick best Sharpe",- "advantages": [- "Adapts to changing market conditions",- "Automatically selects best-performing algorithm",- "Built-in validation prevents overfitting",- ],- },- "turbulence_switching": {- "description": "Switch between aggressive/defensive algorithms based on turbulence",- "algorithms_used": ["PPO (normal)", "A2C (high turbulence)"],- "method": "Use turbulence index to toggle between algorithms",- "advantages": [- "Risk-aware strategy switching",- "Protects against market crashes",- ],- },- "weighted_average": {- "description": "Weighted average of all algorithm predictions",- "algorithms_used": ["A2C", "PPO", "DDPG", "SAC", "TD3"],- "method": "Combine actions from all algorithms with Sharpe-ratio weights",- "advantages": [- "Diversification of strategies",- "Smoothed trading signals",- ],- },-}---# ============================================================================-# Ensemble Training-# ============================================================================--def train_ensemble(- df: pd.DataFrame,- train_start: str,- train_end: str,- test_start: str,- test_end: str,- tickers: list = None,- hmax: int = 100,- initial_amount: float = 1_000_000,- buy_cost_pct: float = 0.001,- sell_cost_pct: float = 0.001,- reward_scaling: float = 1e-4,- tech_indicator_list: list = None,- rebalance_window: int = 63,- validation_window: int = 63,- a2c_params: dict = None,- ppo_params: dict = None,- ddpg_params: dict = None,- sac_params: dict = None,- td3_params: dict = None,- timesteps_dict: dict = None,- print_verbosity: int = 10,-) -> dict:- """- Train an ensemble of DRL agents with rolling validation.-- Uses DRLEnsembleAgent which trains A2C, PPO, DDPG, SAC, TD3 and selects- the best per rebalance window using validation Sharpe ratio.- """- if tech_indicator_list is None:- tech_indicator_list = list(INDICATORS)- if tickers is None:- tickers = df["tic"].unique().tolist() if "tic" in df.columns else ["AAPL"]-- stock_dim = len(tickers)- state_space = 1 + 2 * stock_dim + len(tech_indicator_list) * stock_dim- action_space = stock_dim-- if a2c_params is None: a2c_params = A2C_PARAMS- if ppo_params is None: ppo_params = PPO_PARAMS- if ddpg_params is None: ddpg_params = DDPG_PARAMS- if sac_params is None: sac_params = SAC_PARAMS- if td3_params is None: td3_params = TD3_PARAMS-- if timesteps_dict is None:- timesteps_dict = {- "a2c": 10_000,- "ppo": 10_000,- "ddpg": 10_000,- "sac": 10_000,- "td3": 10_000,- }-- # Build train/test periods- train_period = (train_start, train_end)- val_test_period = (test_start, test_end)-- try:- ensemble_agent = DRLEnsembleAgent(- df=df,- train_period=train_period,- val_test_period=val_test_period,- rebalance_window=rebalance_window,- validation_window=validation_window,- stock_dim=stock_dim,- hmax=hmax,- initial_amount=initial_amount,- buy_cost_pct=[buy_cost_pct] * stock_dim,- sell_cost_pct=[sell_cost_pct] * stock_dim,- reward_scaling=reward_scaling,- state_space=state_space,- action_space=action_space,- tech_indicator_list=tech_indicator_list,- print_verbosity=print_verbosity,- )-- df_summary = ensemble_agent.run_ensemble_strategy(- A2C_model_kwargs=a2c_params,- PPO_model_kwargs=ppo_params,- DDPG_model_kwargs=ddpg_params,- SAC_model_kwargs=sac_params,- TD3_model_kwargs=td3_params,- timesteps_dict=timesteps_dict,- )-- # Save results- os.makedirs(RESULTS_DIR, exist_ok=True)- if isinstance(df_summary, pd.DataFrame):- df_summary.to_csv(os.path.join(RESULTS_DIR, "ensemble_results.csv"), index=False)-- return {- "status": "success",- "strategy": "validation_sharpe",- "rebalance_window": rebalance_window,- "validation_window": validation_window,- "algorithms": ["a2c", "ppo", "ddpg", "sac", "td3"],- "stock_dim": stock_dim,- "train_period": train_period,- "test_period": val_test_period,- "timesteps_per_algo": timesteps_dict,- "results_saved_to": RESULTS_DIR,- }-- except Exception as e:- return {"error": str(e), "status": "failed"}---# ============================================================================-# Turbulence-Based Switching-# ============================================================================--def turbulence_ensemble(- df_test: pd.DataFrame,- models: dict,- stock_dim: int,- hmax: int = 100,- initial_amount: int = 1_000_000,- tech_indicator_list: list = None,- turbulence_threshold: float = 150.0,- normal_algo: str = "ppo",- defensive_algo: str = "a2c",-) -> dict:- """- Ensemble strategy that switches algorithms based on turbulence.- Uses aggressive algo in normal conditions, defensive in high turbulence.- """- if tech_indicator_list is None:- tech_indicator_list = list(INDICATORS)-- from finrl_environments import create_stock_trading_env- from stable_baselines3 import A2C, PPO, DDPG, SAC, TD3-- MODEL_MAP = {"a2c": A2C, "ppo": PPO, "ddpg": DDPG, "sac": SAC, "td3": TD3}-- # Load models- loaded_models = {}- for algo, path in models.items():- model_class = MODEL_MAP.get(algo.lower(), PPO)- loaded_models[algo] = model_class.load(path)-- # Create environment- env = create_stock_trading_env(- df=df_test, stock_dim=stock_dim, hmax=hmax,- initial_amount=initial_amount,- tech_indicator_list=tech_indicator_list,- )-- obs, _ = env.reset()- account_values = [initial_amount]- algo_used = []- done = False-- while not done:- # Check turbulence- turbulence = 0- if hasattr(env, 'turbulence'):- turbulence = env.turbulence- elif "turbulence" in df_test.columns:- day = min(env.day if hasattr(env, 'day') else 0, len(df_test) - 1)- turbulence = df_test.iloc[day].get("turbulence", 0)-- # Select algorithm- if turbulence > turbulence_threshold:- current_algo = defensive_algo- else:- current_algo = normal_algo-- algo_used.append(current_algo)-- # Get action- model = loaded_models.get(current_algo, loaded_models.get(normal_algo))- action, _ = model.predict(obs, deterministic=True)- obs, reward, done, truncated, info = env.step(action)-- # Get final account values- if hasattr(env, 'save_asset_memory'):- df_account = env.save_asset_memory()- if isinstance(df_account, pd.DataFrame) and "account_value" in df_account.columns:- account_values = df_account["account_value"].tolist()-- values = np.array(account_values, dtype=float)- total_return = (values[-1] / values[0] - 1) * 100 if len(values) > 1 else 0-- from collections import Counter- algo_counts = Counter(algo_used)-- return {- "status": "success",- "strategy": "turbulence_switching",- "turbulence_threshold": turbulence_threshold,- "normal_algo": normal_algo,- "defensive_algo": defensive_algo,- "initial_value": float(values[0]),- "final_value": float(values[-1]),- "total_return_pct": round(float(total_return), 4),- "algo_usage": dict(algo_counts),- "algo_switches": sum(1 for i in range(1, len(algo_used)) if algo_used[i] != algo_used[i - 1]),- "total_steps": len(algo_used),- }---# ============================================================================-# Weighted Average Ensemble-# ============================================================================--def weighted_average_actions(- models: dict,- obs: np.ndarray,- weights: dict = None,- deterministic: bool = True,-) -> np.ndarray:- """Compute weighted average of actions from multiple models."""- if weights is None:- weights = {k: 1.0 / len(models) for k in models}-- total_weight = sum(weights.values())- weighted_action = None-- for algo_name, model in models.items():- action, _ = model.predict(obs, deterministic=deterministic)- w = weights.get(algo_name, 0) / total_weight- if weighted_action is None:- weighted_action = action * w- else:- weighted_action += action * w-- return weighted_action---# ============================================================================-# Comparison: Single vs Ensemble-# ============================================================================--def compare_single_vs_ensemble(- df_test: pd.DataFrame,- model_dir: str,- algorithms: list = None,- stock_dim: int = None,- initial_amount: int = 1_000_000,- tech_indicator_list: list = None,- tickers: list = None,-) -> dict:- """Compare individual algorithms against ensemble approaches."""- if algorithms is None:- algorithms = ["a2c", "ppo", "sac"]- if tech_indicator_list is None:- tech_indicator_list = list(INDICATORS)- if stock_dim is None:- stock_dim = df_test["tic"].nunique() if "tic" in df_test.columns else 1-- from finrl_backtest import backtest_agent-- results = {}- for algo in algorithms:- # Find model- for ext in [".zip", "", "_agent.zip", "_agent"]:- path = os.path.join(model_dir, f"{algo}{ext}")- if os.path.exists(path):- try:- metrics = backtest_agent(- model_path=path, df=df_test, algorithm=algo,- stock_dim=stock_dim, initial_amount=initial_amount,- tech_indicator_list=tech_indicator_list,- save_results=False, tickers=tickers,- )- results[algo] = metrics- except Exception as e:- results[algo] = {"error": str(e)}- break-- # Find best single- valid = {k: v for k, v in results.items() if "sharpe_ratio" in v}- if valid:- best_single = max(valid.items(), key=lambda x: x[1]["sharpe_ratio"])- results["_comparison"] = {- "best_single_algorithm": best_single[0],- "best_single_sharpe": best_single[1]["sharpe_ratio"],- "best_single_return": best_single[1]["total_return_pct"],- "num_algorithms_tested": len(algorithms),- }-- return results---# ============================================================================-# CLI Entry Point-# ============================================================================--def main():- parser = argparse.ArgumentParser(description="FinRL Ensemble Strategies")- parser.add_argument("--action", required=True,- choices=["list_strategies", "train", "backtest",- "compare_single_vs_ensemble"])- parser.add_argument("--data_file", help="CSV data file")- parser.add_argument("--tickers", nargs="*")- parser.add_argument("--algorithms", nargs="*", default=["a2c", "ppo", "sac"])- parser.add_argument("--train_start", default="2018-01-01")- parser.add_argument("--train_end", default="2022-06-30")- parser.add_argument("--test_start", default="2022-07-01")- parser.add_argument("--test_end", default="2023-12-31")- parser.add_argument("--rebalance_window", type=int, default=63)- parser.add_argument("--validation_window", type=int, default=63)- parser.add_argument("--initial_amount", type=float, default=1_000_000)- parser.add_argument("--hmax", type=int, default=100)- parser.add_argument("--timesteps", type=int, default=10_000)- parser.add_argument("--model_dir", default=TRAINED_MODEL_DIR)- parser.add_argument("--indicators", nargs="*")-- args = parser.parse_args()-- if args.action == "list_strategies":- result = ENSEMBLE_STRATEGIES-- elif args.action == "train":- if not args.data_file or not os.path.exists(args.data_file):- result = {"error": "Provide valid --data_file"}- else:- df = pd.read_csv(args.data_file)- ts = args.timesteps- result = train_ensemble(- df=df,- train_start=args.train_start,- train_end=args.train_end,- test_start=args.test_start,- test_end=args.test_end,- tickers=args.tickers,- hmax=args.hmax,- initial_amount=args.initial_amount,- tech_indicator_list=args.indicators,- rebalance_window=args.rebalance_window,- validation_window=args.validation_window,- timesteps_dict={a: ts for a in ["a2c", "ppo", "ddpg", "sac", "td3"]},- )-- elif args.action == "compare_single_vs_ensemble":- if not args.data_file or not os.path.exists(args.data_file):- result = {"error": "Provide valid --data_file"}- else:- df = pd.read_csv(args.data_file)- result = compare_single_vs_ensemble(- df_test=df,- model_dir=args.model_dir,- algorithms=args.algorithms,- initial_amount=int(args.initial_amount),- tech_indicator_list=args.indicators,- tickers=args.tickers,- )-- else:- result = {"error": f"Unknown action: {args.action}"}-- print(json.dumps(result, indent=2, default=str))---if __name__ == "__main__":- main()diff --git a/fincept-qt/scripts/Analytics/finrl/finrl_environments.py b/fincept-qt/scripts/Analytics/finrl/finrl_environments.pydeleted file mode 100644index 52f20d296..000000000--- a/fincept-qt/scripts/Analytics/finrl/finrl_environments.py+++ /dev/null@@ -1,510 +0,0 @@-"""-FinRL Trading Environments-Wrapper around all FinRL gymnasium environments.--Covers:-- StockTradingEnv: Standard stock trading with buy/sell/hold actions-- StockTradingEnvCashPenalty: Penalizes holding excess cash-- StockTradingEnvStopLoss: Built-in stop-loss and profit-taking-- StockTradingEnvNP: NumPy-based high-performance environment-- StockPortfolioEnv: Portfolio allocation with weight-based actions-- PortfolioOptimizationEnv: Advanced portfolio optimization-- CryptoEnv: Multi-cryptocurrency trading-- BitcoinEnv: Single BTC trading with CCXT data--Usage (CLI):- python finrl_environments.py --action list_envs- python finrl_environments.py --action create --env stock_trading --data_file train.csv --tickers AAPL MSFT --initial_amount 1000000- python finrl_environments.py --action info --env stock_trading --stock_dim 30- python finrl_environments.py --action create --env crypto --data_file crypto.csv- python finrl_environments.py --action create --env portfolio --data_file train.csv --tickers AAPL MSFT GOOGL- python finrl_environments.py --action create --env cash_penalty --data_file train.csv- python finrl_environments.py --action create --env stop_loss --data_file train.csv-"""--import json-import sys-import os-import argparse-import numpy as np-import pandas as pd--from finrl.meta.env_stock_trading.env_stocktrading import (- StockTradingEnv,-)-from finrl.meta.env_stock_trading.env_stocktrading_cashpenalty import (- StockTradingEnvCashpenalty,-)-from finrl.meta.env_stock_trading.env_stocktrading_stoploss import (- StockTradingEnvStopLoss,-)-from finrl.meta.env_stock_trading.env_stocktrading_np import (- StockTradingEnv as StockTradingEnvNP,-)-from finrl.meta.env_portfolio_allocation.env_portfolio import (- StockPortfolioEnv,-)-from finrl.meta.env_portfolio_optimization.env_portfolio_optimization import (- PortfolioOptimizationEnv,-)-from finrl.meta.env_cryptocurrency_trading.env_multiple_crypto import (- CryptoEnv,-)-from finrl.meta.env_cryptocurrency_trading.env_btc_ccxt import (- BitcoinEnv,-)-from finrl.config import INDICATORS--from stable_baselines3.common.vec_env import DummyVecEnv---# ============================================================================-# Environment Registry-# ============================================================================--ENVIRONMENT_REGISTRY = {- "stock_trading": {- "class": "StockTradingEnv",- "description": "Standard stock trading with discrete buy/sell/hold per stock",- "action_type": "continuous [-hmax, hmax] per stock",- "features": ["Technical indicators", "Turbulence threshold", "Make plots", "State memory"],- },- "cash_penalty": {- "class": "StockTradingEnvCashpenalty",- "description": "Stock trading that penalizes holding too much cash",- "action_type": "continuous or discrete",- "features": ["Cash penalty", "Random start", "Patient mode", "Multi-process support"],- },- "stop_loss": {- "class": "StockTradingEnvStopLoss",- "description": "Stock trading with automatic stop-loss and profit-taking",- "action_type": "continuous or discrete",- "features": ["Stop-loss", "Profit-loss ratio", "Cash penalty", "Random start"],- },- "stock_np": {- "class": "StockTradingEnvNP",- "description": "High-performance NumPy-based stock trading (for ElegantRL)",- "action_type": "continuous with sigmoid",- "features": ["NumPy arrays", "Turbulence", "High performance"],- },- "portfolio": {- "class": "StockPortfolioEnv",- "description": "Portfolio allocation with weight-based actions (softmax normalized)",- "action_type": "portfolio weights [0, 1] summing to 1",- "features": ["Softmax normalization", "Covariance matrix state", "Lookback window"],- },- "portfolio_optimization": {- "class": "PortfolioOptimizationEnv",- "description": "Advanced portfolio optimization with time windows",- "action_type": "portfolio weights",- "features": ["Commission fees", "Time windows", "Multiple normalization modes"],- },- "crypto": {- "class": "CryptoEnv",- "description": "Multi-cryptocurrency trading environment",- "action_type": "continuous per crypto",- "features": ["Multiple cryptocurrencies", "Lookback period", "CCXT integration"],- },- "bitcoin": {- "class": "BitcoinEnv",- "description": "Bitcoin single-asset trading with CCXT data",- "action_type": "continuous [-1, 1]",- "features": ["Sub-minute data", "Cumulative return tracking"],- },-}---# ============================================================================-# Environment Factory Functions-# ============================================================================--def create_stock_trading_env(- df: pd.DataFrame,- stock_dim: int,- hmax: int = 100,- initial_amount: int = 1_000_000,- buy_cost_pct: list = None,- sell_cost_pct: list = None,- reward_scaling: float = 1e-4,- tech_indicator_list: list = None,- turbulence_threshold: float = None,- risk_indicator_col: str = "turbulence",- make_plots: bool = False,- print_verbosity: int = 10,- day: int = 0,- initial: bool = True,- num_stock_shares: list = None,- vectorize: bool = False,-) -> StockTradingEnv:- """Create standard StockTradingEnv."""- if tech_indicator_list is None:- tech_indicator_list = list(INDICATORS)- if buy_cost_pct is None:- buy_cost_pct = [0.001] * stock_dim- if sell_cost_pct is None:- sell_cost_pct = [0.001] * stock_dim- if num_stock_shares is None:- num_stock_shares = [0] * stock_dim-- state_space = 1 + 2 * stock_dim + len(tech_indicator_list) * stock_dim- action_space = stock_dim-- env = StockTradingEnv(- df=df,- stock_dim=stock_dim,- hmax=hmax,- initial_amount=initial_amount,- num_stock_shares=num_stock_shares,- buy_cost_pct=buy_cost_pct,- sell_cost_pct=sell_cost_pct,- reward_scaling=reward_scaling,- state_space=state_space,- action_space=action_space,- tech_indicator_list=tech_indicator_list,- turbulence_threshold=turbulence_threshold,- risk_indicator_col=risk_indicator_col,- make_plots=make_plots,- print_verbosity=print_verbosity,- day=day,- initial=initial,- )-- if vectorize:- return DummyVecEnv([lambda: env])- return env---def create_cash_penalty_env(- df: pd.DataFrame,- hmax: int = 10,- initial_amount: float = 1_000_000,- buy_cost_pct: float = 0.003,- sell_cost_pct: float = 0.003,- cash_penalty_proportion: float = 0.1,- daily_info_cols: list = None,- discrete_actions: bool = False,- random_start: bool = True,- patient: bool = False,- turbulence_threshold: float = None,- print_verbosity: int = 10,- currency: str = "$",- vectorize: bool = False,-) -> StockTradingEnvCashpenalty:- """Create cash-penalty stock trading environment."""- if daily_info_cols is None:- daily_info_cols = ["open", "close", "high", "low", "volume"]-- env = StockTradingEnvCashpenalty(- df=df,- buy_cost_pct=buy_cost_pct,- sell_cost_pct=sell_cost_pct,- hmax=hmax,- discrete_actions=discrete_actions,- turbulence_threshold=turbulence_threshold,- print_verbosity=print_verbosity,- initial_amount=initial_amount,- daily_information_cols=daily_info_cols,- cash_penalty_proportion=cash_penalty_proportion,- random_start=random_start,- patient=patient,- currency=currency,- )-- if vectorize:- return DummyVecEnv([lambda: env])- return env---def create_stop_loss_env(- df: pd.DataFrame,- hmax: int = 10,- initial_amount: float = 1_000_000,- buy_cost_pct: float = 0.003,- sell_cost_pct: float = 0.003,- stoploss_penalty: float = 0.9,- profit_loss_ratio: float = 2.0,- cash_penalty_proportion: float = 0.1,- daily_info_cols: list = None,- discrete_actions: bool = False,- random_start: bool = True,- patient: bool = False,- turbulence_threshold: float = None,- print_verbosity: int = 10,- currency: str = "$",- vectorize: bool = False,-) -> StockTradingEnvStopLoss:- """Create stop-loss stock trading environment."""- if daily_info_cols is None:- daily_info_cols = ["open", "close", "high", "low", "volume"]-- env = StockTradingEnvStopLoss(- df=df,- buy_cost_pct=buy_cost_pct,- sell_cost_pct=sell_cost_pct,- hmax=hmax,- discrete_actions=discrete_actions,- stoploss_penalty=stoploss_penalty,- profit_loss_ratio=profit_loss_ratio,- turbulence_threshold=turbulence_threshold,- print_verbosity=print_verbosity,- initial_amount=initial_amount,- daily_information_cols=daily_info_cols,- cash_penalty_proportion=cash_penalty_proportion,- random_start=random_start,- patient=patient,- currency=currency,- )-- if vectorize:- return DummyVecEnv([lambda: env])- return env---def create_portfolio_env(- df: pd.DataFrame,- stock_dim: int,- hmax: int = 100,- initial_amount: int = 1_000_000,- transaction_cost_pct: float = 0.001,- reward_scaling: float = 1e-4,- tech_indicator_list: list = None,- turbulence_threshold: float = None,- lookback: int = 252,- vectorize: bool = False,-) -> StockPortfolioEnv:- """Create portfolio allocation environment."""- if tech_indicator_list is None:- tech_indicator_list = list(INDICATORS)-- state_space = stock_dim- action_space = stock_dim-- env = StockPortfolioEnv(- df=df,- stock_dim=stock_dim,- hmax=hmax,- initial_amount=initial_amount,- transaction_cost_pct=transaction_cost_pct,- reward_scaling=reward_scaling,- state_space=state_space,- action_space=action_space,- tech_indicator_list=tech_indicator_list,- turbulence_threshold=turbulence_threshold,- lookback=lookback,- )-- if vectorize:- return DummyVecEnv([lambda: env])- return env---def create_portfolio_optimization_env(- df: pd.DataFrame,- initial_amount: float = 1_000_000,- features: list = None,- time_window: int = 1,- commission_fee_pct: float = 0.0,- commission_fee_model: str = "trf",- normalize_df: str = "by_previous_time",- reward_scaling: float = 1.0,- return_last_action: bool = False,- tics_in_portfolio: str = "all",- vectorize: bool = False,-) -> PortfolioOptimizationEnv:- """Create advanced portfolio optimization environment."""- if features is None:- features = ["close", "high", "low"]-- env = PortfolioOptimizationEnv(- df=df,- initial_amount=initial_amount,- features=features,- time_window=time_window,- comission_fee_pct=commission_fee_pct,- comission_fee_model=commission_fee_model,- normalize_df=normalize_df,- reward_scaling=reward_scaling,- return_last_action=return_last_action,- tics_in_portfolio=tics_in_portfolio,- )-- if vectorize:- return DummyVecEnv([lambda: env])- return env---def create_crypto_env(- config: dict,- lookback: int = 1,- initial_capital: float = 1_000_000,- buy_cost_pct: float = 0.001,- sell_cost_pct: float = 0.001,- gamma: float = 0.99,-) -> CryptoEnv:- """Create multi-cryptocurrency trading environment."""- return CryptoEnv(- config=config,- lookback=lookback,- initial_capital=initial_capital,- buy_cost_pct=buy_cost_pct,- sell_cost_pct=sell_cost_pct,- gamma=gamma,- )---def create_bitcoin_env(- price_ary=None,- tech_ary=None,- time_frequency: int = 15,- initial_account: float = 1_000_000,- max_stock: float = 100.0,- transaction_fee_percent: float = 0.001,- mode: str = "train",- gamma: float = 0.99,-) -> BitcoinEnv:- """Create Bitcoin trading environment."""- return BitcoinEnv(- price_ary=price_ary,- tech_ary=tech_ary,- time_frequency=time_frequency,- initial_account=initial_account,- max_stock=max_stock,- transaction_fee_percent=transaction_fee_percent,- mode=mode,- gamma=gamma,- )---# ============================================================================-# Environment Info-# ============================================================================--def get_env_info(env_type: str, stock_dim: int = 30, indicators: list = None) -> dict:- """Get computed environment dimensions."""- if indicators is None:- indicators = list(INDICATORS)-- if env_type in ("stock_trading", "stock_np"):- state_space = 1 + 2 * stock_dim + len(indicators) * stock_dim- action_space = stock_dim- elif env_type in ("portfolio", "portfolio_optimization"):- state_space = stock_dim- action_space = stock_dim- elif env_type == "crypto":- state_space = stock_dim * (1 + len(indicators))- action_space = stock_dim- elif env_type == "bitcoin":- state_space = 1 + 1 + len(indicators)- action_space = 1- else:- state_space = 1 + 2 * stock_dim + len(indicators) * stock_dim- action_space = stock_dim-- return {- "env_type": env_type,- "env_info": ENVIRONMENT_REGISTRY.get(env_type, {}),- "stock_dim": stock_dim,- "state_space": state_space,- "action_space": action_space,- "num_indicators": len(indicators),- "indicators": indicators,- }---# ============================================================================-# CLI Entry Point-# ============================================================================--def main():- parser = argparse.ArgumentParser(description="FinRL Environment Manager")- parser.add_argument("--action", required=True,- choices=["list_envs", "create", "info"])- parser.add_argument("--env", default="stock_trading",- choices=list(ENVIRONMENT_REGISTRY.keys()))- parser.add_argument("--data_file", help="CSV data file path")- parser.add_argument("--tickers", nargs="*")- parser.add_argument("--stock_dim", type=int, default=30)- parser.add_argument("--initial_amount", type=float, default=1_000_000)- parser.add_argument("--hmax", type=int, default=100)- parser.add_argument("--buy_cost", type=float, default=0.001)- parser.add_argument("--sell_cost", type=float, default=0.001)- parser.add_argument("--reward_scaling", type=float, default=1e-4)- parser.add_argument("--turbulence_threshold", type=float, default=None)- parser.add_argument("--indicators", nargs="*")-- args = parser.parse_args()-- if args.action == "list_envs":- result = {name: info for name, info in ENVIRONMENT_REGISTRY.items()}-- elif args.action == "info":- result = get_env_info(args.env, args.stock_dim, args.indicators)-- elif args.action == "create":- if not args.data_file or not os.path.exists(args.data_file):- result = {"error": "Provide valid --data_file CSV path"}- print(json.dumps(result, indent=2))- return-- df = pd.read_csv(args.data_file)- stock_dim = args.stock_dim- if args.tickers:- stock_dim = len(args.tickers)- elif "tic" in df.columns:- stock_dim = df["tic"].nunique()-- indicators = args.indicators or list(INDICATORS)-- if args.env == "stock_trading":- env = create_stock_trading_env(- df=df, stock_dim=stock_dim, hmax=args.hmax,- initial_amount=int(args.initial_amount),- tech_indicator_list=indicators,- turbulence_threshold=args.turbulence_threshold,- reward_scaling=args.reward_scaling,- )- elif args.env == "cash_penalty":- env = create_cash_penalty_env(- df=df, hmax=args.hmax, initial_amount=args.initial_amount,- buy_cost_pct=args.buy_cost, sell_cost_pct=args.sell_cost,- )- elif args.env == "stop_loss":- env = create_stop_loss_env(- df=df, hmax=args.hmax, initial_amount=args.initial_amount,- buy_cost_pct=args.buy_cost, sell_cost_pct=args.sell_cost,- )- elif args.env == "portfolio":- env = create_portfolio_env(- df=df, stock_dim=stock_dim, hmax=args.hmax,- initial_amount=int(args.initial_amount),- tech_indicator_list=indicators,- )- elif args.env == "portfolio_optimization":- env = create_portfolio_optimization_env(- df=df, initial_amount=args.initial_amount,- )- else:- result = {"error": f"Environment '{args.env}' requires array-based config, use Python API"}- print(json.dumps(result, indent=2))- return-- obs_shape = env.observation_space.shape if hasattr(env.observation_space, 'shape') else "N/A"- act_shape = env.action_space.shape if hasattr(env.action_space, 'shape') else "N/A"-- result = {- "status": "created",- "env_type": args.env,- "observation_space": str(obs_shape),- "action_space": str(act_shape),- "stock_dim": stock_dim,- "data_rows": len(df),- }-- else:- result = {"error": f"Unknown action: {args.action}"}-- print(json.dumps(result, indent=2, default=str))---if __name__ == "__main__":- main()diff --git a/fincept-qt/scripts/Analytics/finrl/finrl_plot.py b/fincept-qt/scripts/Analytics/finrl/finrl_plot.pydeleted file mode 100644index 17df2d2ff..000000000--- a/fincept-qt/scripts/Analytics/finrl/finrl_plot.py+++ /dev/null@@ -1,382 +0,0 @@-"""-FinRL Visualization & Performance Charts-Custom plotting since pyfolio is incompatible with Python 3.13.--Covers:-- Equity curve plotting-- Drawdown chart-- Daily/monthly returns analysis-- Algorithm comparison charts-- Portfolio weight visualization-- Action heatmaps-- Risk metrics radar chart-- Export to JSON for frontend rendering--Usage (CLI):- python finrl_plot.py --action equity_curve --data_file results/account_values.csv --output results/equity_curve.json- python finrl_plot.py --action drawdown --data_file results/account_values.csv --output results/drawdown.json- python finrl_plot.py --action returns_analysis --data_file results/account_values.csv --output results/returns.json- python finrl_plot.py --action comparison --data_dir results --algorithms a2c ppo sac --output results/comparison.json- python finrl_plot.py --action full_report --data_file results/account_values.csv --output results/full_report.json-"""--import json-import sys-import os-import argparse-import numpy as np-import pandas as pd-from datetime import datetime---# ============================================================================-# Data Extraction-# ============================================================================--def load_account_values(filepath: str) -> np.ndarray:- """Load account values from CSV."""- df = pd.read_csv(filepath)- if "account_value" in df.columns:- return df["account_value"].values.astype(float)- return df.iloc[:, -1].values.astype(float)---def load_account_df(filepath: str) -> pd.DataFrame:- """Load full account dataframe."""- return pd.read_csv(filepath)---# ============================================================================-# Equity Curve-# ============================================================================--def generate_equity_curve(values: np.ndarray, dates: list = None, benchmark: np.ndarray = None) -> dict:- """Generate equity curve data for frontend charting."""- n = len(values)- if dates is None:- dates = list(range(n))-- # Normalize to percentage returns from initial- normalized = (values / values[0] - 1) * 100-- result = {- "chart_type": "equity_curve",- "title": "Portfolio Equity Curve",- "x_label": "Date",- "y_label": "Return (%)",- "series": [- {- "name": "Portfolio",- "data": [{"x": str(dates[i]), "y": round(float(normalized[i]), 2)} for i in range(n)],- "color": "#FF8800",- }- ],- "summary": {- "initial_value": round(float(values[0]), 2),- "final_value": round(float(values[-1]), 2),- "total_return_pct": round(float(normalized[-1]), 2),- "peak_return_pct": round(float(np.max(normalized)), 2),- },- }-- if benchmark is not None:- bench_norm = (benchmark / benchmark[0] - 1) * 100- result["series"].append({- "name": "Benchmark",- "data": [{"x": str(dates[i]), "y": round(float(bench_norm[i]), 2)} for i in range(min(n, len(benchmark)))],- "color": "#787878",- })- result["summary"]["benchmark_return_pct"] = round(float(bench_norm[-1]), 2)- result["summary"]["alpha_pct"] = round(float(normalized[-1] - bench_norm[-1]), 2)-- return result---# ============================================================================-# Drawdown Chart-# ============================================================================--def generate_drawdown_chart(values: np.ndarray, dates: list = None) -> dict:- """Generate drawdown chart data."""- n = len(values)- if dates is None:- dates = list(range(n))-- running_max = np.maximum.accumulate(values)- drawdowns = (values - running_max) / running_max * 100-- max_dd_idx = np.argmin(drawdowns)- max_dd_start = np.argmax(values[:max_dd_idx + 1]) if max_dd_idx > 0 else 0-- return {- "chart_type": "drawdown",- "title": "Underwater (Drawdown) Chart",- "x_label": "Date",- "y_label": "Drawdown (%)",- "series": [- {- "name": "Drawdown",- "data": [{"x": str(dates[i]), "y": round(float(drawdowns[i]), 2)} for i in range(n)],- "color": "#FF3B3B",- "fill": True,- }- ],- "summary": {- "max_drawdown_pct": round(float(np.min(drawdowns)), 2),- "max_drawdown_date": str(dates[max_dd_idx]),- "max_drawdown_peak_date": str(dates[max_dd_start]),- "current_drawdown_pct": round(float(drawdowns[-1]), 2),- "avg_drawdown_pct": round(float(np.mean(drawdowns)), 2),- },- }---# ============================================================================-# Returns Analysis-# ============================================================================--def generate_returns_analysis(values: np.ndarray, dates: list = None) -> dict:- """Generate returns distribution and time-series analysis."""- daily_returns = np.diff(values) / values[:-1] * 100- n = len(daily_returns)-- # Distribution- hist, bin_edges = np.histogram(daily_returns, bins=50)- distribution = [- {"x": round(float((bin_edges[i] + bin_edges[i + 1]) / 2), 3),- "y": int(hist[i])}- for i in range(len(hist))- ]-- # Rolling metrics (21-day window)- window = min(21, n)- rolling_mean = pd.Series(daily_returns).rolling(window).mean().values- rolling_std = pd.Series(daily_returns).rolling(window).std().values- rolling_sharpe = rolling_mean / (rolling_std + 1e-8) * np.sqrt(252)-- dates_r = dates[1:] if dates and len(dates) > 1 else list(range(n))-- return {- "chart_type": "returns_analysis",- "title": "Returns Analysis",- "distribution": {- "title": "Daily Returns Distribution",- "data": distribution,- "stats": {- "mean_pct": round(float(np.mean(daily_returns)), 4),- "median_pct": round(float(np.median(daily_returns)), 4),- "std_pct": round(float(np.std(daily_returns)), 4),- "skewness": round(float(pd.Series(daily_returns).skew()), 4),- "kurtosis": round(float(pd.Series(daily_returns).kurtosis()), 4),- "best_day_pct": round(float(np.max(daily_returns)), 2),- "worst_day_pct": round(float(np.min(daily_returns)), 2),- "positive_days": int(np.sum(daily_returns > 0)),- "negative_days": int(np.sum(daily_returns < 0)),- },- },- "rolling_sharpe": {- "title": f"Rolling {window}-Day Sharpe Ratio",- "data": [- {"x": str(dates_r[i]), "y": round(float(rolling_sharpe[i]), 2)}- for i in range(n) if not np.isnan(rolling_sharpe[i])- ],- "color": "#00E5FF",- },- "monthly_returns": _calc_monthly_returns(values, dates),- }---def _calc_monthly_returns(values: np.ndarray, dates: list = None) -> dict:- """Calculate monthly return heatmap data."""- if dates is None or len(dates) < 2:- total = (values[-1] / values[0] - 1) * 100 if len(values) > 1 else 0- return {"data": [], "total_return_pct": round(float(total), 2)}-- try:- df = pd.DataFrame({"date": pd.to_datetime(dates[:len(values)]), "value": values})- df.set_index("date", inplace=True)- monthly = df.resample("M").last()- monthly_ret = monthly.pct_change().dropna() * 100-- data = []- for idx, row in monthly_ret.iterrows():- data.append({- "year": idx.year,- "month": idx.month,- "return_pct": round(float(row["value"]), 2),- })- return {"title": "Monthly Returns Heatmap", "data": data}- except Exception:- return {"data": []}---# ============================================================================-# Algorithm Comparison-# ============================================================================--def generate_comparison_chart(results: dict) -> dict:- """Generate comparison data for multiple algorithms."""- series = []- metrics_table = []-- for algo_name, data in results.items():- values = np.array(data["values"], dtype=float)- normalized = (values / values[0] - 1) * 100- dates = data.get("dates", list(range(len(values))))-- series.append({- "name": algo_name.upper(),- "data": [{"x": str(dates[i]), "y": round(float(normalized[i]), 2)}- for i in range(len(values))],- })-- daily_ret = np.diff(values) / values[:-1]- sharpe = np.mean(daily_ret) / (np.std(daily_ret) + 1e-8) * np.sqrt(252)- max_dd = np.min(values / np.maximum.accumulate(values) - 1) * 100-- metrics_table.append({- "algorithm": algo_name.upper(),- "total_return_pct": round(float(normalized[-1]), 2),- "sharpe_ratio": round(float(sharpe), 4),- "max_drawdown_pct": round(float(max_dd), 2),- "final_value": round(float(values[-1]), 2),- })-- # Sort by Sharpe- metrics_table.sort(key=lambda x: x["sharpe_ratio"], reverse=True)-- return {- "chart_type": "comparison",- "title": "Algorithm Comparison",- "series": series,- "metrics_table": metrics_table,- "best_by_return": max(metrics_table, key=lambda x: x["total_return_pct"])["algorithm"],- "best_by_sharpe": metrics_table[0]["algorithm"],- "best_by_drawdown": min(metrics_table, key=lambda x: abs(x["max_drawdown_pct"]))["algorithm"],- }---# ============================================================================-# Full Report-# ============================================================================--def generate_full_report(- values: np.ndarray,- dates: list = None,- algorithm: str = "DRL Agent",- benchmark_values: np.ndarray = None,- actions: np.ndarray = None,- tickers: list = None,-) -> dict:- """Generate a complete performance report with all charts."""- report = {- "generated_at": datetime.now().isoformat(),- "algorithm": algorithm,- "equity_curve": generate_equity_curve(values, dates, benchmark_values),- "drawdown": generate_drawdown_chart(values, dates),- "returns_analysis": generate_returns_analysis(values, dates),- }-- # Performance metrics- daily_ret = np.diff(values) / values[:-1]- report["performance_summary"] = {- "initial_value": round(float(values[0]), 2),- "final_value": round(float(values[-1]), 2),- "total_return_pct": round(float((values[-1] / values[0] - 1) * 100), 2),- "sharpe_ratio": round(float(np.mean(daily_ret) / (np.std(daily_ret) + 1e-8) * np.sqrt(252)), 4),- "max_drawdown_pct": round(float(np.min(values / np.maximum.accumulate(values) - 1) * 100), 2),- "volatility_annual_pct": round(float(np.std(daily_ret) * np.sqrt(252) * 100), 2),- "win_rate_pct": round(float(np.sum(daily_ret > 0) / len(daily_ret) * 100), 2),- "total_trading_days": int(len(daily_ret)),- }-- # Action analysis- if actions is not None:- actions = np.array(actions)- report["action_summary"] = {- "total_actions": int(actions.size),- "buy_actions": int(np.sum(actions > 0)),- "sell_actions": int(np.sum(actions < 0)),- "hold_actions": int(np.sum(actions == 0)),- }-- return report---# ============================================================================-# CLI Entry Point-# ============================================================================--def main():- parser = argparse.ArgumentParser(description="FinRL Visualization & Reporting")- parser.add_argument("--action", required=True,- choices=["equity_curve", "drawdown", "returns_analysis",- "comparison", "full_report"])- parser.add_argument("--data_file", help="CSV with account values")- parser.add_argument("--data_dir", help="Directory with multiple algorithm results")- parser.add_argument("--algorithms", nargs="*", default=["a2c", "ppo", "sac"])- parser.add_argument("--benchmark_file", help="CSV with benchmark values")- parser.add_argument("--algorithm_name", default="DRL Agent")- parser.add_argument("--output", help="Output JSON file path")-- args = parser.parse_args()-- if args.action in ("equity_curve", "drawdown", "returns_analysis", "full_report"):- if not args.data_file or not os.path.exists(args.data_file):- result = {"error": "Provide valid --data_file"}- print(json.dumps(result, indent=2))- return-- values = load_account_values(args.data_file)- df = load_account_df(args.data_file)- dates = df["date"].tolist() if "date" in df.columns else None-- benchmark = None- if args.benchmark_file and os.path.exists(args.benchmark_file):- benchmark = load_account_values(args.benchmark_file)-- if args.action == "equity_curve":- result = generate_equity_curve(values, dates, benchmark)- elif args.action == "drawdown":- result = generate_drawdown_chart(values, dates)- elif args.action == "returns_analysis":- result = generate_returns_analysis(values, dates)- elif args.action == "full_report":- result = generate_full_report(values, dates, args.algorithm_name, benchmark)-- elif args.action == "comparison":- if not args.data_dir or not os.path.exists(args.data_dir):- result = {"error": "Provide valid --data_dir"}- print(json.dumps(result, indent=2))- return-- results = {}- for algo in args.algorithms:- filepath = os.path.join(args.data_dir, f"{algo}_account_values.csv")- if os.path.exists(filepath):- df = pd.read_csv(filepath)- col = "account_value" if "account_value" in df.columns else df.columns[-1]- dates_col = df["date"].tolist() if "date" in df.columns else None- results[algo] = {"values": df[col].tolist(), "dates": dates_col}-- if not results:- result = {"error": f"No result files found in {args.data_dir}"}- else:- result = generate_comparison_chart(results)-- else:- result = {"error": f"Unknown action: {args.action}"}-- output = json.dumps(result, indent=2, default=str)-- if args.output:- os.makedirs(os.path.dirname(args.output) or ".", exist_ok=True)- with open(args.output, "w") as f:- f.write(output)- print(json.dumps({"status": "saved", "output": args.output}))- else:- print(output)---if __name__ == "__main__":- main()diff --git a/fincept-qt/scripts/Analytics/finrl/finrl_portfolio.py b/fincept-qt/scripts/Analytics/finrl/finrl_portfolio.pydeleted file mode 100644index 9f529a90c..000000000--- a/fincept-qt/scripts/Analytics/finrl/finrl_portfolio.py+++ /dev/null@@ -1,466 +0,0 @@-"""-FinRL Portfolio Optimization with DRL-Wrapper around finrl.meta.env_portfolio_allocation, env_portfolio_optimization,-and finrl.agents.portfolio_optimization.--Covers:-- Portfolio allocation environment (softmax weight-based)-- Portfolio optimization environment (advanced, with time windows)-- DRL-based portfolio management training-- Mean-variance comparison-- Portfolio weight analysis-- Rebalancing strategies--Usage (CLI):- python finrl_portfolio.py --action list_modes- python finrl_portfolio.py --action train_allocation --data_file train.csv --tickers AAPL MSFT GOOGL AMZN --algorithm ppo --timesteps 50000- python finrl_portfolio.py --action train_optimization --data_file train.csv --tickers AAPL MSFT GOOGL --time_window 5- python finrl_portfolio.py --action backtest --model_path trained_models/portfolio_ppo.zip --data_file test.csv --tickers AAPL MSFT GOOGL- python finrl_portfolio.py --action analyze_weights --weights_file results/portfolio_weights.csv-"""--import json-import sys-import os-import argparse-import numpy as np-import pandas as pd--from finrl.agents.stablebaselines3.models import DRLAgent-from finrl.config import INDICATORS, TRAINED_MODEL_DIR, RESULTS_DIR-from stable_baselines3.common.vec_env import DummyVecEnv--from finrl_environments import (- create_portfolio_env,- create_portfolio_optimization_env,-)---# ============================================================================-# Portfolio Mode Registry-# ============================================================================--PORTFOLIO_MODES = {- "allocation": {- "description": "Portfolio allocation using softmax-normalized weights",- "action_type": "Continuous weights -> softmax normalized to sum to 1",- "env_class": "StockPortfolioEnv",- "features": [- "Covariance matrix in state",- "Lookback window (default 252 days)",- "Turbulence threshold support",- "Transaction cost support",- ],- },- "optimization": {- "description": "Advanced portfolio optimization with time windows and commission models",- "action_type": "Portfolio weights with commission fee models",- "env_class": "PortfolioOptimizationEnv",- "features": [- "Time window for multi-period state",- "Multiple normalization modes (by_previous_time, by_COLUMN, by_fist_time)",- "Commission fee models (trf = transaction remainder factor, wvm = worth vector model)",- "Custom feature selection (close, high, low, volume, etc.)",- ],- },-}---# ============================================================================-# Training Functions-# ============================================================================--def train_portfolio_allocation(- df: pd.DataFrame,- stock_dim: int = None,- hmax: int = 100,- initial_amount: int = 1_000_000,- transaction_cost_pct: float = 0.001,- reward_scaling: float = 1e-4,- tech_indicator_list: list = None,- turbulence_threshold: float = None,- lookback: int = 252,- algorithm: str = "ppo",- total_timesteps: int = 50_000,- policy: str = "MlpPolicy",- save_path: str = None,- verbose: int = 1,-) -> dict:- """Train a DRL agent for portfolio allocation."""- if tech_indicator_list is None:- tech_indicator_list = list(INDICATORS)- if stock_dim is None:- stock_dim = df["tic"].nunique() if "tic" in df.columns else 1-- # Create environment- env = create_portfolio_env(- df=df,- stock_dim=stock_dim,- hmax=hmax,- initial_amount=initial_amount,- transaction_cost_pct=transaction_cost_pct,- reward_scaling=reward_scaling,- tech_indicator_list=tech_indicator_list,- turbulence_threshold=turbulence_threshold,- lookback=lookback,- )-- # Wrap for SB3- env_train, _ = env.get_sb_env()-- # Train- agent = DRLAgent(env=env_train)- from finrl.agents.stablebaselines3.models import MODEL_KWARGS- model_kwargs = MODEL_KWARGS.get(algorithm.lower(), {})-- model = agent.get_model(- model_name=algorithm.lower(),- policy=policy,- model_kwargs=model_kwargs,- verbose=verbose,- )- trained_model = agent.train_model(- model=model,- tb_log_name=f"portfolio_{algorithm}",- total_timesteps=total_timesteps,- )-- # Save- if save_path is None:- os.makedirs(TRAINED_MODEL_DIR, exist_ok=True)- save_path = os.path.join(TRAINED_MODEL_DIR, f"portfolio_{algorithm}")- trained_model.save(save_path)-- return {- "status": "success",- "mode": "allocation",- "algorithm": algorithm,- "stock_dim": stock_dim,- "lookback": lookback,- "total_timesteps": total_timesteps,- "model_saved_to": save_path,- }---def train_portfolio_optimization(- df: pd.DataFrame,- initial_amount: float = 1_000_000,- features: list = None,- time_window: int = 5,- commission_fee_pct: float = 0.001,- commission_fee_model: str = "trf",- normalize_df: str = "by_previous_time",- reward_scaling: float = 1.0,- algorithm: str = "ppo",- total_timesteps: int = 50_000,- policy: str = "MlpPolicy",- save_path: str = None,- verbose: int = 1,-) -> dict:- """Train a DRL agent for advanced portfolio optimization."""- if features is None:- features = ["close", "high", "low"]-- # Create environment- env = create_portfolio_optimization_env(- df=df,- initial_amount=initial_amount,- features=features,- time_window=time_window,- commission_fee_pct=commission_fee_pct,- commission_fee_model=commission_fee_model,- normalize_df=normalize_df,- reward_scaling=reward_scaling,- )-- # Wrap for SB3- env_train, _ = env.get_sb_env()-- # Train- agent = DRLAgent(env=env_train)- from finrl.agents.stablebaselines3.models import MODEL_KWARGS- model_kwargs = MODEL_KWARGS.get(algorithm.lower(), {})-- model = agent.get_model(- model_name=algorithm.lower(),- policy=policy,- model_kwargs=model_kwargs,- verbose=verbose,- )- trained_model = agent.train_model(- model=model,- tb_log_name=f"portfolio_opt_{algorithm}",- total_timesteps=total_timesteps,- )-- # Save- if save_path is None:- os.makedirs(TRAINED_MODEL_DIR, exist_ok=True)- save_path = os.path.join(TRAINED_MODEL_DIR, f"portfolio_opt_{algorithm}")- trained_model.save(save_path)-- stock_dim = df["tic"].nunique() if "tic" in df.columns else 1-- return {- "status": "success",- "mode": "optimization",- "algorithm": algorithm,- "stock_dim": stock_dim,- "time_window": time_window,- "features": features,- "commission_model": commission_fee_model,- "total_timesteps": total_timesteps,- "model_saved_to": save_path,- }---# ============================================================================-# Backtesting-# ============================================================================--def backtest_portfolio(- model_path: str,- df: pd.DataFrame,- mode: str = "allocation",- algorithm: str = "ppo",- stock_dim: int = None,- initial_amount: int = 1_000_000,- tech_indicator_list: list = None,- features: list = None,- time_window: int = 5,- lookback: int = 252,- deterministic: bool = True,- save_results: bool = True,- results_dir: str = None,- tickers: list = None,-) -> dict:- """Backtest a trained portfolio agent."""- if results_dir is None:- results_dir = RESULTS_DIR- if stock_dim is None:- stock_dim = df["tic"].nunique() if "tic" in df.columns else 1-- if mode == "allocation":- if tech_indicator_list is None:- tech_indicator_list = list(INDICATORS)- env = create_portfolio_env(- df=df, stock_dim=stock_dim,- initial_amount=initial_amount,- tech_indicator_list=tech_indicator_list,- lookback=lookback,- )- else:- if features is None:- features = ["close", "high", "low"]- env = create_portfolio_optimization_env(- df=df, initial_amount=float(initial_amount),- features=features, time_window=time_window,- )-- # Load model- from stable_baselines3 import A2C, PPO, DDPG, SAC, TD3- MODEL_MAP = {"a2c": A2C, "ppo": PPO, "ddpg": DDPG, "sac": SAC, "td3": TD3}- model_class = MODEL_MAP.get(algorithm.lower(), PPO)- model = model_class.load(model_path)-- # Run backtest- obs = env.reset()- account_values = [initial_amount]- all_weights = []-- done = False- while not done:- action, _states = model.predict(obs, deterministic=deterministic)- obs, reward, done, info = env.step(action)-- # Collect weights (softmax of actions)- weights = np.exp(action) / np.sum(np.exp(action))- all_weights.append(weights.tolist() if hasattr(weights, 'tolist') else list(weights))-- # Get account memory- if hasattr(env, 'save_asset_memory'):- df_account = env.save_asset_memory()- if isinstance(df_account, pd.DataFrame) and "account_value" in df_account.columns:- account_values = df_account["account_value"].values.tolist()-- # Metrics- values = np.array(account_values, dtype=float)- total_return = (values[-1] / values[0] - 1) * 100 if len(values) > 1 else 0- daily_returns = np.diff(values) / values[:-1] if len(values) > 1 else np.array([0])- sharpe = np.mean(daily_returns) / (np.std(daily_returns) + 1e-8) * np.sqrt(252)- max_dd = np.min(values / np.maximum.accumulate(values) - 1) * 100 if len(values) > 1 else 0-- result = {- "status": "success",- "mode": mode,- "algorithm": algorithm,- "stock_dim": stock_dim,- "initial_value": float(values[0]),- "final_value": float(values[-1]),- "total_return_pct": round(float(total_return), 4),- "sharpe_ratio": round(float(sharpe), 4),- "max_drawdown_pct": round(float(max_dd), 4),- "num_rebalances": len(all_weights),- }-- # Weight analysis- if all_weights:- weights_arr = np.array(all_weights)- avg_weights = np.mean(weights_arr, axis=0)- ticker_names = tickers[:stock_dim] if tickers else [f"Stock_{i}" for i in range(stock_dim)]- result["average_weights"] = {- ticker_names[i]: round(float(avg_weights[i]), 4)- for i in range(min(len(avg_weights), len(ticker_names)))- }- result["weight_turnover"] = round(float(np.mean(np.sum(np.abs(np.diff(weights_arr, axis=0)), axis=1))), 4) if len(weights_arr) > 1 else 0-- # Save- if save_results:- os.makedirs(results_dir, exist_ok=True)- if all_weights:- pd.DataFrame(all_weights, columns=tickers[:stock_dim] if tickers else None).to_csv(- os.path.join(results_dir, f"portfolio_weights_{algorithm}.csv"), index=False)- result["results_saved_to"] = results_dir-- return result---# ============================================================================-# Weight Analysis-# ============================================================================--def analyze_portfolio_weights(weights_df: pd.DataFrame, tickers: list = None) -> dict:- """Analyze portfolio weight history."""- weights = weights_df.values- num_periods, num_assets = weights.shape-- if tickers is None:- tickers = list(weights_df.columns) if all(not str(c).isdigit() for c in weights_df.columns) else [f"Asset_{i}" for i in range(num_assets)]-- avg_weights = np.mean(weights, axis=0)- std_weights = np.std(weights, axis=0)- min_weights = np.min(weights, axis=0)- max_weights = np.max(weights, axis=0)-- # Concentration: Herfindahl index- hhi = np.mean([np.sum(w ** 2) for w in weights])-- # Turnover- turnover = np.mean(np.sum(np.abs(np.diff(weights, axis=0)), axis=1)) if num_periods > 1 else 0-- per_asset = []- for i in range(num_assets):- per_asset.append({- "ticker": tickers[i] if i < len(tickers) else f"Asset_{i}",- "avg_weight_pct": round(float(avg_weights[i]) * 100, 2),- "std_weight_pct": round(float(std_weights[i]) * 100, 2),- "min_weight_pct": round(float(min_weights[i]) * 100, 2),- "max_weight_pct": round(float(max_weights[i]) * 100, 2),- })-- return {- "num_periods": num_periods,- "num_assets": num_assets,- "avg_concentration_hhi": round(float(hhi), 4),- "effective_num_assets": round(1.0 / (hhi + 1e-8), 2),- "avg_turnover": round(float(turnover), 4),- "per_asset_stats": per_asset,- }---# ============================================================================-# CLI Entry Point-# ============================================================================--def main():- parser = argparse.ArgumentParser(description="FinRL Portfolio Optimization")- parser.add_argument("--action", required=True,- choices=["list_modes", "train_allocation", "train_optimization",- "backtest", "analyze_weights"])- parser.add_argument("--data_file", help="CSV data file")- parser.add_argument("--tickers", nargs="*")- parser.add_argument("--algorithm", default="ppo")- parser.add_argument("--timesteps", type=int, default=50_000)- parser.add_argument("--initial_amount", type=float, default=1_000_000)- parser.add_argument("--lookback", type=int, default=252)- parser.add_argument("--time_window", type=int, default=5)- parser.add_argument("--features", nargs="*", default=["close", "high", "low"])- parser.add_argument("--commission_model", default="trf", choices=["trf", "wvm"])- parser.add_argument("--commission_pct", type=float, default=0.001)- parser.add_argument("--model_path", help="Trained model path")- parser.add_argument("--mode", default="allocation", choices=["allocation", "optimization"])- parser.add_argument("--weights_file", help="CSV with portfolio weights")- parser.add_argument("--save_path")- parser.add_argument("--results_dir", default=RESULTS_DIR)- parser.add_argument("--indicators", nargs="*")-- args = parser.parse_args()-- if args.action == "list_modes":- result = PORTFOLIO_MODES-- elif args.action == "train_allocation":- if not args.data_file or not os.path.exists(args.data_file):- result = {"error": "Provide valid --data_file"}- else:- df = pd.read_csv(args.data_file)- result = train_portfolio_allocation(- df=df, algorithm=args.algorithm,- total_timesteps=args.timesteps,- initial_amount=int(args.initial_amount),- lookback=args.lookback,- tech_indicator_list=args.indicators,- save_path=args.save_path,- )-- elif args.action == "train_optimization":- if not args.data_file or not os.path.exists(args.data_file):- result = {"error": "Provide valid --data_file"}- else:- df = pd.read_csv(args.data_file)- result = train_portfolio_optimization(- df=df, algorithm=args.algorithm,- total_timesteps=args.timesteps,- initial_amount=args.initial_amount,- features=args.features,- time_window=args.time_window,- commission_fee_pct=args.commission_pct,- commission_fee_model=args.commission_model,- save_path=args.save_path,- )-- elif args.action == "backtest":- if not args.model_path or not args.data_file:- result = {"error": "Provide --model_path and --data_file"}- elif not os.path.exists(args.data_file):- result = {"error": f"Data file not found: {args.data_file}"}- else:- df = pd.read_csv(args.data_file)- result = backtest_portfolio(- model_path=args.model_path,- df=df, mode=args.mode,- algorithm=args.algorithm,- initial_amount=int(args.initial_amount),- tech_indicator_list=args.indicators,- features=args.features,- time_window=args.time_window,- lookback=args.lookback,- tickers=args.tickers,- results_dir=args.results_dir,- )-- elif args.action == "analyze_weights":- if not args.weights_file or not os.path.exists(args.weights_file):- result = {"error": "Provide valid --weights_file"}- else:- weights_df = pd.read_csv(args.weights_file)- result = analyze_portfolio_weights(weights_df, args.tickers)-- else:- result = {"error": f"Unknown action: {args.action}"}-- print(json.dumps(result, indent=2, default=str))---if __name__ == "__main__":- main()diff --git a/fincept-qt/scripts/Analytics/finrl/finrl_trading.py b/fincept-qt/scripts/Analytics/finrl/finrl_trading.pydeleted file mode 100644index 587c34e8d..000000000--- a/fincept-qt/scripts/Analytics/finrl/finrl_trading.py+++ /dev/null@@ -1,364 +0,0 @@-"""-FinRL Paper & Live Trading-Wrapper around finrl.trade and finrl.meta.env_stock_trading.env_stock_papertrading.--Covers:-- Alpaca paper trading with trained DRL agents-- Live trading deployment-- Trade mode backtesting (via finrl.trade)-- Crypto trading with CCXT-- Trading session management--Usage (CLI):- python finrl_trading.py --action list_modes- python finrl_trading.py --action paper_trade --algorithm ppo --tickers AAPL MSFT GOOGL --api_key YOUR_KEY --api_secret YOUR_SECRET- python finrl_trading.py --action trade_backtest --algorithm ppo --tickers AAPL MSFT --trade_start 2023-01-01 --trade_end 2023-12-31- python finrl_trading.py --action crypto_config --pairs BTC/USDT ETH/USDT-"""--import json-import sys-import os-import argparse--from finrl.trade import trade as finrl_trade-from finrl.config import (- INDICATORS,- ALPACA_API_KEY, ALPACA_API_SECRET, ALPACA_API_BASE_URL,- TRADE_START_DATE, TRADE_END_DATE,- TRAINED_MODEL_DIR,-)---# ============================================================================-# Trade Mode Registry-# ============================================================================--TRADE_MODES = {- "paper_alpaca": {- "description": "Paper trading on Alpaca Markets",- "requirements": ["Alpaca API key & secret", "Trained model"],- "supported_assets": "US equities",- "market_hours": "9:30 AM - 4:00 PM ET",- "features": [- "Real-time market data via Alpaca API",- "Automatic order submission",- "Position tracking",- "Turbulence-based risk management",- "Latency testing",- ],- },- "backtesting": {- "description": "Trade-mode backtesting with historical data",- "requirements": ["Trained model", "Data source"],- "supported_assets": "Any supported data source",- "features": [- "Replay historical data",- "Evaluate trained models",- "Multiple data sources",- ],- },- "crypto_ccxt": {- "description": "Cryptocurrency trading via CCXT",- "requirements": ["Exchange API credentials", "CCXT library"],- "supported_assets": "100+ crypto exchanges",- "features": [- "24/7 trading",- "Multiple exchanges",- "Sub-minute data",- ],- },-}---# ============================================================================-# Paper Trading-# ============================================================================--def setup_paper_trading(- algorithm: str = "ppo",- tickers: list = None,- time_interval: str = "1Min",- tech_indicator_list: list = None,- turbulence_threshold: float = 30.0,- max_stock: float = 100.0,- api_key: str = None,- api_secret: str = None,- api_base_url: str = None,- model_dir: str = None,- net_dim: int = 512,- drl_lib: str = "stable_baselines3",-) -> dict:- """- Configure Alpaca paper trading session.- Returns configuration dict (does not start trading).- """- if tickers is None:- tickers = ["AAPL", "MSFT", "GOOGL"]- if tech_indicator_list is None:- tech_indicator_list = list(INDICATORS)- if api_key is None:- api_key = ALPACA_API_KEY- if api_secret is None:- api_secret = ALPACA_API_SECRET- if api_base_url is None:- api_base_url = ALPACA_API_BASE_URL- if model_dir is None:- model_dir = TRAINED_MODEL_DIR-- stock_dim = len(tickers)- state_dim = 1 + 2 * stock_dim + len(tech_indicator_list) * stock_dim- action_dim = stock_dim-- config = {- "algorithm": algorithm,- "drl_lib": drl_lib,- "tickers": tickers,- "stock_dim": stock_dim,- "state_dim": state_dim,- "action_dim": action_dim,- "time_interval": time_interval,- "tech_indicators": tech_indicator_list,- "turbulence_threshold": turbulence_threshold,- "max_stock": max_stock,- "net_dim": net_dim,- "model_dir": model_dir,- "api_base_url": api_base_url,- "api_key_configured": bool(api_key and api_key != "xxx"),- "api_secret_configured": bool(api_secret and api_secret != "xxx"),- }-- return {"status": "configured", "config": config}---def start_paper_trading(- algorithm: str = "ppo",- tickers: list = None,- time_interval: str = "1Min",- tech_indicator_list: list = None,- turbulence_threshold: float = 30.0,- max_stock: float = 100.0,- api_key: str = None,- api_secret: str = None,- api_base_url: str = None,- model_dir: str = None,- net_dim: int = 512,- drl_lib: str = "stable_baselines3",-) -> dict:- """- Start Alpaca paper trading with a trained DRL agent.- WARNING: This starts a live trading loop.- """- if tickers is None:- tickers = ["AAPL", "MSFT", "GOOGL"]- if tech_indicator_list is None:- tech_indicator_list = list(INDICATORS)- if api_key is None or api_key == "xxx":- return {"error": "Valid Alpaca API key required. Set --api_key"}- if api_secret is None or api_secret == "xxx":- return {"error": "Valid Alpaca API secret required. Set --api_secret"}- if api_base_url is None:- api_base_url = ALPACA_API_BASE_URL- if model_dir is None:- model_dir = TRAINED_MODEL_DIR-- stock_dim = len(tickers)- state_dim = 1 + 2 * stock_dim + len(tech_indicator_list) * stock_dim- action_dim = stock_dim-- try:- from finrl.meta.env_stock_trading.env_stock_papertrading import AlpacaPaperTrading-- paper_trader = AlpacaPaperTrading(- ticker_list=tickers,- time_interval=time_interval,- drl_lib=drl_lib,- agent=algorithm,- cwd=model_dir,- net_dim=net_dim,- state_dim=state_dim,- action_dim=action_dim,- API_KEY=api_key,- API_SECRET=api_secret,- API_BASE_URL=api_base_url,- tech_indicator_list=tech_indicator_list,- turbulence_thresh=turbulence_threshold,- max_stock=max_stock,- )-- paper_trader.run()-- return {"status": "trading_session_ended"}-- except Exception as e:- return {"error": str(e), "status": "failed"}---# ============================================================================-# Trade-mode Backtesting-# ============================================================================--def trade_backtest(- algorithm: str = "ppo",- tickers: list = None,- trade_start: str = None,- trade_end: str = None,- data_source: str = "yahoofinance",- time_interval: str = "1D",- indicators: list = None,- if_vix: bool = True,- api_key: str = "",- api_secret: str = "",- api_base_url: str = "",- **kwargs,-) -> dict:- """Run trade-mode backtesting via finrl.trade."""- if tickers is None:- tickers = ["AAPL", "MSFT", "GOOGL"]- if indicators is None:- indicators = list(INDICATORS)- if trade_start is None:- trade_start = TRADE_START_DATE- if trade_end is None:- trade_end = TRADE_END_DATE-- from finrl.meta.env_stock_trading.env_stocktrading import StockTradingEnv-- try:- result = finrl_trade(- start_date=trade_start,- end_date=trade_end,- ticker_list=tickers,- data_source=data_source,- time_interval=time_interval,- technical_indicator_list=indicators,- drl_lib="stable_baselines3",- env=StockTradingEnv,- model_name=algorithm.lower(),- API_KEY=api_key,- API_SECRET=api_secret,- API_BASE_URL=api_base_url,- trade_mode="backtesting",- if_vix=if_vix,- **kwargs,- )-- return {- "status": "success",- "algorithm": algorithm,- "trade_period": f"{trade_start} to {trade_end}",- "tickers": tickers,- "data_source": data_source,- }-- except Exception as e:- return {"error": str(e), "status": "failed"}---# ============================================================================-# Crypto Trading Config-# ============================================================================--def build_crypto_config(- pairs: list = None,- exchange: str = "binance",- initial_capital: float = 1_000_000,- lookback: int = 1,- buy_cost_pct: float = 0.001,- sell_cost_pct: float = 0.001,- time_interval: str = "1h",-) -> dict:- """Build configuration for crypto trading environment."""- if pairs is None:- pairs = ["BTC/USDT", "ETH/USDT"]-- return {- "pairs": pairs,- "exchange": exchange,- "num_assets": len(pairs),- "initial_capital": initial_capital,- "lookback": lookback,- "buy_cost_pct": buy_cost_pct,- "sell_cost_pct": sell_cost_pct,- "time_interval": time_interval,- "data_source": "ccxt",- "note": "Use finrl_data.py to download crypto data, then finrl_agents.py to train",- }---# ============================================================================-# CLI Entry Point-# ============================================================================--def main():- parser = argparse.ArgumentParser(description="FinRL Trading Manager")- parser.add_argument("--action", required=True,- choices=["list_modes", "setup_paper", "start_paper",- "trade_backtest", "crypto_config"])- parser.add_argument("--algorithm", default="ppo")- parser.add_argument("--tickers", nargs="*", default=["AAPL", "MSFT", "GOOGL"])- parser.add_argument("--pairs", nargs="*", default=["BTC/USDT", "ETH/USDT"])- parser.add_argument("--time_interval", default="1Min")- parser.add_argument("--api_key", default=None)- parser.add_argument("--api_secret", default=None)- parser.add_argument("--api_base_url", default=ALPACA_API_BASE_URL)- parser.add_argument("--model_dir", default=TRAINED_MODEL_DIR)- parser.add_argument("--trade_start", default=TRADE_START_DATE)- parser.add_argument("--trade_end", default=TRADE_END_DATE)- parser.add_argument("--source", default="yahoofinance")- parser.add_argument("--turbulence_threshold", type=float, default=30.0)- parser.add_argument("--max_stock", type=float, default=100.0)- parser.add_argument("--exchange", default="binance")- parser.add_argument("--initial_capital", type=float, default=1_000_000)- parser.add_argument("--indicators", nargs="*")-- args = parser.parse_args()-- if args.action == "list_modes":- result = TRADE_MODES-- elif args.action == "setup_paper":- result = setup_paper_trading(- algorithm=args.algorithm, tickers=args.tickers,- time_interval=args.time_interval,- tech_indicator_list=args.indicators,- turbulence_threshold=args.turbulence_threshold,- max_stock=args.max_stock,- api_key=args.api_key, api_secret=args.api_secret,- api_base_url=args.api_base_url,- model_dir=args.model_dir,- )-- elif args.action == "start_paper":- result = start_paper_trading(- algorithm=args.algorithm, tickers=args.tickers,- time_interval=args.time_interval,- tech_indicator_list=args.indicators,- turbulence_threshold=args.turbulence_threshold,- max_stock=args.max_stock,- api_key=args.api_key, api_secret=args.api_secret,- api_base_url=args.api_base_url,- model_dir=args.model_dir,- )-- elif args.action == "trade_backtest":- result = trade_backtest(- algorithm=args.algorithm, tickers=args.tickers,- trade_start=args.trade_start, trade_end=args.trade_end,- data_source=args.source, indicators=args.indicators,- )-- elif args.action == "crypto_config":- result = build_crypto_config(- pairs=args.pairs, exchange=args.exchange,- initial_capital=args.initial_capital,- )-- else:- result = {"error": f"Unknown action: {args.action}"}-- print(json.dumps(result, indent=2, default=str))---if __name__ == "__main__":- main()diff --git a/fincept-qt/src/python/PythonSetupManager.cpp b/fincept-qt/src/python/PythonSetupManager.cppindex e361f3526..eb52313e6 100644--- a/fincept-qt/src/python/PythonSetupManager.cpp+++ b/fincept-qt/src/python/PythonSetupManager.cpp@@ -57,8 +57,17 @@ QString PythonSetupManager::base_python_path() const {proc.setProcessEnvironment(env);proc.start(uv_path(), {"python", "find", kPythonVersion});if (proc.waitForFinished(10000) && proc.exitCode() == 0) {- cached_python_path_ = QString::fromUtf8(proc.readAllStandardOutput()).trimmed();- return cached_python_path_;+ // Only accept a path that actually exists. `uv python find` can exit 0+ // while printing nothing usable to stdout; caching that empty string+ // and returning it here skipped the directory scan below and surfaced+ // as a bogus "Python not found after UV install" on a healthy install.+ const QString found = QString::fromUtf8(proc.readAllStandardOutput()).trimmed();+ if (!found.isEmpty() && QFileInfo::exists(found)) {+ cached_python_path_ = found;+ return cached_python_path_;+ }+ LOG_WARN("PythonSetup", QString("uv python find gave an unusable path (%1) — falling back to directory scan")+ .arg(found.isEmpty() ? QStringLiteral("<empty>") : found));}// Fallback: scan known install directory for the cpython-3.11 subdirectory.@@ -399,8 +408,23 @@ void PythonSetupManager::run_setup() {.arg(status.venv_numpy1_ready ? "YES" : "NO").arg(status.venv_numpy2_ready ? "YES" : "NO"));+ // ── Step 0: bail out on hosts we cannot set up at all ────────────────+ // Checked before any download so an impossible platform reports why+ // instead of burning a download and failing at step 2.+ const QString blocked = self->unsupported_platform_reason();+ if (!blocked.isEmpty()) {+ self->emit_progress("python", 0, blocked, true);+ fail(blocked);+ return;+ }+// ── Step 1: Download UV standalone binary (~13MB) ────────────────────- if (!status.uv_installed) {+ // Existence alone is not enough to skip this step — SetupStatus::uv_installed+ // is a bare file check (cheap enough for the fast path). A truncated+ // archive or a lost +x bit from an earlier failed run leaves a file that+ // exists and cannot run, which made every retry skip step 1 and then die+ // at step 2 with "Failed to install Python" instead of re-downloading.+ if (!status.uv_installed || !self->uv_runnable()) {self->emit_progress("uv", 0, "Downloading UV package manager...");if (!self->download_uv()) {self->emit_progress("uv", 0, "Failed to download UV", true);@@ -536,10 +560,48 @@ void PythonSetupManager::run_setup() {});}+QString PythonSetupManager::unsupported_platform_reason() const {+#if defined(__linux__)+ // musl libc (Alpine and friends). uv 0.7.12's download metadata carries+ // gnu / darwin / windows CPython targets only — there is no musl build of+ // 3.11.9 to install — and PyPI has no musl wheels for torch either, so both+ // requirement sets are unresolvable regardless. Detect it up front: failing+ // here beats a 13 MB uv download followed by a bare "Failed to install Python".+ const QDir loader_dir(QStringLiteral("/lib"));+ const QStringList musl_loaders =+ loader_dir.entryList({QStringLiteral("ld-musl-*.so.1")}, QDir::Files | QDir::System);+ if (!musl_loaders.isEmpty()) {+ LOG_ERROR("PythonSetup", "musl libc detected (" + musl_loaders.join(", ") + ") — setup cannot proceed");+ return QStringLiteral("musl libc detected — Fincept Terminal needs a glibc-based Linux. "+ "The Python 3.11 runtime we install is published for glibc only, and "+ "several analytics libraries (PyTorch among them) ship no musl wheels.");+ }+#endif+ return {};+}++bool PythonSetupManager::uv_runnable() const {+ if (!QFileInfo::exists(uv_path()))+ return false;++ QString stderr_out;+ if (run_command_capture(uv_path(), {"--version"}, {}, stderr_out))+ return true;++ LOG_WARN("PythonSetup", QString("UV binary exists but will not run (%1) — re-downloading. Stderr: %2")+ .arg(uv_path(), stderr_out.trimmed().left(200)));+ return false;+}+bool PythonSetupManager::download_uv() {QString dir = install_dir() + "/uv";QDir().mkpath(dir);+ // Clear any leftover from a previous failed run before extracting — the+ // Windows branch below only moves the nested uv.exe up when the target is+ // absent, so a broken file left in place would survive the re-download.+ QFile::remove(uv_path());+// Determine platform-specific archiveQString target;QString ext;diff --git a/fincept-qt/src/python/PythonSetupManager.h b/fincept-qt/src/python/PythonSetupManager.hindex 9a2296fef..74c74b5ca 100644--- a/fincept-qt/src/python/PythonSetupManager.h+++ b/fincept-qt/src/python/PythonSetupManager.h@@ -73,6 +73,14 @@ class PythonSetupManager : public QObject {void emit_progress(const QString& step, int pct, const QString& msg, bool err = false);// Installation steps+ // Non-empty when the host can never complete setup (currently musl libc only) —+ // the string is user-facing. Checked before the first download.+ QString unsupported_platform_reason() const;+ // Spawns `uv --version`. SetupStatus::uv_installed is a bare file-existence+ // check so the fast path stays process-spawn-free; call this before *skipping*+ // the download so a broken leftover binary gets replaced instead of failing+ // the next step.+ bool uv_runnable() const;bool download_uv();bool install_python_via_uv();bool create_venv(const QString& venv_name);