FinceptTerminal is a modern finance application offering advanced market analytics, investment research, and economic data tools, designed for interactive exploration and data-driven decision-making in a user-friendly environment.
C++ ai-agentsalgorithmic-tradingbloomberg-terminalcppfinancefinancial-marketsfintechgood-first-issueinvestmentinvestment-researchmachine-learningopensourcepythonqtquantitative-financestock-markettrading
general updates and bug fixes
tilakpatel22bd5c467c97a4
diff --git a/fincept-qt/docs/DEBUGGING_CRASHES.md b/fincept-qt/docs/DEBUGGING_CRASHES.mddeleted file mode 100644index f28b0e3b5..000000000--- a/fincept-qt/docs/DEBUGGING_CRASHES.md+++ /dev/null@@ -1,144 +0,0 @@-# Debugging long-running crashes on macOS--This page is a runbook for diagnosing the class of crash where the app runs-for hours and then dies inside Qt internals with `EXC_BAD_ACCESS` — typically-inside `doActivate`, `channelReadyRead`, or `qt_mac_socket_callback`. These-are almost always use-after-free / wild-pointer reads against an object that-was destroyed minutes-to-hours earlier; the stack trace shows the *use* site-but tells you nothing about *which allocation went bad*.--The runbook below pinpoints the exact alloc + free pair with full backtraces.-------## TL;DR--```bash-# 1. Launch the signed bundle under macOS malloc instrumentation.-./scripts/diagnose-macos-crash.sh--# 2. Use the app normally until it crashes (could be minutes to hours).--# 3. Open the crash report:-# ~/Library/Logs/DiagnosticReports/FinceptTerminal-*.ips-# Find the faulting address (look for `far:` in the ARM thread state,-# or `Exception Subtype: KERN_INVALID_ADDRESS at 0x…`).-#-# With MallocScribble enabled, a use-after-free shows up as the address-# `0x5555…` or pattern-derived bytes — that's the smoking gun.--# 4. Get the alloc + free backtraces for that address:-malloc_history $(pgrep -n FinceptTerminal) -allEvents 0x<address>-# If the process has already exited, run against the snapshot:-malloc_history ~/Library/Logs/DiagnosticReports/FinceptTerminal-*.ips 0x<address>-```--You now have:-- the line of code that **allocated** the object,-- the line of code that **freed** it,-- the line of code that **tried to use it after free** (from the crash stack).--Root cause is a 3-way grep, not a guess.-------## Why the malloc-debug approach is right here--| Approach | Catches this bug? | Requires rebuild? | Works on signed bundle? |-|---|---|---|---|-| **MallocStackLoggingNoCompact + MallocScribble** | yes (UAF) | no | **yes** |-| AddressSanitizer | yes (UAF + UB) | yes (incl. Qt) | no |-| Crashpad / Sentry telemetry | statistically, across many users | no | yes |-| ThreadSanitizer | only data races | yes | no |-| Static analysis (clang-tidy) | sometimes (patterns) | no | no |-| Code review / guess | no — wastes time | no | no |--`MallocScribble` is the killer feature: every `free()` writes `0x55` over the-returned region, so a subsequent read crashes **at the use site** instead of-2+ hours later when the region happens to be reused for an int. The crash-becomes deterministic in the sense that the bad read becomes immediate.--For details on each env var see Apple's-[Malloc Debug Environment Variables](https://developer.apple.com/library/archive/releasenotes/DeveloperTools/RN-MallocOptions/)-and the-[`malloc_history(1)` man page](https://keith.github.io/xcode-man-pages/malloc_history.1.html).-------## When to reach for AddressSanitizer instead--ASan beats `malloc_history` when:--- You can **reproduce the crash on demand** (under a synthetic stress harness, in- a unit test, etc.).-- You care about **uninitialised reads / heap overflows / stack UAF**, which- `MallocScribble` doesn't cover.--To build it:--```bash-cmake --preset macos-asan-cmake --build build/macos-asan-./scripts/diagnose-macos-crash.sh --asan-```--The `macos-asan` preset is defined in `CMakePresets.json`. It uses-`-fsanitize=address,undefined` and disables unity builds so backtraces are-clean.--> Caveat: ASan only instruments **your** code by default — Qt's frameworks-> are untouched. Most QObject UAFs still get caught (ASan flags the use, even-> if the alloc is in Qt) but bugs inside Qt itself are invisible. For those,-> stay on `malloc_history`.-------## When to ship telemetry instead--If the crash only happens for **specific users** (specific brokers configured,-specific LLM providers, specific OS versions) and you can't repro locally:--- Integrate **Crashpad** (Chromium's crash reporter, used by Slack/Spotify).- ~200 LOC to wire to `QApplication`, uploads to your own backend or- Backtrace/Sentry. After a week of real users you'll have N similar crashes- with full state.--This is more work than the malloc-debug recipe, but it's the right answer-when "use the app for 2 hours then it crashes" can't be reproduced on a-developer machine.-------## Reading the crash report--The `.ips` files under `~/Library/Logs/DiagnosticReports/` are JSON-formatted-crash reports. Useful fields:--```-exception.type: EXC_BAD_ACCESS / EXC_BREAKPOINT / ...-exception.subtype: "KERN_INVALID_ADDRESS at 0x…" ← the faulting address-faultingThread: <index into threads[]>-threads[*].frames: array of {imageOffset, symbol, …} ← per-frame symbols-threadState.far: the address that triggered the page fault on ARM64-```--For the kind of crash this runbook covers:--- `far` will be a small number (`0x3f`, `0x40`, etc.) — that means the object- pointer was 0 and the code read a member at that offset, **or** it'll be- a `0x5555…` pattern — that's `MallocScribble`'s signature confirming the- region was freed before reuse.-- The frame at index 0 is the *use* site. It's the most-recently-executed- code touching the bad pointer, **not** the bug location.-- The bug is at the line that *freed* the object too early. That's what- `malloc_history -allEvents <far>` tells you.-------## Background reading--- [Apple — Enabling the Malloc Debugging Features](https://developer.apple.com/library/archive/documentation/Performance/Conceptual/ManagingMemory/Articles/MallocDebug.html)-- [Apple — Malloc Debug Environment Variables Release Notes](https://developer.apple.com/library/archive/releasenotes/DeveloperTools/RN-MallocOptions/)-- [`malloc_history(1)` man page](https://keith.github.io/xcode-man-pages/malloc_history.1.html)-- [Qt forum — Random crashes on Apple Silicon (Qt 6)](https://forum.qt.io/topic/135220/random-application-crash-on-apple-silicon-m1-qt-6-2-3/42)-- [KDAB — Building Qt with AddressSanitizer](https://www.kdab.com/qt-asan-windows/)diff --git a/fincept-qt/scripts/diagnose-macos-crash.sh b/fincept-qt/scripts/diagnose-macos-crash.shdeleted file mode 100755index 5a6b02b26..000000000--- a/fincept-qt/scripts/diagnose-macos-crash.sh+++ /dev/null@@ -1,143 +0,0 @@-#!/usr/bin/env bash-# diagnose-macos-crash.sh — launch FinceptTerminal under macOS' malloc-# debugging instrumentation so a use-after-free / wild-pointer crash-# fails LOUDLY at the use site (not 2+ hours later when the freed-# region happens to get reused).-#-# Usage:-# scripts/diagnose-macos-crash.sh # signed bundle from build/macos-release-# scripts/diagnose-macos-crash.sh path/to/Fincept.app # any bundle-# scripts/diagnose-macos-crash.sh --asan # build/macos-asan binary-#-# When the app crashes:-# 1. Note the crash address (often shown as `far=0x…` or in x-registers).-# 2. If the address is `0x55…55` — that's MallocScribble's signature.-# The freed region was written before reuse → confirmed UAF.-# 3. Run:-# malloc_history $(pgrep FinceptTerminal) -allEvents <addr>-# (or pass a memgraph file if the app already exited).-# 4. The output shows the EXACT alloc backtrace + free backtrace-# for that address. Root cause becomes a single grep.-#-# References:-# - https://developer.apple.com/library/archive/documentation/Performance/Conceptual/ManagingMemory/Articles/MallocDebug.html-# - https://keith.github.io/xcode-man-pages/malloc_history.1.html-# - https://forum.qt.io/topic/135220/random-application-crash-on-apple-silicon-m1-qt-6-2-3/42--set -euo pipefail--# ── Locate the .app bundle ────────────────────────────────────────────────────-SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"-REPO_DIR="$(cd "$SCRIPT_DIR/.." && pwd)"--MODE="signed"-TARGET=""-while [[ $# -gt 0 ]]; do- case "$1" in- --asan)- MODE="asan"- shift- ;;- --help|-h)- sed -n '2,/^set -e/p' "$0" | sed 's/^# \{0,1\}//'- exit 0- ;;- *)- TARGET="$1"- shift- ;;- esac-done--if [[ -z "${TARGET}" ]]; then- if [[ "${MODE}" == "asan" ]]; then- TARGET="${REPO_DIR}/build/macos-asan/FinceptTerminal.app"- else- TARGET="${REPO_DIR}/build/macos-release/FinceptTerminal.app"- fi-fi--if [[ ! -d "${TARGET}" ]]; then- echo "error: bundle not found at ${TARGET}" >&2- echo " build it first:" >&2- if [[ "${MODE}" == "asan" ]]; then- echo " cmake --preset macos-asan && cmake --build build/macos-asan" >&2- else- echo " cmake --preset macos-release && cmake --build build/macos-release" >&2- fi- exit 1-fi--BIN="${TARGET}/Contents/MacOS/FinceptTerminal"-if [[ ! -x "${BIN}" ]]; then- echo "error: executable missing or not executable: ${BIN}" >&2- exit 1-fi--# ── Malloc instrumentation (works on signed bundles too) ──────────────────────-# MallocStackLoggingNoCompact: record every alloc + free with full backtrace.-# MallocScribble: fill freed memory with 0x55 → crash at use site.-# MallocGuardEdges: place guard pages around allocations.-# MallocCheckHeapEach: periodic heap consistency check (slows runtime).-#-# These work on shipped, signed, notarised apps — no rebuild required.-# Performance cost: app runs ~2-5× slower; memory usage ~2-3× higher.-# That's fine for a diagnostic session; it's not a release config.-export MallocStackLoggingNoCompact=1-export MallocScribble=1-export MallocGuardEdges=1-export MallocErrorAbort=1-export MallocCheckHeapEach=10000--# ── Qt logging — surface socket / network / TLS lifecycle ─────────────────────-# These categories print to stderr; redirect to a file via the wrapper below.-# Disabled by default in Release builds; explicit rules re-enable them.-export QT_LOGGING_RULES="qt.network.*=true;qt.qpa.events=false;qt.qpa.input.events=false"-export QT_FATAL_WARNINGS=0 # set to 1 to abort on first Qt warning (sometimes useful)--# ── ASan options if running an ASan build ─────────────────────────────────────-if [[ "${MODE}" == "asan" ]]; then- export ASAN_OPTIONS="detect_leaks=0:halt_on_error=1:abort_on_error=1:print_stacktrace=1:strict_string_checks=1:detect_stack_use_after_return=1:check_initialization_order=1:strict_init_order=1"- export UBSAN_OPTIONS="halt_on_error=1:abort_on_error=1:print_stacktrace=1"-fi--# ── Launch ────────────────────────────────────────────────────────────────────-LOG_DIR="${REPO_DIR}/diagnostic-logs"-mkdir -p "${LOG_DIR}"-STAMP="$(date +%Y%m%d-%H%M%S)"-LOG_FILE="${LOG_DIR}/fincept-${MODE}-${STAMP}.log"--echo "──────────────────────────────────────────────────────────────────────"-echo " Diagnostic launch: ${MODE}"-echo " Binary: ${BIN}"-echo " Log file: ${LOG_FILE}"-echo ""-echo " When it crashes:"-echo " 1. Note the crash address (look for far= or x19/x20 in crash log)"-echo " 2. Run: malloc_history \$(pgrep FinceptTerminal) -allEvents <addr>"-echo " (or post-mortem on the ips file under ~/Library/Logs/DiagnosticReports/)"-echo " 3. The alloc + free stacks pinpoint the leaking owner."-echo "──────────────────────────────────────────────────────────────────────"--# Use `tee` so we can both watch the run live AND keep a permanent log.-# `script` would capture pty colors but we want plain text for grep.-"${BIN}" 2>&1 | tee "${LOG_FILE}"-EXIT_CODE=${PIPESTATUS[0]}--if [[ "${EXIT_CODE}" -ne 0 ]]; then- echo ""- echo "── Process exited with code ${EXIT_CODE} ──────────────────────────────"- echo "Log saved to: ${LOG_FILE}"- # If the crash registered with ReportCrash, point at it.- LATEST_IPS="$(ls -t ~/Library/Logs/DiagnosticReports/FinceptTerminal-*.ips 2>/dev/null | head -1 || true)"- if [[ -n "${LATEST_IPS}" ]]; then- echo "Latest crash report: ${LATEST_IPS}"- echo ""- echo "Faulting addresses (from crash report):"- # 'far' is the Translation-fault target address on ARM64; x19/x20 are- # commonly the stale-pointer holders in doActivate-style crashes.- grep -E 'far:|x19:|x20:' "${LATEST_IPS}" | head -5 || true- fi-fi--exit "${EXIT_CODE}"