"Vibe-Trading: Your Personal Trading Agent"
Python ai-agentalgorithmic-tradingbacktestingfintechllmmcpmulti-agentpythonquantitative-financetrading
feat(tools): add cross-market ETF holdings look-through
Haozhe Wubbff62b345b8
diff --git a/agent/src/tools/etf_holdings_tool.py b/agent/src/tools/etf_holdings_tool.pynew file mode 100644index 000000000..34654b1c6--- /dev/null+++ b/agent/src/tools/etf_holdings_tool.py@@ -0,0 +1,1845 @@+"""Read-only ETF look-through: fund search + constituent holdings, US and A-share.++An ETF price series says nothing about what the fund actually owns. This tool+answers that second question on both sides of the Pacific from free, no-auth+sources, and — because every ETF holdings disclosure is *stale by construction*+— stamps every answer with the report period it belongs to.++United States — SEC N-PORT+--------------------------+Registered funds file Form NPORT-P, whose ``primary_doc.xml`` carries the+**entire** portfolio (``invstOrSecs/invstOrSec``) plus total/net assets. Three+verified facts drive the US path:++* ``company_tickers.json`` (used by :mod:`backtest.loaders.sec_edgar_client`)+ does **not** list most ETFs — IVV, VOO, ARKK and XLK are all absent, because+ the reporting entity is the trust, not the fund. The ticker index that does+ cover them is the Investment Company Series and Class file, which maps+ ``Class Ticker -> (CIK, Series ID, Series Name, Entity Name)``.+* A trust files one NPORT-P per series per period (iShares Trust files+ hundreds), so the filing list must be filtered by ``Series ID``. The only+ endpoint that does that is the EDGAR browse CGI in ``output=atom`` mode.+* ``repPdEnd`` is the fund's **fiscal year end**, not the report period.+ IVV's amendment ``0002071691-26-015790`` carries ``repPdEnd`` 2026-03-31 with+ ``repPdDate`` 2025-09-30, and EDGAR's own "Period of Report" for it reads+ 2025-09-30. ``repPdDate`` is therefore the as-of date; reading ``repPdEnd``+ would have stamped September holdings with a March date.+* The newest *filed* document is not the newest *period*: that same amendment+ was filed 2026-07-13, two months after the 2026-03-31 report. Filings are+ therefore ranked by their indexed period, not by filing date.+* Disclosure lags: IVV's period ending 2026-03-31 was filed 2026-05-28.++China A-share — Eastmoney fund archives+---------------------------------------+``fundf10.eastmoney.com/FundArchivesDatas.aspx?type=jjcc`` returns the+stock-holding detail as a JS ``var apidata={content:"<html>",...}`` envelope.+Traps, all confirmed against the live endpoint:++* the request 404s without a ``Referer`` on ``fundf10.eastmoney.com``;+* an out-of-range ``year`` is **silently ignored** and the current year is+ returned instead, so the report period is always read back out of the payload+ and never echoed from the request;+* ``topline`` does nothing on its own. The parameter that unlocks the whole book+ is ``month``: the page's own "显示全部持仓明细" control calls+ ``LoadMore(this, <month>, LoadStockPos)``, which appends that month to+ ``params.month``. ``month=6,12`` expands the June and December periods,+ ``month=3,6,9,12`` expands all four.++Chinese funds disclose the **complete** portfolio in the interim (半年度/中期,+period end 06-30) and annual (年度, period end 12-31) reports, and only the top+holdings in the quarterly ones. Measured on the expanded June/December periods:+510300 → 342 holdings summing 98.66% of net assets, 510500 → 541 / 95.35%,+159915 → 118 / 99.96%. That is the full book, not a top-N slice.++Two things make the "is this the whole book?" question harder than reading the+month off the period end, and both are handled rather than assumed:++* **Row provenance.** Eastmoney merges a second source into the table and says+ so in its own footnote: "注:加*号代表进入上市公司的十大流通股东却没有进入单只+ 基金前十大重仓股的个股" — a starred ``序号`` means the row came from the+ *issuer's* top-10 float-holder disclosure, not from the fund's report. On an+ expanded quarterly period 510050 returns 10 fund-reported rows plus 33 starred+ ones. Those rows are real disclosure, but they are a different filing with+ different coverage, so every holding carries ``disclosure_source`` and the+ disclosed-percentage total counts fund-reported rows only.+* **Publication lag.** A period ending 06-30 is served long before its interim+ report exists — until then Eastmoney answers it with the quarterly data. On+ 2026-08-04 the expanded 2026-06-30 period for 510300 is 15 rows / 23.26%+ (top-10 index investment + top-5 active), because the interim report is not+ due until 08-31. ``api.fund.eastmoney.com/f10/JJGG?type=3`` lists the fund's+ periodic reports by title ("…2025年年度报告", "…2025年中期报告"), which+ settles authoritatively whether the full portfolio has been published and when.+ That endpoint also needs a ``Referer``; without one it answers ``ErrCode -999``+ with ``Data`` as an empty **string** rather than a list.++Coverage is therefore decided from evidence, never from the calendar alone, and+is deliberately conservative — see :func:`_annotate_cn_period`. Two candidate+signals were tested and rejected: the ``t2`` table class tracks the requested+year rather than detail availability, and a disclosed-percentage threshold+cannot work because a concentrated fund's top-10 alone reaches 91.5% (513050).++Everything routes through :mod:`backtest.loaders._http` so the SEC and Eastmoney+per-host throttles, sessions and User-Agent policy are the project's existing+ones, not a second stack. No optional package is required.+"""++from __future__ import annotations++import csv+import io+import json+import logging+import re+import threading+from datetime import date, datetime, timezone+from typing import Any++from defusedxml import ElementTree as DefusedET++from backtest.loaders import sec_edgar_client+from backtest.loaders._http import resolve_min_interval, throttled_get+from backtest.loaders.eastmoney_client import get_json, resolve_secid+from src.agent.tools import BaseTool+from src.tools._result_paging import fit_records++logger = logging.getLogger(__name__)++# ── SEC endpoints ────────────────────────────────────────────────────────────++# Ticker -> (CIK, Series ID, Series Name) for every registered fund share class.+# Published per calendar year; the current year is tried first and the previous+# year is the fallback for the window early in January before it is posted.+_SEC_SERIES_INDEX_URL = (+ "https://www.sec.gov/files/investment/data/other/"+ "investment-company-series-class-information/"+ "investment-company-series-class-{year}.csv"+)+# The only EDGAR index that can be filtered to a single fund series.+_SEC_BROWSE_URL = "https://www.sec.gov/cgi-bin/browse-edgar"+_SEC_ARCHIVES_BASE = "https://www.sec.gov/Archives/edgar/data"+# Full-text search: the one endpoint that reports each accession's period of+# report in bulk, so the newest *period* can be picked without downloading a+# half-megabyte filing per candidate.+_SEC_FTS_URL = "https://efts.sec.gov/LATEST/search-index"++_NPORT_HOST_KEY = "sec"++# ── Eastmoney endpoints ──────────────────────────────────────────────────────++_EM_HOLDINGS_URL = "https://fundf10.eastmoney.com/FundArchivesDatas.aspx"+# Periodic-report announcement shelf (``type``/``NEWCATEGORY`` 3 = 定期报告).+# The one authority on whether a period's full portfolio has been published.+_EM_ANNOUNCEMENT_URL = "https://api.fund.eastmoney.com/f10/JJGG"+_EM_PERIODIC_REPORT_TYPE = "3"+# A fund files four periodic reports a year; 80 covers two decades.+_EM_ANNOUNCEMENT_PAGE_SIZE = 80+# Quote hosts in preference order. ``push2delay`` is tried first because the+# other edges intermittently drop a Python HTTP client before responding+# (measured 0/4 on ``push2`` and ``push2his`` against 4/4 on ``push2delay``,+# while curl reached all three) — the delayed edge is also the one AKShare+# itself uses for the fund list, and delayed quotes are irrelevant here since+# only the fund's name and size are read.+_EM_QUOTE_HOSTS = ("push2delay.eastmoney.com", "push2.eastmoney.com")+_EM_QUOTE_PATH = "/api/qt/stock/get"+_EM_LIST_PATH = "/api/qt/clist/get"+_EM_HOST_KEY = "eastmoney"+_EM_MIN_INTERVAL_ENV = "VIBE_TRADING_EASTMONEY_MIN_INTERVAL"+_EM_DEFAULT_MIN_INTERVAL = 1.0++# Eastmoney board filters for exchange-traded funds, and the per-page cap the+# server enforces (``pz`` above 100 is silently clamped to 100).+_EM_ETF_BOARDS = "b:MK0021,b:MK0022,b:MK0023,b:MK0024,b:MK0827"+_EM_PAGE_SIZE = 100+_EM_MAX_PAGES = 30++# Exchange-listed fund code prefixes, mirroring+# ``backtest.loaders._symbol_utils._ETF_PREFIXES`` but resolved to the exchange+# so a bare six-digit code can be addressed without a suffix.+_CN_PREFIX_EXCHANGE = {+ "15": "SZ", "16": "SZ",+ "50": "SH", "51": "SH", "52": "SH", "56": "SH", "58": "SH",+}++# ── Result shaping ───────────────────────────────────────────────────────────++# The ceiling has to clear a real full portfolio, or ``disclosure='full'`` hands+# back a truncated book while still calling itself complete. Measured on the+# 2025 annual reports: 510500 → 541, 512100 → 1041, 159845 → 1044, and the+# CSI-2000 tracker 159531 → 2031. A broad-market fund is therefore well past a+# thousand names, so the cap is set high enough that no measured A-share+# portfolio reaches it; ``fit_records`` still pages the answer down to one+# result's worth, so a large cap costs nothing per response.+_MAX_TOP_N = 6000+_DEFAULT_TOP_N = 25+_MAX_LOOKUP = 40+_DEFAULT_LOOKUP = 10+# NPORT-P filings are quarterly; ten entries is ~2.5 years of report periods.+_SEC_FILING_COUNT = 10++_MODES = ("lookup", "holdings")+_MARKETS = ("auto", "US", "CN")+_DISCLOSURES = ("latest", "full", "cross_reference")++# ``month`` presets for the A-share archive. The default expands only the two+# periods whose reports carry the complete portfolio; the wide one additionally+# expands the quarterlies, which is where the issuer cross-referenced rows live.+_EM_FULL_DISCLOSURE_MONTHS = "6,12"+_EM_ALL_QUARTER_MONTHS = "3,6,9,12"++# Period ends whose report — when published — discloses the complete portfolio.+_FULL_DISCLOSURE_PERIOD_ENDS = ("-06-30", "-12-31")++# A quarterly report discloses the top ten holdings, plus for an index fund the+# top five active-investment holdings. More fund-reported rows than that on a+# June/December period can only have come from the interim or annual report.+_QUARTERLY_DISCLOSURE_CAP = 15++# How many extra years to fetch when hunting backwards for a full portfolio.+_MAX_FULL_LOOKBACK_FETCHES = 3++# Row provenance, per Eastmoney's own footnote on the starred 序号.+_SOURCE_FUND_REPORT = "fund_report"+_SOURCE_ISSUER_CROSS_REF = "issuer_top10_float_holders"++_COVERAGE_FULL = "full_portfolio"+_COVERAGE_TOP_N = "top_n_disclosed"++# CJK unified ideographs — a lookup query containing one is a Chinese fund name.+_HAN_RE = re.compile(r"[一-鿿]")++# Neither source publishes an expense ratio, so it is declared missing rather+# than estimated. Fund size is only available on the paths noted below.+_NOTE_LAG = (+ "ETF holdings are disclosed with a lag. 'as_of' is the report period the "+ "holdings belong to, NOT today's portfolio."+)+++def _error(message: str) -> str:+ """Build the failure envelope as a JSON string.++ Args:+ message: Human-readable error description.++ Returns:+ A ``{"ok": false, "error": ...}`` JSON string.+ """+ return json.dumps({"ok": False, "error": message}, ensure_ascii=False)+++# ── Symbol / market classification ───────────────────────────────────────────+++def _classify_market(symbol: str) -> str:+ """Route a symbol to the A-share or the US path.++ The repository's shared classifier+ (``backtest.engines._market_hooks._detect_market``) is deliberately not used+ here: it defaults an unrecognized symbol to ``a_share``, which would send a+ bare ``IVV`` down the Eastmoney path. This tool needs the opposite default.++ Args:+ symbol: An ETF symbol such as ``"510050.SH"``, ``"510050"`` or ``"IVV"``.++ Returns:+ ``"CN"`` for a ``.SH`` / ``.SZ`` suffix or a bare six-digit code,+ ``"US"`` otherwise.+ """+ upper = symbol.strip().upper()+ if upper.endswith((".SH", ".SZ")):+ return "CN"+ if len(upper) == 6 and upper.isdigit():+ return "CN"+ return "US"+++def _classify_query_market(query: str) -> str:+ """Route a free-text lookup query to the A-share or the US path.++ Adds one rule to :func:`_classify_market`: a query containing Han+ characters is an A-share fund search, since the SEC index holds no Chinese+ fund names and would silently answer "no matches".++ Args:+ query: A ticker, fund name, or theme keyword.++ Returns:+ ``"CN"`` or ``"US"``.+ """+ if _HAN_RE.search(query):+ return "CN"+ return _classify_market(query)+++def _cn_code_and_secid(symbol: str) -> tuple[str, str | None]:+ """Split an A-share fund symbol into its bare code and Eastmoney secid.++ A bare six-digit code carries no exchange, so the exchange is derived from+ the listed-fund prefix table before delegating to the shared Eastmoney+ resolver.++ Args:+ symbol: ``"510050.SH"``, ``"510050"`` or ``"159915.sz"``.++ Returns:+ ``(bare_code, secid)``; ``secid`` is ``None`` when the exchange cannot+ be determined.+ """+ upper = symbol.strip().upper()+ if upper.endswith((".SH", ".SZ")):+ return upper.split(".")[0], resolve_secid(upper)+ exchange = _CN_PREFIX_EXCHANGE.get(upper[:2])+ if exchange is None:+ return upper, None+ return upper, resolve_secid(f"{upper}.{exchange}")+++# ── HTTP transport (reuses the shared per-host throttles) ────────────────────+++def _sec_get_text(url: str, params: dict[str, Any] | None = None) -> str:+ """GET an SEC URL as text through the shared ``"sec"`` throttle bucket.++ The frozen SEC client only decodes JSON, while both endpoints this tool+ needs return XML/CSV, so the transport is reused rather than reimplemented:+ same bucket, same interval floor, same contact User-Agent (env-overridable+ via ``VIBE_TRADING_SEC_UA``).++ Args:+ url: Fully-qualified ``sec.gov`` URL.+ params: Optional query parameters.++ Returns:+ The decoded response body.++ Raises:+ requests.RequestException: Network failure or non-2xx status.+ """+ response = throttled_get(+ url,+ host_key=_NPORT_HOST_KEY,+ min_interval=sec_edgar_client._min_interval(),+ params=params,+ headers={"User-Agent": sec_edgar_client._user_agent()},+ timeout=60.0,+ )+ response.raise_for_status()+ return response.text+++def _em_get_text(url: str, *, params: dict[str, Any], referer: str) -> str:+ """GET an Eastmoney URL as text through the shared ``"eastmoney"`` throttle.++ Args:+ url: Fully-qualified Eastmoney URL.+ params: Query parameters.+ referer: Referer header — ``fundf10`` returns 404 without one.++ Returns:+ The decoded response body.++ Raises:+ requests.RequestException: Network failure or non-2xx status.+ """+ response = throttled_get(+ url,+ host_key=_EM_HOST_KEY,+ min_interval=resolve_min_interval(_EM_MIN_INTERVAL_ENV, _EM_DEFAULT_MIN_INTERVAL),+ params=params,+ headers={"Referer": referer},+ )+ response.raise_for_status()+ response.encoding = "utf-8"+ return response.text+++# ── US: series/class ticker index ────────────────────────────────────────────++_US_INDEX_CACHE: tuple[int, list[dict[str, Any]]] | None = None+_US_INDEX_LOCK = threading.Lock()+++def _parse_series_index(body: str) -> list[dict[str, Any]]:+ """Parse the SEC series/class CSV into ticker-bearing fund records.++ The header carries a UTF-8 BOM and the columns are ``Reporting File+ Number``, ``CIK Number``, ``Entity Name``, ``Entity Org Type``, ``Series+ ID``, ``Series Name``, ``Class ID``, ``Class Name``, ``Class Ticker`` and+ address fields. Rows without a ticker are share classes that cannot be+ searched by symbol and are dropped.++ A row carrying more fields than the header lands under ``DictReader``'s+ rest-key as a *list*, so values are type-checked rather than assumed to be+ strings — one malformed row would otherwise take the whole US path down.++ Args:+ body: Decoded CSV text.++ Returns:+ A list of ``{ticker, cik, series_id, series_name, class_id, class_name,+ entity_name, file_number}`` dicts.+ """+ rows: list[dict[str, Any]] = []+ for raw in csv.DictReader(io.StringIO(body)):+ record = {+ (k or "").lstrip("").strip(): (v.strip() if isinstance(v, str) else "")+ for k, v in raw.items()+ }+ ticker = record.get("Class Ticker", "")+ series_id = record.get("Series ID", "")+ if not ticker or not series_id:+ continue+ rows.append(+ {+ "ticker": ticker.upper(),+ "cik": record.get("CIK Number") or None,+ "series_id": series_id,+ "series_name": record.get("Series Name") or None,+ "class_id": record.get("Class ID") or None,+ # Vanguard's ETF share classes sit inside a mutual fund whose+ # series name says "Fund" (VOO -> "Vanguard 500 Index Fund"),+ # so the class name is the only ETF signal on those rows.+ "class_name": record.get("Class Name") or None,+ "entity_name": record.get("Entity Name") or None,+ "file_number": record.get("Reporting File Number") or None,+ }+ )+ return rows+++def _us_series_index() -> tuple[int, list[dict[str, Any]]]:+ """Return ``(index_year, records)`` for the SEC series/class index, memoized.++ Returns:+ The calendar year of the file actually used and its parsed records.++ Raises:+ RuntimeError: When neither the current nor the previous year's file can+ be fetched.+ """+ global _US_INDEX_CACHE+ if _US_INDEX_CACHE is not None:+ return _US_INDEX_CACHE+ with _US_INDEX_LOCK:+ if _US_INDEX_CACHE is None:+ this_year = date.today().year+ failures: list[str] = []+ for year in (this_year, this_year - 1):+ try:+ body = _sec_get_text(_SEC_SERIES_INDEX_URL.format(year=year))+ except Exception as exc: # noqa: BLE001 - try the older vintage+ failures.append(f"{year}: {exc}")+ continue+ records = _parse_series_index(body)+ if records:+ _US_INDEX_CACHE = (year, records)+ break+ failures.append(f"{year}: index parsed to zero rows")+ else:+ raise RuntimeError(+ "SEC investment-company series/class index unavailable ("+ + "; ".join(failures)+ + ")"+ )+ return _US_INDEX_CACHE+++def _is_etf_class(record: dict[str, Any]) -> bool:+ """Report whether an index row looks like an exchange-traded share class.++ The SEC series/class file has no ETF flag, so the only available signal is+ the wording: an ETF is named one either in its series name (``iShares+ Semiconductor ETF``) or in its class name (VOO is ``ETF Shares`` of the+ ``Vanguard 500 Index Fund``). This is a ranking heuristic only — nothing is+ dropped on the strength of it, and the underlying names are returned so the+ caller can judge for itself.++ Args:+ record: One parsed index row.++ Returns:+ ``True`` when either name carries the word "ETF".+ """+ return "ETF" in f"{record.get('series_name') or ''} {record.get('class_name') or ''}".upper()+++def _us_match(query: str, limit: int) -> list[dict[str, Any]]:+ """Find fund share classes by exact ticker, then by name substring.++ A theme query such as "semiconductor" matches far more mutual fund share+ classes than ETFs, and the index is in filer order, so an unranked answer+ fills the whole first page with non-ETF classes and hides SOXX/SMH/XSD+ behind it. Name matches are therefore ordered ETFs-first, index order kept+ within each group. Exact ticker hits still outrank everything.++ Args:+ query: A ticker, a fund name, or a theme keyword.+ limit: Maximum matches to return.++ Returns:+ Matching index records, exact ticker hits first and ETF share classes+ ahead of other classes.+ """+ _, records = _us_series_index()+ needle = query.strip().upper()+ exact = [r for r in records if r["ticker"] == needle]+ if len(exact) >= limit:+ return exact[:limit]+ seen = {id(r) for r in exact}+ partial = [+ r+ for r in records+ if id(r) not in seen+ and (+ needle in (r["series_name"] or "").upper()+ or needle in (r["entity_name"] or "").upper()+ or (len(needle) >= 2 and r["ticker"].startswith(needle))+ )+ ]+ partial.sort(key=lambda r: not _is_etf_class(r))+ return (exact + partial)[:limit]+++# ── US: N-PORT filing index + holdings ───────────────────────────────────────+++def _local_name(tag: str) -> str:+ """Strip the XML namespace from a tag, returning its local name."""+ return tag.rpartition("}")[2]+++def _us_nport_filings(series_id: str) -> list[dict[str, Any]]:+ """List a fund series' NPORT-P filings, newest first.++ Args:+ series_id: SEC series identifier, e.g. ``"S000004310"``.++ Returns:+ A list of ``{form, accession, filing_date}`` dicts. Empty when the+ series has filed none.++ Raises:+ requests.RequestException: Network failure or non-2xx status.+ """+ body = _sec_get_text(+ _SEC_BROWSE_URL,+ params={+ "action": "getcompany",+ "CIK": series_id,+ "type": "NPORT-P",+ "dateb": "",+ "owner": "include",+ "count": str(_SEC_FILING_COUNT),+ "output": "atom",+ },+ )+ try:+ root = DefusedET.fromstring(body.encode("utf-8", "replace"))+ except Exception as exc: # noqa: BLE001 - a malformed feed is a data error+ raise RuntimeError(f"EDGAR filing feed did not parse: {exc}") from exc++ filings: list[dict[str, Any]] = []+ for element in root.iter():+ if _local_name(element.tag) != "content":+ continue+ fields = {_local_name(c.tag): (c.text or "").strip() for c in element}+ accession = fields.get("accession-number")+ if not accession:+ continue+ filings.append(+ {+ "form": fields.get("filing-type") or None,+ "accession": accession,+ "filing_date": fields.get("filing-date") or None,+ }+ )+ return filings+++def _us_indexed_periods(series_id: str) -> dict[str, str]:+ """Map accession -> period of report for a series, from EDGAR full-text search.++ The atom filing index carries no period, and the newest-filed NPORT-P is+ routinely an amendment of an older period, so the periods are read from the+ full-text index in one request instead of by downloading each candidate+ filing. This is a best-effort enrichment: a failure returns an empty map and+ the caller falls back to filing order.++ Args:+ series_id: SEC series identifier, e.g. ``"S000004310"``.++ Returns:+ Mapping of accession number to ``YYYY-MM-DD`` period of report.+ """+ try:+ body = _sec_get_text(_SEC_FTS_URL, params={"q": f'"{series_id}"', "forms": "NPORT-P"})+ payload = json.loads(body)+ except Exception as exc: # noqa: BLE001 - enrichment only, never fatal+ logger.warning("EDGAR full-text period lookup failed for %s: %s", series_id, exc)+ return {}++ hits = ((payload.get("hits") or {}).get("hits") or []) if isinstance(payload, dict) else []+ periods: dict[str, str] = {}+ for hit in hits:+ if not isinstance(hit, dict):+ continue+ accession = str(hit.get("_id") or "").split(":", 1)[0]+ period = (hit.get("_source") or {}).get("period_ending")+ if accession and period:+ periods[accession] = str(period)+ return periods+++def _select_nport_filing(filings: list[dict[str, Any]], periods: dict[str, str]) -> dict[str, Any]:+ """Pick the filing covering the newest report period, newest vintage first.++ Args:+ filings: Atom-derived filings for one series, newest filed first.+ periods: Accession -> period of report, possibly empty or partial.++ Returns:+ The chosen filing dict, annotated with ``indexed_period``,+ ``period_source`` ("edgar_full_text_index" when a period was known,+ "filing_order" when the newest-filed had to be assumed) and+ ``candidates_without_period``, the number of listed filings the index+ could not date.+ """+ dated = [f for f in filings if periods.get(f["accession"])]+ undated = len(filings) - len(dated)+ if not dated:+ chosen = dict(filings[0])+ chosen["indexed_period"] = None+ chosen["period_source"] = "filing_order"+ chosen["candidates_without_period"] = undated+ return chosen+ # Newest period wins; among filings for the same period the latest vintage+ # wins, so an amendment supersedes the original it corrects.+ chosen = dict(+ max(dated, key=lambda f: (periods[f["accession"]], f["filing_date"] or ""))+ )+ chosen["indexed_period"] = periods[chosen["accession"]]+ chosen["period_source"] = "edgar_full_text_index"+ # The full-text index lags EDGAR's own filing feed, so a just-filed report+ # can be listed with no period yet. Such filings are skipped by the ranking+ # above, which would otherwise quietly answer with the previous quarter —+ # the count is surfaced so a stale answer is visibly stale.+ chosen["candidates_without_period"] = undated+ return chosen+++def _nport_document_url(cik: str, accession: str) -> str:+ """Build the ``primary_doc.xml`` URL for one N-PORT accession.++ Args:+ cik: The registrant CIK, padded or not; leading zeros are dropped.+ accession: Accession number such as ``"0002071691-26-012459"``.++ Returns:+ The fully-qualified EDGAR archive URL.+ """+ return (+ f"{_SEC_ARCHIVES_BASE}/{str(cik).lstrip('0') or '0'}/"+ f"{accession.replace('-', '')}/primary_doc.xml"+ )+++def _text_of(parent: Any, name: str) -> str | None:+ """Return the stripped text of ``parent``'s first child named ``name``."""+ if parent is None:+ return None+ for child in parent:+ if _local_name(child.tag) == name:+ text = (child.text or "").strip()+ return text or None+ return None+++def _child(parent: Any, name: str) -> Any:+ """Return ``parent``'s first child element named ``name``, or ``None``."""+ if parent is None:+ return None+ for element in parent:+ if _local_name(element.tag) == name:+ return element+ return None+++def _iso_date(value: Any) -> str | None:+ """Normalize an Eastmoney ``YYYYMMDD`` field to ``YYYY-MM-DD``.++ Args:+ value: The raw field, e.g. ``20260804`` or ``"20260804"``.++ Returns:+ The ISO date, the original string when it is not eight digits, or+ ``None`` when empty.+ """+ text = str(value).strip() if value not in (None, "") else ""+ if not text:+ return None+ if len(text) == 8 and text.isdigit():+ return f"{text[:4]}-{text[4:6]}-{text[6:]}"+ return text+++def _to_float(value: Any) -> float | None:+ """Coerce a reported value to ``float``, or ``None`` when non-numeric."""+ if value is None or value == "":+ return None+ try:+ return float(str(value).replace(",", ""))+ except (TypeError, ValueError):+ return None+++def _parse_nport_holding(node: Any) -> dict[str, Any]:+ """Shape one ``invstOrSec`` element into a holding record.++ Absent fields are omitted rather than defaulted, so a missing identifier is+ visibly missing instead of silently zero or blank.++ Args:+ node: An ``invstOrSec`` element.++ Returns:+ A holding dict keyed by ``name`` / ``pct_of_net_assets`` / ``value_usd``+ and whatever identifiers the filer supplied.+ """+ identifiers = _child(node, "identifiers")+ isin = ticker = None+ if identifiers is not None:+ for element in identifiers:+ local = _local_name(element.tag)+ if local == "isin":+ isin = (element.get("value") or "").strip() or None+ elif local == "ticker":+ ticker = (element.get("value") or "").strip() or None++ holding: dict[str, Any] = {+ "name": _text_of(node, "name"),+ "title": _text_of(node, "title"),+ "cusip": _text_of(node, "cusip"),+ "isin": isin,+ "ticker": ticker,+ # pctVal is already expressed in percent: IVV's Merck line reports+ # 0.5329 against valUSD 3.84e9 / netAssets 7.21e11.+ "pct_of_net_assets": _to_float(_text_of(node, "pctVal")),+ "value_usd": _to_float(_text_of(node, "valUSD")),+ "balance": _to_float(_text_of(node, "balance")),+ "balance_units": _text_of(node, "units"),+ "currency": _text_of(node, "curCd"),+ "payoff_profile": _text_of(node, "payoffProfile"),+ "asset_category": _text_of(node, "assetCat"),+ "issuer_category": _text_of(node, "issuerCat"),+ "country": _text_of(node, "invCountry"),+ }+ return {k: v for k, v in holding.items() if v is not None}+++def _parse_nport(xml_text: str) -> dict[str, Any]:+ """Parse an N-PORT ``primary_doc.xml`` into fund metadata plus holdings.++ Args:+ xml_text: The raw filing document.++ Returns:+ ``{series_id, series_name, registrant, as_of, fiscal_year_end,+ total_assets_usd, net_assets_usd, holdings}``.++ Raises:+ RuntimeError: When the document is not parseable XML.+ """+ try:+ root = DefusedET.fromstring(xml_text.encode("utf-8", "replace"))+ except Exception as exc: # noqa: BLE001 - a malformed filing is a data error+ raise RuntimeError(f"N-PORT document did not parse: {exc}") from exc++ form_data = _child(root, "formData")+ gen_info = _child(form_data, "genInfo")+ fund_info = _child(form_data, "fundInfo")+ securities = _child(form_data, "invstOrSecs")++ holdings = []+ if securities is not None:+ holdings = [+ _parse_nport_holding(node)+ for node in securities+ if _local_name(node.tag) == "invstOrSec"+ ]++ return {+ "series_id": _text_of(gen_info, "seriesId"),+ "series_name": _text_of(gen_info, "seriesName"),+ "registrant": _text_of(gen_info, "regName"),+ # repPdDate is the date the holdings are reported as of; repPdEnd is the+ # fund's fiscal year end. They diverge (IVV: repPdEnd 2026-03-31 with+ # repPdDate 2025-09-30 on the amendment), and EDGAR's own "Period of+ # Report" tracks repPdDate, so that is the as-of.+ "as_of": _text_of(gen_info, "repPdDate"),+ "fiscal_year_end": _text_of(gen_info, "repPdEnd"),+ "total_assets_usd": _to_float(_text_of(fund_info, "totAssets")),+ "net_assets_usd": _to_float(_text_of(fund_info, "netAssets")),+ "holdings": holdings,+ }+++def _lag_days(as_of: str | None, filing_date: str | None) -> int | None:+ """Return the whole days between a report period end and its filing date."""+ if not as_of or not filing_date:+ return None+ try:+ return (+ datetime.strptime(filing_date, "%Y-%m-%d").date()+ - datetime.strptime(as_of, "%Y-%m-%d").date()+ ).days+ except ValueError:+ return None+++# ── A-share: Eastmoney fund archives ─────────────────────────────────────────++# ``var apidata={ content:"<html>",arryear:[2026,...],curyear:2026};``+_APIDATA_RE = re.compile(r'content:"(?P<content>.*?)",\s*arryear:\[(?P<years>[^\]]*)\]', re.S)+_TAG_RE = re.compile(r"<[^>]+>")+_ROW_RE = re.compile(r"<tr>(?P<row>.*?)</tr>", re.S)+# A cell ends at its closing tag, at the next cell's opening tag, or at the end+# of the row. The last two alternatives are not defensiveness for its own sake:+# Eastmoney emits rows whose 相关资讯 cell is never closed — 510500's 2025 annual+# report carries ``<td class='xglj'>…<td class='tor'>0.00%</td>`` for the NEEQ+# line 400174 中天3 — and requiring ``</td>`` merges that cell with the next one,+# shifting every later column left by one. That silently reported the holding's+# share count (1,877.51) as its weight in percent.+_CELL_RE = re.compile(r"<t[dh][^>]*>(?P<cell>.*?)(?=</t[dh]>|<t[dh][^>]*>|\Z)", re.S)+# Quote links carry the Eastmoney market prefix: 1 = Shanghai, 0 = Shenzhen.+_QUOTE_LINK_RE = re.compile(r"quote\.eastmoney\.com/unify/r/(?P<market>\d)\.(?P<code>\d{6})")+_ASOF_RE = re.compile(r"截止至:\s*<font[^>]*>(?P<date>[\d-]+)</font>")+_FUND_NAME_RE = re.compile(r"title='(?P<name>[^']*)'")+_PERIOD_RE = re.compile(r"(?P<label>\d{4}年\d季度[^<]*)</label>")+_EM_MARKET_SUFFIX = {"1": "SH", "0": "SZ"}+# Present while a period is still collapsed to its top rows; its absence means+# this payload already carries everything Eastmoney holds for that period.+_EXPAND_LINK = "显示全部持仓明细"++# Periodic-report titles, e.g. "华泰柏瑞沪深300交易型开放式指数证券投资基金2025年+# 年度报告". The 摘要 (abstract) editions are excluded: they are a companion to+# the full report, and only the full report carries the portfolio schedule.+_ANNUAL_REPORT_RE = re.compile(r"(?P<year>\d{4})年年度报告(?!摘要)")+_INTERIM_REPORT_RE = re.compile(r"(?P<year>\d{4})年(?:中期|半年度)报告(?!摘要)")+++def _strip_html(fragment: str) -> str:+ """Flatten an HTML fragment to its visible text."""+ return _TAG_RE.sub("", fragment).replace(" ", " ").replace("&", "&").strip()+++def _cn_column_map(header_row: str) -> dict[str, tuple[int, float]]:+ """Map logical holding columns to ``(index, unit_scale)`` from the table head.++ Eastmoney reports 持股数 in 万股 and 持仓市值 in 万元 (both ten-thousands),+ but the exact header wording varies between funds, so the scale is read from+ the header text rather than assumed.++ Args:+ header_row: The raw ``<tr>`` inner HTML of the table head.++ Returns:+ Mapping of ``"code"`` / ``"name"`` / ``"pct"`` / ``"shares"`` /+ ``"value"`` to ``(column_index, scale)``. Absent columns are omitted.+ """+ columns: dict[str, tuple[int, float]] = {}+ for index, match in enumerate(_CELL_RE.finditer(header_row)):+ text = _strip_html(match.group("cell"))+ scale = 10_000.0 if "万" in text else 1.0+ if "序号" in text:+ # Carries the provenance marker: a trailing '*' means the row came+ # from the issuer's float-holder disclosure, not the fund's report.+ columns["seq"] = (index, 1.0)+ elif "股票代码" in text:+ columns["code"] = (index, 1.0)+ elif "股票名称" in text:+ columns["name"] = (index, 1.0)+ elif "占净值" in text:+ columns["pct"] = (index, 1.0)+ elif "持股数" in text:+ columns["shares"] = (index, scale)+ elif "持仓市值" in text:+ columns["value"] = (index, scale)+ return columns+++def _parse_cn_period(block: str) -> dict[str, Any] | None:+ """Parse one report period's holdings block from the Eastmoney fund archive.++ Every holding is stamped with the disclosure it came from, because the table+ merges two of them: unstarred rows are the fund's own periodic report, and a+ starred ``序号`` is Eastmoney cross-referencing the *issuer's* top-10 float+ holders. Mixing the two would inflate the apparent coverage of the fund's+ filing, so the totals are kept apart rather than summed.++ Args:+ block: One ``boxitem`` fragment, header table included.++ Returns:+ ``{as_of, report_label, fund_name, holdings, unparseable_rows,+ expandable, source_labelled, fund_report_holdings,+ cross_referenced_holdings, pct_of_net_assets_disclosed,+ pct_of_net_assets_cross_referenced}``, or ``None`` when the block+ carries no parseable table.+ """+ rows = _ROW_RE.findall(block)+ if len(rows) < 2:+ return None+ columns = _cn_column_map(rows[0])+ if "code" not in columns or "pct" not in columns:+ return None++ width = len(_CELL_RE.findall(rows[0]))+ holdings: list[dict[str, Any]] = []+ malformed = 0+ for row in rows[1:]:+ cells = [m.group("cell") for m in _CELL_RE.finditer(row)]+ # Column indices come from the header, so a row of a different width has+ # them pointing at the wrong values. Dropping it loses one holding;+ # keeping it reports one column's number under another column's name.+ # The count is returned so the loss is never silent.+ if len(cells) != width:+ malformed += 1+ continue++ def cell(key: str) -> tuple[str, float] | None:+ spec = columns.get(key)+ if spec is None or spec[0] >= len(cells):+ return None+ return _strip_html(cells[spec[0]]), spec[1]++ code_cell = cell("code")+ if code_cell is None or not code_cell[0]:+ continue+ link = _QUOTE_LINK_RE.search(row)+ suffix = _EM_MARKET_SUFFIX.get(link.group("market")) if link else None+ code = code_cell[0]++ seq_cell = cell("seq")+ cross_referenced = bool(seq_cell and "*" in seq_cell[0])+ holding: dict[str, Any] = {+ "symbol": f"{code}.{suffix}" if suffix else code,+ "disclosure_source": (+ _SOURCE_ISSUER_CROSS_REF if cross_referenced else _SOURCE_FUND_REPORT+ ),+ }+ name_cell = cell("name")+ if name_cell and name_cell[0]:+ holding["name"] = name_cell[0]+ pct_cell = cell("pct")+ if pct_cell:+ holding["pct_of_net_assets"] = _to_float(pct_cell[0].rstrip("%"))+ shares_cell = cell("shares")+ shares = _to_float(shares_cell[0]) if shares_cell else None+ if shares is not None:+ # Rounded because the ten-thousands rescale otherwise leaks binary+ # float noise into a share count (6816530699.999999 CNY).+ holding["shares"] = round(shares * shares_cell[1], 2)+ value_cell = cell("value")+ value = _to_float(value_cell[0]) if value_cell else None+ if value is not None:+ holding["market_value_cny"] = round(value * value_cell[1], 2)+ holdings.append({k: v for k, v in holding.items() if v is not None})++ if not holdings:+ return None+ as_of = _ASOF_RE.search(block)+ label = _PERIOD_RE.search(block)+ name = _FUND_NAME_RE.search(block)+ fund_rows = [h for h in holdings if h["disclosure_source"] == _SOURCE_FUND_REPORT]+ return {+ "as_of": as_of.group("date") if as_of else None,+ "report_label": label.group("label").strip() if label else None,+ "fund_name": name.group("name") if name else None,+ "holdings": holdings,+ "unparseable_rows": malformed,+ # Still collapsed: Eastmoney holds more rows for this period than this+ # payload carries, so nothing here may be called a complete portfolio.+ "expandable": _EXPAND_LINK in block,+ # Without the 序号 column there is no way to tell the two disclosures+ # apart, so the period is barred from claiming full coverage.+ "source_labelled": "seq" in columns,+ "fund_report_holdings": len(fund_rows),+ "cross_referenced_holdings": len(holdings) - len(fund_rows),+ "pct_of_net_assets_disclosed": round(+ sum(h.get("pct_of_net_assets") or 0.0 for h in fund_rows), 4+ ),+ "pct_of_net_assets_cross_referenced": round(+ sum(+ h.get("pct_of_net_assets") or 0.0+ for h in holdings+ if h["disclosure_source"] == _SOURCE_ISSUER_CROSS_REF+ ),+ 4,+ ),+ }+++def _cn_holdings_periods(code: str, year: int | None, months: str) -> list[dict[str, Any]]:+ """Fetch a fund's disclosed stock holdings by period, newest period first.++ The endpoint silently ignores an out-of-range ``year`` and answers with the+ current one instead, so the caller must read ``as_of`` off each returned+ period rather than assuming the year it asked for.++ Args:+ code: Bare six-digit fund code, e.g. ``"510050"``.+ year: Calendar year to request, or ``None`` for the endpoint default.+ months: Report months to expand past their top rows, as the page's own+ control sends them — ``"6,12"`` for the interim/annual periods,+ ``"3,6,9,12"`` for every quarter. An unexpanded period comes back+ collapsed to its top ten rows.++ Returns:+ Parsed period blocks in the order Eastmoney returns them (newest first).++ Raises:+ requests.RequestException: Network failure or non-2xx status.+ """+ body = _em_get_text(+ _EM_HOLDINGS_URL,+ params={+ "type": "jjcc",+ "code": code,+ "topline": str(_MAX_TOP_N),+ "year": str(year) if year else "",+ "month": months,+ },+ referer=f"https://fundf10.eastmoney.com/ccmx_{code}.html",+ )+ match = _APIDATA_RE.search(body)+ if match is None:+ return []+ content = match.group("content")+ blocks = content.split("<div class='boxitem")[1:]+ periods = [p for p in (_parse_cn_period(b) for b in blocks) if p is not None]+ return periods+++def _cn_periodic_reports(code: str) -> dict[str, dict[str, str]]:+ """Map period end to the interim/annual report that discloses it in full.++ This is the only authority on whether a June or December period's complete+ portfolio is actually public yet: the archive serves such a period from the+ quarterly data until the interim/annual report lands, and looks no different+ when it does. Best-effort by design — a failure returns an empty map, and+ the caller then declines to claim full coverage rather than guessing.++ Args:+ code: Bare six-digit fund code.++ Returns:+ ``{"YYYY-06-30": {"report": title, "published": "YYYY-MM-DD"}}``. When a+ report was published more than once, the earliest date wins, since that+ is when the holdings became public.+ """+ try:+ body = _em_get_text(+ _EM_ANNOUNCEMENT_URL,+ params={+ "fundcode": code,+ "pageIndex": "1",+ "pageSize": str(_EM_ANNOUNCEMENT_PAGE_SIZE),+ "type": _EM_PERIODIC_REPORT_TYPE,+ },+ referer=f"https://fundf10.eastmoney.com/jjgg_{code}_3.html",+ )+ payload = json.loads(body)+ except Exception as exc: # noqa: BLE001 - corroboration only, never fatal+ logger.warning("Eastmoney periodic-report index failed for %s: %s", code, exc)+ return {}++ # Without a Referer the endpoint answers 200 with ErrCode -999 and Data as+ # an empty *string*, so the shape is checked rather than assumed.+ records = payload.get("Data") if isinstance(payload, dict) else None+ if payload.get("ErrCode") != 0 or not isinstance(records, list):+ logger.warning("Eastmoney periodic-report index rejected the request for %s", code)+ return {}++ reports: dict[str, dict[str, str]] = {}+ for record in records:+ if not isinstance(record, dict):+ continue+ title = str(record.get("TITLE") or "")+ published = str(record.get("PUBLISHDATEDesc") or "").strip()+ if annual := _ANNUAL_REPORT_RE.search(title):+ period_end = f"{annual.group('year')}-12-31"+ elif interim := _INTERIM_REPORT_RE.search(title):+ period_end = f"{interim.group('year')}-06-30"+ else:+ continue+ known = reports.get(period_end)+ if known is None or (published and published < known["published"]):+ reports[period_end] = {"report": title, "published": published}+ return reports+++def _annotate_cn_period(period: dict[str, Any], reports: dict[str, dict[str, str]]) -> None:+ """Decide in place what a parsed period's rows actually cover.++ ``full_portfolio`` is claimed only when every one of these holds, so the+ label errs towards understating coverage rather than overstating it:++ * the period end is a June or December one, the only reports that carry the+ complete portfolio;+ * the block was expanded, so no rows are being withheld;+ * row provenance was readable, so fund-reported rows are separable from the+ issuer cross-referenced ones;+ * more fund-reported rows came back than a quarterly report may disclose+ (:data:`_QUARTERLY_DISCLOSURE_CAP`), which is what distinguishes a+ published interim/annual report from the quarterly data still standing in+ for it — on 2026-08-04 the expanded 2026-06-30 period is 15 rows;+ * the announcement index does not contradict it. A fund whose complete book+ genuinely runs to fifteen names or fewer is therefore reported as+ ``top_n_disclosed``; the disclosed percentage is published alongside so+ that understatement is visible rather than misleading.++ Args:+ period: A parsed period block, mutated with its coverage verdict.+ reports: Output of :func:`_cn_periodic_reports`, possibly empty.+ """+ as_of = period["as_of"] or ""+ report = reports.get(as_of)+ # None = the announcement index was unreachable, so it neither confirms nor+ # denies; False = it was read and holds no such report for this period.+ confirmed = None if not reports else report is not None++ period["report"] = report["report"] if report else None+ period["report_published"] = report["published"] if report else None+ period["report_confirmed"] = confirmed+ period["disclosure_lag_days"] = _lag_days(+ as_of, report["published"] if report else None+ )+ period["coverage"] = (+ _COVERAGE_FULL+ if (+ as_of.endswith(_FULL_DISCLOSURE_PERIOD_ENDS)+ and not period["expandable"]+ and period["source_labelled"]+ and period["fund_report_holdings"] > _QUARTERLY_DISCLOSURE_CAP+ and confirmed is not False+ )+ else _COVERAGE_TOP_N+ )+++def _em_push_json(path: str, params: dict[str, Any]) -> Any:+ """Call an Eastmoney quote endpoint, trying each edge host in turn.++ Args:+ path: Endpoint path such as ``"/api/qt/stock/get"``.+ params: Query parameters.++ Returns:+ The decoded JSON payload from the first host that answers.++ Raises:+ requests.RequestException: When every host fails; the last failure is+ re-raised so the caller reports a real cause.+ """+ last: Exception | None = None+ for host in _EM_QUOTE_HOSTS:+ try:+ return get_json(f"https://{host}{path}", params=params)+ except Exception as exc: # noqa: BLE001 - fall through to the next edge+ logger.warning("eastmoney %s failed on %s: %s", path, host, exc)+ last = exc+ raise last if last is not None else RuntimeError("no Eastmoney quote host configured")+++def _cn_quote(secid: str) -> dict[str, Any]:+ """Fetch one A-share fund's name and market size from the Eastmoney quote API.++ Args:+ secid: Eastmoney secid such as ``"1.510050"``.++ Returns:+ The ``data`` mapping (``f57`` code, ``f58`` name, ``f43`` last price,+ ``f116`` total market value, ``f117`` free-float market value), or an+ empty dict when the payload carries none.++ Raises:+ requests.RequestException: Network failure or non-2xx status.+ """+ payload = _em_push_json(+ _EM_QUOTE_PATH,+ {"secid": secid, "fltt": "2", "invt": "2", "fields": "f43,f57,f58,f86,f116,f117"},+ )+ data = payload.get("data") if isinstance(payload, dict) else None+ return data if isinstance(data, dict) else {}+++_CN_LIST_CACHE: list[dict[str, Any]] | None = None+_CN_LIST_LOCK = threading.Lock()+++def _cn_etf_list() -> tuple[list[dict[str, Any]], bool]:+ """Return every exchange-traded fund Eastmoney lists, memoized when complete.++ The quote server caps a page at 100 rows regardless of the requested ``pz``,+ so the ~1,500-row universe takes about sixteen throttled requests. A run+ that dies part-way is returned as-is with ``complete=False`` and is *not*+ cached, so a truncated universe can neither be mistaken for the whole one+ nor poison later calls.++ Returns:+ ``(rows, complete)`` where each row is ``{symbol, code, name,+ total_market_cap_cny, quote_date}``.++ Raises:+ requests.RequestException: Network failure on the very first page.+ """+ global _CN_LIST_CACHE+ if _CN_LIST_CACHE is not None:+ return _CN_LIST_CACHE, True+ with _CN_LIST_LOCK:+ if _CN_LIST_CACHE is not None:+ return _CN_LIST_CACHE, True+ rows: list[dict[str, Any]] = []+ expected: int | None = None+ for page in range(1, _EM_MAX_PAGES + 1):+ try:+ payload = _em_push_json(+ _EM_LIST_PATH,+ {+ "pn": str(page),+ "pz": str(_EM_PAGE_SIZE),+ "po": "1",+ "np": "1",+ "fltt": "2",+ "invt": "2",+ "fid": "f12",+ "fs": _EM_ETF_BOARDS,+ "fields": "f12,f13,f14,f20,f21,f297",+ },+ )+ except Exception:+ if page == 1:+ raise+ logger.warning("eastmoney ETF universe truncated at page %d", page)+ return rows, False+ data = payload.get("data") if isinstance(payload, dict) else None+ page_rows = data.get("diff") if isinstance(data, dict) else None+ if not isinstance(page_rows, list) or not page_rows:+ break+ total = data.get("total") if isinstance(data, dict) else None+ if isinstance(total, int) and total > 0:+ expected = total+ for row in page_rows:+ if not isinstance(row, dict):+ continue+ code = str(row.get("f12") or "").strip()+ if not code:+ continue+ suffix = _EM_MARKET_SUFFIX.get(str(row.get("f13")))+ rows.append(+ {+ "symbol": f"{code}.{suffix}" if suffix else code,+ "code": code,+ "name": row.get("f14"),+ "total_market_cap_cny": _to_float(row.get("f20")),+ "quote_date": _iso_date(row.get("f297")),+ }+ )+ if expected is not None and len(rows) >= expected:+ break+ complete = expected is None or len(rows) >= expected+ if complete:+ _CN_LIST_CACHE = rows+ return rows, complete+++# ── Mode handlers ────────────────────────────────────────────────────────────+++def _lookup_us(query: str, limit: int, offset: int) -> str:+ """Search US fund share classes by ticker or name and render the envelope."""+ try:+ index_year, _ = _us_series_index()+ matches = _us_match(query, limit)+ except Exception as exc: # noqa: BLE001 - surface any fetch failure as envelope+ return _error(f"SEC series/class index lookup failed: {exc}")++ def build(page: list[Any], paging: dict[str, Any]) -> dict[str, Any]:+ return {+ "ok": True,+ "mode": "lookup",+ "market": "US",+ "source": "sec_investment_company_series_class_index",+ "as_of": {"index_year": index_year},+ "missing_fields": ["net_assets", "expense_ratio"],+ "notes": (+ "The SEC series/class index carries identity only. Call "+ "mode='holdings' for net assets and the portfolio; no free SEC "+ "endpoint publishes an expense ratio."+ ),+ "paging": paging,+ "data": {"query": query, "matches": page},+ }++ return fit_records(matches, offset, build)+++def _lookup_cn(query: str, limit: int, offset: int) -> str:+ """Search A-share ETFs by code or name and render the envelope."""+ needle = query.strip()+ matches: list[dict[str, Any]] = []+ quote_as_of: Any = None+ universe_complete = True++ if _classify_market(needle) == "CN":+ code, secid = _cn_code_and_secid(needle)+ if secid is None:+ return _error(f"'{needle}' is not a recognizable A-share fund code")+ try:+ quote = _cn_quote(secid)+ except Exception as exc: # noqa: BLE001 - surface as envelope+ return _error(f"Eastmoney quote request failed: {exc}")+ if quote:+ record = {+ "symbol": f"{code}.{_CN_PREFIX_EXCHANGE.get(code[:2], '')}".rstrip("."),+ "code": quote.get("f57") or code,+ "name": quote.get("f58"),+ "last_price": _to_float(quote.get("f43")),+ # Shares outstanding x last price, i.e. an exchange-side size+ # proxy — not the fund's officially struck NAV-based AUM.+ "fund_size_cny": _to_float(quote.get("f116")),+ }+ matches = [{k: v for k, v in record.items() if v is not None}]+ # f86 is the quote's epoch second; an ISO instant is what downstream+ # reasoning can actually compare against a report period.+ epoch = _to_float(quote.get("f86"))+ if epoch is not None:+ quote_as_of = datetime.fromtimestamp(epoch, tz=timezone.utc).isoformat()+ else:+ try:+ universe, universe_complete = _cn_etf_list()+ except Exception as exc: # noqa: BLE001 - surface as envelope+ return _error(f"Eastmoney ETF universe request failed: {exc}")+ lowered = needle.lower()+ matches = [row for row in universe if lowered in str(row.get("name") or "").lower()][:limit]+ quote_as_of = next((m.get("quote_date") for m in matches), None)++ notes = (+ "fund_size_cny / total_market_cap_cny is shares outstanding x last "+ "traded price, an exchange-side size proxy rather than the fund's "+ "struck NAV. Eastmoney publishes no expense ratio on this endpoint."+ )+ if not universe_complete:+ notes += (+ " WARNING: the ETF universe fetch was cut short by an upstream "+ "failure, so these matches are drawn from a PARTIAL list and a "+ "matching fund may be missing. Retry before concluding none exists."+ )++ def build(page: list[Any], paging: dict[str, Any]) -> dict[str, Any]:+ return {+ "ok": True,+ "mode": "lookup",+ "market": "CN",+ "source": "eastmoney_quote",+ "as_of": quote_as_of,+ "universe_complete": universe_complete,+ "missing_fields": ["expense_ratio"],+ "notes": notes,+ "paging": paging,+ "data": {"query": needle, "matches": page},+ }++ return fit_records(matches, offset, build)+++def _holdings_us(symbol: str, top_n: int, offset: int) -> str:+ """Fetch and render one US ETF's latest N-PORT portfolio."""+ try:+ matches = _us_match(symbol, 1)+ except Exception as exc: # noqa: BLE001 - surface as envelope+ return _error(f"SEC series/class index lookup failed: {exc}")+ if not matches or matches[0]["ticker"] != symbol.strip().upper():+ return _error(+ f"'{symbol}' is not a fund share class in the SEC series/class index. "+ "Note that unit investment trusts (SPY among them) do not appear there "+ "and file no NPORT-P. Use mode='lookup' to find the right ticker."+ )+ fund = matches[0]++ try:+ filings = _us_nport_filings(fund["series_id"])+ except Exception as exc: # noqa: BLE001 - surface as envelope+ return _error(f"EDGAR NPORT-P filing index request failed: {exc}")+ if not filings:+ return _error(f"no NPORT-P filing found for {fund['ticker']} ({fund['series_id']})")++ filing = _select_nport_filing(filings, _us_indexed_periods(fund["series_id"]))+ document_url = _nport_document_url(fund["cik"] or "0", filing["accession"])+ try:+ parsed = _parse_nport(_sec_get_text(document_url))+ except Exception as exc: # noqa: BLE001 - surface as envelope+ return _error(f"N-PORT document request failed: {exc}")++ if parsed["series_id"] and parsed["series_id"] != fund["series_id"]:+ return _error(+ f"filing {filing['accession']} reports series {parsed['series_id']}, "+ f"expected {fund['series_id']} — refusing to attribute it to {fund['ticker']}"+ )++ holdings = parsed["holdings"]+ total_holdings = len(holdings)+ holdings = sorted(+ holdings, key=lambda h: h.get("pct_of_net_assets") or 0.0, reverse=True+ )[:top_n]+ # The document is primary; the full-text index is only how the filing was+ # chosen, so a disagreement is reported rather than reconciled silently.+ as_of = parsed["as_of"] or filing["indexed_period"]++ notes = _NOTE_LAG+ if filing["candidates_without_period"]:+ notes += (+ f" WARNING: {filing['candidates_without_period']} of the "+ f"{len(filings)} listed NPORT-P filings carry no period in EDGAR's "+ "full-text index and were skipped when picking the newest period. "+ "If one of them is a just-filed report the index has not caught up "+ "with, a newer portfolio than this one exists."+ )++ def build(page: list[Any], paging: dict[str, Any]) -> dict[str, Any]:+ return {+ "ok": True,+ "mode": "holdings",+ "market": "US",+ "source": "sec_nport",+ "as_of": as_of,+ "coverage": "full_portfolio",+ "notes": notes,+ "missing_fields": ["expense_ratio"],+ "paging": paging,+ "data": {+ "symbol": fund["ticker"],+ "fund": {+ "series_name": parsed["series_name"] or fund["series_name"],+ "registrant": parsed["registrant"] or fund["entity_name"],+ "cik": fund["cik"],+ "series_id": fund["series_id"],+ "class_id": fund["class_id"],+ "fiscal_year_end": parsed["fiscal_year_end"],+ "total_assets_usd": parsed["total_assets_usd"],+ "net_assets_usd": parsed["net_assets_usd"],+ },+ "filing": {+ "form": filing["form"],+ "accession": filing["accession"],+ "filing_date": filing["filing_date"],+ "disclosure_lag_days": _lag_days(as_of, filing["filing_date"]),+ "period_source": filing["period_source"],+ "indexed_period": filing["indexed_period"],+ "candidates_without_period": filing["candidates_without_period"],+ "document_url": document_url,+ },+ "holdings_in_filing": total_holdings,+ "holdings_returned_of_top_n": len(holdings),+ "holdings": page,+ },+ }++ return fit_records(holdings, offset, build)+++def _cn_fetch_year(+ code: str, year: int | None, months: str, reports: dict[str, dict[str, str]]+) -> list[dict[str, Any]]:+ """Fetch one calendar year of report periods, each annotated with coverage."""+ periods = _cn_holdings_periods(code, year, months)+ for period in periods:+ _annotate_cn_period(period, reports)+ return periods+++def _newest_full_period(reports: dict[str, dict[str, str]]) -> dict[str, str] | None:+ """Return the newest period the announcement index says is fully disclosed."""+ if not reports:+ return None+ as_of = max(reports)+ return {"as_of": as_of, **reports[as_of]}+++def _holdings_cn(+ symbol: str, top_n: int, offset: int, year: int | None, disclosure: str+) -> str:+ """Fetch and render one A-share ETF's disclosed holdings for a report period."""+ code, _ = _cn_code_and_secid(symbol)+ if not (len(code) == 6 and code.isdigit()):+ return _error(f"'{symbol}' is not a six-digit A-share fund code")++ months = (+ _EM_ALL_QUARTER_MONTHS+ if disclosure == "cross_reference"+ else _EM_FULL_DISCLOSURE_MONTHS+ )+ reports = _cn_periodic_reports(code)+ try:+ periods = _cn_fetch_year(code, year, months, reports)+ # A June/December period is served from the quarterly data until its+ # report lands, so the newest full portfolio is routinely in an earlier+ # year. The announcement index says which year to ask for, so the walk+ # back is a targeted refetch rather than a blind sweep.+ if disclosure == "full" and year is None:+ fetched = {int(p["as_of"][:4]) for p in periods if p["as_of"]}+ # One request serves a whole year, and a year contributes both its+ # June and December report, so the years are de-duplicated — the+ # fetch budget must not be spent asking for the same one twice.+ candidates: list[int] = []+ for as_of in sorted(reports, reverse=True):+ year_of = int(as_of[:4])+ if year_of not in fetched and year_of not in candidates:+ candidates.append(year_of)+ for candidate in candidates[:_MAX_FULL_LOOKBACK_FETCHES]:+ if any(p["coverage"] == _COVERAGE_FULL for p in periods):+ break+ periods = _cn_fetch_year(code, candidate, months, reports)+ except Exception as exc: # noqa: BLE001 - surface as envelope+ return _error(f"Eastmoney fund-holdings request failed: {exc}")+ if not periods:+ return _error(+ f"Eastmoney published no stock-holding detail for fund {code}. This "+ "endpoint covers equity holdings only, so a commodity fund (gold, "+ "e.g. 518880.SH) or a pure bond fund legitimately has none."+ )++ # Eastmoney returns the periods newest-first, but the report period is what+ # the answer is stamped with, so it is picked by date rather than by+ # trusting the server's ordering.+ full_periods = [p for p in periods if p["coverage"] == _COVERAGE_FULL]+ if disclosure == "full":+ if not full_periods:+ known = _newest_full_period(reports)+ scope = f" in {year}" if year is not None else ""+ if not known:+ hint = (+ " The announcement index lists no interim or annual report "+ "for this fund yet."+ )+ elif year is not None and not known["as_of"].startswith(str(year)):+ hint = (+ f" The fund's newest one is '{known['report']}' (period "+ f"{known['as_of']}), which the requested year excludes — drop "+ "'year' to reach it."+ )+ else:+ hint = (+ f" The fund's newest such report is '{known['report']}' "+ f"(published {known['published']}, period {known['as_of']}), "+ "but the archive is not yet serving its holdings detail."+ )+ return _error(+ f"no fully-disclosed portfolio is available for fund {code}{scope}. "+ "Only the interim (June) and annual (December) reports carry the "+ "complete portfolio." + hint + " Call again with "+ "disclosure='latest' for the most recent partial disclosure."+ )+ chosen = max(full_periods, key=lambda p: p["as_of"] or "")+ else:+ chosen = max(periods, key=lambda p: p["as_of"] or "")++ holdings = chosen["holdings"][:top_n]+ # ``top_n`` cuts the period's rows before paging, so paging reports the cut+ # length as its total and a caller who pages to the end sees no sign that+ # anything was withheld. That is exactly how a complete portfolio turns into+ # a silently partial one, so the cut is reported rather than assumed benign.+ truncated = len(chosen["holdings"]) - len(holdings)+ complete = chosen["coverage"] == _COVERAGE_FULL++ if complete:+ notes = (+ _NOTE_LAG+ + " These are ALL the fund's disclosed stock holdings for this "+ "period: Chinese funds publish the complete portfolio in the interim "+ "and annual reports, and pct_of_net_assets_disclosed shows how much "+ "of net assets they add up to."+ )+ else:+ notes = (+ _NOTE_LAG+ + " This period is a quarterly disclosure, which carries only the "+ "fund's largest holdings — pct_of_net_assets_disclosed shows how "+ "much of the portfolio that covers. The complete portfolio exists "+ "only for interim (June) and annual (December) periods, once their "+ "report is published; call again with disclosure='full' for the "+ "newest one."+ )+ if chosen["cross_referenced_holdings"]:+ notes += (+ f" {chosen['cross_referenced_holdings']} of these rows are marked "+ f"disclosure_source='{_SOURCE_ISSUER_CROSS_REF}': the fund's report "+ "does not list them, they are positions large enough to put the fund "+ "in the ISSUER's top-10 float holders. Real disclosure, different "+ "filing, and not counted in pct_of_net_assets_disclosed."+ )+ if chosen["unparseable_rows"]:+ notes += (+ f" WARNING: {chosen['unparseable_rows']} row(s) in this period had a "+ "cell count the table header does not explain and were dropped "+ "rather than read against mismatched columns, so this period is "+ "missing that many holdings."+ )+ if chosen["report_confirmed"] is None:+ notes += (+ " The periodic-report announcement index was unreachable, so the "+ "report behind this period could not be confirmed and its "+ "publication date is unknown."+ )+ if truncated:+ notes += (+ f" NOTE: top_n={top_n} cut {truncated} of this period's "+ f"{len(chosen['holdings'])} disclosed rows before paging, so paging "+ "to the end yields the largest top_n holdings rather than the whole "+ f"period. Raise top_n (max {_MAX_TOP_N}) to page through all of it."+ )++ missing = ["expense_ratio", "net_assets"]+ if not complete or truncated:+ missing.insert(1, "full_portfolio")+ newest_full = _newest_full_period(reports)++ def build(page: list[Any], paging: dict[str, Any]) -> dict[str, Any]:+ return {+ "ok": True,+ "mode": "holdings",+ "market": "CN",+ "source": "eastmoney_fund_archives",+ "as_of": chosen["as_of"],+ "coverage": chosen["coverage"],+ "notes": notes,+ "missing_fields": missing,+ "paging": paging,+ "data": {+ "symbol": symbol.strip().upper(),+ "fund": {"code": code, "name": chosen["fund_name"]},+ "report_label": chosen["report_label"],+ "disclosure": {+ "requested": disclosure,+ "report": chosen["report"],+ "report_published": chosen["report_published"],+ "report_confirmed": chosen["report_confirmed"],+ "disclosure_lag_days": chosen["disclosure_lag_days"],+ },+ "holdings_in_period": chosen["fund_report_holdings"],+ "cross_referenced_in_period": chosen["cross_referenced_holdings"],+ "unparseable_rows": chosen["unparseable_rows"],+ # Rows this period holds that top_n cut before paging ever saw+ # them; 0 means paging really can reach the whole period.+ "holdings_withheld_by_top_n": truncated,+ "pct_of_net_assets_disclosed": chosen["pct_of_net_assets_disclosed"],+ "pct_of_net_assets_cross_referenced": (+ chosen["pct_of_net_assets_cross_referenced"]+ ),+ # What the caller could have instead, so a partial answer never+ # hides the existence of the complete one.+ "full_portfolio_available": None if complete else newest_full,+ "available_periods": [+ {+ "as_of": p["as_of"],+ "report_label": p["report_label"],+ "coverage": p["coverage"],+ "holdings_in_period": p["fund_report_holdings"],+ "pct_of_net_assets_disclosed": p["pct_of_net_assets_disclosed"],+ }+ for p in periods+ ],+ "holdings": page,+ },+ }++ return fit_records(holdings, offset, build)+++class EtfHoldingsTool(BaseTool):+ """Look through an ETF to its constituents, or search the ETF universe."""++ name = "etf_holdings"+ description = (+ "ETF look-through across two markets. mode='holdings' returns an ETF's "+ "constituent holdings (security, weight, market value) — for U.S. funds "+ "the FULL portfolio from the SEC N-PORT filing, for A-share funds the "+ "holdings disclosed in the fund's own periodic report. "+ "mode='lookup' searches funds by ticker, name or theme and returns their "+ "identity and size. Market is auto-detected: '.SH'/'.SZ' or a bare "+ "six-digit code goes to the A-share path, anything else to the U.S. path. "+ "For A-share funds, disclosure='full' returns the COMPLETE portfolio from "+ "the newest interim or annual report (hundreds of names, ~95-100% of net "+ "assets) while the default disclosure='latest' returns the most recent "+ "period, which between reports is a quarterly top-N — read 'coverage' and "+ "'pct_of_net_assets_disclosed' to tell which you got. "+ "Every answer carries 'as_of', the report period the holdings belong to — "+ "ETF holdings are disclosed with a lag and are never live, and the full "+ "A-share portfolio is the more complete but older of the two. Fields the "+ "source does not publish (expense ratio in particular) are listed under "+ "'missing_fields' and never estimated. "+ 'Examples: {"mode": "holdings", "symbol": "IVV", "top_n": 20} or '+ '{"mode": "holdings", "symbol": "510300.SH", "disclosure": "full", '+ '"top_n": 400} or {"mode": "lookup", "query": "semiconductor"}.'+ )+ parameters = {+ "type": "object",+ "properties": {+ "mode": {+ "type": "string",+ "enum": list(_MODES),+ "description": (+ "'lookup' searches the ETF universe by ticker/name/theme; "+ "'holdings' returns one fund's constituent holdings."+ ),+ },+ "symbol": {+ "type": "string",+ "description": (+ "Required for mode='holdings'. A U.S. ETF ticker ('IVV', "+ "'SOXX', 'ARKK') or an A-share fund code ('510050.SH', "+ "'159915.SZ', or bare '510050')."+ ),+ },+ "query": {+ "type": "string",+ "description": (+ "Required for mode='lookup'. A ticker, fund name or theme "+ "keyword, e.g. 'IVV', 'semiconductor', '黄金'."+ ),+ },+ "market": {+ "type": "string",+ "enum": list(_MARKETS),+ "description": (+ "Override the auto-detected market (case-insensitive). In "+ "'auto' a lookup query goes to the A-share path when it is "+ "a six-digit fund code or contains Chinese characters. "+ "A-share fund names are Chinese, so a CN keyword search "+ "only matches a Chinese query ('黄金', '半导体') — a "+ "Latin-script keyword against market='CN' legitimately "+ "returns zero matches."+ ),+ "default": "auto",+ },+ "top_n": {+ "type": "integer",+ "description": (+ "For mode='holdings', how many largest holdings by weight to "+ f"consider (1-{_MAX_TOP_N}). Defaults to {_DEFAULT_TOP_N}."+ ),+ "default": _DEFAULT_TOP_N,+ },+ "limit": {+ "type": "integer",+ "description": (+ f"For mode='lookup', maximum matches (1-{_MAX_LOOKUP}). "+ f"Defaults to {_DEFAULT_LOOKUP}."+ ),+ "default": _DEFAULT_LOOKUP,+ },+ "disclosure": {+ "type": "string",+ "enum": list(_DISCLOSURES),+ "description": (+ "A-share holdings only; ignored for U.S. funds, which are "+ "always the full portfolio. 'latest' (default) returns the "+ "newest report period, which between reports is a quarterly "+ "top-N disclosure. 'full' returns the newest period whose "+ "COMPLETE portfolio has been published — only the interim "+ "(June) and annual (December) reports carry it, so this is "+ "the fuller but older answer; it searches back through "+ "earlier years automatically unless 'year' pins one. "+ "'cross_reference' additionally expands the quarterly "+ "periods to include positions disclosed by the ISSUER "+ "rather than the fund (the fund entered that company's "+ "top-10 float holders); those rows are real but come from a "+ "different filing and are labelled disclosure_source on "+ "every holding."+ ),+ "default": "latest",+ },+ "year": {+ "type": "integer",+ "description": (+ "A-share holdings only: calendar year of the reports to "+ "request. Omit for the most recent. The endpoint silently "+ "ignores an unavailable year, so always read the returned "+ "'as_of' rather than assuming this value. Pinning a year "+ "also stops disclosure='full' from searching earlier ones."+ ),+ },+ "offset": {+ "type": "integer",+ "description": (+ "Index of the first record to return. Records are returned "+ "whole and only as many as fit one result — read "+ "paging.next_offset and call again to continue."+ ),+ "default": 0,+ },+ },+ "required": ["mode"],+ }++ def execute(self, **kwargs: Any) -> str:+ """Route to the requested mode and market, returning a JSON envelope.++ Args:+ **kwargs: ``mode`` ("lookup"|"holdings", required), ``symbol``+ (required for holdings), ``query`` (required for lookup),+ ``market`` ("auto"|"US"|"CN"), ``disclosure``+ ("latest"|"full"|"cross_reference", A-share holdings only),+ ``top_n``, ``limit``, ``year`` and ``offset``.++ Returns:+ A JSON string. On success ``{"ok": true, "mode", "market",+ "source", "as_of", "coverage"?, "missing_fields", "paging",+ "data"}``; on failure ``{"ok": false, "error": str}``.+ """+ mode = kwargs.get("mode")+ if mode not in _MODES:+ return _error(f"mode must be one of {list(_MODES)}")++ # Models routinely send 'us'/'cn'/'Auto'; case is not a user error.+ market = str(kwargs.get("market") or "auto").strip()+ market = "auto" if market.lower() == "auto" else market.upper()+ if market not in _MARKETS:+ return _error(f"market must be one of {list(_MARKETS)}")++ try:+ offset = max(int(kwargs.get("offset") or 0), 0)+ except (TypeError, ValueError):+ return _error("offset must be an integer")++ if mode == "lookup":+ query = kwargs.get("query")+ if not isinstance(query, str) or not query.strip():+ return _error("'query' is required and must be a non-empty string for mode='lookup'")+ limit = _clamp(kwargs.get("limit", _DEFAULT_LOOKUP), _DEFAULT_LOOKUP, _MAX_LOOKUP)+ resolved = market if market != "auto" else _classify_query_market(query)+ return _lookup_cn(query, limit, offset) if resolved == "CN" else _lookup_us(query, limit, offset)++ symbol = kwargs.get("symbol")+ if not isinstance(symbol, str) or not symbol.strip():+ return _error("'symbol' is required and must be a non-empty string for mode='holdings'")+ top_n = _clamp(kwargs.get("top_n", _DEFAULT_TOP_N), _DEFAULT_TOP_N, _MAX_TOP_N)+ resolved = market if market != "auto" else _classify_market(symbol)+ if resolved == "CN":+ year = kwargs.get("year")+ if year is not None and (not isinstance(year, int) or isinstance(year, bool)):+ return _error("year must be an integer")+ disclosure = str(kwargs.get("disclosure") or "latest").strip().lower()+ if disclosure not in _DISCLOSURES:+ return _error(f"disclosure must be one of {list(_DISCLOSURES)}")+ return _holdings_cn(symbol, top_n, offset, year, disclosure)+ return _holdings_us(symbol, top_n, offset)+++def _clamp(value: Any, default: int, maximum: int) -> int:+ """Coerce a requested count into ``1..maximum``, falling back to ``default``."""+ try:+ n = int(value)+ except (TypeError, ValueError, OverflowError):+ return default+ return max(1, min(n, maximum))diff --git a/agent/tests/test_etf_holdings_tool.py b/agent/tests/test_etf_holdings_tool.pynew file mode 100644index 000000000..b9c2f59d6--- /dev/null+++ b/agent/tests/test_etf_holdings_tool.py@@ -0,0 +1,1112 @@+"""Tests for etf_holdings_tool: routing, parsing, as-of honesty, degradation.++Every fixture below is a trimmed copy of a real payload captured from the live+endpoint, and all HTTP is mocked at the module's two transport seams+(:func:`_sec_get_text` / :func:`_em_get_text`) plus the shared Eastmoney JSON+helper, so no test touches the network.++The regressions these tests exist for:++* N-PORT's ``repPdEnd`` is the fund's fiscal year end, not the report period.+ IVV's amendment carries ``repPdEnd`` 2026-03-31 with ``repPdDate``+ 2025-09-30, so reading the wrong tag stamps September holdings with a March+ date — and picking the newest *filed* document returns that September+ portfolio when a March one exists.+* On the A-share side the whole portfolio is only ever disclosed by the interim+ and annual reports, and Eastmoney serves a June/December period from the+ quarterly data until that report is published — identically shaped, an order+ of magnitude less complete. ``_JJCC_2026`` is the live 2026-06-30 payload for+ 510300: fifteen rows totalling 23.26% of net assets, on a period end whose+ calendar month says "interim report". Nothing may call that a full portfolio.+* The table merges a second disclosure: a starred ``序号`` is a position taken+ from the *issuer's* top-10 float holders, not from the fund's report.+* Eastmoney emits rows whose 相关资讯 cell is never closed. Requiring ``</td>``+ merges it with the next cell and shifts every later column left by one, which+ reported 510500's share count for 400174 中天3 (1,877.51) as a 1877.51%+ weight — one row inflating the fund's disclosed total to 1972.86%.+"""++from __future__ import annotations++import json+from unittest.mock import patch++import pytest++import src.tools.etf_holdings_tool as etf+from src.tools.etf_holdings_tool import EtfHoldingsTool++# ── Fixtures shaped exactly like the live payloads ───────────────────────────++_SERIES_CSV = (+ "Reporting File Number,CIK Number,Entity Name,Entity Org Type,Series ID,"+ "Series Name,Class ID,Class Name,Class Ticker,Address_1,Address_2,City,State,Zip Code\n"+ "811-09729,0001100663,iSHARES TRUST,30,S000004310,iShares Core S&P 500 ETF,"+ "C000012040,iShares Core S&P 500 ETF,IVV,400 HOWARD STREET,,SAN FRANCISCO,CA,94105\n"+ "811-09729,0001100663,iSHARES TRUST,30,S000004354,iShares Semiconductor ETF,"+ "C000012084,iShares Semiconductor ETF,SOXX,400 HOWARD STREET,,SAN FRANCISCO,CA,94105\n"+ "811-00001,0000000001,NO TICKER TRUST,30,S000000001,Unlisted Series,C000000001,"+ "Class A,,1 MAIN ST,,NEW YORK,NY,10001\n"+)++_ATOM = """<?xml version="1.0" encoding="ISO-8859-1" ?>+<feed xmlns="http://www.w3.org/2005/Atom">+ <company-info><cik>0001100663</cik><conformed-name>iSHARES TRUST</conformed-name></company-info>+ <entry>+ <content type="text/xml">+ <accession-number>0002071691-26-015790</accession-number>+ <filing-date>2026-07-13</filing-date>+ <filing-type>NPORT-P/A</filing-type>+ </content>+ </entry>+ <entry>+ <content type="text/xml">+ <accession-number>0002071691-26-012459</accession-number>+ <filing-date>2026-05-28</filing-date>+ <filing-type>NPORT-P</filing-type>+ </content>+ </entry>+</feed>+"""++# The amendment was filed later but covers an earlier period.+_FTS = json.dumps(+ {+ "hits": {+ "total": {"value": 2, "relation": "eq"},+ "hits": [+ {+ "_id": "0002071691-26-015790:primary_doc.xml",+ "_source": {"period_ending": "2025-09-30", "file_date": "2026-07-13"},+ },+ {+ "_id": "0002071691-26-012459:primary_doc.xml",+ "_source": {"period_ending": "2026-03-31", "file_date": "2026-05-28"},+ },+ ],+ }+ }+)+++def _nport(series_id: str, rep_pd_date: str, rep_pd_end: str) -> str:+ """Build an N-PORT document with two holdings, real tag names and namespace."""+ return f"""<?xml version="1.0" encoding="UTF-8"?>+<edgarSubmission xmlns="http://www.sec.gov/edgar/nport">+ <headerData><submissionType>NPORT-P</submissionType></headerData>+ <formData>+ <genInfo>+ <regName>iShares Trust</regName>+ <seriesName>iShares Core S&P 500 ETF</seriesName>+ <seriesId>{series_id}</seriesId>+ <repPdEnd>{rep_pd_end}</repPdEnd>+ <repPdDate>{rep_pd_date}</repPdDate>+ </genInfo>+ <fundInfo>+ <totAssets>721570012380.01</totAssets>+ <netAssets>720543356320.99</netAssets>+ </fundInfo>+ <invstOrSecs>+ <invstOrSec>+ <name>NVIDIA Corp.</name>+ <title>NVIDIA Corp.</title>+ <cusip>67066G104</cusip>+ <identifiers><isin value="US67066G1040"/><other otherDesc="x" value="y"/></identifiers>+ <balance>312526688.00000000</balance>+ <units>NS</units>+ <curCd>USD</curCd>+ <valUSD>54504654387.20000000</valUSD>+ <pctVal>7.564382338558</pctVal>+ <payoffProfile>Long</payoffProfile>+ <assetCat>EC</assetCat>+ <invCountry>US</invCountry>+ </invstOrSec>+ <invstOrSec>+ <name>CBRE Group, Inc.</name>+ <cusip>12504L109</cusip>+ <identifiers><isin value="US12504L1098"/></identifiers>+ <valUSD>506139381.54000000</valUSD>+ <pctVal>0.070244125783</pctVal>+ <assetCat>EC</assetCat>+ </invstOrSec>+ </invstOrSecs>+ </formData>+</edgarSubmission>+"""+++_FUND_NAME = "沪深300ETF华泰柏瑞"+++def _cn_row(+ seq: str,+ code: str,+ market: str,+ name: str,+ pct: str,+ shares: str,+ value: str,+ *,+ close_info_cell: bool = True,+) -> str:+ """One holdings row in Eastmoney's live seven-column markup.++ Args:+ seq: The 序号 cell; a trailing ``*`` marks an issuer cross-referenced row.+ code: Six-digit stock code.+ market: Eastmoney market prefix — ``"1"`` Shanghai, ``"0"`` Shenzhen.+ name: Stock name.+ pct: 占净值比例 cell text, e.g. ``"7.29%"``.+ shares: 持股数 cell text in 万股.+ value: 持仓市值 cell text in 万元.+ close_info_cell: When ``False``, omit the 相关资讯 cell's ``</td>``, the+ way the live payload does for 510500's 400174 line.+ """+ link = f"//quote.eastmoney.com/unify/r/{market}.{code}"+ info = "<td class='xglj'><a href='#'>股吧</a><a href='#'>行情</a>"+ if close_info_cell:+ info += "</td>"+ return (+ f"<tr><td>{seq}</td><td><a href='{link}'>{code}</a></td>"+ f"<td class='tol'><a href='{link}'>{name}</a></td>{info}"+ f"<td class='tor'>{pct}</td><td class='tor'>{shares}</td>"+ f"<td class='tor'>{value}</td></tr>"+ )+++def _cn_block(label: str, as_of: str, rows: list[str], *, expandable: bool) -> str:+ """One report period's ``boxitem`` fragment.++ Args:+ label: The 季度 label, e.g. ``"2025年4季度股票投资明细"``.+ as_of: Period end as ``YYYY-MM-DD``.+ rows: Rendered ``<tr>`` rows.+ expandable: Whether the "显示全部持仓明细" control is still present,+ which is how the live payload says it is withholding rows.+ """+ control = "显示全部持仓明细>>" if expandable else "收起持仓明细>>"+ return (+ "<div class='box'><div class='boxitem w790'><h4 class='t'><label class='left'>"+ f"<a title='{_FUND_NAME}' href='#'>{_FUND_NAME}</a> {label}</label>"+ "<label class='right lab2 xq505'> 来源:天天基金 截止至:"+ f"<font class='px12'>{as_of}</font></label></h4><div class='space0'></div>"+ "<table class='w782 comm tzxq t2'><thead><tr><th class='first'>序号</th>"+ "<th>股票代码</th><th>股票名称</th><th class='xglj'>相关资讯</th>"+ "<th>占净值<br />比例</th><th class='cgs'>持股数<br />(万股)</th>"+ "<th class='last ccs'>持仓市值<br />(万元)</th></tr></thead><tbody>"+ + "".join(rows)+ + "</tbody></table><div class='hide' id='gpdmList'>1.600519,</div>"+ f"<div class='tfoot'><a onclick='LoadMore(this,6,LoadStockPos)'>{control}</a>"+ "</div></div></div></div>"+ )+++def _jjcc(*blocks: str, curyear: int = 2026) -> str:+ """Wrap period blocks in the ``var apidata={...}`` envelope the endpoint returns."""+ footnote = (+ "<div style='padding:5px 10px'>注:加*号代表进入上市公司的十大流通股东"+ "却没有进入单只基金前十大重仓股的个股。</div>"+ )+ return (+ 'var apidata={ content:"'+ + "".join(blocks)+ + footnote+ + f'",arryear:[2026,2025],curyear:{curyear}}};'+ )+++# 2026-06-30 as the live endpoint serves it on 2026-08-04: the interim report is+# not due until 08-31, so the period end says "interim" while the rows are the+# quarterly top-10 index plus top-5 active — 15 rows, 23.26% of net assets.+_Q2_2026_ROWS = [+ _cn_row(str(i + 1), f"60000{i}", "1", f"大盘股{i}", "2.00%", "100.00", "12,000.00")+ for i in range(10)+] + [+ _cn_row(str(i + 11), f"68800{i}", "1", f"新股{i}", "0.652%", "1.00", "80.00")+ for i in range(5)+]++_JJCC_2026 = _jjcc(+ _cn_block("2026年2季度股票投资明细", "2026-06-30", _Q2_2026_ROWS, expandable=False),+ _cn_block(+ "2026年1季度股票投资明细",+ "2026-03-31",+ [+ _cn_row("1", "600519", "1", "贵州茅台", "7.29%", "121.09", "143,549.56"),+ _cn_row("2", "300750", "0", "宁德时代", "4.99%", "2,059.75", "98,332.49"),+ ],+ expandable=True,+ ),+)++# 2025-12-31 once the annual report is out: twenty fund-reported rows, past the+# fifteen a quarterly report may carry, summing to 97.96% of net assets.+_ANNUAL_2025_ROWS = [+ _cn_row("1", "600519", "1", "贵州茅台", "7.29%", "121.09", "143,549.56"),+ _cn_row("2", "300750", "0", "宁德时代", "4.99%", "2,059.75", "98,332.49"),+] + [+ _cn_row(str(i + 3), f"60100{i}", "1", f"成分股{i}", "4.76%", "500.00", "60,000.00")+ for i in range(18)+]++_JJCC_2025 = _jjcc(+ _cn_block("2025年4季度股票投资明细", "2025-12-31", _ANNUAL_2025_ROWS, expandable=False),+ _cn_block(+ "2025年3季度股票投资明细",+ "2025-09-30",+ [_cn_row("1", "600519", "1", "贵州茅台", "8.10%", "100.00", "120,000.00")],+ expandable=True,+ ),+ curyear=2025,+)++_PREFIX = "华泰柏瑞沪深300交易型开放式指数证券投资基金"++# type=3 is the periodic-report shelf. The 摘要 editions are companions to the+# full report and carry no portfolio schedule, so they must not be read as one.+_JJGG = json.dumps(+ {+ "Data": [+ {"TITLE": f"{_PREFIX}2026年第2季度报告", "PUBLISHDATEDesc": "2026-07-21"},+ {"TITLE": f"{_PREFIX}2025年年度报告", "PUBLISHDATEDesc": "2026-03-31"},+ {"TITLE": f"{_PREFIX}2025年年度报告摘要", "PUBLISHDATEDesc": "2026-03-31"},+ {"TITLE": f"{_PREFIX}2025年中期报告", "PUBLISHDATEDesc": "2025-08-30"},+ ],+ "ErrCode": 0,+ "TotalCount": 4,+ },+ ensure_ascii=False,+)+++def _em_router(url, *, params, referer):+ """Serve the Eastmoney fixtures by URL and requested year, as the live host does."""+ if url == etf._EM_ANNOUNCEMENT_URL:+ return _JJGG+ return _JJCC_2025 if str(params.get("year")) == "2025" else _JJCC_2026++_QUOTE = {+ "data": {+ "f43": 2.983,+ "f57": "510050",+ "f58": "上证50ETF华夏",+ "f86": 1785831111,+ "f116": 22902480112.128004,+ "f117": 22902480112.128004,+ }+}+++def _list_page(page: int) -> dict:+ """Two-row pages of the Eastmoney ETF universe, total 3."""+ rows = [+ [{"f12": "518880", "f13": 1, "f14": "黄金ETF华安", "f20": 9.2e10, "f297": 20260804},+ {"f12": "159915", "f13": 0, "f14": "创业板ETF易方达", "f20": 7.1e10, "f297": 20260804}],+ [{"f12": "588170", "f13": 1, "f14": "科创半导体ETF华夏", "f20": 3.9e10, "f297": 20260804}],+ ]+ return {"data": {"total": 3, "diff": rows[page - 1] if page <= len(rows) else []}}++[email protected](autouse=True)+def _clear_caches():+ """Reset the process-wide memoized indexes between tests."""+ etf._US_INDEX_CACHE = None+ etf._CN_LIST_CACHE = None+ yield+ etf._US_INDEX_CACHE = None+ etf._CN_LIST_CACHE = None+++def _sec_router(url, params=None):+ """Serve the SEC fixtures by URL, mirroring the live endpoint layout."""+ if "series-class" in url:+ return _SERIES_CSV+ if "browse-edgar" in url:+ return _ATOM+ if "search-index" in url:+ return _FTS+ if "015790" in url:+ return _nport("S000004310", "2025-09-30", "2026-03-31")+ return _nport("S000004310", "2026-03-31", "2026-03-31")+++class TestMarketRouting:+ """Symbols and queries reach the intended market path."""++ @pytest.mark.parametrize(+ "symbol,expected",+ [+ ("510050.SH", "CN"),+ ("159915.sz", "CN"),+ ("510050", "CN"),+ ("IVV", "US"),+ ("SOXX", "US"),+ ("BRK.B", "US"),+ ],+ )+ def test_symbol_classification(self, symbol, expected):+ assert etf._classify_market(symbol) == expected++ def test_han_query_routes_to_a_share(self):+ # The SEC index holds no Chinese fund names, so a Han query must not be+ # answered with a confident "no matches" from the US path.+ assert etf._classify_query_market("黄金") == "CN"+ assert etf._classify_query_market("semiconductor") == "US"++ def test_bare_six_digit_code_resolves_to_an_exchange(self):+ assert etf._cn_code_and_secid("510050") == ("510050", "1.510050")+ assert etf._cn_code_and_secid("159915") == ("159915", "0.159915")+ assert etf._cn_code_and_secid("123456")[1] is None++ def test_explicit_market_overrides_detection(self):+ with patch.object(etf, "_sec_get_text", side_effect=_sec_router):+ payload = json.loads(EtfHoldingsTool().execute(mode="lookup", query="510050", market="US"))+ assert payload["market"] == "US"+++class TestUsHoldings:+ """N-PORT parsing, period selection and the as-of contract."""++ def test_report_period_comes_from_rep_pd_date_not_rep_pd_end(self):+ parsed = etf._parse_nport(_nport("S000004310", "2025-09-30", "2026-03-31"))+ assert parsed["as_of"] == "2025-09-30"+ assert parsed["fiscal_year_end"] == "2026-03-31"++ def test_newest_period_wins_over_newest_filing_date(self):+ with patch.object(etf, "_sec_get_text", side_effect=_sec_router):+ payload = json.loads(EtfHoldingsTool().execute(mode="holdings", symbol="IVV", top_n=5))+ assert payload["ok"] is True+ # The amendment was filed 2026-07-13 but covers 2025-09-30; the answer+ # must be the 2026-03-31 original.+ assert payload["as_of"] == "2026-03-31"+ assert payload["data"]["filing"]["accession"] == "0002071691-26-012459"+ assert payload["data"]["filing"]["period_source"] == "edgar_full_text_index"++ def test_falls_back_to_filing_order_when_periods_unknown(self):+ chosen = etf._select_nport_filing(+ [+ {"form": "NPORT-P/A", "accession": "A", "filing_date": "2026-07-13"},+ {"form": "NPORT-P", "accession": "B", "filing_date": "2026-05-28"},+ ],+ {},+ )+ assert chosen["accession"] == "A"+ assert chosen["period_source"] == "filing_order"++ def test_a_filing_the_index_cannot_date_is_counted_not_silently_skipped(self):+ # EDGAR's full-text index lags the filing feed, so the newest NPORT-P+ # can be listed with no period yet. Ranking by period drops it, which+ # would answer with last quarter's portfolio and look authoritative.+ thin_fts = json.dumps(+ {+ "hits": {+ "hits": [+ {+ "_id": "0002071691-26-012459:primary_doc.xml",+ "_source": {"period_ending": "2026-03-31"},+ }+ ]+ }+ }+ )++ def router(url, params=None):+ return thin_fts if "search-index" in url else _sec_router(url, params)++ with patch.object(etf, "_sec_get_text", side_effect=router):+ payload = json.loads(EtfHoldingsTool().execute(mode="holdings", symbol="IVV"))+ assert payload["data"]["filing"]["candidates_without_period"] == 1+ assert "WARNING" in payload["notes"]++ def test_a_fully_indexed_filing_list_carries_no_warning(self):+ with patch.object(etf, "_sec_get_text", side_effect=_sec_router):+ payload = json.loads(EtfHoldingsTool().execute(mode="holdings", symbol="IVV"))+ assert payload["data"]["filing"]["candidates_without_period"] == 0+ assert "WARNING" not in payload["notes"]++ def test_envelope_carries_lag_coverage_and_full_portfolio_count(self):+ with patch.object(etf, "_sec_get_text", side_effect=_sec_router):+ payload = json.loads(EtfHoldingsTool().execute(mode="holdings", symbol="ivv", top_n=1))+ assert payload["coverage"] == "full_portfolio"+ assert payload["data"]["filing"]["disclosure_lag_days"] == 58+ assert payload["data"]["holdings_in_filing"] == 2+ assert payload["data"]["fund"]["net_assets_usd"] == 720543356320.99++ def test_holdings_are_ranked_by_weight_and_omit_absent_fields(self):+ with patch.object(etf, "_sec_get_text", side_effect=_sec_router):+ payload = json.loads(EtfHoldingsTool().execute(mode="holdings", symbol="IVV"))+ holdings = payload["data"]["holdings"]+ assert [h["name"] for h in holdings] == ["NVIDIA Corp.", "CBRE Group, Inc."]+ assert holdings[0]["pct_of_net_assets"] == 7.564382338558+ assert holdings[0]["isin"] == "US67066G1040"+ # The second holding reports no balance/units; those keys are absent+ # rather than defaulted to zero.+ assert "balance" not in holdings[1]+ assert "ticker" not in holdings[0]++ def test_series_mismatch_refuses_to_attribute_the_filing(self):+ def router(url, params=None):+ if "primary_doc" in url:+ return _nport("S000099999", "2026-03-31", "2026-03-31")+ return _sec_router(url, params)++ with patch.object(etf, "_sec_get_text", side_effect=router):+ payload = json.loads(EtfHoldingsTool().execute(mode="holdings", symbol="IVV"))+ assert payload["ok"] is False+ assert "S000099999" in payload["error"]++ def test_unlisted_ticker_names_the_uit_gap(self):+ with patch.object(etf, "_sec_get_text", side_effect=_sec_router):+ payload = json.loads(EtfHoldingsTool().execute(mode="holdings", symbol="SPY"))+ assert payload["ok"] is False+ assert "SPY" in payload["error"]++ def test_document_url_drops_cik_padding_and_accession_dashes(self):+ assert etf._nport_document_url("0001100663", "0002071691-26-012459") == (+ "https://www.sec.gov/Archives/edgar/data/1100663/"+ "000207169126012459/primary_doc.xml"+ )++ def test_large_portfolio_is_paged_not_truncated(self):+ many = "".join(+ f"<invstOrSec><name>Holding {i}</name><cusip>{i:09d}</cusip>"+ f"<valUSD>1000.0</valUSD><pctVal>{100 - i * 0.1}</pctVal>"+ f"<assetCat>EC</assetCat><invCountry>US</invCountry>"+ f"<payoffProfile>Long</payoffProfile><curCd>USD</curCd>"+ f"<units>NS</units><balance>10.0</balance></invstOrSec>"+ for i in range(150)+ )+ doc = _nport("S000004310", "2026-03-31", "2026-03-31").replace(+ "</invstOrSecs>", many + "</invstOrSecs>"+ )++ def router(url, params=None):+ return doc if "primary_doc" in url else _sec_router(url, params)++ with patch.object(etf, "_sec_get_text", side_effect=router):+ text = EtfHoldingsTool().execute(mode="holdings", symbol="IVV", top_n=200)+ payload = json.loads(text)+ assert len(text) <= 10_000+ assert payload["paging"]["complete"] is False+ assert payload["paging"]["next_offset"] == payload["paging"]["returned"]+ assert payload["data"]["holdings_in_filing"] == 152+++class TestUsLookup:+ """Ticker/name search over the SEC series-class index."""++ def test_exact_ticker_ranks_first_and_name_search_works(self):+ with patch.object(etf, "_sec_get_text", side_effect=_sec_router):+ payload = json.loads(EtfHoldingsTool().execute(mode="lookup", query="semiconductor"))+ assert payload["market"] == "US"+ assert payload["data"]["matches"][0]["ticker"] == "SOXX"+ assert payload["data"]["matches"][0]["series_id"] == "S000004354"++ def test_missing_fields_are_declared_never_estimated(self):+ with patch.object(etf, "_sec_get_text", side_effect=_sec_router):+ payload = json.loads(EtfHoldingsTool().execute(mode="lookup", query="IVV"))+ assert "expense_ratio" in payload["missing_fields"]+ assert "expense_ratio" not in payload["data"]["matches"][0]+ assert payload["as_of"] == {"index_year": etf.date.today().year}++ def test_rows_without_a_ticker_are_dropped(self):+ records = etf._parse_series_index(_SERIES_CSV)+ assert {r["ticker"] for r in records} == {"IVV", "SOXX"}++ def test_etf_share_classes_outrank_mutual_fund_classes_on_a_theme_query(self):+ # Live check: "semiconductor" matches 31 classes, and in filer order the+ # first eight are Fidelity mutual fund classes, so an unranked default+ # page of ten never reaches SOXX/SMH/XSD.+ csv_body = (+ _SERIES_CSV+ + "811-03010,0000315700,FIDELITY ADVISOR SERIES VII,30,S000005327,"+ "Fidelity Advisor Semiconductors Fund,C000014549,Class A,FELAX,"+ "245 SUMMER STREET,,BOSTON,MA,02210\n"+ # VOO's ETF-ness is only visible in the class name.+ "811-02652,0000036405,VANGUARD INDEX FUNDS,30,S000002839,"+ "Vanguard 500 Semiconductor Index Fund,C000092055,ETF Shares,VOO,"+ "PO BOX 2600,V26,VALLEY FORGE,PA,19482\n"+ )++ def router(url, params=None):+ return csv_body if "series-class" in url else _sec_router(url, params)++ with patch.object(etf, "_sec_get_text", side_effect=router):+ payload = json.loads(+ EtfHoldingsTool().execute(mode="lookup", query="semiconductor", limit=3)+ )+ assert [m["ticker"] for m in payload["data"]["matches"]] == ["SOXX", "VOO", "FELAX"]++ def test_a_row_with_more_fields_than_the_header_does_not_kill_the_index(self):+ # csv.DictReader files the overflow under a None key as a *list*; the+ # whole US path used to die on ``.strip()`` if the SEC ever emitted one.+ records = etf._parse_series_index(+ _SERIES_CSV + "811-1,0000000002,STRAY TRUST,30,S000000002,Stray ETF,"+ "C000000002,Class A,STRY,1 MAIN ST,,NEW YORK,NY,10001,extra,extra2\n"+ )+ assert {r["ticker"] for r in records} == {"IVV", "SOXX", "STRY"}++ def test_index_falls_back_to_the_previous_year(self):+ seen: list[str] = []++ def router(url, params=None):+ seen.append(url)+ if f"{etf.date.today().year}.csv" in url:+ raise RuntimeError("404 not posted yet")+ return _SERIES_CSV++ with patch.object(etf, "_sec_get_text", side_effect=router):+ year, records = etf._us_series_index()+ assert year == etf.date.today().year - 1+ assert len(records) == 2+ assert len(seen) == 2+++class TestCnHoldings:+ """Eastmoney fund-archive parsing and its disclosed-coverage honesty."""++ def test_latest_period_is_parsed_with_its_own_as_of(self):+ with patch.object(etf, "_em_get_text", side_effect=_em_router):+ payload = json.loads(EtfHoldingsTool().execute(mode="holdings", symbol="510300.SH"))+ assert payload["ok"] is True+ assert payload["market"] == "CN"+ assert payload["as_of"] == "2026-06-30"+ assert payload["data"]["report_label"] == "2026年2季度股票投资明细"+ assert payload["data"]["fund"]["name"] == _FUND_NAME++ def test_units_are_rescaled_from_ten_thousands_and_exchange_inferred(self):+ with patch.object(etf, "_em_get_text", side_effect=_em_router):+ payload = json.loads(+ EtfHoldingsTool().execute(mode="holdings", symbol="510300", year=2025)+ )+ first, second = payload["data"]["holdings"][:2]+ assert first == {+ "symbol": "600519.SH",+ "disclosure_source": "fund_report",+ "name": "贵州茅台",+ "pct_of_net_assets": 7.29,+ "shares": 1210900.0,+ "market_value_cny": 1435495600.0,+ }+ # Market prefix 0 in the quote link means Shenzhen.+ assert second["symbol"] == "300750.SZ"++ def test_a_june_period_served_from_quarterly_data_is_not_called_full(self):+ # The regression: 2026-06-30 is an interim period end, but on 2026-08-04+ # the interim report is not due yet and Eastmoney answers it with the+ # quarterly rows. Reading the calendar month alone would publish 15 rows+ # covering 23.26% of net assets as the fund's complete portfolio.+ with patch.object(etf, "_em_get_text", side_effect=_em_router):+ payload = json.loads(EtfHoldingsTool().execute(mode="holdings", symbol="510300.SH"))+ assert payload["as_of"] == "2026-06-30"+ assert payload["coverage"] == "top_n_disclosed"+ assert payload["data"]["holdings_in_period"] == 15+ assert payload["data"]["pct_of_net_assets_disclosed"] == 23.26+ assert "full_portfolio" in payload["missing_fields"]++ def test_a_partial_answer_names_the_full_portfolio_it_is_not(self):+ with patch.object(etf, "_em_get_text", side_effect=_em_router):+ payload = json.loads(EtfHoldingsTool().execute(mode="holdings", symbol="510300.SH"))+ assert payload["data"]["full_portfolio_available"] == {+ "as_of": "2025-12-31",+ "report": f"{_PREFIX}2025年年度报告",+ "published": "2026-03-31",+ }+ assert "disclosure='full'" in payload["notes"]++ def test_full_walks_back_a_year_and_returns_the_complete_portfolio(self):+ with patch.object(etf, "_em_get_text", side_effect=_em_router):+ payload = json.loads(+ EtfHoldingsTool().execute(+ mode="holdings", symbol="510300.SH", disclosure="full", top_n=100+ )+ )+ assert payload["as_of"] == "2025-12-31"+ assert payload["coverage"] == "full_portfolio"+ assert payload["data"]["holdings_in_period"] == 20+ assert payload["data"]["pct_of_net_assets_disclosed"] == 97.96+ assert payload["data"]["full_portfolio_available"] is None+ assert "full_portfolio" not in payload["missing_fields"]++ def test_the_lookback_asks_for_each_year_at_most_once(self):+ # A year contributes both its interim and its annual report, and one+ # request serves the whole year, so an undeduplicated candidate list+ # burns the fetch budget asking for 2025 twice.+ years: list[str] = []++ def router(url, *, params, referer):+ if url == etf._EM_ANNOUNCEMENT_URL:+ return _JJGG+ years.append(str(params.get("year")))+ return _JJCC_2025 if str(params.get("year")) == "2025" else _JJCC_2026++ with patch.object(etf, "_em_get_text", side_effect=router):+ payload = json.loads(+ EtfHoldingsTool().execute(+ mode="holdings", symbol="510300.SH", disclosure="full"+ )+ )+ # The index lists 2025-12-31 and 2025-06-30; both name the same year.+ assert years == ["", "2025"]+ assert payload["as_of"] == "2025-12-31"++ def test_full_reports_the_filing_and_its_real_publication_lag(self):+ with patch.object(etf, "_em_get_text", side_effect=_em_router):+ payload = json.loads(+ EtfHoldingsTool().execute(+ mode="holdings", symbol="510300.SH", disclosure="full"+ )+ )+ # 2025-12-31 -> published 2026-03-31 is 90 days, taken from the+ # announcement index rather than assumed from the filing deadline.+ assert payload["data"]["disclosure"] == {+ "requested": "full",+ "report": f"{_PREFIX}2025年年度报告",+ "report_published": "2026-03-31",+ "report_confirmed": True,+ "disclosure_lag_days": 90,+ }++ def test_a_period_still_offering_more_rows_is_never_full(self):+ # The 2025-09-30 block keeps its "显示全部持仓明细" control, so rows are+ # being withheld and no claim of completeness can be made about it.+ period = etf._parse_cn_period(_JJCC_2025.split("<div class='boxitem")[2])+ assert period["expandable"] is True+ etf._annotate_cn_period(period, {"2025-09-30": {"report": "x", "published": "y"}})+ assert period["coverage"] == "top_n_disclosed"++ def test_announcement_index_contradicting_the_period_blocks_the_full_claim(self):+ # Twenty expanded rows on a December period, but the index is readable+ # and holds no annual report for it — only the interim one. That is a+ # contradiction rather than an unknown, so fail closed.+ no_annual = json.dumps(+ {+ "Data": [+ {"TITLE": f"{_PREFIX}2025年中期报告", "PUBLISHDATEDesc": "2025-08-30"},+ {"TITLE": f"{_PREFIX}2025年第4季度报告", "PUBLISHDATEDesc": "2026-01-22"},+ ],+ "ErrCode": 0,+ },+ ensure_ascii=False,+ )++ def router(url, *, params, referer):+ return no_annual if url == etf._EM_ANNOUNCEMENT_URL else _JJCC_2025++ with patch.object(etf, "_em_get_text", side_effect=router):+ payload = json.loads(+ EtfHoldingsTool().execute(mode="holdings", symbol="510300.SH", year=2025)+ )+ assert payload["as_of"] == "2025-12-31"+ assert payload["coverage"] == "top_n_disclosed"+ assert payload["data"]["disclosure"]["report_confirmed"] is False++ def test_an_unreachable_announcement_index_says_so_instead_of_going_quiet(self):+ def router(url, *, params, referer):+ if url == etf._EM_ANNOUNCEMENT_URL:+ raise RuntimeError("edge dropped the connection")+ return _JJCC_2025++ with patch.object(etf, "_em_get_text", side_effect=router):+ payload = json.loads(+ EtfHoldingsTool().execute(mode="holdings", symbol="510300.SH", year=2025)+ )+ # The structural evidence still stands on its own, but the filing behind+ # it is unconfirmed and the answer has to admit that.+ assert payload["coverage"] == "full_portfolio"+ assert payload["data"]["disclosure"]["report_confirmed"] is None+ assert payload["data"]["disclosure"]["disclosure_lag_days"] is None+ assert "could not be confirmed" in payload["notes"]++ def test_announcement_index_rejects_the_request_without_crashing(self):+ # No Referer -> 200 with ErrCode -999 and Data as an empty *string*,+ # which a naive list walk would blow up on.+ refused = json.dumps({"Data": "", "ErrCode": -999, "ErrMsg": "", "TotalCount": 0})++ def router(url, *, params, referer):+ return refused if url == etf._EM_ANNOUNCEMENT_URL else _JJCC_2025++ with patch.object(etf, "_em_get_text", side_effect=router):+ assert etf._cn_periodic_reports("510300") == {}++ def test_report_abstracts_are_not_read_as_the_full_report(self):+ with patch.object(etf, "_em_get_text", side_effect=_em_router):+ reports = etf._cn_periodic_reports("510300")+ assert reports == {+ "2025-12-31": {"report": f"{_PREFIX}2025年年度报告", "published": "2026-03-31"},+ "2025-06-30": {"report": f"{_PREFIX}2025年中期报告", "published": "2025-08-30"},+ }++ def test_issuer_cross_referenced_rows_are_labelled_and_kept_out_of_the_total(self):+ # Eastmoney's own footnote: a starred 序号 is a position disclosed by the+ # ISSUER's top-10 float holders, not by the fund's report. Counting it in+ # the disclosed percentage would overstate what the filing covers.+ rows = [+ _cn_row("1", "600519", "1", "贵州茅台", "7.29%", "121.09", "143,549.56"),+ _cn_row("11*", "600036", "1", "招商银行", "1.50%", "500.00", "20,000.00"),+ ]+ period = etf._parse_cn_period(+ _cn_block("2026年1季度股票投资明细", "2026-03-31", rows, expandable=False).split(+ "<div class='boxitem"+ )[1]+ )+ assert [h["disclosure_source"] for h in period["holdings"]] == [+ "fund_report",+ "issuer_top10_float_holders",+ ]+ assert period["fund_report_holdings"] == 1+ assert period["cross_referenced_holdings"] == 1+ assert period["pct_of_net_assets_disclosed"] == 7.29+ assert period["pct_of_net_assets_cross_referenced"] == 1.5++ def test_cross_reference_mode_expands_every_quarter(self):+ seen: list[str] = []++ def router(url, *, params, referer):+ if url == etf._EM_ANNOUNCEMENT_URL:+ return _JJGG+ seen.append(params["month"])+ return _JJCC_2026++ with patch.object(etf, "_em_get_text", side_effect=router):+ EtfHoldingsTool().execute(+ mode="holdings", symbol="510300.SH", disclosure="cross_reference"+ )+ assert seen == ["3,6,9,12"]++ def test_default_modes_expand_only_the_full_disclosure_periods(self):+ seen: list[str] = []++ def router(url, *, params, referer):+ if url == etf._EM_ANNOUNCEMENT_URL:+ return _JJGG+ seen.append(params["month"])+ return _JJCC_2026++ with patch.object(etf, "_em_get_text", side_effect=router):+ EtfHoldingsTool().execute(mode="holdings", symbol="510300.SH")+ assert seen == ["6,12"]++ def test_an_unclosed_info_cell_does_not_shift_the_columns(self):+ # Live regression: 510500's 2025 annual report leaves the 相关资讯 cell+ # of 400174 中天3 unclosed. Merging it with the next cell read the share+ # count as the weight, turning 0.00% into 1877.51% and the fund's+ # disclosed total into 1972.86%.+ row = _cn_row(+ "501", "400174", "1", "中天3", "0.00%", "1,877.51", "262.85",+ close_info_cell=False,+ )+ period = etf._parse_cn_period(+ _cn_block("2025年4季度股票投资明细", "2025-12-31", [row], expandable=False).split(+ "<div class='boxitem"+ )[1]+ )+ assert period["unparseable_rows"] == 0+ assert period["holdings"][0]["pct_of_net_assets"] == 0.0+ assert period["holdings"][0]["shares"] == 18775100.0+ assert period["holdings"][0]["market_value_cny"] == 2628500.0++ def test_a_row_the_header_cannot_explain_is_dropped_and_counted(self):+ good = _cn_row("1", "600519", "1", "贵州茅台", "7.29%", "121.09", "143,549.56")+ # An extra trailing cell leaves the column map pointing at the wrong+ # values; reporting it would file one column's number under another's.+ wrong_width = good.replace("</tr>", "<td class='tor'>stray</td></tr>")+ period = etf._parse_cn_period(+ _cn_block(+ "2025年4季度股票投资明细", "2025-12-31", [good, wrong_width], expandable=False+ ).split("<div class='boxitem")[1]+ )+ assert len(period["holdings"]) == 1+ assert period["unparseable_rows"] == 1++ def test_dropped_rows_are_surfaced_in_the_answer_not_swallowed(self):+ broken = _JJCC_2025.replace(+ "<td class='tor'>60,000.00</td></tr>",+ "<td class='tor'>60,000.00</td><td>stray</td></tr>",+ 1,+ )++ def router(url, *, params, referer):+ return _JJGG if url == etf._EM_ANNOUNCEMENT_URL else broken++ with patch.object(etf, "_em_get_text", side_effect=router):+ payload = json.loads(+ EtfHoldingsTool().execute(mode="holdings", symbol="510300.SH", year=2025)+ )+ assert payload["data"]["unparseable_rows"] == 1+ assert "WARNING" in payload["notes"]++ def test_every_period_carries_its_own_coverage_verdict(self):+ with patch.object(etf, "_em_get_text", side_effect=_em_router):+ payload = json.loads(+ EtfHoldingsTool().execute(+ mode="holdings", symbol="510300.SH", disclosure="full"+ )+ )+ assert payload["data"]["available_periods"] == [+ {+ "as_of": "2025-12-31",+ "report_label": "2025年4季度股票投资明细",+ "coverage": "full_portfolio",+ "holdings_in_period": 20,+ "pct_of_net_assets_disclosed": 97.96,+ },+ {+ "as_of": "2025-09-30",+ "report_label": "2025年3季度股票投资明细",+ "coverage": "top_n_disclosed",+ "holdings_in_period": 1,+ "pct_of_net_assets_disclosed": 8.1,+ },+ ]++ def test_full_refuses_rather_than_passing_off_a_partial_book(self):+ def router(url, *, params, referer):+ return _JJGG if url == etf._EM_ANNOUNCEMENT_URL else _JJCC_2026++ with patch.object(etf, "_em_get_text", side_effect=router):+ payload = json.loads(+ EtfHoldingsTool().execute(+ mode="holdings", symbol="510300.SH", disclosure="full", year=2026+ )+ )+ assert payload["ok"] is False+ assert "no fully-disclosed portfolio" in payload["error"]+ assert "2025-12-31" in payload["error"]++ def test_column_positions_are_read_from_the_header_not_assumed(self):+ # A six-column period: no 相关资讯 column, so fixed indices would slip.+ block = (+ "<div class='boxitem w790'><h4 class='t'><label class='left'>"+ f"<a title='{_FUND_NAME}' href='#'>{_FUND_NAME}</a> "+ "2026年1季度股票投资明细</label><label class='right'>截止至:"+ "<font class='px12'>2026-03-31</font></label></h4>"+ "<table class='w782 comm tzxq'><thead><tr><th>序号</th><th>股票代码</th>"+ "<th>股票名称</th><th>占净值<br />比例</th><th>持股数<br />(万股)</th>"+ "<th>持仓市值<br />(万元)</th></tr></thead><tbody>"+ "<tr><td>1</td><td><a href='//quote.eastmoney.com/unify/r/1.600519'>600519</a>"+ "</td><td class='tol'>贵州茅台</td><td class='tor'>8.10%</td>"+ "<td class='tor'>100.00</td><td class='tor'>120,000.00</td></tr>"+ "</tbody></table></div>"+ )+ period = etf._parse_cn_period(block)+ assert period["as_of"] == "2026-03-31"+ assert period["holdings"][0]["pct_of_net_assets"] == 8.10+ assert period["holdings"][0]["market_value_cny"] == 1200000000.0++ def test_newest_period_wins_even_if_the_server_reorders_the_blocks(self):+ # The answer's as_of must come from the dates in the payload, not from+ # the position Eastmoney happened to put the block in.+ reordered = _jjcc(+ _cn_block(+ "2026年1季度股票投资明细",+ "2026-03-31",+ [_cn_row("1", "600519", "1", "贵州茅台", "8.10%", "100.00", "120,000.00")],+ expandable=True,+ ),+ _cn_block("2026年2季度股票投资明细", "2026-06-30", _Q2_2026_ROWS, expandable=False),+ )++ def router(url, *, params, referer):+ return _JJGG if url == etf._EM_ANNOUNCEMENT_URL else reordered++ with patch.object(etf, "_em_get_text", side_effect=router):+ payload = json.loads(EtfHoldingsTool().execute(mode="holdings", symbol="510300.SH"))+ assert payload["as_of"] == "2026-06-30"+ assert payload["data"]["holdings_in_period"] == 15++ def test_requested_year_is_never_echoed_as_the_period(self):+ # The endpoint silently ignores an unavailable year and answers with the+ # current one, so the payload's own date must win.+ with patch.object(etf, "_em_get_text", side_effect=_em_router):+ payload = json.loads(+ EtfHoldingsTool().execute(mode="holdings", symbol="510300.SH", year=2099)+ )+ assert payload["as_of"] == "2026-06-30"++ def test_referer_is_sent_because_the_endpoint_404s_without_one(self):+ seen: dict[str, str] = {}++ def router(url, *, params, referer):+ seen[url] = referer+ return _JJGG if url == etf._EM_ANNOUNCEMENT_URL else _JJCC_2026++ with patch.object(etf, "_em_get_text", side_effect=router):+ EtfHoldingsTool().execute(mode="holdings", symbol="510300.SH")+ assert seen[etf._EM_HOLDINGS_URL] == "https://fundf10.eastmoney.com/ccmx_510300.html"+ # The announcement API answers ErrCode -999 without one.+ assert seen[etf._EM_ANNOUNCEMENT_URL] == (+ "https://fundf10.eastmoney.com/jjgg_510300_3.html"+ )++ def test_empty_payload_is_an_error_not_an_empty_portfolio(self):+ def router(url, *, params, referer):+ if url == etf._EM_ANNOUNCEMENT_URL:+ return _JJGG+ return 'var apidata={ content:"",arryear:[]};'++ with patch.object(etf, "_em_get_text", side_effect=router):+ payload = json.loads(EtfHoldingsTool().execute(mode="holdings", symbol="518880.SH"))+ assert payload["ok"] is False+ assert "no stock-holding detail" in payload["error"]+++class TestCnLookup:+ """A-share fund search by code and by keyword."""++ def test_code_lookup_uses_the_single_quote_endpoint(self):+ with patch.object(etf, "_em_push_json", return_value=_QUOTE) as mock_json:+ payload = json.loads(EtfHoldingsTool().execute(mode="lookup", query="510050"))+ assert mock_json.call_args[0][0] == etf._EM_QUOTE_PATH+ match = payload["data"]["matches"][0]+ assert match["name"] == "上证50ETF华夏"+ assert match["fund_size_cny"] == 22902480112.128004+ assert payload["as_of"].startswith("2026-08-04T")++ def test_keyword_lookup_scans_the_paged_universe(self):+ with patch.object(etf, "_em_push_json", side_effect=lambda p, q: _list_page(int(q["pn"]))):+ payload = json.loads(EtfHoldingsTool().execute(mode="lookup", query="黄金"))+ assert payload["universe_complete"] is True+ assert payload["data"]["matches"] == [+ {+ "symbol": "518880.SH",+ "code": "518880",+ "name": "黄金ETF华安",+ "total_market_cap_cny": 9.2e10,+ "quote_date": "2026-08-04",+ }+ ]++ def test_partial_universe_is_flagged_and_not_cached(self):+ calls = {"n": 0}++ def flaky(path, params):+ calls["n"] += 1+ if int(params["pn"]) == 1:+ return _list_page(1)+ raise RuntimeError("edge dropped the connection")++ with patch.object(etf, "_em_push_json", side_effect=flaky):+ payload = json.loads(EtfHoldingsTool().execute(mode="lookup", query="半导体"))+ assert payload["universe_complete"] is False+ assert "PARTIAL" in payload["notes"]+ assert etf._CN_LIST_CACHE is None++ def test_first_page_failure_surfaces_as_an_error(self):+ with patch.object(etf, "_em_push_json", side_effect=RuntimeError("boom")):+ payload = json.loads(EtfHoldingsTool().execute(mode="lookup", query="黄金"))+ assert payload["ok"] is False+ assert "ETF universe" in payload["error"]++ def test_quote_host_fallback_tries_the_next_edge(self):+ attempted: list[str] = []++ def flaky(url, params=None):+ attempted.append(url)+ if url.startswith(f"https://{etf._EM_QUOTE_HOSTS[0]}"):+ raise RuntimeError("remote end closed connection")+ return _QUOTE++ with patch.object(etf, "get_json", side_effect=flaky):+ result = etf._em_push_json(etf._EM_QUOTE_PATH, {"secid": "1.510050"})+ assert result == _QUOTE+ assert len(attempted) == 2+++class TestArgumentValidation:+ """Bad input yields the error envelope rather than an exception."""++ @pytest.mark.parametrize(+ "kwargs,fragment",+ [+ ({}, "mode must be one of"),+ ({"mode": "nope"}, "mode must be one of"),+ ({"mode": "lookup"}, "'query' is required"),+ ({"mode": "lookup", "query": " "}, "'query' is required"),+ ({"mode": "holdings"}, "'symbol' is required"),+ ({"mode": "holdings", "symbol": "IVV", "market": "MARS"}, "market must be one of"),+ ({"mode": "holdings", "symbol": "510050", "year": "2025"}, "year must be an integer"),+ (+ {"mode": "holdings", "symbol": "510050", "disclosure": "everything"},+ "disclosure must be one of",+ ),+ ({"mode": "lookup", "query": "IVV", "offset": "x"}, "offset must be an integer"),+ ],+ )+ def test_invalid_arguments(self, kwargs, fragment):+ payload = json.loads(EtfHoldingsTool().execute(**kwargs))+ assert payload["ok"] is False+ assert fragment in payload["error"]++ @pytest.mark.parametrize(+ "value,expected",+ [(None, 25), (0, 1), (-5, 1), (10_000, 6000), ("30", 30), ("junk", 25)],+ )+ def test_top_n_is_clamped(self, value, expected):+ assert etf._clamp(value, 25, etf._MAX_TOP_N) == expected++ def test_top_n_ceiling_clears_a_real_full_portfolio(self):+ # Measured on the live 2025 annual reports: 510500 → 541, 159845 → 1044,+ # and the CSI-2000 tracker 159531 → 2031 holdings. A ceiling at or below+ # any of those hands back a truncated book still labelled+ # ``full_portfolio``, so it has to clear the largest one with headroom.+ assert etf._MAX_TOP_N >= 2031++ def test_a_top_n_cut_full_portfolio_says_it_was_cut(self):+ # The regression: top_n slices the period's rows before paging, so+ # paging reports the slice as its total and a caller who pages to+ # ``complete`` sees nothing to suggest rows were withheld — a complete+ # portfolio silently served as a partial one. Measured live on 159531,+ # whose 2031-row annual report came back as 1000 rows still labelled+ # ``full_portfolio`` with paging.total 1000.+ with patch.object(etf, "_em_get_text", side_effect=_em_router):+ payload = json.loads(+ EtfHoldingsTool().execute(+ mode="holdings", symbol="510300.SH", disclosure="full", top_n=5+ )+ )+ assert payload["coverage"] == "full_portfolio"+ assert payload["data"]["holdings_in_period"] == 20+ assert payload["data"]["holdings_withheld_by_top_n"] == 15+ assert payload["paging"]["total"] == 5+ # The caller is told the whole book is not in hand, both ways.+ assert "full_portfolio" in payload["missing_fields"]+ assert "cut 15 of this period's 20 disclosed rows" in payload["notes"]++ def test_an_untruncated_full_portfolio_reports_nothing_withheld(self):+ with patch.object(etf, "_em_get_text", side_effect=_em_router):+ payload = json.loads(+ EtfHoldingsTool().execute(+ mode="holdings", symbol="510300.SH", disclosure="full", top_n=100+ )+ )+ assert payload["data"]["holdings_withheld_by_top_n"] == 0+ assert "full_portfolio" not in payload["missing_fields"]+ assert "top_n=" not in payload["notes"]++ @pytest.mark.parametrize("value", ["FULL", "Full", " full "])+ def test_disclosure_is_case_and_space_insensitive(self, value):+ with patch.object(etf, "_em_get_text", side_effect=_em_router):+ payload = json.loads(+ EtfHoldingsTool().execute(+ mode="holdings", symbol="510300.SH", disclosure=value+ )+ )+ assert payload["ok"] is True+ assert payload["coverage"] == "full_portfolio"++ @pytest.mark.parametrize("value", ["us", "US", "Us"])+ def test_market_override_is_case_insensitive(self, value):+ with patch.object(etf, "_sec_get_text", side_effect=_sec_router):+ payload = json.loads(+ EtfHoldingsTool().execute(mode="lookup", query="510050", market=value)+ )+ assert payload["market"] == "US"++ def test_tool_contract(self):+ tool = EtfHoldingsTool()+ assert tool.name == "etf_holdings"+ assert tool.is_readonly is True+ assert tool.check_available() is True+ assert tool.parameters["required"] == ["mode"]+ assert json.dumps(tool.to_openai_schema())