RAGFlow is an agent-native database: a Retrieval-Augmented Generation (RAG) engine that grew into context infrastructure.
Go agent-harnessagent-native-databaseagentic-aiagentic-retrievalagentic-searchaiai-agentscontext-enginecontext-engineeringcontext-managementharness-engineeringknowledge-compilationragretrieval-augmented-generationretrieval-harnesssearch-harness
test(sdk): audit chunk management tests (#21635)
Sbaaoui Idriss3f6939db1a35
diff --git a/test/testcases/test_sdk_api/test_chunk_management_within_dataset/README.md b/test/testcases/test_sdk_api/test_chunk_management_within_dataset/README.mdnew file mode 100644index 0000000000..0183cf0ad7--- /dev/null+++ b/test/testcases/test_sdk_api/test_chunk_management_within_dataset/README.md@@ -0,0 +1,170 @@+# Chunk SDK contract audit++This folder targets the checked-in Python SDK and Go API. It does not use the+legacy Python server as a contract oracle. No production code or full SDK CI is+changed. Dataset/document audit PRs #21619 and #21633 are not prerequisites.++## Running safely++Run from the repository root with **both** the local config and `--confcutdir`.+This excludes ancestor fixtures that bootstrap credentials, mutate configuration,+parse shared documents, or delete every dataset belonging to the test tenant.++```sh+CHUNKS=test/testcases/test_sdk_api/test_chunk_management_within_dataset+python -m pytest -c "$CHUNKS/pytest.ini" --confcutdir="$CHUNKS" "$CHUNKS" --collect-only -q+python -m pytest -c "$CHUNKS/pytest.ini" --confcutdir="$CHUNKS" "$CHUNKS" -m 'not integration' -q+```++Integration requires a disposable **Go backend and dedicated tenant**, an embedding+model configured for that tenant, and its search/storage dependencies. Set+`RAGFLOW_SDK_ISOLATED=1`, `RAGFLOW_SDK_BASE_URL`, and `RAGFLOW_SDK_API_KEY` only after+verifying ownership of that environment, then run the same command without `-m`.+The guard fails setup rather than disguising missing infrastructure as a defect+skip. `RAGFLOW_SDK_INDEX_TIMEOUT` controls the indexing deadline (default 60 seconds).+Request connect/read timeouts are 5/60 seconds. API errors are never retried as+indexing lag. The 1,000-chunk deletion case is intentionally substantial.++The valid reranker case also requires `RAGFLOW_SDK_RERANK_ID`; keyword expansion+requires a configured tenant chat model and `RAGFLOW_SDK_CHAT_CONFIGURED=1`.+Missing model configuration fails the relevant case; it is not a production skip.+Run against Elasticsearch and Infinity separately. Current `internal/engine/global.go`+has no OpenSearch backend; obsolete OpenSearch count branches are removed.++Fixtures create a unique dataset per case and delete only that dataset ID, including+when upload fails. Cleanup errors remain visible and deletion is verified by ID.+Chunks are created manually; there is no ingestion run or `document.run == "DONE"`+assumption. Listing and retrieval readiness are checked independently. Raw list+responses are used for persistence probes because the SDK list schema is defective;+a separate skipped test retains the correct public SDK schema assertions.++## Contract evidence++The audited call paths are:++- `sdk/python/ragflow_sdk/modules/document.py`: `add_chunk`, `delete_chunks`,+ `list_chunks`; `modules/chunk.py`: `Chunk.update`, `ChunkUpdateError`;+ `ragflow.py`: `RAGFlow.retrieve`.+- `internal/handler/chunk.go`: request binding, query validation, update type+ handling; `internal/service/chunk/chunk.go`: add/list/update/remove semantics,+ ownership, image storage, engine requests and response fields.+- `internal/handler/dataset.go`: `SearchDatasets` typed request validation;+ `internal/service/dataset/search.go` and `internal/service/nlp/`: retrieval,+ candidate selection, score ordering, document/metadata/availability filtering.+- `internal/engine/elasticsearch/chunk.go`, `internal/engine/infinity/chunk.go`:+ lexical/vector matching and fusion. `top_k` is a candidate limit, not a promised+ response count; `page_size` is a response limit.+- `docs/references/python_api_reference.md` and `http_api_reference.md`: chunk and+ retrieval contracts. Python-specific exception examples are obsolete. The old+ empty-question example also conflicts with current Go required-field validation.+- `internal/handler/chunk_test.go`, `internal/service/chunk/chunk_test.go`,+ `internal/handler/dataset_search_test.go`, and `test/api/retrieval_api_test.go`:+ existing Go validation, ownership, idempotency, schema and retrieval coverage.++These sources establish: add rejects empty/non-string content and invalid list+payloads; update accepts empty string content but must reject invalid types;+questions are trimmed and empty entries removed; duplicate adds use a stable ID;+delete is document-scoped and unknown/repeated IDs are idempotent; numeric list+query strings are accepted, whereas retrieval JSON numeric strings are rejected.+Updates return `None` and do not mutate the SDK object's cached attributes.++Retrieval tests use a small owned corpus with a shared lexical query, threshold+zero and text-only readiness checks. They assert identity, ownership, content,+finite ordered scores, pagination, filtering and response limits. They do not+require arbitrary four/five-result counts from normal hybrid retrieval. The+single-result lexical list filters use unique terms in the owned fixture.++## Original test inventory and decisions++All six original files, their fixtures/helpers, and every parameter branch were+reviewed: **36 original test methods, 134 collected parameter cases**. Classification+below is by original method: **20 UPDATE, 16 REPLACE, 0 unchanged KEEP, 0 whole-test+DELETE**. Valid checks within rewritten methods remain. Consolidated duplicates+include engine-specific lexical/top-k alternatives and repeated missing-argument+checks; missing-argument behavior now has a service-free test. SKIP is an orthogonal+case-level outcome, detailed below, rather than deletion of the original assertions.++| Original file / method | Original cases | Decision | Coverage after audit |+| --- | ---: | --- | --- |+| add / content | 6 | UPDATE | Null, empty, whitespace, integer, text and punctuation; failed requests do not add IDs. |+| add / important_keywords | 6 | UPDATE | String lists, empty members, duplicates, non-string members and scalar rejection. |+| add / questions | 6 | UPDATE | Same boundaries plus normalization of blank questions. |+| add / tag_kwd | 6 | UPDATE | String-list contract and exact Go validation messages. |+| add / repeated_add_chunk | 1 | UPDATE | Same content returns the same ID and persists once. |+| add / add_chunk_to_deleted_document | 1 | UPDATE | Current document ownership error. |+| add / concurrent_add_chunk | 1 | UPDATE | Consume all 50 futures, unique IDs and persisted ID set. |+| delete / delete_partial_invalid_id | 3 | REPLACE | Unknown IDs at three positions do not prevent owned valid deletions. |+| delete / repeated_deletion | 1 | REPLACE | Repeated deletion is idempotent. |+| delete / duplicate_deletion | 1 | REPLACE | Duplicate IDs are idempotent. |+| delete / concurrent_deletion | 1 | UPDATE | All futures succeed and persistence becomes empty. |+| delete / delete_1k | 1 | UPDATE | Preserve bulk coverage; paginated readiness and deletion probes. |+| delete / basic_scenarios | 6 | REPLACE | Unknown/single/all IDs, explicit delete_all, safe no-op and Python kwargs boundary. |+| list / page | 7 | UPDATE | Default/null, valid pages, numeric strings, zero/negative/non-numeric validation. |+| list / page_size | 7 | UPDATE | Default/null, bounds, numeric strings and invalid values. |+| list / keywords | 7 | REPLACE | Empty/default, shared term, unique term, numeric term and no-match; no engine count exceptions. |+| list / id | 4 | REPLACE | Empty IDs remain active; known/unknown filters retain correct expectations under targeted skips. |+| list / concurrent_list | 1 | UPDATE | Consume each future and validate owned fixture count. |+| list / default | 1 | UPDATE | 31 explicit chunks establish the default limit of 30; no sleep. |+| list / invalid_document_id_raises | 1 | UPDATE | Assert meaningful current Go error text. |+| retrieval / basic_scenarios | 4 | REPLACE | Owned results with/without document filter; Python argument errors tested separately. |+| retrieval / page | 7 | REPLACE | Compare slices of a controlled baseline; retain invalid inputs in shared negative matrix. |+| retrieval / page_size | 5 | REPLACE | Defaults, single result, caps and typed/range errors. |+| retrieval / vector_similarity_weight | 4 | UPDATE | Weights 0, 0.5, 1; finite ordered owned results and invalid ranges/types. |+| retrieval / top_k | 5 | REPLACE | Candidate limits 1/10/2048, response cap; invalid values, no engine result-count branches. |+| retrieval / rerank_id | 2 | REPLACE | Configured reranker and explicit unknown-model rejection; no blanket skip. |+| retrieval / keyword | 5 | REPLACE | False/null/true, configured expansion, string-bool rejection; no blanket skip. |+| retrieval / concurrent_retrieval | 1 | UPDATE | Consume all futures and verify nonempty owned ordered results. |+| update / content | 6 | UPDATE | Persist text/punctuation/empty/space; null/integer rejection remains skipped for defect. |+| update / important_keywords | 6 | REPLACE | Shared typed list matrix; seed then clear; invalid cases retain rejection assertions. |+| update / questions | 6 | REPLACE | Shared matrix with trimming/blank normalization and invalid-type skips. |+| update / tag_kwd | 6 | REPLACE | Shared matrix with active strict negative cases. |+| update / available | 6 | REPLACE | Both bool transitions persist; string/integer rejection retained under defect skip. |+| update / repeated_update_chunk | 1 | UPDATE | Every sequential/repeated write is verified in storage. |+| update / concurrent_update_chunk | 1 | UPDATE | 50 contending writes across four chunks; all futures observed and final values submitted. |+| update / update_chunk_to_deleted_document | 1 | UPDATE | `ChunkUpdateError`, current code/message rather than generic exception. |++New coverage includes add list-null/empty and image payload boundaries; delete_all+conflict/document isolation; list response schema; retrieval negative matrix,+document/metadata/availability filters and pagination uniqueness; unsupported update+fields; SDK payload/default/error contracts; polling and cleanup failure behavior.++## Skip audit++Historical references were inspected rather than carried forward based on issue+status: #6411, #6509, #6499, #6554 and #10692 concern the legacy Python backend;+PR #6376 addressed duplicate-ID deletion validation. Their old skips are removed+or replaced by the current defects below. The old `**"not json"` case never reaches+HTTP; a service-free TypeError check replaces that skip. Old list negative-input+skips now assert Go validation, and rerank/keyword groups execute with explicit+model prerequisites. No retrieval group is blanket-skipped.++**20 cases remain skipped for source-confirmed contract defects:**++| Cases | Defect and source boundary | Correct assertion retained |+| ---: | --- | --- |+| 3 | `RemoveChunks` rejects omitted/null/empty IDs without delete_all, contrary to both API references. | Safe no-op preserves IDs. |+| 2 | SDK `list_chunks(id=...)` sends `id`; Go `ListChunks` reads `chunk_ids`. | Known ID selects one; unknown ID returns none. |+| 1 | Go list returns storage names (`chunk_id`, `content_with_weight`, `important_kwd`, `question_kwd`) and omits document/dataset IDs; SDK constructs `Chunk` without mapping. | SDK identity/content/ownership/keyword/question schema. |+| 2 | Go `UpdateChunk` silently ignores null/integer content. | Typed `ChunkUpdateError` mentioning content. |+| 4 | Go update ignores invalid keyword scalar/null types or converts non-string members to empty strings. | List-type validation error. |+| 4 | Go update has the same invalid questions handling. | List-type validation error. |+| 4 | Go update silently ignores string/integer availability. | Typed `ChunkUpdateError` mentioning availability. |++No production fix is included. Defects are confirmed by current source paths, not+by integration execution in this audit environment. Correct assertions remain+inside every skipped case.++## Validation boundary++The resulting suite collects **195 cases**: add 39, delete 17, list 28, retrieval+48, update 40, and 23 service-free SDK/fixture cases. Python syntax and focused+Ruff checks pass. All **23 service-free cases pass**, including payloads, SDK error+propagation, cleanup ownership/failures, deadlines and pagination-loop detection.++A full invocation with no isolated backend configured produces **23 passed,+20 skipped, 152 setup errors, zero assertion failures**. Every setup error is the+explicit integration isolation guard; report these as **152 environment-blocked+cases**, not production defects or successful integration tests. The existing+host stack is not verified disposable and was not used. Elasticsearch/Infinity,+model-backed retrieval/reranking/keyword expansion, 1,000-chunk deletion, images+and concurrency still require execution in the isolated environment described above.diff --git a/test/testcases/test_sdk_api/test_chunk_management_within_dataset/conftest.py b/test/testcases/test_sdk_api/test_chunk_management_within_dataset/conftest.pyindex 4727c85a48..940e99a693 100644--- a/test/testcases/test_sdk_api/test_chunk_management_within_dataset/conftest.py+++ b/test/testcases/test_sdk_api/test_chunk_management_within_dataset/conftest.py@@ -15,41 +15,96 @@#+import os+from time import monotonic, sleep+from uuid import uuid4+import pytest-from common import batch_add_chunks, delete_all_chunks-from pytest import FixtureRequest-from ragflow_sdk import Chunk, DataSet, Document-from utils import wait_for+import requests+from ragflow_sdk import RAGFlow+++def wait_until(probe, timeout=None):+ if timeout is None:+ timeout = float(os.getenv("RAGFLOW_SDK_INDEX_TIMEOUT", "60"))+ assert timeout > 0, "Index timeout must be positive"+ deadline = monotonic() + timeout+ while True:+ value = probe()+ if value:+ return value+ if monotonic() >= deadline:+ pytest.fail("Chunk indexing deadline exceeded")+ sleep(0.25)+++def raw_chunks(document, page=1, page_size=100, **params):+ response = document.get(+ f"/datasets/{document.dataset_id}/documents/{document.id}/chunks",+ {"page": page, "page_size": page_size, **params},+ ).json()+ assert response["code"] == 0, response+ return response["data"]["chunks"]+++def raw_ids(document):+ ids = set()+ page = 1+ while True:+ chunks = raw_chunks(document, page=page)+ page_ids = {chunk["chunk_id"] for chunk in chunks}+ assert len(page_ids) == len(chunks), "Chunk page contains duplicate IDs"+ assert not ids.intersection(page_ids), "Chunk pagination repeated IDs"+ ids.update(page_ids)+ if len(chunks) < 100:+ return ids+ page += 1+++def wait_ids(document, expected):+ expected = set(expected)+ wait_until(lambda: raw_ids(document) == expected)++[email protected](autouse=True)+def network_boundary(request, monkeypatch):+ original = requests.sessions.Session.request+ def bounded(session, *args, **kwargs):+ if request.node.get_closest_marker("integration") is None:+ pytest.fail("Service-free chunk tests must not open network connections")+ kwargs.setdefault("timeout", (5, 60))+ return original(session, *args, **kwargs)-@wait_for(30, 1, "Document parsing timeout")-def condition(_dataset: DataSet):- documents = _dataset.list_documents(page_size=100)- for document in documents:- if document.run != "DONE":- return False- return True+ monkeypatch.setattr(requests.sessions.Session, "request", bounded)-@wait_for(30, 1, "Chunk indexing timeout")-def chunks_visible(_document: Document, _chunk_ids: list[str]):- visible_ids = {chunk.id for chunk in _document.list_chunks(page_size=100)}- return set(_chunk_ids).issubset(visible_ids)+def client():+ if os.getenv("RAGFLOW_SDK_ISOLATED") != "1" or not all(os.getenv(key) for key in ("RAGFLOW_SDK_BASE_URL", "RAGFLOW_SDK_API_KEY")):+ pytest.fail("Integration blocked: set RAGFLOW_SDK_ISOLATED=1, RAGFLOW_SDK_BASE_URL and RAGFLOW_SDK_API_KEY for an isolated Go backend")+ return RAGFlow(os.environ["RAGFLOW_SDK_API_KEY"], os.environ["RAGFLOW_SDK_BASE_URL"])[email protected](scope="function")-def add_chunks_func(request: FixtureRequest, add_document: tuple[DataSet, Document]) -> tuple[DataSet, Document, list[Chunk]]:- def cleanup():- try:- delete_all_chunks(document)- except Exception:- pass+def add_document(client):+ dataset = client.create_dataset(name=f"sdk-chunks-{uuid4().hex}")+ try:+ documents = dataset.upload_documents([{"display_name": "chunks.txt", "blob": b"Owned chunk SDK fixture."}])+ assert len(documents) == 1+ yield dataset, documents[0]+ finally:+ client.delete_datasets(ids=[dataset.id])+ assert client.list_datasets(id=dataset.id) == [], "Owned dataset cleanup failed"- request.addfinalizer(cleanup)+def add_chunks(add_document):dataset, document = add_document- dataset.async_parse_documents([document.id])- condition(dataset)- chunks = batch_add_chunks(document, 4)- chunks_visible(document, [chunk.id for chunk in chunks])+ # Manual chunk operations do not need ingestion or a parser status wait.+ chunks = [+ document.add_chunk(content=f"Coriander botanical archive {label} entry {i}.", important_keywords=["coriander"], questions=["What is coriander?"])+ for i, label in enumerate(("saffron", "cumin", "basil", "thyme"))+ ]+ wait_ids(document, [chunk.id for chunk in chunks])return dataset, document, chunksdiff --git a/test/testcases/test_sdk_api/test_chunk_management_within_dataset/pytest.ini b/test/testcases/test_sdk_api/test_chunk_management_within_dataset/pytest.ininew file mode 100644index 0000000000..4434268134--- /dev/null+++ b/test/testcases/test_sdk_api/test_chunk_management_within_dataset/pytest.ini@@ -0,0 +1,8 @@+[pytest]+pythonpath = ../../../../sdk/python+addopts = --strict-markers+markers =+ integration: requires an explicitly isolated Go backend and dedicated tenant+ p1: high priority+ p2: medium priority+ p3: low prioritydiff --git a/test/testcases/test_sdk_api/test_chunk_management_within_dataset/test_add_chunk.py b/test/testcases/test_sdk_api/test_chunk_management_within_dataset/test_add_chunk.pyindex 2db1022752..8cc923204e 100644--- a/test/testcases/test_sdk_api/test_chunk_management_within_dataset/test_add_chunk.py+++ b/test/testcases/test_sdk_api/test_chunk_management_within_dataset/test_add_chunk.py@@ -14,7 +14,7 @@# limitations under the License.#from concurrent.futures import ThreadPoolExecutor, as_completed-from time import sleep+from conftest import raw_ids, wait_idsimport pytestfrom ragflow_sdk import Chunk@@ -32,14 +32,17 @@ def validate_chunk_details(dataset_id: str, document_id: str, payload: dict, chuassert chunk.tag_kwd == payload["tag_kwd"]+pytestmark = pytest.mark.integration++class TestAddChunk:@pytest.mark.p1@pytest.mark.parametrize("payload, expected_message",[- ({"content": None}, "not instance of"),+ ({"content": None}, "`content` is required"),({"content": ""}, "`content` is required"),- ({"content": 1}, "not instance of"),+ ({"content": 1}, "`content` must be a string"),({"content": "a"}, ""),({"content": " "}, "`content` is required"),({"content": "\n!?。;!?\"'"}, ""),@@ -47,19 +50,18 @@ class TestAddChunk:)def test_content(self, add_document, payload, expected_message):dataset, document = add_document- chunks_count = len(document.list_chunks())+ initial_ids = raw_ids(document)if expected_message:with pytest.raises(Exception) as exception_info:document.add_chunk(**payload)assert expected_message in str(exception_info.value), str(exception_info.value)+ assert raw_ids(document) == initial_idselse:chunk = document.add_chunk(**payload)validate_chunk_details(dataset.id, document.id, payload, chunk)- sleep(1)- chunks = document.list_chunks()- assert len(chunks) == chunks_count + 1, str(chunks)+ wait_ids(document, initial_ids | {chunk.id})@pytest.mark.p2@pytest.mark.parametrize(@@ -67,27 +69,26 @@ class TestAddChunk:[({"content": "chunk test important_keywords 1", "important_keywords": ["a", "b", "c"]}, ""),({"content": "chunk test important_keywords 2", "important_keywords": [""]}, ""),- ({"content": "chunk test important_keywords 3", "important_keywords": [1]}, "not instance of"),+ ({"content": "chunk test important_keywords 3", "important_keywords": [1]}, "`important_keywords` must be a list of strings"),({"content": "chunk test important_keywords 4", "important_keywords": ["a", "a"]}, ""),- ({"content": "chunk test important_keywords 5", "important_keywords": "abc"}, "not instance of"),- ({"content": "chunk test important_keywords 6", "important_keywords": 123}, "not instance of"),+ ({"content": "chunk test important_keywords 5", "important_keywords": "abc"}, "`important_keywords` is required to be a list"),+ ({"content": "chunk test important_keywords 6", "important_keywords": 123}, "`important_keywords` is required to be a list"),],)def test_important_keywords(self, add_document, payload, expected_message):dataset, document = add_document- chunks_count = len(document.list_chunks())+ initial_ids = raw_ids(document)if expected_message:with pytest.raises(Exception) as exception_info:document.add_chunk(**payload)assert expected_message in str(exception_info.value), str(exception_info.value)+ assert raw_ids(document) == initial_idselse:chunk = document.add_chunk(**payload)validate_chunk_details(dataset.id, document.id, payload, chunk)- sleep(1)- chunks = document.list_chunks()- assert len(chunks) == chunks_count + 1, str(chunks)+ wait_ids(document, initial_ids | {chunk.id})@pytest.mark.p3@pytest.mark.parametrize(@@ -95,27 +96,26 @@ class TestAddChunk:[({"content": "chunk test test_questions 1", "questions": ["a", "b", "c"]}, ""),({"content": "chunk test test_questions 2", "questions": [""]}, ""),- ({"content": "chunk test test_questions 3", "questions": [1]}, "not instance of"),+ ({"content": "chunk test test_questions 3", "questions": [1]}, "`questions` must be a list of strings"),({"content": "chunk test test_questions 4", "questions": ["a", "a"]}, ""),- ({"content": "chunk test test_questions 5", "questions": "abc"}, "not instance of"),- ({"content": "chunk test test_questions 6", "questions": 123}, "not instance of"),+ ({"content": "chunk test test_questions 5", "questions": "abc"}, "`questions` is required to be a list"),+ ({"content": "chunk test test_questions 6", "questions": 123}, "`questions` is required to be a list"),],)def test_questions(self, add_document, payload, expected_message):dataset, document = add_document- chunks_count = len(document.list_chunks())+ initial_ids = raw_ids(document)if expected_message:with pytest.raises(Exception) as exception_info:document.add_chunk(**payload)assert expected_message in str(exception_info.value), str(exception_info.value)+ assert raw_ids(document) == initial_idselse:chunk = document.add_chunk(**payload)validate_chunk_details(dataset.id, document.id, payload, chunk)- sleep(1)- chunks = document.list_chunks()- assert len(chunks) == chunks_count + 1, str(chunks)+ wait_ids(document, initial_ids | {chunk.id})@pytest.mark.p2@pytest.mark.parametrize(@@ -123,45 +123,41 @@ class TestAddChunk:[({"content": "chunk test test_tag_kwd 1", "tag_kwd": ["tag1", "tag2"]}, ""),({"content": "chunk test test_tag_kwd 2", "tag_kwd": [""]}, ""),- ({"content": "chunk test test_tag_kwd 3", "tag_kwd": [1]}, "not instance of"),+ ({"content": "chunk test test_tag_kwd 3", "tag_kwd": [1]}, "`tag_kwd` must be a list of strings"),({"content": "chunk test test_tag_kwd 4", "tag_kwd": ["tag", "tag"]}, ""),- ({"content": "chunk test test_tag_kwd 5", "tag_kwd": "abc"}, "not instance of"),- ({"content": "chunk test test_tag_kwd 6", "tag_kwd": 123}, "not instance of"),+ ({"content": "chunk test test_tag_kwd 5", "tag_kwd": "abc"}, "`tag_kwd` is required to be a list"),+ ({"content": "chunk test test_tag_kwd 6", "tag_kwd": 123}, "`tag_kwd` is required to be a list"),],)def test_tag_kwd(self, add_document, payload, expected_message):dataset, document = add_document- chunks_count = len(document.list_chunks())+ initial_ids = raw_ids(document)if expected_message:with pytest.raises(Exception) as exception_info:document.add_chunk(**payload)assert expected_message in str(exception_info.value), str(exception_info.value)+ assert raw_ids(document) == initial_idselse:chunk = document.add_chunk(**payload)validate_chunk_details(dataset.id, document.id, payload, chunk)- sleep(1)- chunks = document.list_chunks()- assert len(chunks) == chunks_count + 1, str(chunks)+ wait_ids(document, initial_ids | {chunk.id})@pytest.mark.p3def test_repeated_add_chunk(self, add_document):payload = {"content": "chunk test repeated_add_chunk"}dataset, document = add_document- chunks_count = len(document.list_chunks())+ initial_ids = raw_ids(document)chunk1 = document.add_chunk(**payload)validate_chunk_details(dataset.id, document.id, payload, chunk1)- sleep(1)- chunks = document.list_chunks()- assert len(chunks) == chunks_count + 1, str(chunks)+ wait_ids(document, initial_ids | {chunk1.id})chunk2 = document.add_chunk(**payload)validate_chunk_details(dataset.id, document.id, payload, chunk2)- sleep(1)- chunks = document.list_chunks()- assert len(chunks) == chunks_count + 1, str(chunks)+ assert chunk2.id == chunk1.id+ wait_ids(document, initial_ids | {chunk1.id})@pytest.mark.p2def test_add_chunk_to_deleted_document(self, add_document):@@ -172,19 +168,53 @@ class TestAddChunk:document.add_chunk(content="chunk test")assert f"you don't own the document {document.id}" in str(exception_info.value), str(exception_info.value)- @pytest.mark.skip(reason="issues/6411")@pytest.mark.p3def test_concurrent_add_chunk(self, add_document):count = 50_, document = add_document- initial_chunk_count = len(document.list_chunks())+ initial_ids = raw_ids(document)def add_chunk_task(i):return document.add_chunk(content=f"chunk test concurrent {i}")with ThreadPoolExecutor(max_workers=5) as executor:futures = [executor.submit(add_chunk_task, i) for i in range(count)]- responses = list(as_completed(futures))- assert len(responses) == count, responses- sleep(5)- assert len(document.list_chunks(page_size=100)) == initial_chunk_count + count+ responses = [future.result() for future in as_completed(futures)]+ assert len({chunk.id for chunk in responses}) == count+ wait_ids(document, initial_ids | {chunk.id for chunk in responses})++ @pytest.mark.parametrize("field", ["important_keywords", "questions", "tag_kwd"])+ @pytest.mark.parametrize("value", [[], None])+ def test_empty_list_and_null(self, add_document, field, value):+ dataset, document = add_document+ payload = {"content": "List boundary", field: value}+ if value is None:+ with pytest.raises(Exception, match=f"`{field}` is required to be a list"):+ document.add_chunk(**payload)+ assert raw_ids(document) == set()+ else:+ chunk = document.add_chunk(**payload)+ validate_chunk_details(dataset.id, document.id, payload, chunk)+ wait_ids(document, [chunk.id])++ @pytest.mark.parametrize(+ "image, error",+ [+ (None, None),+ ("iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAIAAACQd1PeAAAADElEQVR4nGP4z8AAAAMBAQDJ/pLvAAAAAElFTkSuQmCC", None),+ ("", "`image_base64` must be a non-empty string"),+ (" ", "`image_base64` must be a non-empty string"),+ ("not-base64", "invalid `image_base64`"),+ (123, "`image_base64` must be a string"),+ ],+ )+ def test_image(self, add_document, image, error):+ dataset, document = add_document+ if error:+ with pytest.raises(Exception, match=error):+ document.add_chunk(content="Image boundary", image_base64=image)+ assert raw_ids(document) == set()+ else:+ chunk = document.add_chunk(content="Image boundary", image_base64=image)+ assert chunk.image_id == (f"{dataset.id}-{chunk.id}" if image else "")+ wait_ids(document, [chunk.id])diff --git a/test/testcases/test_sdk_api/test_chunk_management_within_dataset/test_chunk_sdk_contract.py b/test/testcases/test_sdk_api/test_chunk_management_within_dataset/test_chunk_sdk_contract.pynew file mode 100644index 0000000000..500c352e94--- /dev/null+++ b/test/testcases/test_sdk_api/test_chunk_management_within_dataset/test_chunk_sdk_contract.py@@ -0,0 +1,208 @@+#+# Copyright 2025 The InfiniFlow Authors. All Rights Reserved.+#+# Licensed under the Apache License, Version 2.0 (the "License");+# you may not use this file except in compliance with the License.+# You may obtain a copy of the License at+#+# http://www.apache.org/licenses/LICENSE-2.0+#+# Unless required by applicable law or agreed to in writing, software+# distributed under the License is distributed on an "AS IS" BASIS,+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.+# See the License for the specific language governing permissions and+# limitations under the License.+#+from unittest.mock import Mock++import pytest+from conftest import add_document, raw_ids, wait_until+from ragflow_sdk import Chunk, Document, RAGFlow+from ragflow_sdk.modules.chunk import ChunkUpdateError+++def sdk():+ return RAGFlow("test-token", "http://unused.invalid")+++def document(sdk):+ return Document(sdk, {"id": "document", "dataset_id": "dataset"})+++def response(body):+ return Mock(json=Mock(return_value=body))+++def test_add_chunk_payload_and_response(sdk, document, monkeypatch):+ body = {"content": "text", "important_keywords": ["key"], "questions": ["why?"], "tag_kwd": ["tag"], "image_base64": "aGVsbG8="}+ row = {**{key: value for key, value in body.items() if key != "image_base64"}, "id": "chunk", "dataset_id": "dataset", "document_id": "document", "image_id": "image"}+ post = Mock(return_value=response({"code": 0, "data": {"chunk": row}}))+ monkeypatch.setattr(sdk, "post", post)+ chunk = document.add_chunk(**body)+ post.assert_called_once_with("/datasets/dataset/documents/document/chunks", body, stream=False, files=None)+ for key, value in row.items():+ assert getattr(chunk, key) == value+++def test_add_chunk_defaults(sdk, document, monkeypatch):+ post = Mock(return_value=response({"code": 0, "data": {"chunk": {"id": "chunk"}}}))+ monkeypatch.setattr(sdk, "post", post)+ document.add_chunk("text")+ assert post.call_args.args[1] == {"content": "text", "important_keywords": [], "questions": [], "tag_kwd": []}+++ "payload,expected",+ [({}, {"chunk_ids": None, "delete_all": False}), ({"ids": ["chunk"]}, {"chunk_ids": ["chunk"], "delete_all": False}), ({"delete_all": True}, {"chunk_ids": None, "delete_all": True})],+)+def test_delete_chunk_payload(sdk, document, monkeypatch, payload, expected):+ delete = Mock(return_value=response({"code": 0}))+ monkeypatch.setattr(sdk, "delete", delete)+ assert document.delete_chunks(**payload) is None+ delete.assert_called_once_with("/datasets/dataset/documents/document/chunks", expected)+++def test_list_chunk_query_and_response(sdk, document, monkeypatch):+ get = Mock(return_value=response({"code": 0, "data": {"chunks": [{"id": "chunk", "content": "text"}]}}))+ monkeypatch.setattr(sdk, "get", get)+ rows = document.list_chunks(page=2, page_size=3, keywords="key", id="chunk")+ get.assert_called_once_with("/datasets/dataset/documents/document/chunks", {"page": 2, "page_size": 3, "keywords": "key", "id": "chunk"})+ assert [(row.id, row.content) for row in rows] == [("chunk", "text")]++[email protected]("body", [{"code": 0}, {"code": 102, "message": "invalid", "details": {"field": "tag_kwd"}}])+def test_chunk_update_result(sdk, monkeypatch, body):+ patch = Mock(return_value=response(body))+ monkeypatch.setattr(sdk, "patch", patch)+ chunk = Chunk(sdk, {"id": "chunk", "dataset_id": "dataset", "document_id": "document", "content": "old"})+ if body["code"]:+ with pytest.raises(ChunkUpdateError) as error:+ chunk.update({"content": "new"})+ assert (error.value.code, error.value.message, error.value.details) == (102, "invalid", {"field": "tag_kwd"})+ else:+ assert chunk.update({"content": "new"}) is None+ patch.assert_called_once_with("/datasets/dataset/documents/document/chunks/chunk", {"content": "new"})+ assert chunk.content == "old" # Updates require a fresh read; the SDK does not mutate this object.++[email protected]("operation", ["add", "delete", "list", "retrieve"])+def test_sdk_propagates_api_error(sdk, document, monkeypatch, operation):+ error = response({"code": 102, "message": "contract failure"})+ for method in ("post", "get", "delete"):+ monkeypatch.setattr(sdk, method, Mock(return_value=error))+ with pytest.raises(Exception, match="^contract failure$"):+ if operation == "add":+ document.add_chunk("text")+ elif operation == "delete":+ document.delete_chunks(ids=["chunk"])+ elif operation == "list":+ document.list_chunks()+ else:+ sdk.retrieve(dataset_ids=["dataset"], question="text")+++ "options",+ [+ {},+ {+ "page": 2,+ "page_size": 1,+ "top_k": 10,+ "vector_similarity_weight": 0.5,+ "similarity_threshold": 0,+ "document_ids": ["document"],+ "rerank_id": "reranker",+ "keyword": True,+ "cross_languages": ["English"],+ "metadata_condition": {"logic": "and", "conditions": []},+ "use_kg": True,+ },+ ],+)+def test_retrieve_payload_and_response(sdk, monkeypatch, options):+ post = Mock(return_value=response({"code": 0, "data": {"chunks": [{"id": "chunk", "content": "text", "similarity": 0.75, "document_id": "document", "dataset_id": "dataset"}]}}))+ monkeypatch.setattr(sdk, "post", post)+ rows = sdk.retrieve(dataset_ids=["dataset"], question="text", **options)+ expected = {+ "dataset_ids": ["dataset"],+ "question": "text",+ "document_ids": [],+ "page": 1,+ "page_size": 30,+ "top_k": 1024,+ "similarity_threshold": 0.2,+ "vector_similarity_weight": 0.3,+ "rerank_id": None,+ "keyword": False,+ "cross_languages": None,+ "metadata_condition": None,+ "use_kg": False,+ **options,+ }+ post.assert_called_once_with("/retrieval", json=expected)+ assert [(row.id, row.content, row.similarity, row.document_id, row.dataset_id) for row in rows] == [("chunk", "text", 0.75, "document", "dataset")]+++def test_missing_datasets_is_python_argument_error(sdk):+ with pytest.raises(TypeError, match="dataset_ids"):+ sdk.retrieve(question="text")+++def test_invalid_kwargs_is_python_argument_error(document):+ with pytest.raises(TypeError, match="mapping"):+ document.delete_chunks(**"not json")+++def test_wait_only_retries_visibility():+ probe = Mock(side_effect=[False, True])+ assert wait_until(probe, timeout=1) is True+ assert probe.call_count == 2+++def test_wait_propagates_errors():+ with pytest.raises(RuntimeError, match="backend unavailable"):+ wait_until(Mock(side_effect=RuntimeError("backend unavailable")))+++def test_wait_deadline():+ with pytest.raises(pytest.fail.Exception, match="deadline exceeded"):+ wait_until(lambda: False, timeout=0.001)+++def test_pagination_detects_repeated_pages(document, monkeypatch):+ monkeypatch.setattr("conftest.raw_chunks", lambda *args, **kwargs: [{"chunk_id": str(i)} for i in range(100)])+ with pytest.raises(AssertionError, match="pagination repeated IDs"):+ raw_ids(document)++[email protected]("delete_fails", [False, True])+def test_fixture_cleanup_is_owned_and_errors_visible(delete_fails):+ client = Mock()+ dataset = client.create_dataset.return_value+ dataset.id = "owned-dataset"+ dataset.upload_documents.return_value = [Mock(id="owned-document")]+ client.list_datasets.return_value = []+ fixture = add_document.__wrapped__(client)+ assert next(fixture) == (dataset, dataset.upload_documents.return_value[0])+ if delete_fails:+ client.delete_datasets.side_effect = RuntimeError("cleanup failure")+ with pytest.raises(RuntimeError, match="cleanup failure"):+ next(fixture)+ else:+ with pytest.raises(StopIteration):+ next(fixture)+ client.list_datasets.assert_called_once_with(id="owned-dataset")+ client.delete_datasets.assert_called_once_with(ids=["owned-dataset"])+++def test_fixture_cleans_up_after_upload_error():+ client = Mock()+ client.create_dataset.return_value.id = "owned-dataset"+ client.create_dataset.return_value.upload_documents.side_effect = RuntimeError("upload failure")+ client.list_datasets.return_value = []+ with pytest.raises(RuntimeError, match="upload failure"):+ next(add_document.__wrapped__(client))+ client.delete_datasets.assert_called_once_with(ids=["owned-dataset"])diff --git a/test/testcases/test_sdk_api/test_chunk_management_within_dataset/test_delete_chunks.py b/test/testcases/test_sdk_api/test_chunk_management_within_dataset/test_delete_chunks.pyindex 20d5dc2aab..9b1c1d076a 100644--- a/test/testcases/test_sdk_api/test_chunk_management_within_dataset/test_delete_chunks.py+++ b/test/testcases/test_sdk_api/test_chunk_management_within_dataset/test_delete_chunks.py@@ -13,104 +13,92 @@# See the License for the specific language governing permissions and# limitations under the License.#-from concurrent.futures import ThreadPoolExecutor, as_completed--import pytest-from common import batch_add_chunks---class TestChunksDeletion:- @pytest.mark.parametrize(- "payload",- [- pytest.param(lambda r: {"ids": ["invalid_id"] + r}, marks=pytest.mark.p3),- pytest.param(lambda r: {"ids": r[:1] + ["invalid_id"] + r[1:4]}, marks=pytest.mark.p3),- pytest.param(lambda r: {"ids": r + ["invalid_id"]}, marks=pytest.mark.p3),- ],- )- def test_delete_partial_invalid_id(self, add_chunks_func, payload):- _, document, chunks = add_chunks_func- chunk_ids = [chunk.id for chunk in chunks]- payload = payload(chunk_ids)- with pytest.raises(Exception) as exception_info:- document.delete_chunks(**payload)- assert "rm_chunk deleted chunks" in str(exception_info.value), str(exception_info.value)- remaining_chunks = document.list_chunks()- assert len(remaining_chunks) == 1, str(remaining_chunks)+from concurrent.futures import ThreadPoolExecutor, as_completed- @pytest.mark.p3- def test_repeated_deletion(self, add_chunks_func):- _, document, chunks = add_chunks_func- chunk_ids = [chunk.id for chunk in chunks]- document.delete_chunks(ids=chunk_ids)+import pytest+from conftest import raw_ids, wait_ids- with pytest.raises(Exception) as exception_info:- document.delete_chunks(ids=chunk_ids)- assert "rm_chunk deleted chunks 0, expect" in str(exception_info.value), str(exception_info.value)+pytestmark = pytest.mark.integration- @pytest.mark.p3- def test_duplicate_deletion(self, add_chunks_func):- _, document, chunks = add_chunks_func- chunk_ids = [chunk.id for chunk in chunks]- document.delete_chunks(ids=chunk_ids * 2)- remaining_chunks = document.list_chunks()- assert len(remaining_chunks) == 1, str(remaining_chunks)- @pytest.mark.p3- def test_concurrent_deletion(self, add_document):- count = 100- _, document = add_document- chunks = batch_add_chunks(document, count)- chunk_ids = [chunk.id for chunk in chunks]-- with ThreadPoolExecutor(max_workers=5) as executor:- futures = [executor.submit(document.delete_chunks, ids=[chunk_id]) for chunk_id in chunk_ids]- responses = list(as_completed(futures))- assert len(responses) == count, responses+class TestChunksDeletion:+ @pytest.mark.parametrize("position", [0, 2, 4])+ def test_delete_partial_invalid_id(self, add_chunks, position):+ _, document, chunks = add_chunks+ ids = [chunk.id for chunk in chunks]+ ids.insert(position, "unknown")+ document.delete_chunks(ids=ids)+ wait_ids(document, [])++ def test_repeated_deletion(self, add_chunks):+ _, document, chunks = add_chunks+ ids = [chunk.id for chunk in chunks]+ document.delete_chunks(ids=ids)+ wait_ids(document, [])+ document.delete_chunks(ids=ids)+ assert raw_ids(document) == set()++ def test_duplicate_deletion(self, add_chunks):+ _, document, chunks = add_chunks+ document.delete_chunks(ids=[chunk.id for chunk in chunks] * 2)+ wait_ids(document, [])++ def test_concurrent_deletion(self, add_chunks):+ _, document, chunks = add_chunks+ with ThreadPoolExecutor(max_workers=4) as executor:+ futures = [executor.submit(document.delete_chunks, ids=[chunk.id]) for chunk in chunks]+ for future in as_completed(futures):+ assert future.result() is None+ wait_ids(document, [])- @pytest.mark.p3def test_delete_1k(self, add_document):- count = 1_000_, document = add_document- chunks = batch_add_chunks(document, count)- chunk_ids = [chunk.id for chunk in chunks]-- from time import sleep-- sleep(1)-- document.delete_chunks(ids=chunk_ids)- remaining_chunks = document.list_chunks()- assert len(remaining_chunks) == 0, str(remaining_chunks)+ chunks = [document.add_chunk(content=f"bulk chunk {i}") for i in range(1000)]+ ids = [chunk.id for chunk in chunks]+ wait_ids(document, ids)+ document.delete_chunks(ids=ids)+ wait_ids(document, [])@pytest.mark.parametrize(- "payload, expected_message, remaining",+ "payload, removed",[- pytest.param(None, "", 5, marks=pytest.mark.p3),- pytest.param({"ids": ["invalid_id"]}, "rm_chunk deleted chunks 0, expect 1", 5, marks=pytest.mark.p3),- pytest.param("not json", "UnboundLocalError", 5, marks=pytest.mark.skip(reason="pull/6376")),- pytest.param(lambda r: {"ids": r[:1]}, "", 4, marks=pytest.mark.p3),- pytest.param(lambda r: {"ids": r}, "", 1, marks=pytest.mark.p3),- pytest.param({"ids": []}, "", 5, marks=pytest.mark.p3),+ ({"ids": ["unknown"]}, 0),+ ("single", 1),+ ("all_ids", 4),+ ({"delete_all": True}, 4),+ ({"ids": [], "delete_all": True}, 4),],)- def test_basic_scenarios(self, add_chunks_func, payload, expected_message, remaining):- _, document, chunks = add_chunks_func- chunk_ids = [chunk.id for chunk in chunks]- if callable(payload):- payload = payload(chunk_ids)-- if expected_message:- with pytest.raises(Exception) as exception_info:- document.delete_chunks(**payload)- assert expected_message in str(exception_info.value), str(exception_info.value)- else:- if payload is None:- document.delete_chunks()- else:- document.delete_chunks(**payload)-- remaining_chunks = document.list_chunks()- assert len(remaining_chunks) == remaining, str(remaining_chunks)+ def test_basic_scenarios(self, add_chunks, payload, removed):+ _, document, chunks = add_chunks+ ids = [chunk.id for chunk in chunks]+ if payload == "single":+ payload = {"ids": ids[:1]}+ elif payload == "all_ids":+ payload = {"ids": ids}+ document.delete_chunks(**payload)+ wait_ids(document, ids[removed:])++ @pytest.mark.skip(reason="Go RemoveChunks rejects omitted/empty IDs instead of documented safe no-op")+ @pytest.mark.parametrize("payload", [{}, {"ids": None}, {"ids": []}])+ def test_empty_selection_is_noop(self, add_chunks, payload):+ _, document, chunks = add_chunks+ document.delete_chunks(**payload)+ assert raw_ids(document) == {chunk.id for chunk in chunks}++ def test_delete_all_rejects_ids(self, add_chunks):+ _, document, chunks = add_chunks+ with pytest.raises(Exception, match="mutually exclusive"):+ document.delete_chunks(ids=[chunks[0].id], delete_all=True)+ assert raw_ids(document) == {chunk.id for chunk in chunks}++ def test_delete_all_is_document_scoped(self, add_chunks):+ dataset, document, _ = add_chunks+ other = dataset.upload_documents([{"display_name": "other.txt", "blob": b"Other owned document"}])[0]+ chunk = other.add_chunk(content="Must survive deletion in another document")+ wait_ids(other, [chunk.id])+ document.delete_chunks(delete_all=True)+ wait_ids(document, [])+ assert raw_ids(other) == {chunk.id}diff --git a/test/testcases/test_sdk_api/test_chunk_management_within_dataset/test_list_chunks.py b/test/testcases/test_sdk_api/test_chunk_management_within_dataset/test_list_chunks.pyindex e4f11dfcb2..eed021b58a 100644--- a/test/testcases/test_sdk_api/test_chunk_management_within_dataset/test_list_chunks.py+++ b/test/testcases/test_sdk_api/test_chunk_management_within_dataset/test_list_chunks.py@@ -13,148 +13,104 @@# See the License for the specific language governing permissions and# limitations under the License.#-import os++from concurrent.futures import ThreadPoolExecutor, as_completedimport pytest-from common import batch_add_chunks-from utils.engine_utils import get_doc_engine+from conftest import wait_ids+from ragflow_sdk import Document++pytestmark = pytest.mark.integrationclass TestChunksList:- @pytest.mark.p3@pytest.mark.parametrize(- "params, expected_page_size, expected_message",+ "params, size, error",[- ({"page": None, "page_size": 2}, 2, ""),- pytest.param({"page": 0, "page_size": 2}, 0, "ValueError('Search does not support negative slicing.')", marks=pytest.mark.skip),- ({"page": 2, "page_size": 2}, 2, ""),- ({"page": 3, "page_size": 2}, 1, ""),- ({"page": "3", "page_size": 2}, 1, ""),- pytest.param({"page": -1, "page_size": 2}, 0, "ValueError('Search does not support negative slicing.')", marks=pytest.mark.skip),- pytest.param({"page": "a", "page_size": 2}, 0, """ValueError("invalid literal for int() with base 10: \'a\'")""", marks=pytest.mark.skip),+ ({"page": None, "page_size": 2}, 2, None),+ ({"page": 0}, 0, "page must be a positive integer"),+ ({"page": 2, "page_size": 2}, 2, None),+ ({"page": 3, "page_size": 2}, 0, None),+ ({"page": "3", "page_size": 2}, 0, None),+ ({"page": -1}, 0, "page must be a positive integer"),+ ({"page": "a"}, 0, "page must be a positive integer"),],)- def test_page(self, add_chunks, params, expected_page_size, expected_message):+ def test_page(self, add_chunks, params, size, error):_, document, _ = add_chunks-- if expected_message:- with pytest.raises(Exception) as exception_info:+ if error:+ with pytest.raises(Exception, match=error):document.list_chunks(**params)- assert expected_message in str(exception_info.value), str(exception_info.value)else:- chunks = document.list_chunks(**params)- assert len(chunks) == expected_page_size, str(chunks)+ assert len(document.list_chunks(**params)) == size- @pytest.mark.p3@pytest.mark.parametrize(- "params, expected_page_size, expected_message",+ "value, size, error",[- ({"page_size": None}, 5, ""),- pytest.param({"page_size": 0}, 5, ""),- ({"page_size": 1}, 1, ""),- ({"page_size": 6}, 5, ""),- ({"page_size": "1"}, 1, ""),- pytest.param({"page_size": -1}, 5, "", marks=pytest.mark.skip),- pytest.param({"page_size": "a"}, 0, """ValueError("invalid literal for int() with base 10: \'a\'")""", marks=pytest.mark.skip),+ (None, 4, None),+ (0, 0, "page_size must be a positive integer"),+ (1, 1, None),+ (6, 4, None),+ ("1", 1, None),+ (-1, 0, "page_size must be a positive integer"),+ ("a", 0, "page_size must be a positive integer"),],)- def test_page_size(self, add_chunks, params, expected_page_size, expected_message):+ def test_page_size(self, add_chunks, value, size, error):_, document, _ = add_chunks-- if expected_message:- with pytest.raises(Exception) as exception_info:- document.list_chunks(**params)- assert expected_message in str(exception_info.value), str(exception_info.value)+ if error:+ with pytest.raises(Exception, match=error):+ document.list_chunks(page_size=value)else:- chunks = document.list_chunks(**params)- assert len(chunks) == expected_page_size, str(chunks)+ assert len(document.list_chunks(page_size=value)) == size- @pytest.mark.p2- @pytest.mark.parametrize(- "params, expected_page_size",- [- ({"keywords": None}, 5),- ({"keywords": ""}, 5),- ({"keywords": "1"}, 1),- ({"keywords": "chunk"}, 4),- pytest.param({"keywords": "ragflow"}, 1, marks=pytest.mark.skipif(os.getenv("DOC_ENGINE") == "infinity", reason="issues/6509")),- pytest.param({"keywords": "ragflow"}, 5, marks=pytest.mark.skipif(os.getenv("DOC_ENGINE") != "infinity", reason="issues/6509")),- ({"keywords": "unknown"}, 0),- ],- )- def test_keywords(self, add_chunks, params, expected_page_size):+ @pytest.mark.parametrize("keywords, size", [(None, 4), ("", 4), ("coriander", 4), ("saffron", 1), ("1", 1), ("unfindablequasar", 0)])+ def test_keywords(self, add_chunks, keywords, size):_, document, _ = add_chunks- if params.get("keywords") == "ragflow":- doc_engine = get_doc_engine(document.rag)- if doc_engine == "infinity" and expected_page_size == 1:- pytest.skip("issues/6509")- if doc_engine != "infinity" and expected_page_size == 5:- pytest.skip("issues/6509")- chunks = document.list_chunks(**params)- assert len(chunks) == expected_page_size, str(chunks)+ assert len(document.list_chunks(keywords=keywords)) == size- @pytest.mark.p1- @pytest.mark.parametrize(- "chunk_id, expected_page_size, expected_message",- [- (None, 5, ""),- ("", 5, ""),- pytest.param(lambda r: r[0], 1, "", marks=pytest.mark.skipif(os.getenv("DOC_ENGINE") == "infinity", reason="issues/6499")),- pytest.param("unknown", 0, """AttributeError("\'NoneType\' object has no attribute \'keys\'")""", marks=pytest.mark.skip),- ],- )- def test_id(self, add_chunks, chunk_id, expected_page_size, expected_message):- _, document, chunks = add_chunks- if callable(chunk_id) and get_doc_engine(document.rag) == "infinity":- pytest.skip("issues/6499")- chunk_ids = [chunk.id for chunk in chunks]- if callable(chunk_id):- params = {"id": chunk_id(chunk_ids)}- else:- params = {"id": chunk_id}+ @pytest.mark.parametrize("chunk_id", [None, ""])+ def test_id_empty(self, add_chunks, chunk_id):+ _, document, _ = add_chunks+ assert len(document.list_chunks(id=chunk_id)) == 4- if expected_message:- with pytest.raises(Exception) as exception_info:- document.list_chunks(**params)- assert expected_message in str(exception_info.value), str(exception_info.value)- else:- chunks = document.list_chunks(**params)- if params["id"] in [None, ""]:- assert len(chunks) == expected_page_size, str(chunks)- else:- assert chunks[0].id == params["id"], str(chunks)+ @pytest.mark.skip(reason="SDK sends id but Go ListChunks only reads chunk_ids; Go list also omits SDK id field")+ @pytest.mark.parametrize("selection", ["existing", "unknown"])+ def test_id(self, add_chunks, selection):+ _, document, chunks = add_chunks+ wanted = chunks[0].id if selection == "existing" else "unknown"+ rows = document.list_chunks(id=wanted)+ assert [chunk.id for chunk in rows] == ([wanted] if selection == "existing" else [])++ @pytest.mark.skip(reason="Go List returns chunk_id/content_with_weight/important_kwd/question_kwd without SDK field mapping")+ def test_list_chunk_schema(self, add_chunks):+ dataset, document, chunks = add_chunks+ rows = document.list_chunks(page_size=100)+ assert {chunk.id for chunk in rows} == {chunk.id for chunk in chunks}+ for chunk in rows:+ expected = next(original for original in chunks if original.id == chunk.id)+ assert chunk.content == expected.content+ assert chunk.document_id == document.id+ assert chunk.dataset_id == dataset.id+ assert chunk.important_keywords == expected.important_keywords+ assert chunk.questions == expected.questions- @pytest.mark.p3def test_concurrent_list(self, add_chunks):_, document, _ = add_chunks- count = 100with ThreadPoolExecutor(max_workers=5) as executor:- futures = [executor.submit(document.list_chunks) for _ in range(count)]-- responses = list(as_completed(futures))- assert len(responses) == count, responses- assert all(len(future.result()) == 5 for future in futures)+ futures = [executor.submit(document.list_chunks) for _ in range(20)]+ for future in as_completed(futures):+ assert len(future.result()) == 4- @pytest.mark.p3def test_default(self, add_document):_, document = add_document- batch_add_chunks(document, 31)-- from time import sleep+ chunks = [document.add_chunk(content=f"default pagination {i}") for i in range(31)]+ wait_ids(document, [chunk.id for chunk in chunks])+ assert len(document.list_chunks()) == 30- sleep(3)-- chunks = document.list_chunks()- assert len(chunks) == 30, str(chunks)-- @pytest.mark.p3def test_list_chunks_invalid_document_id_raises(self, add_chunks):- _, document, _ = add_chunks- invalid_document = document.__class__(- document.rag,- {"id": "missing-document-id-for-chunks", "dataset_id": document.dataset_id},- )- with pytest.raises(Exception) as exception_info:- invalid_document.list_chunks()- assert str(exception_info.value), exception_info+ dataset, _, _ = add_chunks+ document = Document(dataset.rag, {"id": "missing-document", "dataset_id": dataset.id})+ with pytest.raises(Exception, match="document not found"):+ document.list_chunks()diff --git a/test/testcases/test_sdk_api/test_chunk_management_within_dataset/test_retrieval_chunks.py b/test/testcases/test_sdk_api/test_chunk_management_within_dataset/test_retrieval_chunks.pyindex 112765b730..9df55cb0f6 100644--- a/test/testcases/test_sdk_api/test_chunk_management_within_dataset/test_retrieval_chunks.py+++ b/test/testcases/test_sdk_api/test_chunk_management_within_dataset/test_retrieval_chunks.py@@ -13,233 +13,194 @@# See the License for the specific language governing permissions and# limitations under the License.#+++import mathimport osfrom concurrent.futures import ThreadPoolExecutor, as_completedimport pytest+from conftest import raw_chunks, wait_until++pytestmark = pytest.mark.integration+-DOC_ENGINE = (os.getenv("DOC_ENGINE") or "").lower()+def retrieval(add_chunks, client):+ dataset, document, chunks = add_chunks+ payload = {"dataset_ids": [dataset.id], "question": "coriander botanical archive", "similarity_threshold": 0, "vector_similarity_weight": 0}+ # Listing visibility does not prove that chunks are searchable.+ wait_until(lambda: {chunk.id for chunk in client.retrieve(**payload)} == {chunk.id for chunk in chunks})+ return payload, dataset, document, chunks+++def assert_results(rows, dataset, originals, limit=30):+ expected = {chunk.id: chunk for chunk in originals}+ assert len(rows) <= limit+ assert len({row.id for row in rows}) == len(rows)+ for row in rows:+ assert row.id in expected+ assert row.dataset_id == dataset.id+ assert row.document_id == expected[row.id].document_id+ assert row.content == expected[row.id].content+ assert math.isfinite(row.similarity)+ scores = [row.similarity for row in rows]+ assert scores == sorted(scores, reverse=True)class TestChunksRetrieval:- @pytest.mark.p3- @pytest.mark.parametrize(- "payload, expected_page_size, expected_message",- [- ({"question": "chunk", "dataset_ids": None}, 4, ""),- ({"question": "chunk", "document_ids": None}, 0, "missing 1 required positional argument"),- ({"question": "chunk", "dataset_ids": None, "document_ids": None}, 4, ""),- ({"question": "chunk"}, 0, "missing 1 required positional argument"),- ],- )- def test_basic_scenarios(self, client, add_chunks, payload, expected_page_size, expected_message):- dataset, document, _ = add_chunks- if "dataset_ids" in payload:- payload["dataset_ids"] = [dataset.id]- if "document_ids" in payload:+ @pytest.mark.parametrize("document_filter", [False, True])+ def test_basic_scenarios(self, client, retrieval, document_filter):+ payload, dataset, document, originals = retrieval+ if document_filter:payload["document_ids"] = [document.id]+ rows = client.retrieve(**payload)+ assert rows+ assert_results(rows, dataset, originals)++ @pytest.mark.parametrize("page", [None, 1, 2, 3])+ def test_page(self, client, retrieval, page):+ payload, dataset, _, originals = retrieval+ baseline = client.retrieve(**payload)+ rows = client.retrieve(**{**payload, "page": page, "page_size": 2})+ effective_page = page or 1+ assert len(rows) == len(baseline[(effective_page - 1) * 2 : effective_page * 2])+ assert {row.id for row in rows} <= {row.id for row in baseline}+ assert_results(rows, dataset, originals, 2)++ @pytest.mark.parametrize("page_size", [None, 1, 5])+ def test_page_size(self, client, retrieval, page_size):+ payload, dataset, _, originals = retrieval+ rows = client.retrieve(**{**payload, "page_size": page_size})+ baseline = client.retrieve(**payload)+ assert len(rows) == len(baseline[:page_size])+ assert {row.id for row in rows} <= {row.id for row in baseline}+ assert rows+ assert_results(rows, dataset, originals, page_size or 30)++ @pytest.mark.parametrize("weight", [0, 0.5, 1])+ def test_vector_similarity_weight(self, client, retrieval, weight):+ payload, dataset, _, originals = retrieval+ rows = client.retrieve(**{**payload, "vector_similarity_weight": weight})+ assert rows+ assert_results(rows, dataset, originals)++ @pytest.mark.parametrize("top_k", [1, 10, 2048])+ def test_top_k(self, client, retrieval, top_k):+ payload, dataset, _, originals = retrieval+ # top_k limits vector candidates; engines can fuse additional text matches.+ rows = client.retrieve(**{**payload, "top_k": top_k, "page_size": 2, "vector_similarity_weight": 0.5})+ assert rows+ assert_results(rows, dataset, originals, 2)- if expected_message:- with pytest.raises(Exception) as exception_info:- client.retrieve(**payload)- assert expected_message in str(exception_info.value), str(exception_info.value)- else:- chunks = client.retrieve(**payload)- assert len(chunks) == expected_page_size, str(chunks)-- @pytest.mark.p3- @pytest.mark.parametrize(- "payload, expected_page_size, expected_message",- [- pytest.param(- {"page": None, "page_size": 2},- 2,- """TypeError("int() argument must be a string, a bytes-like object or a real number, not \'NoneType\'")""",- marks=pytest.mark.skip,- ),- pytest.param(- {"page": 0, "page_size": 2},- 0,- "ValueError('Search does not support negative slicing.')",- marks=pytest.mark.skip,- ),- ({"page": 2, "page_size": 2}, 2, ""),- ({"page": 3, "page_size": 2}, 0, ""),- ({"page": "3", "page_size": 2}, 0, ""),- pytest.param(- {"page": -1, "page_size": 2},- 0,- "ValueError('Search does not support negative slicing.')",- marks=pytest.mark.skip,- ),- pytest.param(- {"page": "a", "page_size": 2},- 0,- """ValueError("invalid literal for int() with base 10: \'a\'")""",- marks=pytest.mark.skip,- ),- ],- )- def test_page(self, client, add_chunks, payload, expected_page_size, expected_message):- dataset, _, _ = add_chunks- payload.update({"question": "chunk", "dataset_ids": [dataset.id]})-- if expected_message:- with pytest.raises(Exception) as exception_info:- client.retrieve(**payload)- assert expected_message in str(exception_info.value), str(exception_info.value)- else:- chunks = client.retrieve(**payload)- assert len(chunks) == expected_page_size, str(chunks)-- @pytest.mark.p3- @pytest.mark.parametrize(- "payload, expected_page_size, expected_message",- [- pytest.param(- {"page_size": None},- 0,- """TypeError("int() argument must be a string, a bytes-like object or a real number, not \'NoneType\'")""",- marks=pytest.mark.skip,- ),- pytest.param({"page_size": 1}, 1, "", marks=pytest.mark.skip(reason="issues/10692")),- ({"page_size": 5}, 4, ""),- pytest.param({"page_size": "1"}, 1, "", marks=pytest.mark.skip(reason="issues/10692")),- pytest.param(- {"page_size": "a"},- 0,- """ValueError("invalid literal for int() with base 10: \'a\'")""",- marks=pytest.mark.skip,- ),- ],- )- def test_page_size(self, client, add_chunks, payload, expected_page_size, expected_message):- dataset, _, _ = add_chunks- payload.update({"question": "chunk", "dataset_ids": [dataset.id]})-- if expected_message:- with pytest.raises(Exception) as exception_info:- client.retrieve(**payload)- assert expected_message in str(exception_info.value), str(exception_info.value)- else:- chunks = client.retrieve(**payload)- assert len(chunks) == expected_page_size, str(chunks)-- @pytest.mark.p3- @pytest.mark.parametrize(- "payload, expected_page_size, expected_message",- [- ({"vector_similarity_weight": 0}, 4, ""),- ({"vector_similarity_weight": 0.5}, 4, ""),- ({"vector_similarity_weight": 10}, 4, ""),- pytest.param(- {"vector_similarity_weight": "a"},- 0,- """ValueError("could not convert string to float: 'a'")""",- marks=pytest.mark.skip,- ),- ],- )- def test_vector_similarity_weight(self, client, add_chunks, payload, expected_page_size, expected_message):- dataset, _, _ = add_chunks- payload.update({"question": "chunk", "dataset_ids": [dataset.id]})-- if expected_message:- with pytest.raises(Exception) as exception_info:- client.retrieve(**payload)- assert expected_message in str(exception_info.value), str(exception_info.value)- else:- chunks = client.retrieve(**payload)- assert len(chunks) == expected_page_size, str(chunks)-- @pytest.mark.p2- @pytest.mark.parametrize(- "payload, expected_page_size, expected_message",- [- ({"top_k": 10}, 4, ""),- pytest.param(- {"top_k": 1},- 4,- "",- marks=pytest.mark.skipif(DOC_ENGINE in ["infinity", "opensearch"], reason="Infinity"),- ),- pytest.param(- {"top_k": 1},- 1,- "",- marks=pytest.mark.skipif(DOC_ENGINE in ["", "opensearch", "elasticsearch"], reason="elasticsearch"),- ),- ({"top_k": -1}, 4, "must be greater than 0"),- pytest.param(- {"top_k": "a"},- 0,- """ValueError("invalid literal for int() with base 10: \'a\'")""",- marks=pytest.mark.skip,- ),- ],- )- def test_top_k(self, client, add_chunks, payload, expected_page_size, expected_message):- dataset, _, _ = add_chunks- payload.update({"question": "chunk", "dataset_ids": [dataset.id]})-- if expected_message:- with pytest.raises(Exception) as exception_info:- client.retrieve(**payload)- assert expected_message in str(exception_info.value), str(exception_info.value)- else:- chunks = client.retrieve(**payload)- assert len(chunks) == expected_page_size, str(chunks)-- @pytest.mark.skip- @pytest.mark.parametrize(- "payload, expected_message",- [- ({"rerank_id": "BAAI/bge-reranker-v2-m3"}, ""),- pytest.param({"rerank_id": "unknown"}, "LookupError('Model(unknown) not authorized')", marks=pytest.mark.skip),- ],- )- def test_rerank_id(self, client, add_chunks, payload, expected_message):- dataset, _, _ = add_chunks- payload.update({"question": "chunk", "dataset_ids": [dataset.id]})-- if expected_message:- with pytest.raises(Exception) as exception_info:- client.retrieve(**payload)- assert expected_message in str(exception_info.value), str(exception_info.value)- else:- chunks = client.retrieve(**payload)- assert len(chunks) > 0, str(chunks)-- @pytest.mark.skip@pytest.mark.parametrize(- "payload, expected_page_size, expected_message",+ "field,value,message",[- ({"keyword": True}, 5, ""),- ({"keyword": "True"}, 5, ""),- ({"keyword": False}, 5, ""),- ({"keyword": "False"}, 5, ""),- ({"keyword": None}, 5, ""),+ ("page", 0, "page must be greater than or equal to 1"),+ ("page", -1, "page must be greater than or equal to 1"),+ ("page", "3", "cannot unmarshal string"),+ ("page", "a", "cannot unmarshal string"),+ ("page_size", 0, "page_size must be greater than or equal to 1"),+ ("page_size", -1, "page_size must be greater than or equal to 1"),+ ("page_size", "1", "cannot unmarshal string"),+ ("page_size", "a", "cannot unmarshal string"),+ ("vector_similarity_weight", 10, "vector_similarity_weight must be between 0 and 1"),+ ("vector_similarity_weight", -0.1, "vector_similarity_weight must be between 0 and 1"),+ ("vector_similarity_weight", "a", "cannot unmarshal string"),+ ("similarity_threshold", -0.1, "similarity_threshold must be between 0 and 1"),+ ("similarity_threshold", 1.1, "similarity_threshold must be between 0 and 1"),+ ("top_k", -1, "must be between 1 and 2048"),+ ("top_k", 0, "must be between 1 and 2048"),+ ("top_k", 2049, "must be between 1 and 2048"),+ ("top_k", "a", "cannot unmarshal string"),+ ("keyword", "True", "cannot unmarshal string"),+ ("keyword", "False", "cannot unmarshal string"),+ ("question", "", "question is required"),+ ("question", " ", "question is required"),+ ("dataset_ids", [], "dataset_ids array cannot be empty"),+ ("dataset_ids", None, "dataset_ids is required"),],)- def test_keyword(self, client, add_chunks, payload, expected_page_size, expected_message):- dataset, _, _ = add_chunks- payload.update({"question": "chunk test", "dataset_ids": [dataset.id]})-- if expected_message:- with pytest.raises(Exception) as exception_info:- client.retrieve(**payload)- assert expected_message in str(exception_info.value), str(exception_info.value)- else:- chunks = client.retrieve(**payload)- assert len(chunks) == expected_page_size, str(chunks)-- @pytest.mark.p3- def test_concurrent_retrieval(self, client, add_chunks):- dataset, _, _ = add_chunks- count = 100- payload = {"question": "chunk", "dataset_ids": [dataset.id]}-+ def test_invalid_parameters(self, client, add_document, field, value, message):+ dataset, _ = add_document+ payload = {"dataset_ids": [dataset.id], "question": "coriander", field: value}+ with pytest.raises(Exception, match=message):+ client.retrieve(**payload)++ def test_unknown_rerank_id(self, client, retrieval):+ payload, _, _, _ = retrieval+ with pytest.raises(Exception, match="failed to get rerank model by rerank_id"):+ client.retrieve(**{**payload, "rerank_id": "sdk-chunks-nonexistent-reranker"})++ def test_rerank_id(self, client, retrieval):+ model = os.getenv("RAGFLOW_SDK_RERANK_ID")+ assert model, "Set RAGFLOW_SDK_RERANK_ID to an authorized reranker for this integration case"+ payload, dataset, _, originals = retrieval+ rows = client.retrieve(**{**payload, "rerank_id": model})+ assert rows+ assert_results(rows, dataset, originals)++ @pytest.mark.parametrize("keyword", [False, None, True])+ def test_keyword(self, client, retrieval, keyword):+ if keyword:+ assert os.getenv("RAGFLOW_SDK_CHAT_CONFIGURED") == "1", "Keyword expansion needs the isolated tenant's configured chat model"+ payload, dataset, _, originals = retrieval+ rows = client.retrieve(**{**payload, "keyword": keyword})+ assert rows+ assert_results(rows, dataset, originals)++ def test_document_filter(self, client, retrieval):+ payload, dataset, document, originals = retrieval+ other = dataset.upload_documents([{"display_name": "filter.txt", "blob": b"Filter fixture"}])[0]+ extra = other.add_chunk(content="Coriander botanical archive excluded document.")+ wait_until(lambda: extra.id in {row.id for row in client.retrieve(**payload)})+ rows = client.retrieve(**{**payload, "document_ids": [document.id]})+ assert rows+ assert_results(rows, dataset, originals)+ assert client.retrieve(**{**payload, "document_ids": ["missing-document"]}) == []++ def test_concurrent_retrieval(self, client, retrieval):+ payload, dataset, _, originals = retrievalwith ThreadPoolExecutor(max_workers=5) as executor:- futures = [executor.submit(client.retrieve, **payload) for _ in range(count)]- responses = list(as_completed(futures))- assert len(responses) == count, responses+ futures = [executor.submit(client.retrieve, **payload) for _ in range(20)]+ for future in as_completed(futures):+ rows = future.result()+ assert rows+ assert_results(rows, dataset, originals)++ def test_pagination_covers_results_once(self, client, retrieval):+ payload, _, _, _ = retrieval+ baseline = client.retrieve(**payload)+ pages = [client.retrieve(**{**payload, "page": page, "page_size": 1}) for page in range(1, len(baseline) + 1)]+ ids = [row.id for page in pages for row in page]+ assert len(ids) == len(set(ids)) == len(baseline)+ assert set(ids) == {row.id for row in baseline}++ def test_metadata_filter(self, client, retrieval):+ payload, dataset, document, originals = retrieval+ result = dataset.patch(+ f"/datasets/{dataset.id}/documents/metadatas", {"selector": {"document_ids": [document.id]}, "updates": [{"key": "author", "value": "chunk-audit"}], "deletes": []}+ ).json()+ assert result["code"] == 0, result+ assert result["data"]["updated"] == result["data"]["matched_docs"] == 1+ for author, match in [("chunk-audit", True), ("other-author", False)]:+ condition = {"logic": "and", "conditions": [{"name": "author", "comparison_operator": "=", "value": author}]}+ rows = client.retrieve(**{**payload, "metadata_condition": condition})+ if match:+ assert rows+ assert_results(rows, dataset, originals)+ else:+ assert rows == []++ def test_unavailable_chunks_are_excluded(self, client, retrieval):+ payload, dataset, document, originals = retrieval+ hidden = originals[0]+ hidden.update({"available": False})+ wait_until(lambda: next(row for row in raw_chunks(document) if row["chunk_id"] == hidden.id)["available"] is False)++ def visible_results():+ rows = client.retrieve(**payload)+ return rows if hidden.id not in {row.id for row in rows} else None++ rows = wait_until(visible_results)+ assert_results(rows, dataset, originals[1:])diff --git a/test/testcases/test_sdk_api/test_chunk_management_within_dataset/test_update_chunk.py b/test/testcases/test_sdk_api/test_chunk_management_within_dataset/test_update_chunk.pyindex 460292da3f..053530fc12 100644--- a/test/testcases/test_sdk_api/test_chunk_management_within_dataset/test_update_chunk.py+++ b/test/testcases/test_sdk_api/test_chunk_management_within_dataset/test_update_chunk.py@@ -13,159 +13,124 @@# See the License for the specific language governing permissions and# limitations under the License.#-import os++from concurrent.futures import ThreadPoolExecutor, as_completed-from random import randintimport pytest+from conftest import raw_chunks, wait_until+from ragflow_sdk.modules.chunk import ChunkUpdateError+pytestmark = pytest.mark.integration-class TestUpdatedChunk:- @pytest.mark.p3- @pytest.mark.parametrize(- "payload, expected_message",- [- ({"content": None}, ""),- ({"content": ""}, "`content` is required"),- pytest.param(- {"content": 1},- "TypeError('expected string or bytes-like object')",- marks=pytest.mark.skip,- ),- ({"content": "update chunk"}, ""),- ({"content": " "}, "`content` is required"),- ({"content": "\n!?。;!?\"'"}, ""),- ],- )- def test_content(self, add_chunks, payload, expected_message):- _, _, chunks = add_chunks- chunk = chunks[0]- if expected_message:- with pytest.raises(Exception) as exception_info:- chunk.update(payload)- assert expected_message in str(exception_info.value), str(exception_info.value)- else:- chunk.update(payload)+def assert_updated(document, chunk, field, expected):+ def persisted():+ rows = raw_chunks(document)+ row = next((row for row in rows if row["chunk_id"] == chunk.id), None)+ return row is not None and row.get(field) == expected- @pytest.mark.p3- @pytest.mark.parametrize(- "payload, expected_message",- [- ({"important_keywords": ["a", "b", "c"]}, ""),- ({"important_keywords": [""]}, ""),- ({"important_keywords": [1]}, "TypeError('sequence item 0: expected str instance, int found')"),- ({"important_keywords": ["a", "a"]}, ""),- ({"important_keywords": "abc"}, "`important_keywords` should be a list"),- ({"important_keywords": 123}, "`important_keywords` should be a list"),- ],- )- def test_important_keywords(self, add_chunks, payload, expected_message):- _, _, chunks = add_chunks- chunk = chunks[0]+ wait_until(persisted)- if expected_message:- with pytest.raises(Exception) as exception_info:- chunk.update(payload)- assert expected_message in str(exception_info.value), str(exception_info.value)- else:- chunk.update(payload)- @pytest.mark.p2- @pytest.mark.parametrize(- "payload, expected_message",- [- ({"questions": ["a", "b", "c"]}, ""),- ({"questions": [""]}, ""),- ({"questions": [1]}, "TypeError('sequence item 0: expected str instance, int found')"),- ({"questions": ["a", "a"]}, ""),- ({"questions": "abc"}, "`questions` should be a list"),- ({"questions": 123}, "`questions` should be a list"),- ],- )- def test_questions(self, add_chunks, payload, expected_message):- _, _, chunks = add_chunks+class TestUpdatedChunk:+ @pytest.mark.parametrize("content", ["update chunk", "\n!?。;!?\"'", "", " "])+ def test_content(self, add_chunks, content):+ _, document, chunks = add_chunkschunk = chunks[0]+ assert chunk.update({"content": content}) is None+ assert_updated(document, chunk, "content_with_weight", content)- if expected_message:- with pytest.raises(Exception) as exception_info:- chunk.update(payload)- assert expected_message in str(exception_info.value), str(exception_info.value)- else:- chunk.update(payload)-- @pytest.mark.p2@pytest.mark.parametrize(- "payload, expected_message",+ "content",[- ({"tag_kwd": ["tag1", "tag2"]}, ""),- ({"tag_kwd": [""]}, ""),- ({"tag_kwd": [1]}, "`tag_kwd` must be a list of strings"),- ({"tag_kwd": ["tag", "tag"]}, ""),- ({"tag_kwd": "tag"}, "`tag_kwd` should be a list"),- ({"tag_kwd": 123}, "`tag_kwd` should be a list"),+ pytest.param(None, marks=pytest.mark.skip(reason="Go UpdateChunk silently ignores non-string content")),+ pytest.param(1, marks=pytest.mark.skip(reason="Go UpdateChunk silently ignores non-string content")),],)- def test_tag_kwd(self, add_chunks, payload, expected_message):+ def test_content_invalid(self, add_chunks, content):_, _, chunks = add_chunks- chunk = chunks[0]+ with pytest.raises(ChunkUpdateError) as error:+ chunks[0].update({"content": content})+ assert error.value.code in (101, 102, 400)+ assert "content" in error.value.message++ @pytest.mark.parametrize("field, stored", [("important_keywords", "important_kwd"), ("questions", "question_kwd"), ("tag_kwd", "tag_kwd")])+ @pytest.mark.parametrize("value", [["a", "b", "c"], [""], ["a", "a"], []])+ def test_list_fields(self, add_chunks, field, stored, value):+ _, document, chunks = add_chunks+ # Seed first so [] must clear existing values rather than preserve defaults.+ chunks[0].update({field: ["seed"]})+ assert_updated(document, chunks[0], stored, ["seed"])+ chunks[0].update({field: value})+ expected = [item.strip() for item in value if item.strip()] if field == "questions" else value+ assert_updated(document, chunks[0], stored, expected)- if expected_message:- with pytest.raises(Exception) as exception_info:- chunk.update(payload)- assert expected_message in str(exception_info.value), str(exception_info.value)- else:- chunk.update(payload)-- @pytest.mark.p2@pytest.mark.parametrize(- "payload, expected_message",+ "field",[- ({"available": True}, ""),- pytest.param({"available": "True"}, """ValueError("invalid literal for int() with base 10: \'True\'")""", marks=pytest.mark.skip),- ({"available": 1}, ""),- ({"available": False}, ""),- pytest.param({"available": "False"}, """ValueError("invalid literal for int() with base 10: \'False\'")""", marks=pytest.mark.skip),- ({"available": 0}, ""),+ pytest.param("important_keywords", marks=pytest.mark.skip(reason="Go UpdateChunk ignores invalid keyword types or replaces non-string elements with empty strings")),+ pytest.param("questions", marks=pytest.mark.skip(reason="Go UpdateChunk ignores invalid question types or replaces non-string elements with empty strings")),+ "tag_kwd",],)- def test_available(self, add_chunks, payload, expected_message):+ @pytest.mark.parametrize("value", [[1], "abc", 123, None])+ def test_list_fields_invalid(self, add_chunks, field, value):_, _, chunks = add_chunks- chunk = chunks[0]-- if expected_message:- with pytest.raises(Exception) as exception_info:- chunk.update(payload)- assert expected_message in str(exception_info.value), str(exception_info.value)- else:- chunk.update(payload)-- @pytest.mark.p3- def test_repeated_update_chunk(self, add_chunks):+ with pytest.raises(ChunkUpdateError) as error:+ chunks[0].update({field: value})+ assert error.value.code == 102+ assert field in error.value.message+ assert "list" in error.value.message++ @pytest.mark.parametrize("available", [True, False])+ def test_available(self, add_chunks, available):+ _, document, chunks = add_chunks+ chunks[0].update({"available": not available})+ assert_updated(document, chunks[0], "available", not available)+ chunks[0].update({"available": available})+ assert_updated(document, chunks[0], "available", available)++ @pytest.mark.skip(reason="Go UpdateChunk only reads bool availability and silently ignores other types")+ @pytest.mark.parametrize("available", ["True", "False", 1, 0])+ def test_available_invalid(self, add_chunks, available):_, _, chunks = add_chunks- chunk = chunks[0]+ with pytest.raises(ChunkUpdateError) as error:+ chunks[0].update({"available": available})+ assert error.value.code in (101, 102, 400)+ assert "available" in error.value.message- chunk.update({"content": "chunk test 1"})- chunk.update({"content": "chunk test 2"})+ def test_repeated_update_chunk(self, add_chunks):+ _, document, chunks = add_chunks+ for content in ("chunk test 1", "chunk test 2", "chunk test 2"):+ chunks[0].update({"content": content})+ assert_updated(document, chunks[0], "content_with_weight", content)- @pytest.mark.p3- @pytest.mark.skipif(os.getenv("DOC_ENGINE") == "infinity", reason="issues/6554")def test_concurrent_update_chunk(self, add_chunks):- count = 50- _, _, chunks = add_chunks-+ _, document, chunks = add_chunks+ values = {chunk.id: set() for chunk in chunks}with ThreadPoolExecutor(max_workers=5) as executor:- futures = [executor.submit(chunks[randint(0, 3)].update, {"content": f"update chunk test {i}"}) for i in range(count)]- responses = list(as_completed(futures))- assert len(responses) == count, responses+ futures = []+ for i in range(50):+ chunk = chunks[i % len(chunks)]+ content = f"concurrent update {i}"+ values[chunk.id].add(content)+ futures.append(executor.submit(chunk.update, {"content": content}))+ for future in as_completed(futures):+ assert future.result() is None+ # Completion order is unspecified; each final value must be a submitted write.+ wait_until(lambda: all(row["content_with_weight"] in values[row["chunk_id"]] for row in raw_chunks(document)))+ assert len(raw_chunks(document)) == len(chunks)- @pytest.mark.p3def test_update_chunk_to_deleted_document(self, add_chunks):dataset, document, chunks = add_chunksdataset.delete_documents(ids=[document.id])-- with pytest.raises(Exception) as exception_info:+ with pytest.raises(ChunkUpdateError, match="document not found") as error:chunks[0].update({})- message = str(exception_info.value)- ownership_message = f"you don't own the document {chunks[0].document_id}"- assert message.rstrip(".") == ownership_message or message == f"Can't find this chunk {chunks[0].id}", message+ assert error.value.code == 500++ def test_unsupported_field(self, add_chunks):+ _, _, chunks = add_chunks+ with pytest.raises(ChunkUpdateError, match="not supported") as error:+ chunks[0].update({"unknown": True})+ assert error.value.code == 400