Skip to content

RAGFlow is an agent-native database: a Retrieval-Augmented Generation (RAG) engine that grew into context infrastructure.

test(sdk): audit chunk management tests (#21635)

Sbaaoui Idriss3f6939db1a35
diff --git a/test/testcases/test_sdk_api/test_chunk_management_within_dataset/README.md b/test/testcases/test_sdk_api/test_chunk_management_within_dataset/README.md
new file mode 100644
index 0000000000..0183cf0ad7
--- /dev/null
+++ b/test/testcases/test_sdk_api/test_chunk_management_within_dataset/README.md
@@ -0,0 +1,170 @@
+# Chunk SDK contract audit
+
+This folder targets the checked-in Python SDK and Go API. It does not use the
+legacy Python server as a contract oracle. No production code or full SDK CI is
+changed. Dataset/document audit PRs #21619 and #21633 are not prerequisites.
+
+## Running safely
+
+Run from the repository root with **both** the local config and `--confcutdir`.
+This excludes ancestor fixtures that bootstrap credentials, mutate configuration,
+parse shared documents, or delete every dataset belonging to the test tenant.
+
+```sh
+CHUNKS=test/testcases/test_sdk_api/test_chunk_management_within_dataset
+python -m pytest -c "$CHUNKS/pytest.ini" --confcutdir="$CHUNKS" "$CHUNKS" --collect-only -q
+python -m pytest -c "$CHUNKS/pytest.ini" --confcutdir="$CHUNKS" "$CHUNKS" -m 'not integration' -q
+```
+
+Integration requires a disposable **Go backend and dedicated tenant**, an embedding
+model configured for that tenant, and its search/storage dependencies. Set
+`RAGFLOW_SDK_ISOLATED=1`, `RAGFLOW_SDK_BASE_URL`, and `RAGFLOW_SDK_API_KEY` only after
+verifying ownership of that environment, then run the same command without `-m`.
+The guard fails setup rather than disguising missing infrastructure as a defect
+skip. `RAGFLOW_SDK_INDEX_TIMEOUT` controls the indexing deadline (default 60 seconds).
+Request connect/read timeouts are 5/60 seconds. API errors are never retried as
+indexing lag. The 1,000-chunk deletion case is intentionally substantial.
+
+The valid reranker case also requires `RAGFLOW_SDK_RERANK_ID`; keyword expansion
+requires a configured tenant chat model and `RAGFLOW_SDK_CHAT_CONFIGURED=1`.
+Missing model configuration fails the relevant case; it is not a production skip.
+Run against Elasticsearch and Infinity separately. Current `internal/engine/global.go`
+has no OpenSearch backend; obsolete OpenSearch count branches are removed.
+
+Fixtures create a unique dataset per case and delete only that dataset ID, including
+when upload fails. Cleanup errors remain visible and deletion is verified by ID.
+Chunks are created manually; there is no ingestion run or `document.run == "DONE"`
+assumption. Listing and retrieval readiness are checked independently. Raw list
+responses are used for persistence probes because the SDK list schema is defective;
+a separate skipped test retains the correct public SDK schema assertions.
+
+## Contract evidence
+
+The audited call paths are:
+
+- `sdk/python/ragflow_sdk/modules/document.py`: `add_chunk`, `delete_chunks`,
+ `list_chunks`; `modules/chunk.py`: `Chunk.update`, `ChunkUpdateError`;
+ `ragflow.py`: `RAGFlow.retrieve`.
+- `internal/handler/chunk.go`: request binding, query validation, update type
+ handling; `internal/service/chunk/chunk.go`: add/list/update/remove semantics,
+ ownership, image storage, engine requests and response fields.
+- `internal/handler/dataset.go`: `SearchDatasets` typed request validation;
+ `internal/service/dataset/search.go` and `internal/service/nlp/`: retrieval,
+ candidate selection, score ordering, document/metadata/availability filtering.
+- `internal/engine/elasticsearch/chunk.go`, `internal/engine/infinity/chunk.go`:
+ lexical/vector matching and fusion. `top_k` is a candidate limit, not a promised
+ response count; `page_size` is a response limit.
+- `docs/references/python_api_reference.md` and `http_api_reference.md`: chunk and
+ retrieval contracts. Python-specific exception examples are obsolete. The old
+ empty-question example also conflicts with current Go required-field validation.
+- `internal/handler/chunk_test.go`, `internal/service/chunk/chunk_test.go`,
+ `internal/handler/dataset_search_test.go`, and `test/api/retrieval_api_test.go`:
+ existing Go validation, ownership, idempotency, schema and retrieval coverage.
+
+These sources establish: add rejects empty/non-string content and invalid list
+payloads; update accepts empty string content but must reject invalid types;
+questions are trimmed and empty entries removed; duplicate adds use a stable ID;
+delete is document-scoped and unknown/repeated IDs are idempotent; numeric list
+query strings are accepted, whereas retrieval JSON numeric strings are rejected.
+Updates return `None` and do not mutate the SDK object's cached attributes.
+
+Retrieval tests use a small owned corpus with a shared lexical query, threshold
+zero and text-only readiness checks. They assert identity, ownership, content,
+finite ordered scores, pagination, filtering and response limits. They do not
+require arbitrary four/five-result counts from normal hybrid retrieval. The
+single-result lexical list filters use unique terms in the owned fixture.
+
+## Original test inventory and decisions
+
+All six original files, their fixtures/helpers, and every parameter branch were
+reviewed: **36 original test methods, 134 collected parameter cases**. Classification
+below is by original method: **20 UPDATE, 16 REPLACE, 0 unchanged KEEP, 0 whole-test
+DELETE**. Valid checks within rewritten methods remain. Consolidated duplicates
+include engine-specific lexical/top-k alternatives and repeated missing-argument
+checks; missing-argument behavior now has a service-free test. SKIP is an orthogonal
+case-level outcome, detailed below, rather than deletion of the original assertions.
+
+| Original file / method | Original cases | Decision | Coverage after audit |
+| --- | ---: | --- | --- |
+| add / content | 6 | UPDATE | Null, empty, whitespace, integer, text and punctuation; failed requests do not add IDs. |
+| add / important_keywords | 6 | UPDATE | String lists, empty members, duplicates, non-string members and scalar rejection. |
+| add / questions | 6 | UPDATE | Same boundaries plus normalization of blank questions. |
+| add / tag_kwd | 6 | UPDATE | String-list contract and exact Go validation messages. |
+| add / repeated_add_chunk | 1 | UPDATE | Same content returns the same ID and persists once. |
+| add / add_chunk_to_deleted_document | 1 | UPDATE | Current document ownership error. |
+| add / concurrent_add_chunk | 1 | UPDATE | Consume all 50 futures, unique IDs and persisted ID set. |
+| delete / delete_partial_invalid_id | 3 | REPLACE | Unknown IDs at three positions do not prevent owned valid deletions. |
+| delete / repeated_deletion | 1 | REPLACE | Repeated deletion is idempotent. |
+| delete / duplicate_deletion | 1 | REPLACE | Duplicate IDs are idempotent. |
+| delete / concurrent_deletion | 1 | UPDATE | All futures succeed and persistence becomes empty. |
+| delete / delete_1k | 1 | UPDATE | Preserve bulk coverage; paginated readiness and deletion probes. |
+| delete / basic_scenarios | 6 | REPLACE | Unknown/single/all IDs, explicit delete_all, safe no-op and Python kwargs boundary. |
+| list / page | 7 | UPDATE | Default/null, valid pages, numeric strings, zero/negative/non-numeric validation. |
+| list / page_size | 7 | UPDATE | Default/null, bounds, numeric strings and invalid values. |
+| list / keywords | 7 | REPLACE | Empty/default, shared term, unique term, numeric term and no-match; no engine count exceptions. |
+| list / id | 4 | REPLACE | Empty IDs remain active; known/unknown filters retain correct expectations under targeted skips. |
+| list / concurrent_list | 1 | UPDATE | Consume each future and validate owned fixture count. |
+| list / default | 1 | UPDATE | 31 explicit chunks establish the default limit of 30; no sleep. |
+| list / invalid_document_id_raises | 1 | UPDATE | Assert meaningful current Go error text. |
+| retrieval / basic_scenarios | 4 | REPLACE | Owned results with/without document filter; Python argument errors tested separately. |
+| retrieval / page | 7 | REPLACE | Compare slices of a controlled baseline; retain invalid inputs in shared negative matrix. |
+| retrieval / page_size | 5 | REPLACE | Defaults, single result, caps and typed/range errors. |
+| retrieval / vector_similarity_weight | 4 | UPDATE | Weights 0, 0.5, 1; finite ordered owned results and invalid ranges/types. |
+| retrieval / top_k | 5 | REPLACE | Candidate limits 1/10/2048, response cap; invalid values, no engine result-count branches. |
+| retrieval / rerank_id | 2 | REPLACE | Configured reranker and explicit unknown-model rejection; no blanket skip. |
+| retrieval / keyword | 5 | REPLACE | False/null/true, configured expansion, string-bool rejection; no blanket skip. |
+| retrieval / concurrent_retrieval | 1 | UPDATE | Consume all futures and verify nonempty owned ordered results. |
+| update / content | 6 | UPDATE | Persist text/punctuation/empty/space; null/integer rejection remains skipped for defect. |
+| update / important_keywords | 6 | REPLACE | Shared typed list matrix; seed then clear; invalid cases retain rejection assertions. |
+| update / questions | 6 | REPLACE | Shared matrix with trimming/blank normalization and invalid-type skips. |
+| update / tag_kwd | 6 | REPLACE | Shared matrix with active strict negative cases. |
+| update / available | 6 | REPLACE | Both bool transitions persist; string/integer rejection retained under defect skip. |
+| update / repeated_update_chunk | 1 | UPDATE | Every sequential/repeated write is verified in storage. |
+| update / concurrent_update_chunk | 1 | UPDATE | 50 contending writes across four chunks; all futures observed and final values submitted. |
+| update / update_chunk_to_deleted_document | 1 | UPDATE | `ChunkUpdateError`, current code/message rather than generic exception. |
+
+New coverage includes add list-null/empty and image payload boundaries; delete_all
+conflict/document isolation; list response schema; retrieval negative matrix,
+document/metadata/availability filters and pagination uniqueness; unsupported update
+fields; SDK payload/default/error contracts; polling and cleanup failure behavior.
+
+## Skip audit
+
+Historical references were inspected rather than carried forward based on issue
+status: #6411, #6509, #6499, #6554 and #10692 concern the legacy Python backend;
+PR #6376 addressed duplicate-ID deletion validation. Their old skips are removed
+or replaced by the current defects below. The old `**"not json"` case never reaches
+HTTP; a service-free TypeError check replaces that skip. Old list negative-input
+skips now assert Go validation, and rerank/keyword groups execute with explicit
+model prerequisites. No retrieval group is blanket-skipped.
+
+**20 cases remain skipped for source-confirmed contract defects:**
+
+| Cases | Defect and source boundary | Correct assertion retained |
+| ---: | --- | --- |
+| 3 | `RemoveChunks` rejects omitted/null/empty IDs without delete_all, contrary to both API references. | Safe no-op preserves IDs. |
+| 2 | SDK `list_chunks(id=...)` sends `id`; Go `ListChunks` reads `chunk_ids`. | Known ID selects one; unknown ID returns none. |
+| 1 | Go list returns storage names (`chunk_id`, `content_with_weight`, `important_kwd`, `question_kwd`) and omits document/dataset IDs; SDK constructs `Chunk` without mapping. | SDK identity/content/ownership/keyword/question schema. |
+| 2 | Go `UpdateChunk` silently ignores null/integer content. | Typed `ChunkUpdateError` mentioning content. |
+| 4 | Go update ignores invalid keyword scalar/null types or converts non-string members to empty strings. | List-type validation error. |
+| 4 | Go update has the same invalid questions handling. | List-type validation error. |
+| 4 | Go update silently ignores string/integer availability. | Typed `ChunkUpdateError` mentioning availability. |
+
+No production fix is included. Defects are confirmed by current source paths, not
+by integration execution in this audit environment. Correct assertions remain
+inside every skipped case.
+
+## Validation boundary
+
+The resulting suite collects **195 cases**: add 39, delete 17, list 28, retrieval
+48, update 40, and 23 service-free SDK/fixture cases. Python syntax and focused
+Ruff checks pass. All **23 service-free cases pass**, including payloads, SDK error
+propagation, cleanup ownership/failures, deadlines and pagination-loop detection.
+
+A full invocation with no isolated backend configured produces **23 passed,
+20 skipped, 152 setup errors, zero assertion failures**. Every setup error is the
+explicit integration isolation guard; report these as **152 environment-blocked
+cases**, not production defects or successful integration tests. The existing
+host stack is not verified disposable and was not used. Elasticsearch/Infinity,
+model-backed retrieval/reranking/keyword expansion, 1,000-chunk deletion, images
+and concurrency still require execution in the isolated environment described above.
diff --git a/test/testcases/test_sdk_api/test_chunk_management_within_dataset/conftest.py b/test/testcases/test_sdk_api/test_chunk_management_within_dataset/conftest.py
index 4727c85a48..940e99a693 100644
--- a/test/testcases/test_sdk_api/test_chunk_management_within_dataset/conftest.py
+++ b/test/testcases/test_sdk_api/test_chunk_management_within_dataset/conftest.py
@@ -15,41 +15,96 @@
#
+import os
+from time import monotonic, sleep
+from uuid import uuid4
+
import pytest
-from common import batch_add_chunks, delete_all_chunks
-from pytest import FixtureRequest
-from ragflow_sdk import Chunk, DataSet, Document
-from utils import wait_for
+import requests
+from ragflow_sdk import RAGFlow
+
+
+def wait_until(probe, timeout=None):
+ if timeout is None:
+ timeout = float(os.getenv("RAGFLOW_SDK_INDEX_TIMEOUT", "60"))
+ assert timeout > 0, "Index timeout must be positive"
+ deadline = monotonic() + timeout
+ while True:
+ value = probe()
+ if value:
+ return value
+ if monotonic() >= deadline:
+ pytest.fail("Chunk indexing deadline exceeded")
+ sleep(0.25)
+
+
+def raw_chunks(document, page=1, page_size=100, **params):
+ response = document.get(
+ f"/datasets/{document.dataset_id}/documents/{document.id}/chunks",
+ {"page": page, "page_size": page_size, **params},
+ ).json()
+ assert response["code"] == 0, response
+ return response["data"]["chunks"]
+
+
+def raw_ids(document):
+ ids = set()
+ page = 1
+ while True:
+ chunks = raw_chunks(document, page=page)
+ page_ids = {chunk["chunk_id"] for chunk in chunks}
+ assert len(page_ids) == len(chunks), "Chunk page contains duplicate IDs"
+ assert not ids.intersection(page_ids), "Chunk pagination repeated IDs"
+ ids.update(page_ids)
+ if len(chunks) < 100:
+ return ids
+ page += 1
+
+
+def wait_ids(document, expected):
+ expected = set(expected)
+ wait_until(lambda: raw_ids(document) == expected)
+
+
[email protected](autouse=True)
+def network_boundary(request, monkeypatch):
+ original = requests.sessions.Session.request
+ def bounded(session, *args, **kwargs):
+ if request.node.get_closest_marker("integration") is None:
+ pytest.fail("Service-free chunk tests must not open network connections")
+ kwargs.setdefault("timeout", (5, 60))
+ return original(session, *args, **kwargs)
-@wait_for(30, 1, "Document parsing timeout")
-def condition(_dataset: DataSet):
- documents = _dataset.list_documents(page_size=100)
- for document in documents:
- if document.run != "DONE":
- return False
- return True
+ monkeypatch.setattr(requests.sessions.Session, "request", bounded)
-@wait_for(30, 1, "Chunk indexing timeout")
-def chunks_visible(_document: Document, _chunk_ids: list[str]):
- visible_ids = {chunk.id for chunk in _document.list_chunks(page_size=100)}
- return set(_chunk_ids).issubset(visible_ids)
+def client():
+ if os.getenv("RAGFLOW_SDK_ISOLATED") != "1" or not all(os.getenv(key) for key in ("RAGFLOW_SDK_BASE_URL", "RAGFLOW_SDK_API_KEY")):
+ pytest.fail("Integration blocked: set RAGFLOW_SDK_ISOLATED=1, RAGFLOW_SDK_BASE_URL and RAGFLOW_SDK_API_KEY for an isolated Go backend")
+ return RAGFlow(os.environ["RAGFLOW_SDK_API_KEY"], os.environ["RAGFLOW_SDK_BASE_URL"])
[email protected](scope="function")
-def add_chunks_func(request: FixtureRequest, add_document: tuple[DataSet, Document]) -> tuple[DataSet, Document, list[Chunk]]:
- def cleanup():
- try:
- delete_all_chunks(document)
- except Exception:
- pass
+def add_document(client):
+ dataset = client.create_dataset(name=f"sdk-chunks-{uuid4().hex}")
+ try:
+ documents = dataset.upload_documents([{"display_name": "chunks.txt", "blob": b"Owned chunk SDK fixture."}])
+ assert len(documents) == 1
+ yield dataset, documents[0]
+ finally:
+ client.delete_datasets(ids=[dataset.id])
+ assert client.list_datasets(id=dataset.id) == [], "Owned dataset cleanup failed"
- request.addfinalizer(cleanup)
+def add_chunks(add_document):
dataset, document = add_document
- dataset.async_parse_documents([document.id])
- condition(dataset)
- chunks = batch_add_chunks(document, 4)
- chunks_visible(document, [chunk.id for chunk in chunks])
+ # Manual chunk operations do not need ingestion or a parser status wait.
+ chunks = [
+ document.add_chunk(content=f"Coriander botanical archive {label} entry {i}.", important_keywords=["coriander"], questions=["What is coriander?"])
+ for i, label in enumerate(("saffron", "cumin", "basil", "thyme"))
+ ]
+ wait_ids(document, [chunk.id for chunk in chunks])
return dataset, document, chunks
diff --git a/test/testcases/test_sdk_api/test_chunk_management_within_dataset/pytest.ini b/test/testcases/test_sdk_api/test_chunk_management_within_dataset/pytest.ini
new file mode 100644
index 0000000000..4434268134
--- /dev/null
+++ b/test/testcases/test_sdk_api/test_chunk_management_within_dataset/pytest.ini
@@ -0,0 +1,8 @@
+[pytest]
+pythonpath = ../../../../sdk/python
+addopts = --strict-markers
+markers =
+ integration: requires an explicitly isolated Go backend and dedicated tenant
+ p1: high priority
+ p2: medium priority
+ p3: low priority
diff --git a/test/testcases/test_sdk_api/test_chunk_management_within_dataset/test_add_chunk.py b/test/testcases/test_sdk_api/test_chunk_management_within_dataset/test_add_chunk.py
index 2db1022752..8cc923204e 100644
--- a/test/testcases/test_sdk_api/test_chunk_management_within_dataset/test_add_chunk.py
+++ b/test/testcases/test_sdk_api/test_chunk_management_within_dataset/test_add_chunk.py
@@ -14,7 +14,7 @@
# limitations under the License.
#
from concurrent.futures import ThreadPoolExecutor, as_completed
-from time import sleep
+from conftest import raw_ids, wait_ids
import pytest
from ragflow_sdk import Chunk
@@ -32,14 +32,17 @@ def validate_chunk_details(dataset_id: str, document_id: str, payload: dict, chu
assert chunk.tag_kwd == payload["tag_kwd"]
+pytestmark = pytest.mark.integration
+
+
class TestAddChunk:
@pytest.mark.p1
@pytest.mark.parametrize(
"payload, expected_message",
[
- ({"content": None}, "not instance of"),
+ ({"content": None}, "`content` is required"),
({"content": ""}, "`content` is required"),
- ({"content": 1}, "not instance of"),
+ ({"content": 1}, "`content` must be a string"),
({"content": "a"}, ""),
({"content": " "}, "`content` is required"),
({"content": "\n!?。;!?\"'"}, ""),
@@ -47,19 +50,18 @@ class TestAddChunk:
)
def test_content(self, add_document, payload, expected_message):
dataset, document = add_document
- chunks_count = len(document.list_chunks())
+ initial_ids = raw_ids(document)
if expected_message:
with pytest.raises(Exception) as exception_info:
document.add_chunk(**payload)
assert expected_message in str(exception_info.value), str(exception_info.value)
+ assert raw_ids(document) == initial_ids
else:
chunk = document.add_chunk(**payload)
validate_chunk_details(dataset.id, document.id, payload, chunk)
- sleep(1)
- chunks = document.list_chunks()
- assert len(chunks) == chunks_count + 1, str(chunks)
+ wait_ids(document, initial_ids | {chunk.id})
@pytest.mark.p2
@pytest.mark.parametrize(
@@ -67,27 +69,26 @@ class TestAddChunk:
[
({"content": "chunk test important_keywords 1", "important_keywords": ["a", "b", "c"]}, ""),
({"content": "chunk test important_keywords 2", "important_keywords": [""]}, ""),
- ({"content": "chunk test important_keywords 3", "important_keywords": [1]}, "not instance of"),
+ ({"content": "chunk test important_keywords 3", "important_keywords": [1]}, "`important_keywords` must be a list of strings"),
({"content": "chunk test important_keywords 4", "important_keywords": ["a", "a"]}, ""),
- ({"content": "chunk test important_keywords 5", "important_keywords": "abc"}, "not instance of"),
- ({"content": "chunk test important_keywords 6", "important_keywords": 123}, "not instance of"),
+ ({"content": "chunk test important_keywords 5", "important_keywords": "abc"}, "`important_keywords` is required to be a list"),
+ ({"content": "chunk test important_keywords 6", "important_keywords": 123}, "`important_keywords` is required to be a list"),
],
)
def test_important_keywords(self, add_document, payload, expected_message):
dataset, document = add_document
- chunks_count = len(document.list_chunks())
+ initial_ids = raw_ids(document)
if expected_message:
with pytest.raises(Exception) as exception_info:
document.add_chunk(**payload)
assert expected_message in str(exception_info.value), str(exception_info.value)
+ assert raw_ids(document) == initial_ids
else:
chunk = document.add_chunk(**payload)
validate_chunk_details(dataset.id, document.id, payload, chunk)
- sleep(1)
- chunks = document.list_chunks()
- assert len(chunks) == chunks_count + 1, str(chunks)
+ wait_ids(document, initial_ids | {chunk.id})
@pytest.mark.p3
@pytest.mark.parametrize(
@@ -95,27 +96,26 @@ class TestAddChunk:
[
({"content": "chunk test test_questions 1", "questions": ["a", "b", "c"]}, ""),
({"content": "chunk test test_questions 2", "questions": [""]}, ""),
- ({"content": "chunk test test_questions 3", "questions": [1]}, "not instance of"),
+ ({"content": "chunk test test_questions 3", "questions": [1]}, "`questions` must be a list of strings"),
({"content": "chunk test test_questions 4", "questions": ["a", "a"]}, ""),
- ({"content": "chunk test test_questions 5", "questions": "abc"}, "not instance of"),
- ({"content": "chunk test test_questions 6", "questions": 123}, "not instance of"),
+ ({"content": "chunk test test_questions 5", "questions": "abc"}, "`questions` is required to be a list"),
+ ({"content": "chunk test test_questions 6", "questions": 123}, "`questions` is required to be a list"),
],
)
def test_questions(self, add_document, payload, expected_message):
dataset, document = add_document
- chunks_count = len(document.list_chunks())
+ initial_ids = raw_ids(document)
if expected_message:
with pytest.raises(Exception) as exception_info:
document.add_chunk(**payload)
assert expected_message in str(exception_info.value), str(exception_info.value)
+ assert raw_ids(document) == initial_ids
else:
chunk = document.add_chunk(**payload)
validate_chunk_details(dataset.id, document.id, payload, chunk)
- sleep(1)
- chunks = document.list_chunks()
- assert len(chunks) == chunks_count + 1, str(chunks)
+ wait_ids(document, initial_ids | {chunk.id})
@pytest.mark.p2
@pytest.mark.parametrize(
@@ -123,45 +123,41 @@ class TestAddChunk:
[
({"content": "chunk test test_tag_kwd 1", "tag_kwd": ["tag1", "tag2"]}, ""),
({"content": "chunk test test_tag_kwd 2", "tag_kwd": [""]}, ""),
- ({"content": "chunk test test_tag_kwd 3", "tag_kwd": [1]}, "not instance of"),
+ ({"content": "chunk test test_tag_kwd 3", "tag_kwd": [1]}, "`tag_kwd` must be a list of strings"),
({"content": "chunk test test_tag_kwd 4", "tag_kwd": ["tag", "tag"]}, ""),
- ({"content": "chunk test test_tag_kwd 5", "tag_kwd": "abc"}, "not instance of"),
- ({"content": "chunk test test_tag_kwd 6", "tag_kwd": 123}, "not instance of"),
+ ({"content": "chunk test test_tag_kwd 5", "tag_kwd": "abc"}, "`tag_kwd` is required to be a list"),
+ ({"content": "chunk test test_tag_kwd 6", "tag_kwd": 123}, "`tag_kwd` is required to be a list"),
],
)
def test_tag_kwd(self, add_document, payload, expected_message):
dataset, document = add_document
- chunks_count = len(document.list_chunks())
+ initial_ids = raw_ids(document)
if expected_message:
with pytest.raises(Exception) as exception_info:
document.add_chunk(**payload)
assert expected_message in str(exception_info.value), str(exception_info.value)
+ assert raw_ids(document) == initial_ids
else:
chunk = document.add_chunk(**payload)
validate_chunk_details(dataset.id, document.id, payload, chunk)
- sleep(1)
- chunks = document.list_chunks()
- assert len(chunks) == chunks_count + 1, str(chunks)
+ wait_ids(document, initial_ids | {chunk.id})
@pytest.mark.p3
def test_repeated_add_chunk(self, add_document):
payload = {"content": "chunk test repeated_add_chunk"}
dataset, document = add_document
- chunks_count = len(document.list_chunks())
+ initial_ids = raw_ids(document)
chunk1 = document.add_chunk(**payload)
validate_chunk_details(dataset.id, document.id, payload, chunk1)
- sleep(1)
- chunks = document.list_chunks()
- assert len(chunks) == chunks_count + 1, str(chunks)
+ wait_ids(document, initial_ids | {chunk1.id})
chunk2 = document.add_chunk(**payload)
validate_chunk_details(dataset.id, document.id, payload, chunk2)
- sleep(1)
- chunks = document.list_chunks()
- assert len(chunks) == chunks_count + 1, str(chunks)
+ assert chunk2.id == chunk1.id
+ wait_ids(document, initial_ids | {chunk1.id})
@pytest.mark.p2
def test_add_chunk_to_deleted_document(self, add_document):
@@ -172,19 +168,53 @@ class TestAddChunk:
document.add_chunk(content="chunk test")
assert f"you don't own the document {document.id}" in str(exception_info.value), str(exception_info.value)
- @pytest.mark.skip(reason="issues/6411")
@pytest.mark.p3
def test_concurrent_add_chunk(self, add_document):
count = 50
_, document = add_document
- initial_chunk_count = len(document.list_chunks())
+ initial_ids = raw_ids(document)
def add_chunk_task(i):
return document.add_chunk(content=f"chunk test concurrent {i}")
with ThreadPoolExecutor(max_workers=5) as executor:
futures = [executor.submit(add_chunk_task, i) for i in range(count)]
- responses = list(as_completed(futures))
- assert len(responses) == count, responses
- sleep(5)
- assert len(document.list_chunks(page_size=100)) == initial_chunk_count + count
+ responses = [future.result() for future in as_completed(futures)]
+ assert len({chunk.id for chunk in responses}) == count
+ wait_ids(document, initial_ids | {chunk.id for chunk in responses})
+
+ @pytest.mark.parametrize("field", ["important_keywords", "questions", "tag_kwd"])
+ @pytest.mark.parametrize("value", [[], None])
+ def test_empty_list_and_null(self, add_document, field, value):
+ dataset, document = add_document
+ payload = {"content": "List boundary", field: value}
+ if value is None:
+ with pytest.raises(Exception, match=f"`{field}` is required to be a list"):
+ document.add_chunk(**payload)
+ assert raw_ids(document) == set()
+ else:
+ chunk = document.add_chunk(**payload)
+ validate_chunk_details(dataset.id, document.id, payload, chunk)
+ wait_ids(document, [chunk.id])
+
+ @pytest.mark.parametrize(
+ "image, error",
+ [
+ (None, None),
+ ("iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAIAAACQd1PeAAAADElEQVR4nGP4z8AAAAMBAQDJ/pLvAAAAAElFTkSuQmCC", None),
+ ("", "`image_base64` must be a non-empty string"),
+ (" ", "`image_base64` must be a non-empty string"),
+ ("not-base64", "invalid `image_base64`"),
+ (123, "`image_base64` must be a string"),
+ ],
+ )
+ def test_image(self, add_document, image, error):
+ dataset, document = add_document
+ if error:
+ with pytest.raises(Exception, match=error):
+ document.add_chunk(content="Image boundary", image_base64=image)
+ assert raw_ids(document) == set()
+ else:
+ chunk = document.add_chunk(content="Image boundary", image_base64=image)
+ assert chunk.image_id == (f"{dataset.id}-{chunk.id}" if image else "")
+ wait_ids(document, [chunk.id])
diff --git a/test/testcases/test_sdk_api/test_chunk_management_within_dataset/test_chunk_sdk_contract.py b/test/testcases/test_sdk_api/test_chunk_management_within_dataset/test_chunk_sdk_contract.py
new file mode 100644
index 0000000000..500c352e94
--- /dev/null
+++ b/test/testcases/test_sdk_api/test_chunk_management_within_dataset/test_chunk_sdk_contract.py
@@ -0,0 +1,208 @@
+#
+# Copyright 2025 The InfiniFlow Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+from unittest.mock import Mock
+
+import pytest
+from conftest import add_document, raw_ids, wait_until
+from ragflow_sdk import Chunk, Document, RAGFlow
+from ragflow_sdk.modules.chunk import ChunkUpdateError
+
+
+def sdk():
+ return RAGFlow("test-token", "http://unused.invalid")
+
+
+def document(sdk):
+ return Document(sdk, {"id": "document", "dataset_id": "dataset"})
+
+
+def response(body):
+ return Mock(json=Mock(return_value=body))
+
+
+def test_add_chunk_payload_and_response(sdk, document, monkeypatch):
+ body = {"content": "text", "important_keywords": ["key"], "questions": ["why?"], "tag_kwd": ["tag"], "image_base64": "aGVsbG8="}
+ row = {**{key: value for key, value in body.items() if key != "image_base64"}, "id": "chunk", "dataset_id": "dataset", "document_id": "document", "image_id": "image"}
+ post = Mock(return_value=response({"code": 0, "data": {"chunk": row}}))
+ monkeypatch.setattr(sdk, "post", post)
+ chunk = document.add_chunk(**body)
+ post.assert_called_once_with("/datasets/dataset/documents/document/chunks", body, stream=False, files=None)
+ for key, value in row.items():
+ assert getattr(chunk, key) == value
+
+
+def test_add_chunk_defaults(sdk, document, monkeypatch):
+ post = Mock(return_value=response({"code": 0, "data": {"chunk": {"id": "chunk"}}}))
+ monkeypatch.setattr(sdk, "post", post)
+ document.add_chunk("text")
+ assert post.call_args.args[1] == {"content": "text", "important_keywords": [], "questions": [], "tag_kwd": []}
+
+
+ "payload,expected",
+ [({}, {"chunk_ids": None, "delete_all": False}), ({"ids": ["chunk"]}, {"chunk_ids": ["chunk"], "delete_all": False}), ({"delete_all": True}, {"chunk_ids": None, "delete_all": True})],
+)
+def test_delete_chunk_payload(sdk, document, monkeypatch, payload, expected):
+ delete = Mock(return_value=response({"code": 0}))
+ monkeypatch.setattr(sdk, "delete", delete)
+ assert document.delete_chunks(**payload) is None
+ delete.assert_called_once_with("/datasets/dataset/documents/document/chunks", expected)
+
+
+def test_list_chunk_query_and_response(sdk, document, monkeypatch):
+ get = Mock(return_value=response({"code": 0, "data": {"chunks": [{"id": "chunk", "content": "text"}]}}))
+ monkeypatch.setattr(sdk, "get", get)
+ rows = document.list_chunks(page=2, page_size=3, keywords="key", id="chunk")
+ get.assert_called_once_with("/datasets/dataset/documents/document/chunks", {"page": 2, "page_size": 3, "keywords": "key", "id": "chunk"})
+ assert [(row.id, row.content) for row in rows] == [("chunk", "text")]
+
+
[email protected]("body", [{"code": 0}, {"code": 102, "message": "invalid", "details": {"field": "tag_kwd"}}])
+def test_chunk_update_result(sdk, monkeypatch, body):
+ patch = Mock(return_value=response(body))
+ monkeypatch.setattr(sdk, "patch", patch)
+ chunk = Chunk(sdk, {"id": "chunk", "dataset_id": "dataset", "document_id": "document", "content": "old"})
+ if body["code"]:
+ with pytest.raises(ChunkUpdateError) as error:
+ chunk.update({"content": "new"})
+ assert (error.value.code, error.value.message, error.value.details) == (102, "invalid", {"field": "tag_kwd"})
+ else:
+ assert chunk.update({"content": "new"}) is None
+ patch.assert_called_once_with("/datasets/dataset/documents/document/chunks/chunk", {"content": "new"})
+ assert chunk.content == "old" # Updates require a fresh read; the SDK does not mutate this object.
+
+
[email protected]("operation", ["add", "delete", "list", "retrieve"])
+def test_sdk_propagates_api_error(sdk, document, monkeypatch, operation):
+ error = response({"code": 102, "message": "contract failure"})
+ for method in ("post", "get", "delete"):
+ monkeypatch.setattr(sdk, method, Mock(return_value=error))
+ with pytest.raises(Exception, match="^contract failure$"):
+ if operation == "add":
+ document.add_chunk("text")
+ elif operation == "delete":
+ document.delete_chunks(ids=["chunk"])
+ elif operation == "list":
+ document.list_chunks()
+ else:
+ sdk.retrieve(dataset_ids=["dataset"], question="text")
+
+
+ "options",
+ [
+ {},
+ {
+ "page": 2,
+ "page_size": 1,
+ "top_k": 10,
+ "vector_similarity_weight": 0.5,
+ "similarity_threshold": 0,
+ "document_ids": ["document"],
+ "rerank_id": "reranker",
+ "keyword": True,
+ "cross_languages": ["English"],
+ "metadata_condition": {"logic": "and", "conditions": []},
+ "use_kg": True,
+ },
+ ],
+)
+def test_retrieve_payload_and_response(sdk, monkeypatch, options):
+ post = Mock(return_value=response({"code": 0, "data": {"chunks": [{"id": "chunk", "content": "text", "similarity": 0.75, "document_id": "document", "dataset_id": "dataset"}]}}))
+ monkeypatch.setattr(sdk, "post", post)
+ rows = sdk.retrieve(dataset_ids=["dataset"], question="text", **options)
+ expected = {
+ "dataset_ids": ["dataset"],
+ "question": "text",
+ "document_ids": [],
+ "page": 1,
+ "page_size": 30,
+ "top_k": 1024,
+ "similarity_threshold": 0.2,
+ "vector_similarity_weight": 0.3,
+ "rerank_id": None,
+ "keyword": False,
+ "cross_languages": None,
+ "metadata_condition": None,
+ "use_kg": False,
+ **options,
+ }
+ post.assert_called_once_with("/retrieval", json=expected)
+ assert [(row.id, row.content, row.similarity, row.document_id, row.dataset_id) for row in rows] == [("chunk", "text", 0.75, "document", "dataset")]
+
+
+def test_missing_datasets_is_python_argument_error(sdk):
+ with pytest.raises(TypeError, match="dataset_ids"):
+ sdk.retrieve(question="text")
+
+
+def test_invalid_kwargs_is_python_argument_error(document):
+ with pytest.raises(TypeError, match="mapping"):
+ document.delete_chunks(**"not json")
+
+
+def test_wait_only_retries_visibility():
+ probe = Mock(side_effect=[False, True])
+ assert wait_until(probe, timeout=1) is True
+ assert probe.call_count == 2
+
+
+def test_wait_propagates_errors():
+ with pytest.raises(RuntimeError, match="backend unavailable"):
+ wait_until(Mock(side_effect=RuntimeError("backend unavailable")))
+
+
+def test_wait_deadline():
+ with pytest.raises(pytest.fail.Exception, match="deadline exceeded"):
+ wait_until(lambda: False, timeout=0.001)
+
+
+def test_pagination_detects_repeated_pages(document, monkeypatch):
+ monkeypatch.setattr("conftest.raw_chunks", lambda *args, **kwargs: [{"chunk_id": str(i)} for i in range(100)])
+ with pytest.raises(AssertionError, match="pagination repeated IDs"):
+ raw_ids(document)
+
+
[email protected]("delete_fails", [False, True])
+def test_fixture_cleanup_is_owned_and_errors_visible(delete_fails):
+ client = Mock()
+ dataset = client.create_dataset.return_value
+ dataset.id = "owned-dataset"
+ dataset.upload_documents.return_value = [Mock(id="owned-document")]
+ client.list_datasets.return_value = []
+ fixture = add_document.__wrapped__(client)
+ assert next(fixture) == (dataset, dataset.upload_documents.return_value[0])
+ if delete_fails:
+ client.delete_datasets.side_effect = RuntimeError("cleanup failure")
+ with pytest.raises(RuntimeError, match="cleanup failure"):
+ next(fixture)
+ else:
+ with pytest.raises(StopIteration):
+ next(fixture)
+ client.list_datasets.assert_called_once_with(id="owned-dataset")
+ client.delete_datasets.assert_called_once_with(ids=["owned-dataset"])
+
+
+def test_fixture_cleans_up_after_upload_error():
+ client = Mock()
+ client.create_dataset.return_value.id = "owned-dataset"
+ client.create_dataset.return_value.upload_documents.side_effect = RuntimeError("upload failure")
+ client.list_datasets.return_value = []
+ with pytest.raises(RuntimeError, match="upload failure"):
+ next(add_document.__wrapped__(client))
+ client.delete_datasets.assert_called_once_with(ids=["owned-dataset"])
diff --git a/test/testcases/test_sdk_api/test_chunk_management_within_dataset/test_delete_chunks.py b/test/testcases/test_sdk_api/test_chunk_management_within_dataset/test_delete_chunks.py
index 20d5dc2aab..9b1c1d076a 100644
--- a/test/testcases/test_sdk_api/test_chunk_management_within_dataset/test_delete_chunks.py
+++ b/test/testcases/test_sdk_api/test_chunk_management_within_dataset/test_delete_chunks.py
@@ -13,104 +13,92 @@
# See the License for the specific language governing permissions and
# limitations under the License.
#
-from concurrent.futures import ThreadPoolExecutor, as_completed
-
-import pytest
-from common import batch_add_chunks
-
-
-class TestChunksDeletion:
- @pytest.mark.parametrize(
- "payload",
- [
- pytest.param(lambda r: {"ids": ["invalid_id"] + r}, marks=pytest.mark.p3),
- pytest.param(lambda r: {"ids": r[:1] + ["invalid_id"] + r[1:4]}, marks=pytest.mark.p3),
- pytest.param(lambda r: {"ids": r + ["invalid_id"]}, marks=pytest.mark.p3),
- ],
- )
- def test_delete_partial_invalid_id(self, add_chunks_func, payload):
- _, document, chunks = add_chunks_func
- chunk_ids = [chunk.id for chunk in chunks]
- payload = payload(chunk_ids)
- with pytest.raises(Exception) as exception_info:
- document.delete_chunks(**payload)
- assert "rm_chunk deleted chunks" in str(exception_info.value), str(exception_info.value)
- remaining_chunks = document.list_chunks()
- assert len(remaining_chunks) == 1, str(remaining_chunks)
+from concurrent.futures import ThreadPoolExecutor, as_completed
- @pytest.mark.p3
- def test_repeated_deletion(self, add_chunks_func):
- _, document, chunks = add_chunks_func
- chunk_ids = [chunk.id for chunk in chunks]
- document.delete_chunks(ids=chunk_ids)
+import pytest
+from conftest import raw_ids, wait_ids
- with pytest.raises(Exception) as exception_info:
- document.delete_chunks(ids=chunk_ids)
- assert "rm_chunk deleted chunks 0, expect" in str(exception_info.value), str(exception_info.value)
+pytestmark = pytest.mark.integration
- @pytest.mark.p3
- def test_duplicate_deletion(self, add_chunks_func):
- _, document, chunks = add_chunks_func
- chunk_ids = [chunk.id for chunk in chunks]
- document.delete_chunks(ids=chunk_ids * 2)
- remaining_chunks = document.list_chunks()
- assert len(remaining_chunks) == 1, str(remaining_chunks)
- @pytest.mark.p3
- def test_concurrent_deletion(self, add_document):
- count = 100
- _, document = add_document
- chunks = batch_add_chunks(document, count)
- chunk_ids = [chunk.id for chunk in chunks]
-
- with ThreadPoolExecutor(max_workers=5) as executor:
- futures = [executor.submit(document.delete_chunks, ids=[chunk_id]) for chunk_id in chunk_ids]
- responses = list(as_completed(futures))
- assert len(responses) == count, responses
+class TestChunksDeletion:
+ @pytest.mark.parametrize("position", [0, 2, 4])
+ def test_delete_partial_invalid_id(self, add_chunks, position):
+ _, document, chunks = add_chunks
+ ids = [chunk.id for chunk in chunks]
+ ids.insert(position, "unknown")
+ document.delete_chunks(ids=ids)
+ wait_ids(document, [])
+
+ def test_repeated_deletion(self, add_chunks):
+ _, document, chunks = add_chunks
+ ids = [chunk.id for chunk in chunks]
+ document.delete_chunks(ids=ids)
+ wait_ids(document, [])
+ document.delete_chunks(ids=ids)
+ assert raw_ids(document) == set()
+
+ def test_duplicate_deletion(self, add_chunks):
+ _, document, chunks = add_chunks
+ document.delete_chunks(ids=[chunk.id for chunk in chunks] * 2)
+ wait_ids(document, [])
+
+ def test_concurrent_deletion(self, add_chunks):
+ _, document, chunks = add_chunks
+ with ThreadPoolExecutor(max_workers=4) as executor:
+ futures = [executor.submit(document.delete_chunks, ids=[chunk.id]) for chunk in chunks]
+ for future in as_completed(futures):
+ assert future.result() is None
+ wait_ids(document, [])
- @pytest.mark.p3
def test_delete_1k(self, add_document):
- count = 1_000
_, document = add_document
- chunks = batch_add_chunks(document, count)
- chunk_ids = [chunk.id for chunk in chunks]
-
- from time import sleep
-
- sleep(1)
-
- document.delete_chunks(ids=chunk_ids)
- remaining_chunks = document.list_chunks()
- assert len(remaining_chunks) == 0, str(remaining_chunks)
+ chunks = [document.add_chunk(content=f"bulk chunk {i}") for i in range(1000)]
+ ids = [chunk.id for chunk in chunks]
+ wait_ids(document, ids)
+ document.delete_chunks(ids=ids)
+ wait_ids(document, [])
@pytest.mark.parametrize(
- "payload, expected_message, remaining",
+ "payload, removed",
[
- pytest.param(None, "", 5, marks=pytest.mark.p3),
- pytest.param({"ids": ["invalid_id"]}, "rm_chunk deleted chunks 0, expect 1", 5, marks=pytest.mark.p3),
- pytest.param("not json", "UnboundLocalError", 5, marks=pytest.mark.skip(reason="pull/6376")),
- pytest.param(lambda r: {"ids": r[:1]}, "", 4, marks=pytest.mark.p3),
- pytest.param(lambda r: {"ids": r}, "", 1, marks=pytest.mark.p3),
- pytest.param({"ids": []}, "", 5, marks=pytest.mark.p3),
+ ({"ids": ["unknown"]}, 0),
+ ("single", 1),
+ ("all_ids", 4),
+ ({"delete_all": True}, 4),
+ ({"ids": [], "delete_all": True}, 4),
],
)
- def test_basic_scenarios(self, add_chunks_func, payload, expected_message, remaining):
- _, document, chunks = add_chunks_func
- chunk_ids = [chunk.id for chunk in chunks]
- if callable(payload):
- payload = payload(chunk_ids)
-
- if expected_message:
- with pytest.raises(Exception) as exception_info:
- document.delete_chunks(**payload)
- assert expected_message in str(exception_info.value), str(exception_info.value)
- else:
- if payload is None:
- document.delete_chunks()
- else:
- document.delete_chunks(**payload)
-
- remaining_chunks = document.list_chunks()
- assert len(remaining_chunks) == remaining, str(remaining_chunks)
+ def test_basic_scenarios(self, add_chunks, payload, removed):
+ _, document, chunks = add_chunks
+ ids = [chunk.id for chunk in chunks]
+ if payload == "single":
+ payload = {"ids": ids[:1]}
+ elif payload == "all_ids":
+ payload = {"ids": ids}
+ document.delete_chunks(**payload)
+ wait_ids(document, ids[removed:])
+
+ @pytest.mark.skip(reason="Go RemoveChunks rejects omitted/empty IDs instead of documented safe no-op")
+ @pytest.mark.parametrize("payload", [{}, {"ids": None}, {"ids": []}])
+ def test_empty_selection_is_noop(self, add_chunks, payload):
+ _, document, chunks = add_chunks
+ document.delete_chunks(**payload)
+ assert raw_ids(document) == {chunk.id for chunk in chunks}
+
+ def test_delete_all_rejects_ids(self, add_chunks):
+ _, document, chunks = add_chunks
+ with pytest.raises(Exception, match="mutually exclusive"):
+ document.delete_chunks(ids=[chunks[0].id], delete_all=True)
+ assert raw_ids(document) == {chunk.id for chunk in chunks}
+
+ def test_delete_all_is_document_scoped(self, add_chunks):
+ dataset, document, _ = add_chunks
+ other = dataset.upload_documents([{"display_name": "other.txt", "blob": b"Other owned document"}])[0]
+ chunk = other.add_chunk(content="Must survive deletion in another document")
+ wait_ids(other, [chunk.id])
+ document.delete_chunks(delete_all=True)
+ wait_ids(document, [])
+ assert raw_ids(other) == {chunk.id}
diff --git a/test/testcases/test_sdk_api/test_chunk_management_within_dataset/test_list_chunks.py b/test/testcases/test_sdk_api/test_chunk_management_within_dataset/test_list_chunks.py
index e4f11dfcb2..eed021b58a 100644
--- a/test/testcases/test_sdk_api/test_chunk_management_within_dataset/test_list_chunks.py
+++ b/test/testcases/test_sdk_api/test_chunk_management_within_dataset/test_list_chunks.py
@@ -13,148 +13,104 @@
# See the License for the specific language governing permissions and
# limitations under the License.
#
-import os
+
+
from concurrent.futures import ThreadPoolExecutor, as_completed
import pytest
-from common import batch_add_chunks
-from utils.engine_utils import get_doc_engine
+from conftest import wait_ids
+from ragflow_sdk import Document
+
+pytestmark = pytest.mark.integration
class TestChunksList:
- @pytest.mark.p3
@pytest.mark.parametrize(
- "params, expected_page_size, expected_message",
+ "params, size, error",
[
- ({"page": None, "page_size": 2}, 2, ""),
- pytest.param({"page": 0, "page_size": 2}, 0, "ValueError('Search does not support negative slicing.')", marks=pytest.mark.skip),
- ({"page": 2, "page_size": 2}, 2, ""),
- ({"page": 3, "page_size": 2}, 1, ""),
- ({"page": "3", "page_size": 2}, 1, ""),
- pytest.param({"page": -1, "page_size": 2}, 0, "ValueError('Search does not support negative slicing.')", marks=pytest.mark.skip),
- pytest.param({"page": "a", "page_size": 2}, 0, """ValueError("invalid literal for int() with base 10: \'a\'")""", marks=pytest.mark.skip),
+ ({"page": None, "page_size": 2}, 2, None),
+ ({"page": 0}, 0, "page must be a positive integer"),
+ ({"page": 2, "page_size": 2}, 2, None),
+ ({"page": 3, "page_size": 2}, 0, None),
+ ({"page": "3", "page_size": 2}, 0, None),
+ ({"page": -1}, 0, "page must be a positive integer"),
+ ({"page": "a"}, 0, "page must be a positive integer"),
],
)
- def test_page(self, add_chunks, params, expected_page_size, expected_message):
+ def test_page(self, add_chunks, params, size, error):
_, document, _ = add_chunks
-
- if expected_message:
- with pytest.raises(Exception) as exception_info:
+ if error:
+ with pytest.raises(Exception, match=error):
document.list_chunks(**params)
- assert expected_message in str(exception_info.value), str(exception_info.value)
else:
- chunks = document.list_chunks(**params)
- assert len(chunks) == expected_page_size, str(chunks)
+ assert len(document.list_chunks(**params)) == size
- @pytest.mark.p3
@pytest.mark.parametrize(
- "params, expected_page_size, expected_message",
+ "value, size, error",
[
- ({"page_size": None}, 5, ""),
- pytest.param({"page_size": 0}, 5, ""),
- ({"page_size": 1}, 1, ""),
- ({"page_size": 6}, 5, ""),
- ({"page_size": "1"}, 1, ""),
- pytest.param({"page_size": -1}, 5, "", marks=pytest.mark.skip),
- pytest.param({"page_size": "a"}, 0, """ValueError("invalid literal for int() with base 10: \'a\'")""", marks=pytest.mark.skip),
+ (None, 4, None),
+ (0, 0, "page_size must be a positive integer"),
+ (1, 1, None),
+ (6, 4, None),
+ ("1", 1, None),
+ (-1, 0, "page_size must be a positive integer"),
+ ("a", 0, "page_size must be a positive integer"),
],
)
- def test_page_size(self, add_chunks, params, expected_page_size, expected_message):
+ def test_page_size(self, add_chunks, value, size, error):
_, document, _ = add_chunks
-
- if expected_message:
- with pytest.raises(Exception) as exception_info:
- document.list_chunks(**params)
- assert expected_message in str(exception_info.value), str(exception_info.value)
+ if error:
+ with pytest.raises(Exception, match=error):
+ document.list_chunks(page_size=value)
else:
- chunks = document.list_chunks(**params)
- assert len(chunks) == expected_page_size, str(chunks)
+ assert len(document.list_chunks(page_size=value)) == size
- @pytest.mark.p2
- @pytest.mark.parametrize(
- "params, expected_page_size",
- [
- ({"keywords": None}, 5),
- ({"keywords": ""}, 5),
- ({"keywords": "1"}, 1),
- ({"keywords": "chunk"}, 4),
- pytest.param({"keywords": "ragflow"}, 1, marks=pytest.mark.skipif(os.getenv("DOC_ENGINE") == "infinity", reason="issues/6509")),
- pytest.param({"keywords": "ragflow"}, 5, marks=pytest.mark.skipif(os.getenv("DOC_ENGINE") != "infinity", reason="issues/6509")),
- ({"keywords": "unknown"}, 0),
- ],
- )
- def test_keywords(self, add_chunks, params, expected_page_size):
+ @pytest.mark.parametrize("keywords, size", [(None, 4), ("", 4), ("coriander", 4), ("saffron", 1), ("1", 1), ("unfindablequasar", 0)])
+ def test_keywords(self, add_chunks, keywords, size):
_, document, _ = add_chunks
- if params.get("keywords") == "ragflow":
- doc_engine = get_doc_engine(document.rag)
- if doc_engine == "infinity" and expected_page_size == 1:
- pytest.skip("issues/6509")
- if doc_engine != "infinity" and expected_page_size == 5:
- pytest.skip("issues/6509")
- chunks = document.list_chunks(**params)
- assert len(chunks) == expected_page_size, str(chunks)
+ assert len(document.list_chunks(keywords=keywords)) == size
- @pytest.mark.p1
- @pytest.mark.parametrize(
- "chunk_id, expected_page_size, expected_message",
- [
- (None, 5, ""),
- ("", 5, ""),
- pytest.param(lambda r: r[0], 1, "", marks=pytest.mark.skipif(os.getenv("DOC_ENGINE") == "infinity", reason="issues/6499")),
- pytest.param("unknown", 0, """AttributeError("\'NoneType\' object has no attribute \'keys\'")""", marks=pytest.mark.skip),
- ],
- )
- def test_id(self, add_chunks, chunk_id, expected_page_size, expected_message):
- _, document, chunks = add_chunks
- if callable(chunk_id) and get_doc_engine(document.rag) == "infinity":
- pytest.skip("issues/6499")
- chunk_ids = [chunk.id for chunk in chunks]
- if callable(chunk_id):
- params = {"id": chunk_id(chunk_ids)}
- else:
- params = {"id": chunk_id}
+ @pytest.mark.parametrize("chunk_id", [None, ""])
+ def test_id_empty(self, add_chunks, chunk_id):
+ _, document, _ = add_chunks
+ assert len(document.list_chunks(id=chunk_id)) == 4
- if expected_message:
- with pytest.raises(Exception) as exception_info:
- document.list_chunks(**params)
- assert expected_message in str(exception_info.value), str(exception_info.value)
- else:
- chunks = document.list_chunks(**params)
- if params["id"] in [None, ""]:
- assert len(chunks) == expected_page_size, str(chunks)
- else:
- assert chunks[0].id == params["id"], str(chunks)
+ @pytest.mark.skip(reason="SDK sends id but Go ListChunks only reads chunk_ids; Go list also omits SDK id field")
+ @pytest.mark.parametrize("selection", ["existing", "unknown"])
+ def test_id(self, add_chunks, selection):
+ _, document, chunks = add_chunks
+ wanted = chunks[0].id if selection == "existing" else "unknown"
+ rows = document.list_chunks(id=wanted)
+ assert [chunk.id for chunk in rows] == ([wanted] if selection == "existing" else [])
+
+ @pytest.mark.skip(reason="Go List returns chunk_id/content_with_weight/important_kwd/question_kwd without SDK field mapping")
+ def test_list_chunk_schema(self, add_chunks):
+ dataset, document, chunks = add_chunks
+ rows = document.list_chunks(page_size=100)
+ assert {chunk.id for chunk in rows} == {chunk.id for chunk in chunks}
+ for chunk in rows:
+ expected = next(original for original in chunks if original.id == chunk.id)
+ assert chunk.content == expected.content
+ assert chunk.document_id == document.id
+ assert chunk.dataset_id == dataset.id
+ assert chunk.important_keywords == expected.important_keywords
+ assert chunk.questions == expected.questions
- @pytest.mark.p3
def test_concurrent_list(self, add_chunks):
_, document, _ = add_chunks
- count = 100
with ThreadPoolExecutor(max_workers=5) as executor:
- futures = [executor.submit(document.list_chunks) for _ in range(count)]
-
- responses = list(as_completed(futures))
- assert len(responses) == count, responses
- assert all(len(future.result()) == 5 for future in futures)
+ futures = [executor.submit(document.list_chunks) for _ in range(20)]
+ for future in as_completed(futures):
+ assert len(future.result()) == 4
- @pytest.mark.p3
def test_default(self, add_document):
_, document = add_document
- batch_add_chunks(document, 31)
-
- from time import sleep
+ chunks = [document.add_chunk(content=f"default pagination {i}") for i in range(31)]
+ wait_ids(document, [chunk.id for chunk in chunks])
+ assert len(document.list_chunks()) == 30
- sleep(3)
-
- chunks = document.list_chunks()
- assert len(chunks) == 30, str(chunks)
-
- @pytest.mark.p3
def test_list_chunks_invalid_document_id_raises(self, add_chunks):
- _, document, _ = add_chunks
- invalid_document = document.__class__(
- document.rag,
- {"id": "missing-document-id-for-chunks", "dataset_id": document.dataset_id},
- )
- with pytest.raises(Exception) as exception_info:
- invalid_document.list_chunks()
- assert str(exception_info.value), exception_info
+ dataset, _, _ = add_chunks
+ document = Document(dataset.rag, {"id": "missing-document", "dataset_id": dataset.id})
+ with pytest.raises(Exception, match="document not found"):
+ document.list_chunks()
diff --git a/test/testcases/test_sdk_api/test_chunk_management_within_dataset/test_retrieval_chunks.py b/test/testcases/test_sdk_api/test_chunk_management_within_dataset/test_retrieval_chunks.py
index 112765b730..9df55cb0f6 100644
--- a/test/testcases/test_sdk_api/test_chunk_management_within_dataset/test_retrieval_chunks.py
+++ b/test/testcases/test_sdk_api/test_chunk_management_within_dataset/test_retrieval_chunks.py
@@ -13,233 +13,194 @@
# See the License for the specific language governing permissions and
# limitations under the License.
#
+
+
+import math
import os
from concurrent.futures import ThreadPoolExecutor, as_completed
import pytest
+from conftest import raw_chunks, wait_until
+
+pytestmark = pytest.mark.integration
+
-DOC_ENGINE = (os.getenv("DOC_ENGINE") or "").lower()
+def retrieval(add_chunks, client):
+ dataset, document, chunks = add_chunks
+ payload = {"dataset_ids": [dataset.id], "question": "coriander botanical archive", "similarity_threshold": 0, "vector_similarity_weight": 0}
+ # Listing visibility does not prove that chunks are searchable.
+ wait_until(lambda: {chunk.id for chunk in client.retrieve(**payload)} == {chunk.id for chunk in chunks})
+ return payload, dataset, document, chunks
+
+
+def assert_results(rows, dataset, originals, limit=30):
+ expected = {chunk.id: chunk for chunk in originals}
+ assert len(rows) <= limit
+ assert len({row.id for row in rows}) == len(rows)
+ for row in rows:
+ assert row.id in expected
+ assert row.dataset_id == dataset.id
+ assert row.document_id == expected[row.id].document_id
+ assert row.content == expected[row.id].content
+ assert math.isfinite(row.similarity)
+ scores = [row.similarity for row in rows]
+ assert scores == sorted(scores, reverse=True)
class TestChunksRetrieval:
- @pytest.mark.p3
- @pytest.mark.parametrize(
- "payload, expected_page_size, expected_message",
- [
- ({"question": "chunk", "dataset_ids": None}, 4, ""),
- ({"question": "chunk", "document_ids": None}, 0, "missing 1 required positional argument"),
- ({"question": "chunk", "dataset_ids": None, "document_ids": None}, 4, ""),
- ({"question": "chunk"}, 0, "missing 1 required positional argument"),
- ],
- )
- def test_basic_scenarios(self, client, add_chunks, payload, expected_page_size, expected_message):
- dataset, document, _ = add_chunks
- if "dataset_ids" in payload:
- payload["dataset_ids"] = [dataset.id]
- if "document_ids" in payload:
+ @pytest.mark.parametrize("document_filter", [False, True])
+ def test_basic_scenarios(self, client, retrieval, document_filter):
+ payload, dataset, document, originals = retrieval
+ if document_filter:
payload["document_ids"] = [document.id]
+ rows = client.retrieve(**payload)
+ assert rows
+ assert_results(rows, dataset, originals)
+
+ @pytest.mark.parametrize("page", [None, 1, 2, 3])
+ def test_page(self, client, retrieval, page):
+ payload, dataset, _, originals = retrieval
+ baseline = client.retrieve(**payload)
+ rows = client.retrieve(**{**payload, "page": page, "page_size": 2})
+ effective_page = page or 1
+ assert len(rows) == len(baseline[(effective_page - 1) * 2 : effective_page * 2])
+ assert {row.id for row in rows} <= {row.id for row in baseline}
+ assert_results(rows, dataset, originals, 2)
+
+ @pytest.mark.parametrize("page_size", [None, 1, 5])
+ def test_page_size(self, client, retrieval, page_size):
+ payload, dataset, _, originals = retrieval
+ rows = client.retrieve(**{**payload, "page_size": page_size})
+ baseline = client.retrieve(**payload)
+ assert len(rows) == len(baseline[:page_size])
+ assert {row.id for row in rows} <= {row.id for row in baseline}
+ assert rows
+ assert_results(rows, dataset, originals, page_size or 30)
+
+ @pytest.mark.parametrize("weight", [0, 0.5, 1])
+ def test_vector_similarity_weight(self, client, retrieval, weight):
+ payload, dataset, _, originals = retrieval
+ rows = client.retrieve(**{**payload, "vector_similarity_weight": weight})
+ assert rows
+ assert_results(rows, dataset, originals)
+
+ @pytest.mark.parametrize("top_k", [1, 10, 2048])
+ def test_top_k(self, client, retrieval, top_k):
+ payload, dataset, _, originals = retrieval
+ # top_k limits vector candidates; engines can fuse additional text matches.
+ rows = client.retrieve(**{**payload, "top_k": top_k, "page_size": 2, "vector_similarity_weight": 0.5})
+ assert rows
+ assert_results(rows, dataset, originals, 2)
- if expected_message:
- with pytest.raises(Exception) as exception_info:
- client.retrieve(**payload)
- assert expected_message in str(exception_info.value), str(exception_info.value)
- else:
- chunks = client.retrieve(**payload)
- assert len(chunks) == expected_page_size, str(chunks)
-
- @pytest.mark.p3
- @pytest.mark.parametrize(
- "payload, expected_page_size, expected_message",
- [
- pytest.param(
- {"page": None, "page_size": 2},
- 2,
- """TypeError("int() argument must be a string, a bytes-like object or a real number, not \'NoneType\'")""",
- marks=pytest.mark.skip,
- ),
- pytest.param(
- {"page": 0, "page_size": 2},
- 0,
- "ValueError('Search does not support negative slicing.')",
- marks=pytest.mark.skip,
- ),
- ({"page": 2, "page_size": 2}, 2, ""),
- ({"page": 3, "page_size": 2}, 0, ""),
- ({"page": "3", "page_size": 2}, 0, ""),
- pytest.param(
- {"page": -1, "page_size": 2},
- 0,
- "ValueError('Search does not support negative slicing.')",
- marks=pytest.mark.skip,
- ),
- pytest.param(
- {"page": "a", "page_size": 2},
- 0,
- """ValueError("invalid literal for int() with base 10: \'a\'")""",
- marks=pytest.mark.skip,
- ),
- ],
- )
- def test_page(self, client, add_chunks, payload, expected_page_size, expected_message):
- dataset, _, _ = add_chunks
- payload.update({"question": "chunk", "dataset_ids": [dataset.id]})
-
- if expected_message:
- with pytest.raises(Exception) as exception_info:
- client.retrieve(**payload)
- assert expected_message in str(exception_info.value), str(exception_info.value)
- else:
- chunks = client.retrieve(**payload)
- assert len(chunks) == expected_page_size, str(chunks)
-
- @pytest.mark.p3
- @pytest.mark.parametrize(
- "payload, expected_page_size, expected_message",
- [
- pytest.param(
- {"page_size": None},
- 0,
- """TypeError("int() argument must be a string, a bytes-like object or a real number, not \'NoneType\'")""",
- marks=pytest.mark.skip,
- ),
- pytest.param({"page_size": 1}, 1, "", marks=pytest.mark.skip(reason="issues/10692")),
- ({"page_size": 5}, 4, ""),
- pytest.param({"page_size": "1"}, 1, "", marks=pytest.mark.skip(reason="issues/10692")),
- pytest.param(
- {"page_size": "a"},
- 0,
- """ValueError("invalid literal for int() with base 10: \'a\'")""",
- marks=pytest.mark.skip,
- ),
- ],
- )
- def test_page_size(self, client, add_chunks, payload, expected_page_size, expected_message):
- dataset, _, _ = add_chunks
- payload.update({"question": "chunk", "dataset_ids": [dataset.id]})
-
- if expected_message:
- with pytest.raises(Exception) as exception_info:
- client.retrieve(**payload)
- assert expected_message in str(exception_info.value), str(exception_info.value)
- else:
- chunks = client.retrieve(**payload)
- assert len(chunks) == expected_page_size, str(chunks)
-
- @pytest.mark.p3
- @pytest.mark.parametrize(
- "payload, expected_page_size, expected_message",
- [
- ({"vector_similarity_weight": 0}, 4, ""),
- ({"vector_similarity_weight": 0.5}, 4, ""),
- ({"vector_similarity_weight": 10}, 4, ""),
- pytest.param(
- {"vector_similarity_weight": "a"},
- 0,
- """ValueError("could not convert string to float: 'a'")""",
- marks=pytest.mark.skip,
- ),
- ],
- )
- def test_vector_similarity_weight(self, client, add_chunks, payload, expected_page_size, expected_message):
- dataset, _, _ = add_chunks
- payload.update({"question": "chunk", "dataset_ids": [dataset.id]})
-
- if expected_message:
- with pytest.raises(Exception) as exception_info:
- client.retrieve(**payload)
- assert expected_message in str(exception_info.value), str(exception_info.value)
- else:
- chunks = client.retrieve(**payload)
- assert len(chunks) == expected_page_size, str(chunks)
-
- @pytest.mark.p2
- @pytest.mark.parametrize(
- "payload, expected_page_size, expected_message",
- [
- ({"top_k": 10}, 4, ""),
- pytest.param(
- {"top_k": 1},
- 4,
- "",
- marks=pytest.mark.skipif(DOC_ENGINE in ["infinity", "opensearch"], reason="Infinity"),
- ),
- pytest.param(
- {"top_k": 1},
- 1,
- "",
- marks=pytest.mark.skipif(DOC_ENGINE in ["", "opensearch", "elasticsearch"], reason="elasticsearch"),
- ),
- ({"top_k": -1}, 4, "must be greater than 0"),
- pytest.param(
- {"top_k": "a"},
- 0,
- """ValueError("invalid literal for int() with base 10: \'a\'")""",
- marks=pytest.mark.skip,
- ),
- ],
- )
- def test_top_k(self, client, add_chunks, payload, expected_page_size, expected_message):
- dataset, _, _ = add_chunks
- payload.update({"question": "chunk", "dataset_ids": [dataset.id]})
-
- if expected_message:
- with pytest.raises(Exception) as exception_info:
- client.retrieve(**payload)
- assert expected_message in str(exception_info.value), str(exception_info.value)
- else:
- chunks = client.retrieve(**payload)
- assert len(chunks) == expected_page_size, str(chunks)
-
- @pytest.mark.skip
- @pytest.mark.parametrize(
- "payload, expected_message",
- [
- ({"rerank_id": "BAAI/bge-reranker-v2-m3"}, ""),
- pytest.param({"rerank_id": "unknown"}, "LookupError('Model(unknown) not authorized')", marks=pytest.mark.skip),
- ],
- )
- def test_rerank_id(self, client, add_chunks, payload, expected_message):
- dataset, _, _ = add_chunks
- payload.update({"question": "chunk", "dataset_ids": [dataset.id]})
-
- if expected_message:
- with pytest.raises(Exception) as exception_info:
- client.retrieve(**payload)
- assert expected_message in str(exception_info.value), str(exception_info.value)
- else:
- chunks = client.retrieve(**payload)
- assert len(chunks) > 0, str(chunks)
-
- @pytest.mark.skip
@pytest.mark.parametrize(
- "payload, expected_page_size, expected_message",
+ "field,value,message",
[
- ({"keyword": True}, 5, ""),
- ({"keyword": "True"}, 5, ""),
- ({"keyword": False}, 5, ""),
- ({"keyword": "False"}, 5, ""),
- ({"keyword": None}, 5, ""),
+ ("page", 0, "page must be greater than or equal to 1"),
+ ("page", -1, "page must be greater than or equal to 1"),
+ ("page", "3", "cannot unmarshal string"),
+ ("page", "a", "cannot unmarshal string"),
+ ("page_size", 0, "page_size must be greater than or equal to 1"),
+ ("page_size", -1, "page_size must be greater than or equal to 1"),
+ ("page_size", "1", "cannot unmarshal string"),
+ ("page_size", "a", "cannot unmarshal string"),
+ ("vector_similarity_weight", 10, "vector_similarity_weight must be between 0 and 1"),
+ ("vector_similarity_weight", -0.1, "vector_similarity_weight must be between 0 and 1"),
+ ("vector_similarity_weight", "a", "cannot unmarshal string"),
+ ("similarity_threshold", -0.1, "similarity_threshold must be between 0 and 1"),
+ ("similarity_threshold", 1.1, "similarity_threshold must be between 0 and 1"),
+ ("top_k", -1, "must be between 1 and 2048"),
+ ("top_k", 0, "must be between 1 and 2048"),
+ ("top_k", 2049, "must be between 1 and 2048"),
+ ("top_k", "a", "cannot unmarshal string"),
+ ("keyword", "True", "cannot unmarshal string"),
+ ("keyword", "False", "cannot unmarshal string"),
+ ("question", "", "question is required"),
+ ("question", " ", "question is required"),
+ ("dataset_ids", [], "dataset_ids array cannot be empty"),
+ ("dataset_ids", None, "dataset_ids is required"),
],
)
- def test_keyword(self, client, add_chunks, payload, expected_page_size, expected_message):
- dataset, _, _ = add_chunks
- payload.update({"question": "chunk test", "dataset_ids": [dataset.id]})
-
- if expected_message:
- with pytest.raises(Exception) as exception_info:
- client.retrieve(**payload)
- assert expected_message in str(exception_info.value), str(exception_info.value)
- else:
- chunks = client.retrieve(**payload)
- assert len(chunks) == expected_page_size, str(chunks)
-
- @pytest.mark.p3
- def test_concurrent_retrieval(self, client, add_chunks):
- dataset, _, _ = add_chunks
- count = 100
- payload = {"question": "chunk", "dataset_ids": [dataset.id]}
-
+ def test_invalid_parameters(self, client, add_document, field, value, message):
+ dataset, _ = add_document
+ payload = {"dataset_ids": [dataset.id], "question": "coriander", field: value}
+ with pytest.raises(Exception, match=message):
+ client.retrieve(**payload)
+
+ def test_unknown_rerank_id(self, client, retrieval):
+ payload, _, _, _ = retrieval
+ with pytest.raises(Exception, match="failed to get rerank model by rerank_id"):
+ client.retrieve(**{**payload, "rerank_id": "sdk-chunks-nonexistent-reranker"})
+
+ def test_rerank_id(self, client, retrieval):
+ model = os.getenv("RAGFLOW_SDK_RERANK_ID")
+ assert model, "Set RAGFLOW_SDK_RERANK_ID to an authorized reranker for this integration case"
+ payload, dataset, _, originals = retrieval
+ rows = client.retrieve(**{**payload, "rerank_id": model})
+ assert rows
+ assert_results(rows, dataset, originals)
+
+ @pytest.mark.parametrize("keyword", [False, None, True])
+ def test_keyword(self, client, retrieval, keyword):
+ if keyword:
+ assert os.getenv("RAGFLOW_SDK_CHAT_CONFIGURED") == "1", "Keyword expansion needs the isolated tenant's configured chat model"
+ payload, dataset, _, originals = retrieval
+ rows = client.retrieve(**{**payload, "keyword": keyword})
+ assert rows
+ assert_results(rows, dataset, originals)
+
+ def test_document_filter(self, client, retrieval):
+ payload, dataset, document, originals = retrieval
+ other = dataset.upload_documents([{"display_name": "filter.txt", "blob": b"Filter fixture"}])[0]
+ extra = other.add_chunk(content="Coriander botanical archive excluded document.")
+ wait_until(lambda: extra.id in {row.id for row in client.retrieve(**payload)})
+ rows = client.retrieve(**{**payload, "document_ids": [document.id]})
+ assert rows
+ assert_results(rows, dataset, originals)
+ assert client.retrieve(**{**payload, "document_ids": ["missing-document"]}) == []
+
+ def test_concurrent_retrieval(self, client, retrieval):
+ payload, dataset, _, originals = retrieval
with ThreadPoolExecutor(max_workers=5) as executor:
- futures = [executor.submit(client.retrieve, **payload) for _ in range(count)]
- responses = list(as_completed(futures))
- assert len(responses) == count, responses
+ futures = [executor.submit(client.retrieve, **payload) for _ in range(20)]
+ for future in as_completed(futures):
+ rows = future.result()
+ assert rows
+ assert_results(rows, dataset, originals)
+
+ def test_pagination_covers_results_once(self, client, retrieval):
+ payload, _, _, _ = retrieval
+ baseline = client.retrieve(**payload)
+ pages = [client.retrieve(**{**payload, "page": page, "page_size": 1}) for page in range(1, len(baseline) + 1)]
+ ids = [row.id for page in pages for row in page]
+ assert len(ids) == len(set(ids)) == len(baseline)
+ assert set(ids) == {row.id for row in baseline}
+
+ def test_metadata_filter(self, client, retrieval):
+ payload, dataset, document, originals = retrieval
+ result = dataset.patch(
+ f"/datasets/{dataset.id}/documents/metadatas", {"selector": {"document_ids": [document.id]}, "updates": [{"key": "author", "value": "chunk-audit"}], "deletes": []}
+ ).json()
+ assert result["code"] == 0, result
+ assert result["data"]["updated"] == result["data"]["matched_docs"] == 1
+ for author, match in [("chunk-audit", True), ("other-author", False)]:
+ condition = {"logic": "and", "conditions": [{"name": "author", "comparison_operator": "=", "value": author}]}
+ rows = client.retrieve(**{**payload, "metadata_condition": condition})
+ if match:
+ assert rows
+ assert_results(rows, dataset, originals)
+ else:
+ assert rows == []
+
+ def test_unavailable_chunks_are_excluded(self, client, retrieval):
+ payload, dataset, document, originals = retrieval
+ hidden = originals[0]
+ hidden.update({"available": False})
+ wait_until(lambda: next(row for row in raw_chunks(document) if row["chunk_id"] == hidden.id)["available"] is False)
+
+ def visible_results():
+ rows = client.retrieve(**payload)
+ return rows if hidden.id not in {row.id for row in rows} else None
+
+ rows = wait_until(visible_results)
+ assert_results(rows, dataset, originals[1:])
diff --git a/test/testcases/test_sdk_api/test_chunk_management_within_dataset/test_update_chunk.py b/test/testcases/test_sdk_api/test_chunk_management_within_dataset/test_update_chunk.py
index 460292da3f..053530fc12 100644
--- a/test/testcases/test_sdk_api/test_chunk_management_within_dataset/test_update_chunk.py
+++ b/test/testcases/test_sdk_api/test_chunk_management_within_dataset/test_update_chunk.py
@@ -13,159 +13,124 @@
# See the License for the specific language governing permissions and
# limitations under the License.
#
-import os
+
+
from concurrent.futures import ThreadPoolExecutor, as_completed
-from random import randint
import pytest
+from conftest import raw_chunks, wait_until
+from ragflow_sdk.modules.chunk import ChunkUpdateError
+pytestmark = pytest.mark.integration
-class TestUpdatedChunk:
- @pytest.mark.p3
- @pytest.mark.parametrize(
- "payload, expected_message",
- [
- ({"content": None}, ""),
- ({"content": ""}, "`content` is required"),
- pytest.param(
- {"content": 1},
- "TypeError('expected string or bytes-like object')",
- marks=pytest.mark.skip,
- ),
- ({"content": "update chunk"}, ""),
- ({"content": " "}, "`content` is required"),
- ({"content": "\n!?。;!?\"'"}, ""),
- ],
- )
- def test_content(self, add_chunks, payload, expected_message):
- _, _, chunks = add_chunks
- chunk = chunks[0]
- if expected_message:
- with pytest.raises(Exception) as exception_info:
- chunk.update(payload)
- assert expected_message in str(exception_info.value), str(exception_info.value)
- else:
- chunk.update(payload)
+def assert_updated(document, chunk, field, expected):
+ def persisted():
+ rows = raw_chunks(document)
+ row = next((row for row in rows if row["chunk_id"] == chunk.id), None)
+ return row is not None and row.get(field) == expected
- @pytest.mark.p3
- @pytest.mark.parametrize(
- "payload, expected_message",
- [
- ({"important_keywords": ["a", "b", "c"]}, ""),
- ({"important_keywords": [""]}, ""),
- ({"important_keywords": [1]}, "TypeError('sequence item 0: expected str instance, int found')"),
- ({"important_keywords": ["a", "a"]}, ""),
- ({"important_keywords": "abc"}, "`important_keywords` should be a list"),
- ({"important_keywords": 123}, "`important_keywords` should be a list"),
- ],
- )
- def test_important_keywords(self, add_chunks, payload, expected_message):
- _, _, chunks = add_chunks
- chunk = chunks[0]
+ wait_until(persisted)
- if expected_message:
- with pytest.raises(Exception) as exception_info:
- chunk.update(payload)
- assert expected_message in str(exception_info.value), str(exception_info.value)
- else:
- chunk.update(payload)
- @pytest.mark.p2
- @pytest.mark.parametrize(
- "payload, expected_message",
- [
- ({"questions": ["a", "b", "c"]}, ""),
- ({"questions": [""]}, ""),
- ({"questions": [1]}, "TypeError('sequence item 0: expected str instance, int found')"),
- ({"questions": ["a", "a"]}, ""),
- ({"questions": "abc"}, "`questions` should be a list"),
- ({"questions": 123}, "`questions` should be a list"),
- ],
- )
- def test_questions(self, add_chunks, payload, expected_message):
- _, _, chunks = add_chunks
+class TestUpdatedChunk:
+ @pytest.mark.parametrize("content", ["update chunk", "\n!?。;!?\"'", "", " "])
+ def test_content(self, add_chunks, content):
+ _, document, chunks = add_chunks
chunk = chunks[0]
+ assert chunk.update({"content": content}) is None
+ assert_updated(document, chunk, "content_with_weight", content)
- if expected_message:
- with pytest.raises(Exception) as exception_info:
- chunk.update(payload)
- assert expected_message in str(exception_info.value), str(exception_info.value)
- else:
- chunk.update(payload)
-
- @pytest.mark.p2
@pytest.mark.parametrize(
- "payload, expected_message",
+ "content",
[
- ({"tag_kwd": ["tag1", "tag2"]}, ""),
- ({"tag_kwd": [""]}, ""),
- ({"tag_kwd": [1]}, "`tag_kwd` must be a list of strings"),
- ({"tag_kwd": ["tag", "tag"]}, ""),
- ({"tag_kwd": "tag"}, "`tag_kwd` should be a list"),
- ({"tag_kwd": 123}, "`tag_kwd` should be a list"),
+ pytest.param(None, marks=pytest.mark.skip(reason="Go UpdateChunk silently ignores non-string content")),
+ pytest.param(1, marks=pytest.mark.skip(reason="Go UpdateChunk silently ignores non-string content")),
],
)
- def test_tag_kwd(self, add_chunks, payload, expected_message):
+ def test_content_invalid(self, add_chunks, content):
_, _, chunks = add_chunks
- chunk = chunks[0]
+ with pytest.raises(ChunkUpdateError) as error:
+ chunks[0].update({"content": content})
+ assert error.value.code in (101, 102, 400)
+ assert "content" in error.value.message
+
+ @pytest.mark.parametrize("field, stored", [("important_keywords", "important_kwd"), ("questions", "question_kwd"), ("tag_kwd", "tag_kwd")])
+ @pytest.mark.parametrize("value", [["a", "b", "c"], [""], ["a", "a"], []])
+ def test_list_fields(self, add_chunks, field, stored, value):
+ _, document, chunks = add_chunks
+ # Seed first so [] must clear existing values rather than preserve defaults.
+ chunks[0].update({field: ["seed"]})
+ assert_updated(document, chunks[0], stored, ["seed"])
+ chunks[0].update({field: value})
+ expected = [item.strip() for item in value if item.strip()] if field == "questions" else value
+ assert_updated(document, chunks[0], stored, expected)
- if expected_message:
- with pytest.raises(Exception) as exception_info:
- chunk.update(payload)
- assert expected_message in str(exception_info.value), str(exception_info.value)
- else:
- chunk.update(payload)
-
- @pytest.mark.p2
@pytest.mark.parametrize(
- "payload, expected_message",
+ "field",
[
- ({"available": True}, ""),
- pytest.param({"available": "True"}, """ValueError("invalid literal for int() with base 10: \'True\'")""", marks=pytest.mark.skip),
- ({"available": 1}, ""),
- ({"available": False}, ""),
- pytest.param({"available": "False"}, """ValueError("invalid literal for int() with base 10: \'False\'")""", marks=pytest.mark.skip),
- ({"available": 0}, ""),
+ pytest.param("important_keywords", marks=pytest.mark.skip(reason="Go UpdateChunk ignores invalid keyword types or replaces non-string elements with empty strings")),
+ pytest.param("questions", marks=pytest.mark.skip(reason="Go UpdateChunk ignores invalid question types or replaces non-string elements with empty strings")),
+ "tag_kwd",
],
)
- def test_available(self, add_chunks, payload, expected_message):
+ @pytest.mark.parametrize("value", [[1], "abc", 123, None])
+ def test_list_fields_invalid(self, add_chunks, field, value):
_, _, chunks = add_chunks
- chunk = chunks[0]
-
- if expected_message:
- with pytest.raises(Exception) as exception_info:
- chunk.update(payload)
- assert expected_message in str(exception_info.value), str(exception_info.value)
- else:
- chunk.update(payload)
-
- @pytest.mark.p3
- def test_repeated_update_chunk(self, add_chunks):
+ with pytest.raises(ChunkUpdateError) as error:
+ chunks[0].update({field: value})
+ assert error.value.code == 102
+ assert field in error.value.message
+ assert "list" in error.value.message
+
+ @pytest.mark.parametrize("available", [True, False])
+ def test_available(self, add_chunks, available):
+ _, document, chunks = add_chunks
+ chunks[0].update({"available": not available})
+ assert_updated(document, chunks[0], "available", not available)
+ chunks[0].update({"available": available})
+ assert_updated(document, chunks[0], "available", available)
+
+ @pytest.mark.skip(reason="Go UpdateChunk only reads bool availability and silently ignores other types")
+ @pytest.mark.parametrize("available", ["True", "False", 1, 0])
+ def test_available_invalid(self, add_chunks, available):
_, _, chunks = add_chunks
- chunk = chunks[0]
+ with pytest.raises(ChunkUpdateError) as error:
+ chunks[0].update({"available": available})
+ assert error.value.code in (101, 102, 400)
+ assert "available" in error.value.message
- chunk.update({"content": "chunk test 1"})
- chunk.update({"content": "chunk test 2"})
+ def test_repeated_update_chunk(self, add_chunks):
+ _, document, chunks = add_chunks
+ for content in ("chunk test 1", "chunk test 2", "chunk test 2"):
+ chunks[0].update({"content": content})
+ assert_updated(document, chunks[0], "content_with_weight", content)
- @pytest.mark.p3
- @pytest.mark.skipif(os.getenv("DOC_ENGINE") == "infinity", reason="issues/6554")
def test_concurrent_update_chunk(self, add_chunks):
- count = 50
- _, _, chunks = add_chunks
-
+ _, document, chunks = add_chunks
+ values = {chunk.id: set() for chunk in chunks}
with ThreadPoolExecutor(max_workers=5) as executor:
- futures = [executor.submit(chunks[randint(0, 3)].update, {"content": f"update chunk test {i}"}) for i in range(count)]
- responses = list(as_completed(futures))
- assert len(responses) == count, responses
+ futures = []
+ for i in range(50):
+ chunk = chunks[i % len(chunks)]
+ content = f"concurrent update {i}"
+ values[chunk.id].add(content)
+ futures.append(executor.submit(chunk.update, {"content": content}))
+ for future in as_completed(futures):
+ assert future.result() is None
+ # Completion order is unspecified; each final value must be a submitted write.
+ wait_until(lambda: all(row["content_with_weight"] in values[row["chunk_id"]] for row in raw_chunks(document)))
+ assert len(raw_chunks(document)) == len(chunks)
- @pytest.mark.p3
def test_update_chunk_to_deleted_document(self, add_chunks):
dataset, document, chunks = add_chunks
dataset.delete_documents(ids=[document.id])
-
- with pytest.raises(Exception) as exception_info:
+ with pytest.raises(ChunkUpdateError, match="document not found") as error:
chunks[0].update({})
- message = str(exception_info.value)
- ownership_message = f"you don't own the document {chunks[0].document_id}"
- assert message.rstrip(".") == ownership_message or message == f"Can't find this chunk {chunks[0].id}", message
+ assert error.value.code == 500
+
+ def test_unsupported_field(self, add_chunks):
+ _, _, chunks = add_chunks
+ with pytest.raises(ChunkUpdateError, match="not supported") as error:
+ chunks[0].update({"unknown": True})
+ assert error.value.code == 400