🪢 Open source agent evals & observability: Trace, evaluate, and improve LLM applications with one open platform.
TypeScript analyticsautogenevaluationlangchainlarge-language-modelsllama-indexllmllm-evaluationllm-observabilityllmopsmonitoringobservabilityopen-sourceopenaiplaygroundprompt-engineeringprompt-managementself-hostedycombinator
Dataset creation from observations
index.tsxowns the modal and its immutable opening source selection. It loads dataset metadata and the appropriate v3/v4 preview; table selection is cleared only after a batch is accepted, without changing the progress view's source.datasetMappingStore.tsowns the selected dataset ID, mapping drafts, create screen and submission state. Schema defaults are prepared when a dataset is selected or created, never when a field mounts or metadata refetches.DatasetMappingEditor.tsxsubscribes to mapping drafts and renders the mapping editor shell, keeping field edits out of the modal's query/lifecycle owner.DatasetCreateStep.tsxadapts the shared dataset form. The form owns its submission and footer; event callbacks guard modal dismissal and return the newly created dataset to the mapping store without a validation-state bridge.prepareDatasetMapping.tsproduces values, JSONPath warnings and validation for all fields. The editor, preview and submission action use this same result; there are no child-to-parent validation effects.submitDatasetBatch.tsguards duplicate or invalid submissions and calls the existing batch API. Failure retains the draft and selection; the tRPC mutation reports the error. The worker remains responsible for all selected items.DatasetItemEditorLayoutin the datasets feature contains the bulk selector, mapping editor and sample preview, followed by background progress after submission. Single-object creation usesNewDatasetItemFormwith one column of editable JSON, multiple datasets, schema checks and media uploads, then calls the existing immediate creation API. Both flows reuse dataset creation; their item editors and submission adapters stay separate.
The existing single-item submit analytics and evaluation analytics remain at their intent seams. Layout changes and cancellation do not add analytics events.