Skip to content

:hedgehog: PostHog is the leading platform for building self-driving products. Our developer tools – AI observability, analytics, session replay, flags, experiments, error tracking, logs, and more – capture all the context agents need to diagnose problems, uncover opportunities, and ship fixes. Steer it all from Slack, web, desktop, or the MCP.

feat(aio): add short scorer names and refine scorer management (#122304)

Radu Raicea8828abd73586
diff --git a/docs/internal/ai-offline-evaluation-reporting.md b/docs/internal/ai-offline-evaluation-reporting.md
index c6165771a7bb..8f38241c722d 100644
--- a/docs/internal/ai-offline-evaluation-reporting.md
+++ b/docs/internal/ai-offline-evaluation-reporting.md
@@ -77,6 +77,13 @@ Each scorer chart displays the loaded versions with results in the selected date
while the summary defaults to the latest matching version. The version arrows change only the summary.
Opening a scorer's history preserves the author and date filters and selects the version shown in the summary.
The scorer list shows each scorer's creator and last editor, and can be filtered by creator.
+By default, scorers are sorted by last updated, newest first.
+Creating and editing scorers stays under Evaluations → Scorers, with a detail heading and a back link to the list.
+Successful creation and edits return to the scorer list; failed saves keep the form open with your changes.
+
+Scorers have an optional **Short name** (up to 40 characters) for experiment table headers and other compact views.
+These views use the full name when the short name is blank. Editing or clearing a short name updates existing
+experiment displays without creating a scorer version or changing scoring rules. Versions appear in a separate badge.
The SQL suite uses these existing scorers:
diff --git a/frontend/snapshots.yml b/frontend/snapshots.yml
index 7f6f29455b1e..554044675cd4 100644
--- a/frontend/snapshots.yml
+++ b/frontend/snapshots.yml
@@ -125,73 +125,85 @@ snapshots:
ai-observability-offline-experiments-scorer-history--narrow--light:
hash: v1.k794b7964.4842515b727957419dffc97a061cd561a640c5b3d12357415b2c6b0c0a5b2729.7EOWporG0PIqDbpe8ArXsT-fl8w_hNdckYUgKv4uHDY
ai-observability-scorer-editor--boolean-detector--dark:
- hash: v1.k794b7964.841ab07e56ee68448ba1df160d2365beed4eec6db48856b984488836ad4ce85f.91iF4r5RJZ81eQDCcoFWZ5N1JTo-dznWBBKCzTSJ-mM
+ hash: v1.k794b7964.bf295f71ee0750c9703c615cbc53f39849426b32b354fbf8ef0c8915f7f42e57.BVPu0uJuGhNx1_q4FgZ7vDXRQ7IllIygH2Hi6qKoTZw
ai-observability-scorer-editor--boolean-detector--light:
- hash: v1.k794b7964.59661b0cff4ffa3256ba4ad9718fef1ee91a69020755c7f65d058d792456d6d4.euNebFHbsx9MWKKQVZb6FSuIkB2v_HMBCDeODOMVuSY
+ hash: v1.k794b7964.e7930dd72a2b07152f4993381b375bcc18645aecfedd5dff075d05ffea319dc2.GJhkHJH_e4oyO9SlOp_jhD-1vl9nYFONru5Y_X2Lo7Y
ai-observability-scorer-editor--legacy-boolean--dark:
- hash: v1.k794b7964.589593f3b43eaab4211e68f705c4f1e8e58402cf72c48660d3b6e85b12ca7f4e.4iTnDz5cGsuxeZ8ip-NPJFtSPynhPmhnn0d4Q1SQsQ8
+ hash: v1.k794b7964.62f407fd23bb7f3575b4fb365e8da51c24f5ab3b4cf28b2cbb6bbd31e26533c7.AumNGtWLAQmqJocDFm3csG7wKlCOQEgUpnG0SKksOLI
ai-observability-scorer-editor--legacy-boolean--light:
- hash: v1.k794b7964.a94659d37c973f53d579471300a61e0c09c50867c38865f1e3d74739f415ac03.BR5-bEASo9aFk-ArydSTfywdvww_4yO4qGOhPsQtRhU
+ hash: v1.k794b7964.ada53f4980031b13daa435b938ec27ae73a50946b1b307908425eb2e0a33215c.4YX2Jm8a-RueXU6paDiBjhKNPDsrQfttuL4LBiiAdCo
ai-observability-scorer-editor--load-error--dark:
- hash: v1.k794b7964.0f8cb6804577e15ea4422940b536fb357fbabb29eacf26afbd87468ec28c90f6.Ly9Rb5gH25bFKYuYUmcSrfPs0DA2Uv-qF6q3gs3n5Bw
+ hash: v1.k794b7964.a235a1f9b2b1d316060b19f4f180f688e7401f0130a6cafcadb2e0c84f62fab9.q6x9rh_Ev3zw6YnCj6RCmzTiEs4dr-MJuubfvSYEF78
ai-observability-scorer-editor--load-error--light:
- hash: v1.k794b7964.0d15a6abe82416de58c525f70d9bc94874d3039b026c30a14c8bb192b2d661cc.a0uei1Wg5xJeyTqmlHUUfxDgQp86PJs8UXAVYVsvvxo
+ hash: v1.k794b7964.76e5d05434b4aa8528472f6745dafa8a26d25f30c6f734d65d05951a4e66d34e.gFLmhVa9zGOKlit8t8cQD_84GoePNvqywiI-4nCHIrI
ai-observability-scorer-editor--narrow--dark:
- hash: v1.k794b7964.ebce9a1a02bd2d41471f7c199d3673d7bafca24abb717f64baf05548fa140fff.DUTL_KGeO-KPWyLyFnob-GLWt0z2Dn2x8KzRw7TbyBM
+ hash: v1.k794b7964.1ef1a481d6092a2559c8c250977321b66ef6af2ef4613ab01c0167e52267ed94.GVj6NNBv57lr_A-t7mR4BuhTwcD60GetvzzFGmB0xzY
ai-observability-scorer-editor--narrow--light:
- hash: v1.k794b7964.172e67c64e21a0c4104ff35eaaef9a440dbfefa859e53b27e7db09e60b9bc9f4.UiC0GZW1jjTKYBDeKg66KN1yfmpiycjhzw6eZywIKJY
+ hash: v1.k794b7964.b6e3dd0dac7783fe3364b3da5c803c30c9d5f824d9ac826b3605e04b9b1d1478.Dbm0eALmSQ_ldkd_VqKxYT3PGZ3fdOcxLtU-pvEPL00
ai-observability-scorer-editor--new-scorer--dark:
- hash: v1.k794b7964.0adbbb66244d361aad28b60dedc3d1664eeed713b385b0d179d016ec65e30f5f.9DiorxJuO21bW-diNrOFoyRjB-S81Tl9BPJi_1y8DGw
+ hash: v1.k794b7964.ffaa02732924d74b925c77f3f6f9f68bfad3f65a14db8579e46563a615593eb3.GDeb1g1JW7TDQACjvzzxMdfEW5iZfF0vPiuiCNr933E
ai-observability-scorer-editor--new-scorer--light:
- hash: v1.k794b7964.05e69e8cbf5a269adecfe7ae4b7bbc35f0529f20d4c3da7b1e7fe81df4992740.pybjXPj0fuu7rAV5lPqdRXFlz28ZiuYrSqvrQkhvcFc
+ hash: v1.k794b7964.d063f0f23200b8ee99a911eef22df427dbddc3bf29852179a705d30bb2b87abd.nT49E25GKviJe65vDCJLCflaS61cn2NKNDggvsexwco
ai-observability-scorer-editor--numeric--dark:
- hash: v1.k794b7964.90b0b440870f4a039c4ab8dc6e78f96b6ad72c30e831a0a3ceb4217509eae46c.t9VTtIqyYW9WaiYD1UfyuUqeOri63YNad8Nr3ELMk6k
+ hash: v1.k794b7964.6201221a2a50239c1c766d7c881a6464f307b41076636fd5f3d376998c642a63.IrTC8HC7JFOp5Dmd79QUKIAsrwl-tUTQHFWiry_OTwI
ai-observability-scorer-editor--numeric--light:
- hash: v1.k794b7964.1422448358f0f0e1c593d6321a90881d7a478e28a219ef782dbd4c3b83521d1d.cZhIHoCNgA6vM5Vn64UY0gmqwvXBNbzQqfAvaSw1r0o
+ hash: v1.k794b7964.bf3a7e6357ebb8ad298b42070564c05376a58704dff4adbc324eb0b8961261a6.XpywbYO1WY1Kg3uFv1E0Dtxf7a7ns25tYiQRIzYymGQ
ai-observability-scorer-form--boolean-detector--dark:
- hash: v1.k794b7964.6a1b98222f1fba6d4c3fdde6156df44c35ec487e83d0d76da7e712fbd56d3d58.pewGEfZSlIaH1aWMKghO0BqRdpfk2sBTz_kB9ZITvrQ
+ hash: v1.k794b7964.137a0396119f6cc65f4d3c3416803ff0a4612810b2ee7dc02e43d53bd59ed6c2.W5jELDiVqyrDWhjl21_PkP8kSGhNYxahW9F83FyH5cc
ai-observability-scorer-form--boolean-detector--light:
- hash: v1.k794b7964.fb38726711fb17b0fa6942f6a0fb059242655b3330a4533ac51dacf912b74432.8dfTh2m0gn9-XfXqxhOo8IeqogXWQGN5jiJaiKmjglk
+ hash: v1.k794b7964.66e481510938c77c155451fb55545d649eedc572a32c998c69543886dee71a39.5qp8fnwq5S7H6uiqXx8XCJp_JGGFmDw7oZiqGJbQy84
ai-observability-scorer-form--categorical--dark:
- hash: v1.k794b7964.04d3da604f06dd1ccb47841a3b47eae1f75306494679130c39507005568aba9f.x9dfhvETwi5MIoWT5ZYpevdY5xRb5qn3agTKLbeCTb4
+ hash: v1.k794b7964.aecae78e08e1a24b6d2c4aae0ff8c2417148c65b1d451bc48f566c4282ea4339.Cd5Pib9m0iUYf-6-okZSuMLSN-qHrHpdY4Pv0OB4xMk
ai-observability-scorer-form--categorical--light:
- hash: v1.k794b7964.cd03b79d9a3af57bfd028822546d869b973873e23090d8afa5b7f037d929ee44.RB5Hko-bMw0gv6oZOyIqBVj3wXhzTgxUoGFycz-gMa0
+ hash: v1.k794b7964.9416f2d2441206bd34c21af37eebaaab6667555b1bba837eb12c76e7f3b2d584.M_MinbQ4sa9SS_zQ8FJ5KeMTgAVfDCBRoldj-62ev6U
ai-observability-scorer-form--categorical-narrow--dark:
- hash: v1.k794b7964.2c87624a84ca9a3c5ca06a165e812fd6b8935e8e8a8558c5d9d397fcb3964947.9ypznBCGhMMzwXsCgC3BARS2wb71r0Pq5airJcY19Ec
+ hash: v1.k794b7964.8d1aa368859710f12058dad9ddb8d0c8db7f7779aea405a43b720395e92f5a58.1ZFR0YlfysW22jFZe409XHthTuuS3h9lv7JGCktu5QE
ai-observability-scorer-form--categorical-narrow--light:
- hash: v1.k794b7964.2268634695c9fc0feb52c4b39b35c47adbf9b9506791b7f5fd9f8a657bd72b1c.1RT_WHWdRV9h60t6Zk3EkZNfOG3v9Hht67ghARoQ9BE
+ hash: v1.k794b7964.70dc74db43bf086c19796bdcbe6cc4c98fc551bd61bff56d65bf228a57563246.MJwcvKnBaNE1dkc_zQQ8I4J-B0FrJu9CLaCsQkGoEMg
ai-observability-scorer-form--narrow--dark:
- hash: v1.k794b7964.b9ab06356a6dfe363ddb32dfc2593919ab8b92b0e85a9fb21a62f5e9ccc25c67.ZeH7cshPLmd_wTh-L6v_J5QqEpydtgd3Drm9igiGHWo
+ hash: v1.k794b7964.900df36db8a8708e8ab8760fbe7e7b77b0925b1511df0178a88d3ab646d91d2c.lh2ncl7BizJIVjnqotdCFej0_7k6FFZIrEd1yeuF4fQ
ai-observability-scorer-form--narrow--light:
- hash: v1.k794b7964.041db37e5229c59717c8ab44092cf686c1fbb7de4fcf1fcd56e66dbd359a3d93.KHYQLy9Od2uZZFPNvN42h9SNDfl2G7f9zxW8DYfkey4
+ hash: v1.k794b7964.c9a92b9fd8a7e9b0dd450975d783c72391aac4ffab7f713b8ce16970413a13da.gYPjzuIaVbaRHHDiTIgNWMvn6jU2AgVzvlkvl9uZVT4
ai-observability-scorer-form--new-scorer--dark:
- hash: v1.k794b7964.5b6ff3f06fa0ca76614e5c68ce0a428a56c97b47c24ea6ce234f7cede16d694d.cFt5UUeLhEXacFnJJHuVSZyzUY8xuyE32szF68aURvY
+ hash: v1.k794b7964.758f03442c0e12602793f45865f112eef2efff4de31558a2bcad4ab62134cb29.kcR5lDX8L79AvPRozacVMwn-4-sZZWd883DlBZ75UXo
ai-observability-scorer-form--new-scorer--light:
- hash: v1.k794b7964.59be9a6a45462b7250fa36e4453d605a6d3468103ebec066fcb9b72425be5651.AwuILo4ym2pWUSlyUSfWcsCWmVmvzxMv7D4HnmfYq48
+ hash: v1.k794b7964.a28f3a58cdc22aa68dfc14668ea7e8d59e62a5a9d9bd4890385259969b938aa1.GPDyUJryXhY62_JFaR0yZcLZxF3VGEO6IyIreTbLzPU
ai-observability-scorer-form--numeric--dark:
- hash: v1.k794b7964.52f6c3a3ad754e436b7c8df02397ff9c87f2fc4b54499ddff3d2f708e1b45219.cpMjcuayuECAfm8Yl7SGQlRn-Kmj7tqOpLeMTB9yX2c
+ hash: v1.k794b7964.e65f83b16cc891d5a32152bad369f05bc3ff345ecb42c5911f714fa9b7791a52.OCufjXhWPcyEHL0MsEdvfDfcPDQLiVIWw8L59oU8gGA
ai-observability-scorer-form--numeric--light:
- hash: v1.k794b7964.18a44cf74dae3bf5133e83407a6b07f379d485ff45096ff965babade020b625b.eEuDBN5MRJv8evVoqoiOpH-Ax35PBgPxD99F3_iGmVI
+ hash: v1.k794b7964.65b7edf34727e09d0d25339cfc9042bd638279d5b0c148507597d9abff25ae53.ZXHFZC17jCkLAt8jpyE74r13YON-iwzDGy6iVtr0rFQ
ai-observability-scorer-form--offline-disabled--dark:
- hash: v1.k794b7964.da97aa78c23020a365ab4eab3ed818e5f8993c46239c1e92977e8a8b28bfe362.M5pd-RjArDI2eiqfGBewTmK8Fbg01zE6uvPXSSJ0RR8
+ hash: v1.k794b7964.48d19bffb07476d2cffbfa36d78ee6b7ad42bf5d3d8ae393242dc6a0c69fdb2e.IVUELjOarFxPOQHCJmO0UzcZk32dfL-25SxUP95fAhQ
ai-observability-scorer-form--offline-disabled--light:
- hash: v1.k794b7964.2dc03d1322685e9016f63454a0a2966ae96614bf08802aa725ebc955af65feaf.J8cmghXsSZoVxurLs2npYwulIcvOVgODD2EKNauo1F4
+ hash: v1.k794b7964.96adcb897ac177576269dae3b6d06084145e2cb49dc9a78b6c180a57fd5981a8.fsK3rW2tDHKnVUOW5t56xwuFMmblsP_sO_2OPkkHsNg
+ ai-observability-scorer-label--full-name-fallback--dark:
+ hash: v1.k794b7964.9315086563b18122437914ae9b43bfe1650d410bdc7fabdf1f0afbbed8afc68f._uyAMXfeYQoJ85sPLvDgxMjorTWgZjAb4B3NNAhRtd0
+ ai-observability-scorer-label--full-name-fallback--light:
+ hash: v1.k794b7964.1055549e480cdedcb2e9455968b0ce91f43e414108f6477d535659395278013a.ILXD_OfNC6dsjgcI47CW30_-EG3IVfX_Og95vEsazA4
+ ai-observability-scorer-label--long-word--dark:
+ hash: v1.k794b7964.64ed9cc0ae80189f0b8f82ff343087510b79c021c6dec0e283c5da94f71e1f05.WFyW3kr3WsF2dkm7eorDqBFg9e1a3VZGL_sZVwc3qr0
+ ai-observability-scorer-label--long-word--light:
+ hash: v1.k794b7964.66cccd5a5a9696147707d0959ff340f75df2a0158881b1495fd820154cedad05.bvmxrb4mUZywPMicvsHtm1D0MlW0ZuRDLLBjoyj6DWE
+ ai-observability-scorer-label--short-name--dark:
+ hash: v1.k794b7964.e3539428ce1e244c0b21c5b06a5525068c04dd71e38ae94e4501df8236a1cafb.RNOWFTR2RXSBOXfxO0gn0dAEA2E8CbiY1Gd9uTxvFS8
+ ai-observability-scorer-label--short-name--light:
+ hash: v1.k794b7964.14a30b76a81a5f4c56dc83ad82034bc75376f086fa9169cec00cc794f17581f3.I3AWEwAP3UCPcYVt0zJUCAAN6QpdjYIat3dC2zcESYM
ai-observability-scorers--default--dark:
- hash: v1.k794b7964.105dd90df7f4dee687709720baa591c4922ca61230af6f69e986ed2639d968e6.TALgmDkTXEx3Dvid49CXQZeoJoqUH700GouwkLSLLZM
+ hash: v1.k794b7964.94a8d71fade50d092b70b7f2692b856d8eedac951c0abb669d0720d386b0ec30.H15J7C1GEqmPGLB9ALlapvTgz8CCe_1piXRkN3SRWYw
ai-observability-scorers--default--light:
- hash: v1.k794b7964.4d0810a94533130a29145429df6b048fbc7a7fa71938d183677fb8c7b74b4af3.zs4r15SouBP9zbQYX7H00T9URFOWoQZMgN3jp8XLSio
+ hash: v1.k794b7964.554e7455e11ba8d37a18ecf0793ad17a099d2eceaa6d976fdfb953ef84778a04.v4j_k-WT9CRnY0c49xgLNK-4_luIdj0kII_6dPjQmss
ai-observability-scorers--empty--dark:
- hash: v1.k794b7964.a6315f5111c58a9e719d05a7bd7388e4190771b6567760b10e2d403192fb7f47.PA7TFbjqzcvJC0mrhJAiTwXxvfaFMBijhkO8kW0xzBo
+ hash: v1.k794b7964.cb0d5980dff31f864bfbddd44069c39a1be34cdb7015a0c6f237f0876f0e0a1a.XZX3LogxPDcRLRUAF8KisxhdtNT4IGMg_-XX6jZihfo
ai-observability-scorers--empty--light:
- hash: v1.k794b7964.afc91ff60397c14759f014fa0504d21622a3ecc31b0fdb411cf2ae8b6cd5531f.Y6GnUWQ3zTGZreML0AlH_jpkG0Cv-QQ2wVCZLsALk0o
+ hash: v1.k794b7964.e6b2559c4832d448b1a8377380be1e3c60fdf0de608ebdcc2cf13dcfa2877d93.9TivokqXim0H2gUS8GrJy9ICMifDDau6vYnKMWl3P64
ai-observability-scorers--narrow--dark:
- hash: v1.k794b7964.072195b7ef69ebf55d6f111083b48eb5b74ae29b2c3664f9db1d79e27c741cd5.UJyx1IfMh79T08W9nsasxZMauUAxUEjatPDW0OFrCnk
+ hash: v1.k794b7964.973c30ded9dec0a11a8ebe20215df56fc0b28703e269e3970490789590b292ba.lob10PY0o6wFBBYN054WY129yJmHhMFWdQLTaEdJnko
ai-observability-scorers--narrow--light:
- hash: v1.k794b7964.0d1bb933147d7910f3f3f16d97980b581a99cd56a51fed4b81194fc547f539b8.KZyTW_wbEDbTR8CXU9fbqjqD9z4hDvUb99tjDqaQW_w
+ hash: v1.k794b7964.1fab459e49a0d19f53899bbcfba14a502df939bb31759df4967225e0b10fa27c.bfCFGkMDz6gY-cdYFAvFL8tCp-8oj2JkqI5Zj8K2oyU
ai-observability-scorers--offline-disabled--dark:
- hash: v1.k794b7964.5811769002dfe17d6dd93b1ebd92284be4b2dbf12c0635533f529c30780981d6.KQirndbX4Ao8xtlOrhBCmJdIAz4rizWfekajwn0jqnM
+ hash: v1.k794b7964.1f13da3f0cb15b238aeaf9d92099954eed6dcdaf5840e04b61be34003c397982.kkJJNPntWvYhJ2NP9FHUMDWNKx1TkMs9VKCq_ZrF7i4
ai-observability-scorers--offline-disabled--light:
- hash: v1.k794b7964.33c4b992e81d24167a2a93ff7fefd446163393d96fee65be672d7f2c37f66e92.-meL1Z1Bu2Q_uZirr9uMOFyFjzcLzQk9c2LohkP0tgs
+ hash: v1.k794b7964.5c51cd13f0ec0c202e5bbe717d6be94051dee11c777d05ce60f5df1018fb9064.wh5kOB_EqY14ioHFumViq5yE2om-fi7ez0hQxaBoNm0
charts-heatmap--activity-by-day-and-hour--dark:
hash: v1.k794b7964.cb3a745c072482c2b24a91879fea9a6f72d195ac683a38efc3f222d944828b9e.jlohG3R6gpgos058qmytizuCZtVRjXramep8NJDaIQE
charts-heatmap--activity-by-day-and-hour--light:
@@ -8887,9 +8899,9 @@ snapshots:
scenes-app-ai-observability-instrumentation-checklist-card--warnings-with-a-dismissal--light:
hash: v1.k794b7964.b0e06944ee7afe4af8cb39363280135111e71ba215aab56a3e410c79de62238b.WtW8suUOWanaTrvjxhV2Z_ANVDRmEW3bqHpTrTe5GoI
scenes-app-ai-observability-offline-experiments-experiment--all-scorers--dark:
- hash: v1.k794b7964.77b8eeb6a6bb9e85a4028d13148b4fe4bcfd18960aa499749bbade387d2dffc2.MTL9sKGqzrSi8kpPYs3haq3i1XgxRiC6yXqFUnKvlzM
+ hash: v1.k794b7964.5e999325599c0186b678a4aaba8423a09a578de4789220ddb18fbf7c64245764.9GJhQsDjcB9bKZSm5iX9O2RQApiIhl3KFVBhf1nv3_I
scenes-app-ai-observability-offline-experiments-experiment--all-scorers--light:
- hash: v1.k794b7964.012426a295152daa24de8516ce67a5ecade2be33a299dc738fc60fc15b6399df.pNBFQGCUrKAgPWtFABTOcNZp7osXurRr3vLfHePiHY0
+ hash: v1.k794b7964.ca7daaf6377489399dff0b6270f03d8f65da9a54deec7f5134c0bb4bff47c501.wfb2AAUXH9N-2yJ3Ts6m1r8SyNLoPbWeou7kGrHLSj0
scenes-app-ai-observability-offline-experiments-experiment--detailed-result--dark:
hash: v1.k794b7964.be36d278ad6c590ede1e5869eadba71cba2fbf4612a0b23a05aceff58c340136.TOcMfeHss0Chw1S7KTVU7eu2GLXiKIQU3euYoRAPUFI
scenes-app-ai-observability-offline-experiments-experiment--detailed-result--light:
@@ -8903,13 +8915,17 @@ snapshots:
scenes-app-ai-observability-offline-experiments-experiment--item-inspector--light:
hash: v1.k794b7964.d5b5031d997fe898f08b76cdbcf9e8cba8c64f95eed1c7cc6e9d07659e3f53d2.MThIVEaUuPRk8atrUxCM-MIndrdmV0NqfZzxQ5hx6LE
scenes-app-ai-observability-offline-experiments-experiment--narrow--dark:
- hash: v1.k794b7964.5673d38bbf3a6d707143df672800761a7a7a5451ddeea6e22cf1be23cbfcf4f3.PQJedGD6_fF2MJ688vh6jNYOLPeyEvNzXSRoEbvYQX8
+ hash: v1.k794b7964.5b7e9e2313647b0d7e5c5ac672ac72e3649fb4adb9dde5c5dc55c9fca5a3a761.hYGkT3JRlbyStq0sW3zufXHtNGuyX3mPTnNwRfPhmWI
scenes-app-ai-observability-offline-experiments-experiment--narrow--light:
- hash: v1.k794b7964.f8f31513b5dc7d6a8f39040052961a4bacaa6e906b9f3a3f601f7f55d601c731.gfmC4IiX-mi4Edik_p63tKftOIXkyB1fMfhDhLIrv1M
+ hash: v1.k794b7964.cc6f3c4a25e8dd22b3b984268a9da3d1981f811a3c79f1f04765f049ab0438b7.nGe67J2TsDbJ5r_CFPyPkSqK6jKK38NoaX8GYjtQQrM
scenes-app-ai-observability-offline-experiments-experiment--request-failure--dark:
hash: v1.k794b7964.501bc38da197ae45e045d236336316a18cb78d533ee3008c678c25afe6f3624a.It6rPCGENocYlEoHpN4OPbTNCP4EOKtWe3DmtpiP8CM
scenes-app-ai-observability-offline-experiments-experiment--request-failure--light:
hash: v1.k794b7964.a31d57c72b94117a24a7a8948a68c6959b4fd00cde6d24aa3cdd206e1ca12eb7.392D8AhKK3JzKggndN4ojaEVt_vsHWraj2vVAWxz5NI
+ scenes-app-ai-observability-offline-experiments-experiment--short-names--dark:
+ hash: v1.k794b7964.4b436d19420df374338c1f484cc2f7ac644177a51aeec84125a3ed8e3f583446.Px8ocO-D-muTEGstgIHFnNjELHqtqNm5idqNh2yc-ew
+ scenes-app-ai-observability-offline-experiments-experiment--short-names--light:
+ hash: v1.k794b7964.ed9eb6e6043d142832b3ffb804ccaa67d157197f0b1db705096c8f4f5f42bd43.aa_vf8j9vALW_L0KFnIa3aooK6hV7ZL8kzyvjMXJijY
scenes-app-ai-observability-offline-experiments-navigation--dataset-detail--dark:
hash: v1.k794b7964.b52ff6e7af233221985da0bd9f25d650d605a85f4689e9bdd254c2ddfb625a7b.ar4RTmAEuV5gpBCtbtPCBwaP_kF2-6X2TFbRzU05Vo0
scenes-app-ai-observability-offline-experiments-navigation--dataset-detail--light:
diff --git a/products/ai_observability/backend/api/offline_experiment_read_serializers.py b/products/ai_observability/backend/api/offline_experiment_read_serializers.py
index 618c28c5c724..e2c138a1f291 100644
--- a/products/ai_observability/backend/api/offline_experiment_read_serializers.py
+++ b/products/ai_observability/backend/api/offline_experiment_read_serializers.py
@@ -266,6 +266,7 @@ class OfflineScorerVersionReadSerializer(serializers.Serializer):
version = serializers.IntegerField(help_text="Version number within the definition.")
kind = serializers.ChoiceField(choices=ScoreDefinition.Kind.choices, help_text="Scorer value kind.")
name = serializers.CharField(help_text="Current scorer display name.")
+ short_name = serializers.CharField(allow_blank=True, help_text="Current compact label. Blank uses the full name.")
description = serializers.CharField(allow_blank=True, help_text="Current scorer description.")
archived = serializers.BooleanField(help_text="Whether the scorer is archived.")
config = ScoreDefinitionConfigField(help_text="Pinned immutable configuration used to interpret these results.")
diff --git a/products/ai_observability/backend/api/score_definitions.py b/products/ai_observability/backend/api/score_definitions.py
index a1df95573d3b..4fbcd98b72fa 100644
--- a/products/ai_observability/backend/api/score_definitions.py
+++ b/products/ai_observability/backend/api/score_definitions.py
@@ -42,6 +42,9 @@ from products.ai_observability.backend.score_definition_configs import ScoreDefi
class ScoreDefinitionSerializer(ResourceTagsReadSerializer, serializers.ModelSerializer):
+ short_name = serializers.CharField(
+ read_only=True, allow_blank=True, help_text="Optional compact label for table headers and other compact views."
+ )
updated_by = UserBasicSerializer(
read_only=True, allow_null=True, help_text="User who last updated the scorer or created a version."
)
@@ -71,6 +74,7 @@ class ScoreDefinitionSerializer(ResourceTagsReadSerializer, serializers.ModelSer
fields = [
"id",
"name",
+ "short_name",
"description",
"tags",
"kind",
@@ -147,6 +151,9 @@ def _score_definition_version_page(
class ScoreDefinitionCreateSerializer(ResourceTagsWriteSerializer):
name = serializers.CharField(max_length=255, help_text="Human-readable scorer name.")
+ short_name = serializers.CharField(
+ max_length=40, required=False, allow_blank=True, help_text="Optional compact label. Blank uses the full name."
+ )
description = serializers.CharField(
required=False,
allow_blank=True,
@@ -178,6 +185,9 @@ class ScoreDefinitionCreateSerializer(ResourceTagsWriteSerializer):
class ScoreDefinitionMetadataSerializer(ResourceTagsWriteSerializer):
name = serializers.CharField(max_length=255, required=False, help_text="Updated scorer name.")
+ short_name = serializers.CharField(
+ max_length=40, required=False, allow_blank=True, help_text="Updated compact label. Blank uses the full name."
+ )
description = serializers.CharField(
required=False,
allow_blank=True,
@@ -197,6 +207,9 @@ class ScoreDefinitionMetadataSerializer(ResourceTagsWriteSerializer):
class ScoreDefinitionNewVersionSerializer(serializers.Serializer):
+ short_name = serializers.CharField(
+ max_length=40, required=False, allow_blank=True, help_text="Updated compact label, saved with this version."
+ )
name = serializers.CharField(
max_length=255, required=False, help_text="Updated scorer name, saved with this version."
)
@@ -228,7 +241,7 @@ class ScoreDefinitionFilter(django_filters.FilterSet):
created_by = django_filters.NumberFilter(field_name="created_by_id", help_text="Filter by the creator's user ID.")
search = django_filters.CharFilter(
method="filter_search",
- help_text="Search scorers by name or description.",
+ help_text="Search scorers by name, short name, or description.",
)
kind = django_filters.ChoiceFilter(
field_name="kind",
@@ -256,7 +269,9 @@ class ScoreDefinitionFilter(django_filters.FilterSet):
def filter_search(self, queryset: QuerySet, _name: str, value: str) -> QuerySet:
if value:
- return queryset.filter(Q(name__icontains=value) | Q(description__icontains=value))
+ return queryset.filter(
+ Q(name__icontains=value) | Q(short_name__icontains=value) | Q(description__icontains=value)
+ )
return queryset
@@ -387,7 +402,12 @@ class ScoreDefinitionViewSet(
# create_new_version holds the row lock until commit. Reload so the metadata comparison sees writes made after get_object().
definition.refresh_from_db()
self._update_definition_metadata(
- definition, {field: validated_data[field] for field in ("name", "description") if field in validated_data}
+ definition,
+ {
+ field: validated_data[field]
+ for field in ("name", "short_name", "description")
+ if field in validated_data
+ },
)
return definition
@@ -397,7 +417,7 @@ class ScoreDefinitionViewSet(
OpenApiParameter(
"search",
OpenApiTypes.STR,
- description="Search scorers by name or description.",
+ description="Search scorers by name, short name, or description.",
examples=[OpenApiExample("Search scorer", value="quality")],
),
OpenApiParameter(
diff --git a/products/ai_observability/backend/api/test/test_offline_experiment_reads.py b/products/ai_observability/backend/api/test/test_offline_experiment_reads.py
index f8385384a3e8..5a85684b44da 100644
--- a/products/ai_observability/backend/api/test/test_offline_experiment_reads.py
+++ b/products/ai_observability/backend/api/test/test_offline_experiment_reads.py
@@ -345,6 +345,8 @@ class TestOfflineExperimentReads(APIBaseTest):
definition = ScoreDefinition.objects.create(team=self.team, name="Category", kind="categorical")
version = definition.create_new_version(config=config, created_by=self.user)
unused_version = definition.create_new_version(config=config, created_by=self.user)
+ definition.short_name = "Category label"
+ definition.save(update_fields=["short_name"])
items = [
self.item,
*OfflineExperimentItem.objects.for_team(self.team.id).bulk_create(
@@ -382,6 +384,9 @@ class TestOfflineExperimentReads(APIBaseTest):
sorted([str(version.id), str(unused_version.id)]),
)
self.assertEqual([scorer["config"] for scorer in response.data["scorer_versions"]], [config, config])
+ self.assertEqual(
+ [scorer["short_name"] for scorer in response.data["scorer_versions"]], ["Category label", "Category label"]
+ )
for item in response.data["results"]:
self.assertEqual(len(item["results"]), 1)
result = item["results"][0]
diff --git a/products/ai_observability/backend/api/test/test_score_definitions.py b/products/ai_observability/backend/api/test/test_score_definitions.py
index 8c2998e46555..d42a9cee234b 100644
--- a/products/ai_observability/backend/api/test/test_score_definitions.py
+++ b/products/ai_observability/backend/api/test/test_score_definitions.py
@@ -104,7 +104,12 @@ class TestScoreDefinitionsApi(APIBaseTest):
response = self.client.patch(
f"{self._endpoint()}{definition.id}/",
- {"name": "Updated quality", "description": "Updated description", "archived": True},
+ {
+ "name": "Updated quality",
+ "short_name": " Quality ",
+ "description": "Updated description",
+ "archived": True,
+ },
format="json",
)
@@ -112,6 +117,7 @@ class TestScoreDefinitionsApi(APIBaseTest):
definition.refresh_from_db()
current_version = self._current_version(definition)
self.assertEqual(definition.name, "Updated quality")
+ self.assertEqual(definition.short_name, "Quality")
self.assertEqual(definition.created_by, original_author)
self.assertEqual(response.data["updated_by"]["id"], self.user.id)
self.assertEqual(definition.description, "Updated description")
@@ -244,6 +250,7 @@ class TestScoreDefinitionsApi(APIBaseTest):
f"{self._endpoint()}{definition.id}/new_version/",
{
"name": "Updated scorer",
+ "short_name": "Quality",
"description": "Updated description",
"base_version": 1,
"config": {
@@ -266,6 +273,7 @@ class TestScoreDefinitionsApi(APIBaseTest):
self.assertEqual(current_version.version, 2)
self.assertEqual(response.data["updated_by"]["id"], self.user.id)
self.assertEqual((definition.name, definition.description), ("Updated scorer", "Updated description"))
+ self.assertEqual(definition.short_name, "Quality")
self.assertEqual(definition.versions.count(), 2)
self.assertEqual(definition.versions.get(version=1).config, original_config)
self.assertEqual(
@@ -588,7 +596,8 @@ class TestScoreDefinitionsApi(APIBaseTest):
self.assertIn(str(active.id), ids)
self.assertNotIn(str(archived.id), ids)
- def test_list_supports_search_kind_archived_and_ordering(self):
+ @parameterized.expand(["resolved", "resolution"])
+ def test_list_supports_search_kind_archived_and_ordering(self, search: str) -> None:
active = self._create_definition(name="Quality", kind="categorical")
archived = self._create_definition(
name="Resolved",
@@ -596,10 +605,12 @@ class TestScoreDefinitionsApi(APIBaseTest):
archived=True,
)
+ archived.short_name = "Resolution"
+ archived.save(update_fields=["short_name"])
response = self.client.get(
self._endpoint(),
{
- "search": "resolved",
+ "search": search,
"kind": "boolean",
"archived": "true",
"order_by": "name",
diff --git a/products/ai_observability/backend/migrations/0067_scorer_short_name.py b/products/ai_observability/backend/migrations/0067_scorer_short_name.py
new file mode 100644
index 000000000000..296e2ee4e5b9
--- /dev/null
+++ b/products/ai_observability/backend/migrations/0067_scorer_short_name.py
@@ -0,0 +1,17 @@
+# Generated by Django 5.2.18 on 2026-10-22 17:40
+
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+ dependencies = [
+ ("ai_observability", "0066_scorer_and_experiment_authors"),
+ ]
+
+ operations = [
+ migrations.AddField(
+ model_name="scoredefinition",
+ name="short_name",
+ field=models.CharField(blank=True, db_default="", default="", max_length=40),
+ ),
+ ]
diff --git a/products/ai_observability/backend/migrations/max_migration.txt b/products/ai_observability/backend/migrations/max_migration.txt
index d62786b97f43..eb7e75cc5f34 100644
--- a/products/ai_observability/backend/migrations/max_migration.txt
+++ b/products/ai_observability/backend/migrations/max_migration.txt
@@ -1 +1 @@
-0066_scorer_and_experiment_authors
+0067_scorer_short_name
diff --git a/products/ai_observability/backend/models/score_definitions.py b/products/ai_observability/backend/models/score_definitions.py
index 1aaab15272c3..870ac552341c 100644
--- a/products/ai_observability/backend/models/score_definitions.py
+++ b/products/ai_observability/backend/models/score_definitions.py
@@ -26,6 +26,7 @@ class ScoreDefinition(Taggable, UUIDModel, CreatedMetaFields, UpdatedMetaFields)
team = models.ForeignKey(Team, on_delete=models.CASCADE, related_name="+")
name = models.CharField(max_length=255)
+ short_name = models.CharField(max_length=40, blank=True, default="", db_default="")
description = models.TextField(blank=True, default="")
kind = models.CharField(max_length=32, choices=Kind)
archived = models.BooleanField(default=False)
diff --git a/products/ai_observability/backend/offline_evaluation_read_service.py b/products/ai_observability/backend/offline_evaluation_read_service.py
index bfb139840fdc..8c54fd8bc1fb 100644
--- a/products/ai_observability/backend/offline_evaluation_read_service.py
+++ b/products/ai_observability/backend/offline_evaluation_read_service.py
@@ -366,6 +366,7 @@ class OfflineEvaluationReadService:
version=version.version,
kind=definition.kind,
name=definition.name,
+ short_name=definition.short_name,
description=definition.description,
archived=definition.archived,
config=cast(dict[str, JSONValue], version.config),
diff --git a/products/ai_observability/backend/offline_evaluation_read_types.py b/products/ai_observability/backend/offline_evaluation_read_types.py
index 4eed70ad5ecb..52b483c32482 100644
--- a/products/ai_observability/backend/offline_evaluation_read_types.py
+++ b/products/ai_observability/backend/offline_evaluation_read_types.py
@@ -114,6 +114,7 @@ class OfflineScorerVersionRead:
version: int
kind: str
name: str
+ short_name: str
description: str
archived: bool
config: dict[str, JSONValue]
diff --git a/products/ai_observability/frontend/generated/api.schemas.ts b/products/ai_observability/frontend/generated/api.schemas.ts
index 353e29ca7293..bd2e7ca5d821 100644
--- a/products/ai_observability/frontend/generated/api.schemas.ts
+++ b/products/ai_observability/frontend/generated/api.schemas.ts
@@ -928,6 +928,8 @@ export interface OfflineScorerVersionReadApi {
kind: ScoreDefinitionKindEnumApi
/** Current scorer display name. */
name: string
+ /** Current compact label. Blank uses the full name. */
+ short_name: string
/** Current scorer description. */
description: string
/** Whether the scorer is archived. */
@@ -3794,6 +3796,8 @@ export interface PatchedReviewQueueUpdateApi {
export interface ScoreDefinitionApi {
readonly id: string
readonly name: string
+ /** Optional compact label for table headers and other compact views. */
+ readonly short_name: string
readonly description: string
/** Project tag names assigned to this resource. */
readonly tags: readonly string[]
@@ -3839,6 +3843,11 @@ export interface ScoreDefinitionCreateApi {
* @maxLength 255
*/
name: string
+ /**
+ * Optional compact label. Blank uses the full name.
+ * @maxLength 40
+ */
+ short_name?: string
/**
* Optional human-readable description.
* @nullable
@@ -3868,6 +3877,11 @@ export interface PatchedScoreDefinitionMetadataApi {
* @maxLength 255
*/
name?: string
+ /**
+ * Updated compact label. Blank uses the full name.
+ * @maxLength 40
+ */
+ short_name?: string
/**
* Updated scorer description.
* @nullable
@@ -3878,6 +3892,11 @@ export interface PatchedScoreDefinitionMetadataApi {
}
export interface ScoreDefinitionNewVersionApi {
+ /**
+ * Updated compact label, saved with this version.
+ * @maxLength 40
+ */
+ short_name?: string
/**
* Updated scorer name, saved with this version.
* @maxLength 255
@@ -5573,7 +5592,7 @@ export type LlmAnalyticsScoreDefinitionsListParams = {
*/
order_by?: string
/**
- * Search scorers by name or description.
+ * Search scorers by name, short name, or description.
*/
search?: string
/**
diff --git a/products/ai_observability/frontend/generated/api.zod.ts b/products/ai_observability/frontend/generated/api.zod.ts
index 8c722c0ee8dc..0e3a707fcd5c 100644
--- a/products/ai_observability/frontend/generated/api.zod.ts
+++ b/products/ai_observability/frontend/generated/api.zod.ts
@@ -2410,6 +2410,8 @@ export const llmAnalyticsScoreDefinitionsCreateBodyTagsMax = 100
export const llmAnalyticsScoreDefinitionsCreateBodyNameMax = 255
+export const llmAnalyticsScoreDefinitionsCreateBodyShortNameMax = 40
+
export const llmAnalyticsScoreDefinitionsCreateBodyArchivedDefault = false
export const llmAnalyticsScoreDefinitionsCreateBodyConfigOneOneOptionsItemKeyMax = 128
@@ -2424,6 +2426,11 @@ export const LlmAnalyticsScoreDefinitionsCreateBody = /* @__PURE__ */ zod.object
.optional()
.describe('Tag names. Replaces existing tags; send [] to remove all tags. Names are trimmed and lowercased.'),
name: zod.string().max(llmAnalyticsScoreDefinitionsCreateBodyNameMax).describe('Human-readable scorer name.'),
+ short_name: zod
+ .string()
+ .max(llmAnalyticsScoreDefinitionsCreateBodyShortNameMax)
+ .optional()
+ .describe('Optional compact label. Blank uses the full name.'),
description: zod.string().nullish().describe('Optional human-readable description.'),
kind: zod
.enum(['categorical', 'numeric', 'boolean'])
@@ -2544,6 +2551,8 @@ export const llmAnalyticsScoreDefinitionsPartialUpdateBodyTagsMax = 100
export const llmAnalyticsScoreDefinitionsPartialUpdateBodyNameMax = 255
+export const llmAnalyticsScoreDefinitionsPartialUpdateBodyShortNameMax = 40
+
export const LlmAnalyticsScoreDefinitionsPartialUpdateBody = /* @__PURE__ */ zod.object({
tags: zod
.array(zod.string().max(llmAnalyticsScoreDefinitionsPartialUpdateBodyTagsItemMax))
@@ -2555,10 +2564,17 @@ export const LlmAnalyticsScoreDefinitionsPartialUpdateBody = /* @__PURE__ */ zod
.max(llmAnalyticsScoreDefinitionsPartialUpdateBodyNameMax)
.optional()
.describe('Updated scorer name.'),
+ short_name: zod
+ .string()
+ .max(llmAnalyticsScoreDefinitionsPartialUpdateBodyShortNameMax)
+ .optional()
+ .describe('Updated compact label. Blank uses the full name.'),
description: zod.string().nullish().describe('Updated scorer description.'),
archived: zod.boolean().optional().describe('Whether the scorer is archived.'),
})
+export const llmAnalyticsScoreDefinitionsNewVersionCreateBodyShortNameMax = 40
+
export const llmAnalyticsScoreDefinitionsNewVersionCreateBodyNameMax = 255
export const llmAnalyticsScoreDefinitionsNewVersionCreateBodyConfigOneOneOptionsItemKeyMax = 128
@@ -2568,6 +2584,11 @@ export const llmAnalyticsScoreDefinitionsNewVersionCreateBodyConfigOneOneOptions
export const llmAnalyticsScoreDefinitionsNewVersionCreateBodyConfigOneOnePassingRuleOneCategoriesItemMax = 128
export const LlmAnalyticsScoreDefinitionsNewVersionCreateBody = /* @__PURE__ */ zod.object({
+ short_name: zod
+ .string()
+ .max(llmAnalyticsScoreDefinitionsNewVersionCreateBodyShortNameMax)
+ .optional()
+ .describe('Updated compact label, saved with this version.'),
name: zod
.string()
.max(llmAnalyticsScoreDefinitionsNewVersionCreateBodyNameMax)
diff --git a/products/ai_observability/frontend/offline-evaluations/OfflineExperimentContent.stories.tsx b/products/ai_observability/frontend/offline-evaluations/OfflineExperimentContent.stories.tsx
index 6e07ee5f2adc..daa702a701a7 100644
--- a/products/ai_observability/frontend/offline-evaluations/OfflineExperimentContent.stories.tsx
+++ b/products/ai_observability/frontend/offline-evaluations/OfflineExperimentContent.stories.tsx
@@ -90,6 +90,31 @@ export default meta
type Story = StoryObj<typeof OfflineExperimentContent>
export const AllScorers: Story = {}
+export const ShortNames: Story = {
+ decorators: [
+ mswDecorator({
+ get: {
+ '/api/projects/:team/ai_observability/offline_experiments/:id/scorer_summaries/': {
+ count: detailSummaries.length,
+ next_cursor: null,
+ results: detailSummaries.map((summary, index) => ({
+ ...summary,
+ scorer: {
+ ...summary.scorer,
+ name:
+ [
+ 'Checks whether the answer addresses every part of the question',
+ 'Checks whether claims are supported by the reference material',
+ 'Response time including tool execution',
+ ][index] || summary.scorer.name,
+ short_name: ['Answer quality', 'Groundedness'][index] || '',
+ },
+ })),
+ },
+ },
+ }),
+ ],
+}
export const Narrow: Story = {
decorators: [
(Story) => (
diff --git a/products/ai_observability/frontend/offline-evaluations/OfflineExperimentContent.test.tsx b/products/ai_observability/frontend/offline-evaluations/OfflineExperimentContent.test.tsx
index 30c314c55aa6..34b7b5b57dee 100644
--- a/products/ai_observability/frontend/offline-evaluations/OfflineExperimentContent.test.tsx
+++ b/products/ai_observability/frontend/offline-evaluations/OfflineExperimentContent.test.tsx
@@ -1,6 +1,6 @@
import '@testing-library/jest-dom'
-import { cleanup, fireEvent, render, screen, waitFor } from '@testing-library/react'
+import { cleanup, fireEvent, render, screen, waitFor, within } from '@testing-library/react'
import { initKeaTests } from '~/test/init'
@@ -37,6 +37,26 @@ describe('OfflineExperimentContent', () => {
)
})
+ it.each(['Groundedness', ''])('uses the compact scorer name or its fallback in headers: %j', async (shortName) => {
+ const fullName = 'Checks whether the answer is supported by the supplied reference material'
+ readSummaries.mockResolvedValue({
+ count: 1,
+ next_cursor: null,
+ results: [
+ {
+ ...detailSummaries[0],
+ scorer: { ...detailSummaries[0].scorer, name: fullName, short_name: shortName },
+ },
+ ],
+ })
+ render(<OfflineExperimentContent teamId={997} experimentId={detailExperiment.id} />)
+ const header = (await screen.findByText(shortName || fullName, { selector: 'th span' })).closest(
+ 'th'
+ ) as HTMLTableCellElement
+ expect(within(header).getByText(shortName || fullName)).toBeInTheDocument()
+ expect(within(header).getByText('v2')).toHaveClass('LemonTag')
+ })
+
it('keeps a scorer summary panel the user opened open after a refresh', async () => {
render(<OfflineExperimentContent teamId={997} experimentId={detailExperiment.id} />)
const title = await screen.findByText('Scorer summaries (25)')
diff --git a/products/ai_observability/frontend/offline-evaluations/OfflineExperimentContent.tsx b/products/ai_observability/frontend/offline-evaluations/OfflineExperimentContent.tsx
index 724297449d9f..155fbd814181 100644
--- a/products/ai_observability/frontend/offline-evaluations/OfflineExperimentContent.tsx
+++ b/products/ai_observability/frontend/offline-evaluations/OfflineExperimentContent.tsx
@@ -11,6 +11,7 @@ import { urls } from 'scenes/urls'
import { AccessControlLevel, AccessControlResourceType } from '~/types'
import type { OfflineScorerSummaryApi } from '../generated/api.schemas'
+import { ScorerLabel } from '../scoreDefinitions/ScorerLabel'
import { OfflineExperimentLogicProps, offlineExperimentLogic } from './offlineExperimentLogic'
import { OfflineExperimentStatus } from './OfflineExperimentStatus'
import { OfflineItemInspector } from './OfflineItemInspector'
@@ -181,7 +182,9 @@ export function OfflineExperimentContent(props: OfflineExperimentLogicProps): JS
<Link
className="break-words"
to={offlineScorerHistoryUrl(summary.scorer, experiment, timezone)}
- >{`${summary.scorer.name} v${summary.scorer.version}`}</Link>
+ >
+ <ScorerLabel scorer={summary.scorer} />
+ </Link>
),
},
{
diff --git a/products/ai_observability/frontend/offline-evaluations/OfflineItemInspector.test.tsx b/products/ai_observability/frontend/offline-evaluations/OfflineItemInspector.test.tsx
index c9979701f331..de3231d3c4c8 100644
--- a/products/ai_observability/frontend/offline-evaluations/OfflineItemInspector.test.tsx
+++ b/products/ai_observability/frontend/offline-evaluations/OfflineItemInspector.test.tsx
@@ -97,11 +97,10 @@ describe('OfflineItemInspector', () => {
onSelectResult={jest.fn()}
/>
)
- const label = `${result.scorer.name} v${result.scorer.version}`
- expect(await screen.findByText(label)).toBeInTheDocument()
+ expect(await screen.findByText(result.scorer.name, { selector: 'button span' })).toBeInTheDocument()
fireEvent.click(screen.getByText('Next'))
expect(await screen.findByText('Results unavailable')).toBeInTheDocument()
- expect(screen.queryByText(label)).not.toBeInTheDocument()
+ expect(screen.queryByText(result.scorer.name, { selector: 'button span' })).not.toBeInTheDocument()
expect(screen.getByText('Next').closest('button')).toHaveAttribute('aria-disabled', 'true')
fireEvent.click(screen.getByText('Next'))
expect(api.aiObservabilityOfflineExperimentsItemsResultsList).toHaveBeenCalledTimes(2)
diff --git a/products/ai_observability/frontend/offline-evaluations/OfflineItemInspector.tsx b/products/ai_observability/frontend/offline-evaluations/OfflineItemInspector.tsx
index 23216a3675f4..daa307148145 100644
--- a/products/ai_observability/frontend/offline-evaluations/OfflineItemInspector.tsx
+++ b/products/ai_observability/frontend/offline-evaluations/OfflineItemInspector.tsx
@@ -7,6 +7,7 @@ import { LemonModalContent, LemonModalFooter, LemonModalHeader } from 'lib/lemon
import { urls } from 'scenes/urls'
import type { OfflineResultReadApi } from '../generated/api.schemas'
+import { ScorerLabel } from '../scoreDefinitions/ScorerLabel'
import { OfflineItemInspectorLogicProps, offlineItemInspectorLogic } from './offlineItemInspectorLogic'
import { OfflinePayload } from './OfflinePayload'
import { OfflineResultTag } from './OfflineResultTag'
@@ -217,7 +218,9 @@ export function OfflineItemInspector({
className="max-w-full"
active={result.id === selectedResult?.id}
onClick={() => onSelectResult(result)}
- >{`${result.scorer.name} v${result.scorer.version}`}</LemonButton>
+ >
+ <ScorerLabel scorer={result.scorer} />
+ </LemonButton>
),
},
{
diff --git a/products/ai_observability/frontend/offline-evaluations/OfflineItemMatrix.tsx b/products/ai_observability/frontend/offline-evaluations/OfflineItemMatrix.tsx
index 04466e3342cd..904d4d8d966a 100644
--- a/products/ai_observability/frontend/offline-evaluations/OfflineItemMatrix.tsx
+++ b/products/ai_observability/frontend/offline-evaluations/OfflineItemMatrix.tsx
@@ -6,6 +6,7 @@ import { LemonBanner, LemonButton, LemonTable } from '@posthog/lemon-ui'
import type { LemonTableColumns } from 'lib/lemon-ui/LemonTable'
import type { OfflineItemReadApi } from '../generated/api.schemas'
+import { ScorerLabel } from '../scoreDefinitions/ScorerLabel'
import { OfflineExperimentLogicProps, offlineExperimentLogic } from './offlineExperimentLogic'
import {
OFFLINE_ITEM_COLUMN_WIDTH,
@@ -78,11 +79,9 @@ export function OfflineItemMatrix(props: OfflineExperimentLogicProps): JSX.Eleme
width: OFFLINE_SCORER_COLUMN_WIDTH,
className: 'min-w-[180px] max-w-[180px]',
title: (
- <span
- data-offline-scorer={scorer.id}
- className="block truncate"
- title={`${scorer.name} v${scorer.version}`}
- >{`${scorer.name} v${scorer.version}`}</span>
+ <div data-offline-scorer={scorer.id}>
+ <ScorerLabel scorer={scorer} />
+ </div>
),
render: (_: unknown, item: OfflineItemReadApi) => {
const batchIndex = Math.floor(index / OFFLINE_SCORER_BATCH_SIZE)
diff --git a/products/ai_observability/frontend/offline-evaluations/OfflineOverviewTrend.tsx b/products/ai_observability/frontend/offline-evaluations/OfflineOverviewTrend.tsx
index 8a0cc195fa7d..10a27676913c 100644
--- a/products/ai_observability/frontend/offline-evaluations/OfflineOverviewTrend.tsx
+++ b/products/ai_observability/frontend/offline-evaluations/OfflineOverviewTrend.tsx
@@ -43,7 +43,7 @@ export function OfflineOverviewTrend(
) : trend ? (
<>
<div className="flex flex-wrap items-center gap-2">
- <h3 className="mb-0 min-w-0 truncate">
+ <h3 className="mb-0 min-w-0 break-words">
<Link
to={
combineUrl(urls.aiObservabilityOfflineScorerHistory(props.scorerId), {
@@ -55,7 +55,7 @@ export function OfflineOverviewTrend(
}
data-attr="offline-score-view-history"
>
- {trend.definition.name}
+ {trend.definition.short_name || trend.definition.name}
</Link>
</h3>
<LemonTag type="muted">{trend.definition.kind}</LemonTag>
diff --git a/products/ai_observability/frontend/offline-evaluations/OfflineScoreChooser.tsx b/products/ai_observability/frontend/offline-evaluations/OfflineScoreChooser.tsx
index ec74d46a143d..da8ef6a12f21 100644
--- a/products/ai_observability/frontend/offline-evaluations/OfflineScoreChooser.tsx
+++ b/products/ai_observability/frontend/offline-evaluations/OfflineScoreChooser.tsx
@@ -116,7 +116,9 @@ export function OfflineScoreChooser(props: OfflineExperimentsLogicProps): JSX.El
)}
{draftScorerIds.map((id, index) => (
<div key={id} className="flex items-center gap-1 min-w-0">
- <span className="flex-1 truncate">{scorersById[id]?.name || id}</span>
+ <span className="flex-1 truncate">
+ {scorersById[id]?.short_name || scorersById[id]?.name || id}
+ </span>
<LemonButton
icon={<IconChevronDown className="rotate-180" />}
aria-label="Move score up"
diff --git a/products/ai_observability/frontend/offline-evaluations/OfflineScoreTrendChart.tsx b/products/ai_observability/frontend/offline-evaluations/OfflineScoreTrendChart.tsx
index c3b68a5a3127..a7f9f170ebfd 100644
--- a/products/ai_observability/frontend/offline-evaluations/OfflineScoreTrendChart.tsx
+++ b/products/ai_observability/frontend/offline-evaluations/OfflineScoreTrendChart.tsx
@@ -8,6 +8,7 @@ import { dayjs } from 'lib/dayjs'
import { cn } from 'lib/utils/css-classes'
import type { OfflineHistoryPointApi } from '../generated/api.schemas'
+import { ScorerLabel } from '../scoreDefinitions/ScorerLabel'
import type { offlineExperimentsLogicType } from './offlineExperimentsLogic'
import { createOfflineScoreTrendTickFormatter } from './offlineScoreTrendAxis'
import { OfflineScoreTrendCrosshair } from './OfflineScoreTrendCrosshair'
@@ -120,7 +121,7 @@ export function OfflineScoreTrendChart({
<div className="mt-2 pt-2 border-t border-current/25">
<div className="flex items-center gap-2 font-medium">
<TooltipSwatch color={point.color} />
- <span className="break-words">{`${summary.scorer.name} · v${summary.scorer.version}`}</span>
+ <ScorerLabel scorer={summary.scorer} />
</div>
<div className="flex items-baseline justify-between gap-4 mt-1">
<span className="opacity-70">{meta.metric}</span>
diff --git a/products/ai_observability/frontend/offline-evaluations/offlineOverviewFixtures.ts b/products/ai_observability/frontend/offline-evaluations/offlineOverviewFixtures.ts
index 47223c9bf3c7..300decc59fad 100644
--- a/products/ai_observability/frontend/offline-evaluations/offlineOverviewFixtures.ts
+++ b/products/ai_observability/frontend/offline-evaluations/offlineOverviewFixtures.ts
@@ -5,6 +5,7 @@ export const overviewScorers: ScoreDefinitionApi[] = ['Answer quality', 'Grounde
tags: ['regression', 'production', 'quality'],
id: `11111111-1111-4111-8111-${String(index + 1).padStart(12, '0')}`,
name,
+ short_name: '',
description: '',
kind: index === 1 ? 'boolean' : 'numeric',
archived: false,
@@ -63,6 +64,7 @@ export function overviewHistory(scorer: ScoreDefinitionApi): OfflineHistoryPoint
id: scorer.current_version_id!,
definition_id: scorer.id,
name: scorer.name,
+ short_name: scorer.short_name,
description: '',
kind: scorer.kind,
archived: false,
diff --git a/products/ai_observability/frontend/offline-evaluations/offlineScoreTrends.fixtures.ts b/products/ai_observability/frontend/offline-evaluations/offlineScoreTrends.fixtures.ts
index eda76dff7471..f881a8e50021 100644
--- a/products/ai_observability/frontend/offline-evaluations/offlineScoreTrends.fixtures.ts
+++ b/products/ai_observability/frontend/offline-evaluations/offlineScoreTrends.fixtures.ts
@@ -3,6 +3,7 @@ import type { OfflineHistoryPointApi, ScoreDefinitionApi, ScoreDefinitionVersion
export const OFFLINE_STORY_DEFINITION: ScoreDefinitionApi = {
tags: [],
id: '10000000-0000-4000-8000-000000000001',
+ short_name: '',
name: 'Answer relevance',
description: 'Relevance to the question, scored from 0 to 5.',
kind: 'numeric',
@@ -62,6 +63,7 @@ export function makeOfflineHistoryPoint(index: number, mean: number | null = 4.2
version: 2,
kind: 'numeric',
name: OFFLINE_STORY_DEFINITION.name,
+ short_name: OFFLINE_STORY_DEFINITION.short_name,
description: OFFLINE_STORY_DEFINITION.description,
archived: false,
config: OFFLINE_STORY_DEFINITION.config,
diff --git a/products/ai_observability/frontend/scoreDefinitions/AIObservabilityScoreDefinitions.tsx b/products/ai_observability/frontend/scoreDefinitions/AIObservabilityScoreDefinitions.tsx
index 8b485636d7e7..4113ddb786d2 100644
--- a/products/ai_observability/frontend/scoreDefinitions/AIObservabilityScoreDefinitions.tsx
+++ b/products/ai_observability/frontend/scoreDefinitions/AIObservabilityScoreDefinitions.tsx
@@ -76,17 +76,8 @@ export function AIObservabilityScoreDefinitions(): JSX.Element {
width: '18%',
render: function renderName(_, scoreDefinition) {
return (
- <div className="space-y-1">
- <div className="font-semibold max-w-64 @max-[50rem]/scorers:max-w-32 @max-[40rem]/scorers:max-w-24 truncate">
- <Link to={urls.aiObservabilityScorer(scoreDefinition.id)}>{scoreDefinition.name}</Link>
- </div>
- {scoreDefinition.description ? (
- <div className="max-w-64 @max-[50rem]/scorers:max-w-32 @max-[40rem]/scorers:max-w-24 truncate text-muted-alt">
- {scoreDefinition.description}
- </div>
- ) : (
- <div className="text-muted">No description</div>
- )}
+ <div className="font-semibold max-w-64 @max-[50rem]/scorers:max-w-32 @max-[40rem]/scorers:max-w-24 truncate">
+ <Link to={urls.aiObservabilityScorer(scoreDefinition.id)}>{scoreDefinition.name}</Link>
</div>
)
},
@@ -194,9 +185,10 @@ export function AIObservabilityScoreDefinitions(): JSX.Element {
return (
<div className="space-y-4 @container/scorers">
- <div className="flex gap-x-4 gap-y-2 items-center flex-wrap py-4 mb-4 border-b justify-between">
+ <div className="flex gap-x-4 gap-y-2 items-center flex-wrap pb-4 mb-4 border-b justify-between">
<div className="flex items-center gap-2 flex-wrap">
<LemonInput
+ size="small"
type="search"
placeholder="Search scorers..."
value={filters.search}
@@ -204,22 +196,28 @@ export function AIObservabilityScoreDefinitions(): JSX.Element {
className="min-w-64"
data-attr="llma-scorers-search-input"
/>
- <ResourceTagFilter filters={filters} onChange={(tags) => setFilters(tags, true, false)} />
+ <ResourceTagFilter
+ size="small"
+ filters={filters}
+ onChange={(tags) => setFilters(tags, true, false)}
+ />
<div data-attr="llma-scorers-created-by-filter">
<MemberSelect
value={filters.created_by}
- size="medium"
+ size="small"
defaultLabel="Any creator"
onChange={(user) => setFilters({ created_by: user?.id ?? null }, true, false)}
/>
</div>
<LemonSelect<ScoreDefinitionKind | ''>
+ size="small"
value={filters.kind}
onChange={(value) => setFilters({ kind: value || '' })}
options={KIND_OPTIONS}
data-attr="llma-scorers-kind-filter"
/>
<LemonSelect<'' | 'false' | 'true'>
+ size="small"
value={filters.archived}
onChange={(value) => setFilters({ archived: value === '' ? '' : value || 'false' })}
options={ARCHIVED_OPTIONS}
@@ -256,6 +254,7 @@ export function AIObservabilityScoreDefinitions(): JSX.Element {
{!scoreDefinitionsError && (
<LemonTable
className="@max-[50rem]/scorers:[&_colgroup]:hidden"
+ size="small"
loading={scoreDefinitionsLoading}
columns={columns}
dataSource={scoreDefinitions.results}
diff --git a/products/ai_observability/frontend/scoreDefinitions/AIObservabilityScorerScene.stories.tsx b/products/ai_observability/frontend/scoreDefinitions/AIObservabilityScorerScene.stories.tsx
index 27c544615e16..3d11b8d7fe48 100644
--- a/products/ai_observability/frontend/scoreDefinitions/AIObservabilityScorerScene.stories.tsx
+++ b/products/ai_observability/frontend/scoreDefinitions/AIObservabilityScorerScene.stories.tsx
@@ -11,6 +11,7 @@ import { AIObservabilityScorerScene } from './AIObservabilityScorerScene'
const definition: ScoreDefinitionApi = {
tags: [],
id: 'b4f331fe-7d3b-4bbe-9f88-52e1b0a8b000',
+ short_name: 'Quality',
name: 'Answer quality',
description: 'How well the response answers the question, scored from 0 to 1.',
kind: 'numeric',
diff --git a/products/ai_observability/frontend/scoreDefinitions/AIObservabilityScorerScene.tsx b/products/ai_observability/frontend/scoreDefinitions/AIObservabilityScorerScene.tsx
index c27352781eb7..76f5907ca716 100644
--- a/products/ai_observability/frontend/scoreDefinitions/AIObservabilityScorerScene.tsx
+++ b/products/ai_observability/frontend/scoreDefinitions/AIObservabilityScorerScene.tsx
@@ -1,6 +1,6 @@
import { useActions, useValues } from 'kea'
-import { IconEllipsis } from '@posthog/icons'
+import { IconArrowLeft, IconEllipsis } from '@posthog/icons'
import { LemonBanner, LemonButton, LemonMenu, LemonTag } from '@posthog/lemon-ui'
import { CopyToClipboardInline } from 'lib/components/CopyToClipboard'
@@ -13,8 +13,6 @@ import { getAccessControlDisabledReason, getProductAccessDisabledReason } from '
import { Scene, SceneExport } from 'scenes/sceneTypes'
import { urls } from 'scenes/urls'
-import { SceneContent } from '~/layout/scenes/components/SceneContent'
-import { SceneTitleSection } from '~/layout/scenes/components/SceneTitleSection'
import { ProductKey } from '~/queries/schema/schema-general'
import { AccessControlLevel, AccessControlResourceType } from '~/types'
@@ -23,6 +21,7 @@ import { QuillAIObservabilityScorerActions } from './QuillAIObservabilityScorerA
import { ScoreDefinitionForm } from './ScoreDefinitionForm'
import { formatKindLabel } from './scoreDefinitionModalUtils'
import { ScorerLogicProps, scorerLogic } from './scorerLogic'
+import { ScorersScene } from './ScorersScene'
export const scene: SceneExport<ScorerLogicProps> = {
component: AIObservabilityScorerScene,
@@ -63,100 +62,114 @@ export function AIObservabilityScorerScene(props: ScorerLogicProps): JSX.Element
if (definitionLoading || (!isNew && !definition && !loadError)) {
return (
- <SceneContent>
+ <ScorersScene>
<LemonSkeleton className="h-8 w-64" />
<LemonSkeleton className="h-96 w-full max-w-3xl" />
- </SceneContent>
+ </ScorersScene>
)
}
if (loadError) {
return (
- <SceneContent>
+ <ScorersScene>
<LemonBanner type="error" action={{ children: 'Try again', onClick: loadDefinition }}>
{loadError}
</LemonBanner>
<LemonButton to={urls.aiObservabilityScorers()}>Back to scorers</LemonButton>
- </SceneContent>
+ </ScorersScene>
)
}
return (
- <SceneContent>
- <SceneTitleSection
- name={isNew ? 'New scorer' : definition?.name || 'Scorer'}
- description="Define the scores used in manual reviews and offline evaluations."
- resourceType={{ type: 'llm_evaluations' }}
- actions={
- todayRailNav ? (
- <QuillAIObservabilityScorerActions {...props} />
- ) : (
- <div className="flex flex-wrap items-center gap-2">
- {!isNew && definition && (
- <LemonMenu
- items={[
- {
- label: 'Duplicate',
- to: urls.aiObservabilityScorer('new', { duplicate: definition.id }),
- disabledReason:
- editDisabledReason || (saving ? 'Saving scorer' : undefined),
- 'data-attr': 'llma-scorer-duplicate',
- },
- {
- label: 'Create new version',
- disabledReason:
- editDisabledReason ||
- (saving
- ? 'Saving scorer'
- : hasChanges
- ? 'Save your changes first'
- : undefined),
- onClick: () =>
- openCreateScorerVersionDialog({
- currentVersion: definition.current_version,
- createVersion,
- }),
- 'data-attr': 'llma-scorer-new-version',
- },
- {
- label: definition.archived ? 'Unarchive' : 'Archive',
- status: definition.archived ? 'default' : 'danger',
- disabledReason:
- editDisabledReason ||
- (saving
- ? 'Saving scorer'
- : hasChanges
- ? 'Save your changes first'
- : undefined),
- onClick: toggleArchive,
- 'data-attr': 'llma-scorer-archive-toggle',
- },
- ]}
- >
- <LemonButton icon={<IconEllipsis />} aria-label="Scorer actions" type="secondary" />
- </LemonMenu>
- )}
- <LemonButton
- to={urls.aiObservabilityScorers()}
- type="secondary"
- disabledReason={saving ? 'Saving scorer' : undefined}
- data-attr="llma-scorer-cancel"
+ <ScorersScene>
+ <div className="flex flex-wrap items-center gap-3">
+ <nav aria-label="Scorer" className="flex min-w-0 flex-1 basis-40 items-center gap-2">
+ <LemonButton
+ size="small"
+ icon={<IconArrowLeft />}
+ aria-label="Back to scorers"
+ tooltip="Back to scorers"
+ to={urls.aiObservabilityScorers()}
+ disabledReason={saving ? 'Saving scorer' : undefined}
+ />
+ <h2 className="text-base font-semibold m-0 truncate" title={definition?.name}>
+ {isNew ? 'New scorer' : definition?.name || 'Scorer'}
+ </h2>
+ </nav>
+ {todayRailNav ? (
+ <QuillAIObservabilityScorerActions {...props} />
+ ) : (
+ <div className="flex flex-wrap items-center gap-2">
+ {!isNew && definition && (
+ <LemonMenu
+ items={[
+ {
+ label: 'Duplicate',
+ to: urls.aiObservabilityScorer('new', { duplicate: definition.id }),
+ disabledReason: editDisabledReason || (saving ? 'Saving scorer' : undefined),
+ 'data-attr': 'llma-scorer-duplicate',
+ },
+ {
+ label: 'Create new version',
+ disabledReason:
+ editDisabledReason ||
+ (saving
+ ? 'Saving scorer'
+ : hasChanges
+ ? 'Save your changes first'
+ : undefined),
+ onClick: () =>
+ openCreateScorerVersionDialog({
+ currentVersion: definition.current_version,
+ createVersion,
+ }),
+ 'data-attr': 'llma-scorer-new-version',
+ },
+ {
+ label: definition.archived ? 'Unarchive' : 'Archive',
+ status: definition.archived ? 'default' : 'danger',
+ disabledReason:
+ editDisabledReason ||
+ (saving
+ ? 'Saving scorer'
+ : hasChanges
+ ? 'Save your changes first'
+ : undefined),
+ onClick: toggleArchive,
+ 'data-attr': 'llma-scorer-archive-toggle',
+ },
+ ]}
>
- {hasChanges || isNew ? 'Cancel' : 'Back to scorers'}
- </LemonButton>
- <LemonButton
- type="primary"
- htmlType="submit"
- form="scorer-form"
- loading={saving}
- disabledReason={editDisabledReason || (!hasChanges ? 'No unsaved changes' : undefined)}
- data-attr="llma-scorer-save"
- >
- {isNew ? 'Create scorer' : 'Save changes'}
- </LemonButton>
- </div>
- )
- }
- />
+ <LemonButton
+ icon={<IconEllipsis />}
+ aria-label="Scorer actions"
+ type="secondary"
+ size="small"
+ />
+ </LemonMenu>
+ )}
+ <LemonButton
+ to={urls.aiObservabilityScorers()}
+ type="secondary"
+ size="small"
+ disabledReason={saving ? 'Saving scorer' : undefined}
+ data-attr="llma-scorer-cancel"
+ >
+ {hasChanges || isNew ? 'Cancel' : 'Back to scorers'}
+ </LemonButton>
+ <LemonButton
+ type="primary"
+ size="small"
+ htmlType="submit"
+ form="scorer-form"
+ loading={saving}
+ disabledReason={editDisabledReason || (!hasChanges ? 'No unsaved changes' : undefined)}
+ data-attr="llma-scorer-save"
+ >
+ {isNew ? 'Create scorer' : 'Save changes'}
+ </LemonButton>
+ </div>
+ )}
+ </div>
<form
id="scorer-form"
className="max-w-3xl w-full space-y-6 pb-8"
@@ -229,6 +242,6 @@ export function AIObservabilityScorerScene(props: ScorerLogicProps): JSX.Element
removeOption={removeOption}
/>
</form>
- </SceneContent>
+ </ScorersScene>
)
}
diff --git a/products/ai_observability/frontend/scoreDefinitions/AIObservabilityScorersScene.stories.tsx b/products/ai_observability/frontend/scoreDefinitions/AIObservabilityScorersScene.stories.tsx
index 354f9f4ee364..3b378956c375 100644
--- a/products/ai_observability/frontend/scoreDefinitions/AIObservabilityScorersScene.stories.tsx
+++ b/products/ai_observability/frontend/scoreDefinitions/AIObservabilityScorersScene.stories.tsx
@@ -11,6 +11,7 @@ import { AIObservabilityScorersScene } from './AIObservabilityScorersScene'
const definition: ScoreDefinitionApi = {
tags: ['production', 'quality', 'regression', 'support', 'long-tag-name-for-overflow'],
id: 'b4f331fe-7d3b-4bbe-9f88-52e1b0a8b000',
+ short_name: '',
name: 'Answer quality',
description: 'Quality of generated answers',
kind: 'numeric',
diff --git a/products/ai_observability/frontend/scoreDefinitions/AIObservabilityScorersScene.tsx b/products/ai_observability/frontend/scoreDefinitions/AIObservabilityScorersScene.tsx
index b3155c22827f..9c186f5205d5 100644
--- a/products/ai_observability/frontend/scoreDefinitions/AIObservabilityScorersScene.tsx
+++ b/products/ai_observability/frontend/scoreDefinitions/AIObservabilityScorersScene.tsx
@@ -1,34 +1,15 @@
-import { useValues } from 'kea'
-
-import { FEATURE_FLAGS } from 'lib/constants'
-import { featureFlagLogic } from 'lib/logic/featureFlagLogic'
-
-import { SceneContent } from '~/layout/scenes/components/SceneContent'
-import { SceneTitleSection } from '~/layout/scenes/components/SceneTitleSection'
import { ProductKey } from '~/queries/schema/schema-general'
import { SceneExport } from '~/scenes/sceneTypes'
-import { EvaluationsTabs } from '../evaluations/EvaluationsTabs'
import { AIObservabilityScoreDefinitions } from './AIObservabilityScoreDefinitions'
+import { ScorersScene } from './ScorersScene'
export const scene: SceneExport = { component: AIObservabilityScorersScene, productKey: ProductKey.AI_OBSERVABILITY }
export function AIObservabilityScorersScene(): JSX.Element {
- const { featureFlags } = useValues(featureFlagLogic)
return (
- <SceneContent>
- <SceneTitleSection
- name="Evaluations"
- description={
- featureFlags[FEATURE_FLAGS.AI_OBSERVABILITY_OFFLINE_EVALUATIONS]
- ? 'Manage score definitions and versions for manual reviews and offline evaluations.'
- : 'Manage score definitions and versions for manual reviews.'
- }
- resourceType={{ type: 'llm_evaluations' }}
- />
- <EvaluationsTabs activeTab="scorers">
- <AIObservabilityScoreDefinitions />
- </EvaluationsTabs>
- </SceneContent>
+ <ScorersScene>
+ <AIObservabilityScoreDefinitions />
+ </ScorersScene>
)
}
diff --git a/products/ai_observability/frontend/scoreDefinitions/ScoreDefinitionForm.tsx b/products/ai_observability/frontend/scoreDefinitions/ScoreDefinitionForm.tsx
index c973a2e07bb5..8e94e528e7bd 100644
--- a/products/ai_observability/frontend/scoreDefinitions/ScoreDefinitionForm.tsx
+++ b/products/ai_observability/frontend/scoreDefinitions/ScoreDefinitionForm.tsx
@@ -67,6 +67,21 @@ export function ScoreDefinitionForm({
data-attr="llma-scorer-name-input"
/>
</LemonField.Pure>
+ <LemonField.Pure
+ label="Short name (optional)"
+ htmlFor={`${id}-short-name`}
+ help="Used in table headers and other compact views. Leave blank to use the full name."
+ >
+ <LemonInput
+ id={`${id}-short-name`}
+ value={draft.shortName}
+ onChange={(value) => setDraftField('shortName', value)}
+ maxLength={40}
+ disabled={disabled}
+ placeholder="e.g. Quality"
+ data-attr="aio-scorer-short-name-input"
+ />
+ </LemonField.Pure>
<LemonField.Pure label="Description (optional)" htmlFor={`${id}-description`}>
<LemonTextArea
id={`${id}-description`}
diff --git a/products/ai_observability/frontend/scoreDefinitions/ScorerLabel.stories.tsx b/products/ai_observability/frontend/scoreDefinitions/ScorerLabel.stories.tsx
new file mode 100644
index 000000000000..2924b2a2d760
--- /dev/null
+++ b/products/ai_observability/frontend/scoreDefinitions/ScorerLabel.stories.tsx
@@ -0,0 +1,34 @@
+import type { Meta, StoryObj } from '@storybook/react'
+
+import { ScorerLabel } from './ScorerLabel'
+
+const meta: Meta<typeof ScorerLabel> = {
+ title: 'AI observability/Scorer label',
+ component: ScorerLabel,
+ decorators: [
+ (Story) => (
+ <div className="w-40">
+ <Story />
+ </div>
+ ),
+ ],
+ args: {
+ scorer: {
+ name: 'Checks whether the answer is supported by the supplied reference material',
+ short_name: 'Groundedness',
+ version: 12,
+ },
+ },
+}
+export default meta
+type Story = StoryObj<typeof ScorerLabel>
+
+export const ShortName: Story = {}
+export const FullNameFallback: Story = {
+ args: { scorer: { ...meta.args!.scorer!, short_name: '' } },
+}
+export const LongWord: Story = {
+ args: {
+ scorer: { name: 'Reference consistency', short_name: 'ReferenceMaterialConsistencyVerification', version: 1 },
+ },
+}
diff --git a/products/ai_observability/frontend/scoreDefinitions/ScorerLabel.tsx b/products/ai_observability/frontend/scoreDefinitions/ScorerLabel.tsx
new file mode 100644
index 000000000000..5a7b3a556d40
--- /dev/null
+++ b/products/ai_observability/frontend/scoreDefinitions/ScorerLabel.tsx
@@ -0,0 +1,18 @@
+import { LemonTag } from '@posthog/lemon-ui'
+
+import type { OfflineScorerVersionReadApi } from '../generated/api.schemas'
+
+export function ScorerLabel({
+ scorer,
+}: {
+ scorer: Pick<OfflineScorerVersionReadApi, 'name' | 'short_name' | 'version'>
+}): JSX.Element {
+ return (
+ <div className="flex flex-wrap items-center gap-1 min-w-0 normal-case">
+ <span className="whitespace-normal break-words min-w-0 max-w-full">{scorer.short_name || scorer.name}</span>
+ <LemonTag type="muted" size="small" className="shrink-0" weight="normal">
+ {`v${scorer.version}`}
+ </LemonTag>
+ </div>
+ )
+}
diff --git a/products/ai_observability/frontend/scoreDefinitions/ScorersScene.tsx b/products/ai_observability/frontend/scoreDefinitions/ScorersScene.tsx
new file mode 100644
index 000000000000..54d92302d9e6
--- /dev/null
+++ b/products/ai_observability/frontend/scoreDefinitions/ScorersScene.tsx
@@ -0,0 +1,28 @@
+import { useValues } from 'kea'
+import type { ReactNode } from 'react'
+
+import { FEATURE_FLAGS } from 'lib/constants'
+import { featureFlagLogic } from 'lib/logic/featureFlagLogic'
+
+import { SceneContent } from '~/layout/scenes/components/SceneContent'
+import { SceneTitleSection } from '~/layout/scenes/components/SceneTitleSection'
+
+import { EvaluationsTabs } from '../evaluations/EvaluationsTabs'
+
+export function ScorersScene({ children }: { children: ReactNode }): JSX.Element {
+ const { featureFlags } = useValues(featureFlagLogic)
+ return (
+ <SceneContent>
+ <SceneTitleSection
+ name="Evaluations"
+ description={
+ featureFlags[FEATURE_FLAGS.AI_OBSERVABILITY_OFFLINE_EVALUATIONS]
+ ? 'Manage score definitions and versions for manual reviews and offline evaluations.'
+ : 'Manage score definitions and versions for manual reviews.'
+ }
+ resourceType={{ type: 'llm_evaluations' }}
+ />
+ <EvaluationsTabs activeTab="scorers">{children}</EvaluationsTabs>
+ </SceneContent>
+ )
+}
diff --git a/products/ai_observability/frontend/scoreDefinitions/aiObservabilityScoreDefinitionsLogic.test.ts b/products/ai_observability/frontend/scoreDefinitions/aiObservabilityScoreDefinitionsLogic.test.ts
index 7e6270371756..a90b8d5b7286 100644
--- a/products/ai_observability/frontend/scoreDefinitions/aiObservabilityScoreDefinitionsLogic.test.ts
+++ b/products/ai_observability/frontend/scoreDefinitions/aiObservabilityScoreDefinitionsLogic.test.ts
@@ -36,6 +36,7 @@ const mockAIObservabilityScoreDefinitionsPartialUpdate =
const mockScoreDefinition: ScoreDefinition = {
tags: [],
id: 'score_def_1',
+ short_name: '',
name: 'Quality',
description: 'Reusable quality scorer',
kind: 'categorical',
@@ -76,7 +77,12 @@ describe('aiObservabilityScoreDefinitionsLogic', () => {
mockAIObservabilityScoreDefinitionsPartialUpdate.mockResolvedValue(mockScoreDefinition)
})
- it('loads score definitions on mount using the default filters', async () => {
+ it.each([
+ { orderBy: undefined, expectedOrder: '-updated_at' },
+ { orderBy: 'invalid', expectedOrder: '-updated_at' },
+ { orderBy: 'name', expectedOrder: 'name' },
+ ])('loads scorers with sort $expectedOrder for URL sort $orderBy', async ({ orderBy, expectedOrder }) => {
+ router.actions.push(urls.aiObservabilityScorers(), { order_by: orderBy })
const logic = aiObservabilityScoreDefinitionsLogic()
logic.mount()
@@ -86,10 +92,15 @@ describe('aiObservabilityScoreDefinitionsLogic', () => {
search: undefined,
kind: undefined,
archived: false,
- order_by: 'name',
+ order_by: expectedOrder,
offset: 0,
limit: SCORE_DEFINITIONS_PER_PAGE,
})
+ expect(logic.values.sorting).toEqual(
+ expectedOrder === 'name' ? { columnKey: 'name', order: 1 } : { columnKey: 'updated_at', order: -1 }
+ )
+ await expectLogic(logic, () => logic.actions.setFilters({ search: 'quality' })).toFinishAllListeners()
+ expect(router.values.searchParams.order_by).toBe(expectedOrder === '-updated_at' ? undefined : expectedOrder)
})
it.each(['saved', 'url'])('waits for the project before restoring %s tag filters', async (source) => {
@@ -136,14 +147,14 @@ describe('aiObservabilityScoreDefinitionsLogic', () => {
created_by: 42,
kind: 'categorical',
archived: 'true',
- order_by: 'name',
+ order_by: '-updated_at',
})
expect(mockAIObservabilityScoreDefinitionsList).toHaveBeenLastCalledWith(String(MOCK_DEFAULT_TEAM.id), {
search: 'quality',
created_by: 42,
kind: 'categorical',
archived: true,
- order_by: 'name',
+ order_by: '-updated_at',
offset: 0,
limit: SCORE_DEFINITIONS_PER_PAGE,
})
diff --git a/products/ai_observability/frontend/scoreDefinitions/aiObservabilityScoreDefinitionsLogic.ts b/products/ai_observability/frontend/scoreDefinitions/aiObservabilityScoreDefinitionsLogic.ts
index 1b88f4e5703c..b48a8b3bf151 100644
--- a/products/ai_observability/frontend/scoreDefinitions/aiObservabilityScoreDefinitionsLogic.ts
+++ b/products/ai_observability/frontend/scoreDefinitions/aiObservabilityScoreDefinitionsLogic.ts
@@ -60,7 +60,7 @@ const ALLOWED_ORDER_BY_VALUES = new Set([
])
function cleanFilters(values: ScoreDefinitionFilterInput): ScoreDefinitionFilters {
- const orderBy = values.order_by && ALLOWED_ORDER_BY_VALUES.has(values.order_by) ? values.order_by : 'name'
+ const orderBy = values.order_by && ALLOWED_ORDER_BY_VALUES.has(values.order_by) ? values.order_by : '-updated_at'
return {
...cleanTagFilters(values),
@@ -95,7 +95,7 @@ function getUrlFilters(filters: ScoreDefinitionFilters): Record<string, unknown>
created_by: filters.created_by || undefined,
kind: filters.kind || undefined,
archived: filters.archived === 'false' ? undefined : filters.archived === '' ? 'all' : 'true',
- order_by: filters.order_by === 'name' ? undefined : filters.order_by,
+ order_by: filters.order_by === '-updated_at' ? undefined : filters.order_by,
}
}
@@ -302,8 +302,8 @@ export const aiObservabilityScoreDefinitionsLogic = kea<aiObservabilityScoreDefi
(filters: ScoreDefinitionFilters): Sorting | null => {
if (!filters.order_by) {
return {
- columnKey: 'name',
- order: 1,
+ columnKey: 'updated_at',
+ order: -1,
}
}
diff --git a/products/ai_observability/frontend/scoreDefinitions/scoreDefinitionModalUtils.test.ts b/products/ai_observability/frontend/scoreDefinitions/scoreDefinitionModalUtils.test.ts
index 10f8cb74f368..e44ae3da03f0 100644
--- a/products/ai_observability/frontend/scoreDefinitions/scoreDefinitionModalUtils.test.ts
+++ b/products/ai_observability/frontend/scoreDefinitions/scoreDefinitionModalUtils.test.ts
@@ -4,6 +4,7 @@ import { buildConfigFromDraft, createDraft, validateDraft } from './scoreDefinit
const definition: ScoreDefinitionApi = {
tags: [],
id: 'scorer-example',
+ short_name: '',
name: 'Answer quality',
description: '',
kind: 'boolean',
diff --git a/products/ai_observability/frontend/scoreDefinitions/scoreDefinitionModalUtils.ts b/products/ai_observability/frontend/scoreDefinitions/scoreDefinitionModalUtils.ts
index 0b2d218d2bf3..a7b9865fd4ef 100644
--- a/products/ai_observability/frontend/scoreDefinitions/scoreDefinitionModalUtils.ts
+++ b/products/ai_observability/frontend/scoreDefinitions/scoreDefinitionModalUtils.ts
@@ -16,6 +16,7 @@ export type CategoricalSelectionMode = 'single' | 'multiple'
export interface ScoreDefinitionDraft {
name: string
+ shortName: string
description: string
kind: ScoreDefinitionKind
options: ScoreDefinitionOption[]
@@ -156,6 +157,7 @@ export function createDraft(
return {
name: mode === 'duplicate' ? duplicatedName : baseDefinition?.name || '',
+ shortName: baseDefinition?.short_name || '',
description: baseDefinition?.description || '',
kind,
options: defaultOptions,
diff --git a/products/ai_observability/frontend/scoreDefinitions/scorerLogic.test.ts b/products/ai_observability/frontend/scoreDefinitions/scorerLogic.test.ts
index 4dde97d000ea..e22d694ebccd 100644
--- a/products/ai_observability/frontend/scoreDefinitions/scorerLogic.test.ts
+++ b/products/ai_observability/frontend/scoreDefinitions/scorerLogic.test.ts
@@ -31,6 +31,7 @@ const update = jest.mocked(llmAnalyticsScoreDefinitionsPartialUpdate)
const definition: ScoreDefinitionApi = {
tags: [],
id: 'scorer-example',
+ short_name: '',
name: 'Answer quality',
description: '',
kind: 'numeric',
@@ -54,23 +55,36 @@ describe('scorer editor', () => {
}
jest.resetAllMocks()
retrieve.mockResolvedValue(definition)
+ router.actions.replace(urls.aiObservabilityScorer(definition.id))
})
- it.each<{ kind: ScoreDefinitionApi['kind']; config: ScoreDefinitionConfigApi; duplicateFrom?: string }>([
+ it.each<{
+ kind: ScoreDefinitionApi['kind']
+ config: ScoreDefinitionConfigApi
+ duplicateFrom?: string
+ field?: 'name' | 'shortName'
+ shortName?: string
+ newValue?: string
+ }>([
{
kind: 'categorical',
config: { options: [{ key: 'good', label: 'Good' }], passing_rule: { categories: ['good'] } },
},
{ kind: 'numeric', config: { min: 0, max: null, passing_rule: null } },
{ kind: 'numeric', config: {}, duplicateFrom: 'another-scorer' },
+ { kind: 'numeric', config: {}, field: 'shortName' },
+ { kind: 'numeric', config: {}, field: 'shortName', shortName: 'Quality', newValue: '' },
{ kind: 'boolean', config: { true_label: '', false_label: 'Needs work', true_is_failure: false } },
{ kind: 'categorical', config: { options: [{ key: 'good', label: 'Good' }], selection_mode: 'single' } },
])(
'saves only metadata when an existing $kind configuration is untouched',
- async ({ kind, config, duplicateFrom }) => {
- const existing = { ...definition, kind, config }
+ async ({ kind, config, duplicateFrom, field = 'name', shortName = '', newValue = 'Renamed scorer' }) => {
+ const existing = { ...definition, kind, config, short_name: shortName }
retrieve.mockResolvedValue(existing)
- update.mockResolvedValue({ ...existing, name: 'Renamed scorer' })
+ const metadata = { name: existing.name, short_name: shortName, description: '' }
+ const originalValue = field === 'name' ? existing.name : shortName
+ metadata[field === 'name' ? 'name' : 'short_name'] = newValue
+ update.mockResolvedValue({ ...existing, ...metadata })
const logic = scorerLogic({ scorerId: definition.id, duplicateFrom })
logic.mount()
await expectLogic(logic).toFinishAllListeners()
@@ -80,22 +94,23 @@ describe('scorer editor', () => {
expect(logic.values.hasChanges).toBe(false)
expect(logic.values.hasUnsavedChanges).toBe(false)
expect(logic.values.willCreateVersion).toBe(false)
- logic.actions.setDraftField('name', 'Renamed scorer')
+ logic.actions.setDraftField(field, newValue)
+ expect(logic.values.hasChanges).toBe(true)
expect(logic.values.hasUnsavedChanges).toBe(true)
expect(logic.values.willCreateVersion).toBe(false)
- logic.actions.setDraftField('name', existing.name)
+ logic.actions.setDraftField(field, originalValue)
expect(logic.values.hasUnsavedChanges).toBe(false)
- logic.actions.setDraftField('name', 'Renamed scorer')
+ logic.actions.setDraftField(field, newValue)
await expectLogic(logic, () => logic.actions.save()).toFinishAllListeners()
- expect(update).toHaveBeenCalledWith(String(MOCK_DEFAULT_TEAM.id), definition.id, {
- name: 'Renamed scorer',
- description: '',
- })
+ expect(update).toHaveBeenCalledWith(String(MOCK_DEFAULT_TEAM.id), definition.id, metadata)
expect(createVersion).not.toHaveBeenCalled()
expect(logic.values.definition?.config).toEqual(config)
expect(logic.values.hasChanges).toBe(false)
expect(logic.values.hasUnsavedChanges).toBe(false)
+ expect(router.values.location.pathname).toBe(
+ `/project/${MOCK_DEFAULT_TEAM.id}${urls.aiObservabilityScorers()}`
+ )
}
)
@@ -130,6 +145,7 @@ describe('scorer editor', () => {
expect(createVersion).toHaveBeenCalledWith(String(MOCK_DEFAULT_TEAM.id), definition.id, {
name: 'Updated quality',
+ short_name: '',
description: '',
config: { passing_rule: { operator: 'gte', threshold: 7 } },
base_version: 2,
@@ -172,6 +188,7 @@ describe('scorer editor', () => {
expect(createVersion).toHaveBeenCalledWith(String(MOCK_DEFAULT_TEAM.id), definition.id, {
name: 'Renamed scorer',
+ short_name: '',
description: '',
config: saved.config,
base_version: 2,
@@ -209,6 +226,7 @@ describe('scorer editor', () => {
expect(create).toHaveBeenCalledWith(String(MOCK_DEFAULT_TEAM.id), {
name: 'Answer quality copy',
+ short_name: '',
description: '',
kind: 'boolean',
config: existing.config,
@@ -254,11 +272,12 @@ describe('scorer editor', () => {
resolve(saved)
await expectLogic(logic).toFinishAllListeners()
expect(logic.values.saving).toBe(false)
- if (isNew) {
+ if (action === 'save') {
expect(router.values.location.pathname).toBe(
- `/project/${MOCK_DEFAULT_TEAM.id}${urls.aiObservabilityScorer(saved.id)}`
+ `/project/${MOCK_DEFAULT_TEAM.id}${urls.aiObservabilityScorers()}`
)
- } else {
+ }
+ if (!isNew) {
expect(logic.values.definition).toEqual(saved)
expect(logic.values.hasUnsavedChanges).toBe(false)
}
@@ -275,6 +294,9 @@ describe('scorer editor', () => {
await expectLogic(logic, () => logic.actions.save()).toFinishAllListeners()
expect(createVersion).toHaveBeenCalledTimes(1)
+ expect(router.values.location.pathname).toBe(
+ `/project/${MOCK_DEFAULT_TEAM.id}${urls.aiObservabilityScorer(definition.id)}`
+ )
expect(logic.values.error).toContain('newer version')
expect(logic.values.draft.numericMax).toBe('10')
expect(logic.values.definition?.current_version).toBe(2)
diff --git a/products/ai_observability/frontend/scoreDefinitions/scorerLogic.ts b/products/ai_observability/frontend/scoreDefinitions/scorerLogic.ts
index 21b80685dd8d..ae116d768366 100644
--- a/products/ai_observability/frontend/scoreDefinitions/scorerLogic.ts
+++ b/products/ai_observability/frontend/scoreDefinitions/scorerLogic.ts
@@ -257,6 +257,7 @@ export const scorerLogic = kea<scorerLogicType>([
isNew ||
configChanged ||
draft.name.trim() !== definition?.name ||
+ draft.shortName.trim() !== (definition?.short_name || '') ||
draft.description.trim() !== definition?.description,
],
willCreateVersion: [
@@ -318,7 +319,11 @@ export const scorerLogic = kea<scorerLogicType>([
performSave: async (_, breakpoint) => {
const teamId = getCurrentProjectId()
const { draft, definition, isNew, willCreateVersion } = values
- const metadata = { name: draft.name.trim(), description: draft.description.trim() }
+ const metadata = {
+ name: draft.name.trim(),
+ short_name: draft.shortName.trim(),
+ description: draft.description.trim(),
+ }
const operation = isNew ? 'create' : willCreateVersion ? 'version' : 'metadata'
actions.setError(null)
posthog.capture('llma scorer save started', { operation, kind: draft.kind })
@@ -345,9 +350,7 @@ export const scorerLogic = kea<scorerLogicType>([
}
posthog.capture('llma scorer saved', { operation, kind: saved.kind })
lemonToast.success(isNew ? 'Scorer created.' : 'Scorer saved.')
- if (isNew) {
- router.actions.replace(urls.aiObservabilityScorer(saved.id))
- }
+ router.actions.replace(urls.aiObservabilityScorers())
} catch (error) {
breakpoint()
const stale = typeof error === 'object' && error !== null && 'status' in error && error.status === 409
diff --git a/products/ai_observability/frontend/traceReviews/traceReviewModalLogic.test.ts b/products/ai_observability/frontend/traceReviews/traceReviewModalLogic.test.ts
index ca41a48b58e7..6003aee13dc9 100644
--- a/products/ai_observability/frontend/traceReviews/traceReviewModalLogic.test.ts
+++ b/products/ai_observability/frontend/traceReviews/traceReviewModalLogic.test.ts
@@ -36,6 +36,7 @@ describe('traceReviewModalLogic', () => {
const booleanDefinition: ScoreDefinition = {
tags: [],
id: 'score_def_1',
+ short_name: '',
name: 'Hallucination',
description: 'Whether the model hallucinated.',
kind: 'boolean',
@@ -56,6 +57,7 @@ describe('traceReviewModalLogic', () => {
const numericDefinition: ScoreDefinition = {
tags: [],
id: 'score_def_2',
+ short_name: '',
name: 'Accuracy',
description: 'How accurate the response was.',
kind: 'numeric',
diff --git a/products/ai_observability/frontend/traceReviews/traceReviewModalLogic.ts b/products/ai_observability/frontend/traceReviews/traceReviewModalLogic.ts
index ef79ef9fbd47..ae9de5f36822 100644
--- a/products/ai_observability/frontend/traceReviews/traceReviewModalLogic.ts
+++ b/products/ai_observability/frontend/traceReviews/traceReviewModalLogic.ts
@@ -94,6 +94,7 @@ function buildScoreDefinitionFromReviewScore(score: TraceReviewScore, teamId: nu
tags: [],
id: score.definition_id,
name: score.definition_name,
+ short_name: '',
description: score.definition_archived
? 'Archived scorer from this review.'
: 'Saved scorer version from this review.',
diff --git a/services/mcp/src/api/generated.ts b/services/mcp/src/api/generated.ts
index 5b046cb74b2d..414182e2d24e 100644
--- a/services/mcp/src/api/generated.ts
+++ b/services/mcp/src/api/generated.ts
@@ -71379,6 +71379,8 @@ export namespace Schemas {
kind: ScoreDefinitionKindEnum;
/** Current scorer display name. */
name: string;
+ /** Current compact label. Blank uses the full name. */
+ short_name: string;
/** Current scorer description. */
description: string;
/** Whether the scorer is archived. */
@@ -75999,6 +76001,8 @@ export namespace Schemas {
export interface ScoreDefinition {
readonly id: string;
readonly name: string;
+ /** Optional compact label for table headers and other compact views. */
+ readonly short_name: string;
readonly description: string;
/** Project tag names assigned to this resource. */
readonly tags: readonly string[];
@@ -87937,6 +87941,11 @@ export namespace Schemas {
* @maxLength 255
*/
name?: string;
+ /**
+ * Updated compact label. Blank uses the full name.
+ * @maxLength 40
+ */
+ short_name?: string;
/**
* Updated scorer description.
* @nullable
@@ -102905,6 +102914,11 @@ export namespace Schemas {
* @maxLength 255
*/
name: string;
+ /**
+ * Optional compact label. Blank uses the full name.
+ * @maxLength 40
+ */
+ short_name?: string;
/**
* Optional human-readable description.
* @nullable
@@ -102923,6 +102937,11 @@ export namespace Schemas {
}
export interface ScoreDefinitionNewVersion {
+ /**
+ * Updated compact label, saved with this version.
+ * @maxLength 40
+ */
+ short_name?: string;
/**
* Updated scorer name, saved with this version.
* @maxLength 255
@@ -131929,7 +131948,7 @@ export namespace Schemas {
*/
order_by?: string;
/**
- * Search scorers by name or description.
+ * Search scorers by name, short name, or description.
*/
search?: string;
/**
diff --git a/services/mcp/src/generated/ai_observability/api.ts b/services/mcp/src/generated/ai_observability/api.ts
index 2185af4720d5..b789533a214d 100644
--- a/services/mcp/src/generated/ai_observability/api.ts
+++ b/services/mcp/src/generated/ai_observability/api.ts
@@ -2912,7 +2912,7 @@ export const LlmAnalyticsScoreDefinitionsListQueryParams = () => zod.object({
limit: zod.number().optional().describe('Number of results to return per page.'),
offset: zod.number().optional().describe('The initial index from which to return the results.'),
order_by: zod.string().optional().describe('Sort by name, kind, created_at, updated_at, or current_version.'),
- search: zod.string().optional().describe('Search scorers by name or description.'),
+ search: zod.string().optional().describe('Search scorers by name, short name, or description.'),
tag_match: zod
.enum(['any', 'all'])
.default(llmAnalyticsScoreDefinitionsListQueryTagMatchDefault)
@@ -2941,6 +2941,8 @@ export const llmAnalyticsScoreDefinitionsCreateBodyTagsMax = 100
export const llmAnalyticsScoreDefinitionsCreateBodyNameMax = 255
+export const llmAnalyticsScoreDefinitionsCreateBodyShortNameMax = 40
+
export const llmAnalyticsScoreDefinitionsCreateBodyArchivedDefault = false
export const llmAnalyticsScoreDefinitionsCreateBodyConfigOneOneOptionsItemKeyMax = 128
@@ -2955,6 +2957,11 @@ export const LlmAnalyticsScoreDefinitionsCreateBody = () => zod.object({
.optional()
.describe('Tag names. Replaces existing tags; send [] to remove all tags. Names are trimmed and lowercased.'),
name: zod.string().max(llmAnalyticsScoreDefinitionsCreateBodyNameMax).describe('Human-readable scorer name.'),
+ short_name: zod
+ .string()
+ .max(llmAnalyticsScoreDefinitionsCreateBodyShortNameMax)
+ .optional()
+ .describe('Optional compact label. Blank uses the full name.'),
description: zod.string().nullish().describe('Optional human-readable description.'),
kind: zod
.enum(['categorical', 'numeric', 'boolean'])
@@ -3093,6 +3100,8 @@ export const llmAnalyticsScoreDefinitionsPartialUpdateBodyTagsMax = 100
export const llmAnalyticsScoreDefinitionsPartialUpdateBodyNameMax = 255
+export const llmAnalyticsScoreDefinitionsPartialUpdateBodyShortNameMax = 40
+
export const LlmAnalyticsScoreDefinitionsPartialUpdateBody = () => zod.object({
tags: zod
.array(zod.string().max(llmAnalyticsScoreDefinitionsPartialUpdateBodyTagsItemMax))
@@ -3104,6 +3113,11 @@ export const LlmAnalyticsScoreDefinitionsPartialUpdateBody = () => zod.object({
.max(llmAnalyticsScoreDefinitionsPartialUpdateBodyNameMax)
.optional()
.describe('Updated scorer name.'),
+ short_name: zod
+ .string()
+ .max(llmAnalyticsScoreDefinitionsPartialUpdateBodyShortNameMax)
+ .optional()
+ .describe('Updated compact label. Blank uses the full name.'),
description: zod.string().nullish().describe('Updated scorer description.'),
archived: zod.boolean().optional().describe('Whether the scorer is archived.'),
})
@@ -3117,6 +3131,8 @@ export const LlmAnalyticsScoreDefinitionsNewVersionCreateParams = () => zod.obje
),
})
+export const llmAnalyticsScoreDefinitionsNewVersionCreateBodyShortNameMax = 40
+
export const llmAnalyticsScoreDefinitionsNewVersionCreateBodyNameMax = 255
export const llmAnalyticsScoreDefinitionsNewVersionCreateBodyConfigOneOneOptionsItemKeyMax = 128
@@ -3126,6 +3142,11 @@ export const llmAnalyticsScoreDefinitionsNewVersionCreateBodyConfigOneOneOptions
export const llmAnalyticsScoreDefinitionsNewVersionCreateBodyConfigOneOnePassingRuleOneCategoriesItemMax = 128
export const LlmAnalyticsScoreDefinitionsNewVersionCreateBody = () => zod.object({
+ short_name: zod
+ .string()
+ .max(llmAnalyticsScoreDefinitionsNewVersionCreateBodyShortNameMax)
+ .optional()
+ .describe('Updated compact label, saved with this version.'),
name: zod
.string()
.max(llmAnalyticsScoreDefinitionsNewVersionCreateBodyNameMax)
diff --git a/services/mcp/src/tools/generated/ai_observability.ts b/services/mcp/src/tools/generated/ai_observability.ts
index d92b87f0867c..c31d051b7f5d 100644
--- a/services/mcp/src/tools/generated/ai_observability.ts
+++ b/services/mcp/src/tools/generated/ai_observability.ts
@@ -2661,6 +2661,9 @@ const llmaScoreDefinitionCreate = (): ToolBase<
if (params.name !== undefined) {
body['name'] = params.name
}
+ if (params.short_name !== undefined) {
+ body['short_name'] = params.short_name
+ }
if (params.description !== undefined) {
body['description'] = params.description
}
@@ -2754,6 +2757,9 @@ const llmaScoreDefinitionNewVersion = (): ToolBase<
handler: async (context: Context, params: z.infer<ReturnType<typeof LlmaScoreDefinitionNewVersionSchema>>) => {
const projectId = await context.stateManager.getProjectId()
const body: Record<string, unknown> = {}
+ if (params.short_name !== undefined) {
+ body['short_name'] = params.short_name
+ }
if (params.name !== undefined) {
body['name'] = params.name
}
@@ -2799,6 +2805,9 @@ const llmaScoreDefinitionUpdate = (): ToolBase<
if (params.name !== undefined) {
body['name'] = params.name
}
+ if (params.short_name !== undefined) {
+ body['short_name'] = params.short_name
+ }
if (params.description !== undefined) {
body['description'] = params.description
}
diff --git a/services/mcp/tests/unit/__snapshots__/tool-schemas/llma-score-definition-create.json b/services/mcp/tests/unit/__snapshots__/tool-schemas/llma-score-definition-create.json
index 33715433a169..61a182ec397e 100644
--- a/services/mcp/tests/unit/__snapshots__/tool-schemas/llma-score-definition-create.json
+++ b/services/mcp/tests/unit/__snapshots__/tool-schemas/llma-score-definition-create.json
@@ -175,6 +175,11 @@
"maxLength": 255,
"type": "string"
},
+ "short_name": {
+ "description": "Optional compact label. Blank uses the full name.",
+ "maxLength": 40,
+ "type": "string"
+ },
"tags": {
"description": "Tag names. Replaces existing tags; send [] to remove all tags. Names are trimmed and lowercased.",
"items": {
diff --git a/services/mcp/tests/unit/__snapshots__/tool-schemas/llma-score-definition-list.json b/services/mcp/tests/unit/__snapshots__/tool-schemas/llma-score-definition-list.json
index 347acb78842e..304212dffbc9 100644
--- a/services/mcp/tests/unit/__snapshots__/tool-schemas/llma-score-definition-list.json
+++ b/services/mcp/tests/unit/__snapshots__/tool-schemas/llma-score-definition-list.json
@@ -26,7 +26,7 @@
"type": "string"
},
"search": {
- "description": "Search scorers by name or description.",
+ "description": "Search scorers by name, short name, or description.",
"type": "string"
},
"tag_match": {
diff --git a/services/mcp/tests/unit/__snapshots__/tool-schemas/llma-score-definition-new-version.json b/services/mcp/tests/unit/__snapshots__/tool-schemas/llma-score-definition-new-version.json
index 24dd2f4839bc..77567a527cff 100644
--- a/services/mcp/tests/unit/__snapshots__/tool-schemas/llma-score-definition-new-version.json
+++ b/services/mcp/tests/unit/__snapshots__/tool-schemas/llma-score-definition-new-version.json
@@ -173,6 +173,11 @@
"description": "Updated scorer name, saved with this version.",
"maxLength": 255,
"type": "string"
+ },
+ "short_name": {
+ "description": "Updated compact label, saved with this version.",
+ "maxLength": 40,
+ "type": "string"
}
},
"required": ["id", "config"],
diff --git a/services/mcp/tests/unit/__snapshots__/tool-schemas/llma-score-definition-update.json b/services/mcp/tests/unit/__snapshots__/tool-schemas/llma-score-definition-update.json
index 4291be7bc6d6..ad77f8c12971 100644
--- a/services/mcp/tests/unit/__snapshots__/tool-schemas/llma-score-definition-update.json
+++ b/services/mcp/tests/unit/__snapshots__/tool-schemas/llma-score-definition-update.json
@@ -25,6 +25,11 @@
"maxLength": 255,
"type": "string"
},
+ "short_name": {
+ "description": "Updated compact label. Blank uses the full name.",
+ "maxLength": 40,
+ "type": "string"
+ },
"tags": {
"description": "Tag names. Replaces existing tags; send [] to remove all tags. Names are trimmed and lowercased.",
"items": {