Skip to content

Multi-tier framework for evaluating AI agent skills and plugins with quality gates, security scanning, harness compatibility checks, semantic overlap detection, synthetic evaluation dataset generation, and live agent evaluation that measures how skills affect agent behavior. Ships reusable CI templates that bring every check to any repository.

Fix empty process-control environment resets (#133)

RyaliNvidia7e189c6bdada
diff --git a/CHANGELOG.md b/CHANGELOG.md
index e3746e20..dbdb43ab 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -19,6 +19,10 @@ All notable changes to SkillEvaluator are documented in this file.
### Fixed
+- Tier 3 local mode now drops evaluator-managed empty process-loader resets
+ while continuing to reject non-empty loader overrides, allowing generated
+ tasks to reach agent execution
+ ([#132](https://github.com/NVIDIA/SkillEvaluator/issues/132)).
- Unpinned-dependency warnings are no longer suppressed by comparison
operators inside PEP 508 environment markers; requirements such as
`pkg; python_version < "3.13"` are now correctly reported, while direct
diff --git a/src/skillevaluator/tier3/harbor/local_environment.py b/src/skillevaluator/tier3/harbor/local_environment.py
index 9be41907..d4da8334 100644
--- a/src/skillevaluator/tier3/harbor/local_environment.py
+++ b/src/skillevaluator/tier3/harbor/local_environment.py
@@ -1013,6 +1013,9 @@ class SkillEvaluatorLocalEnvironment(BaseEnvironment):
for key, value in env.items():
normalized = key.upper()
if normalized in _BLOCKED_COMMAND_ENV_NAMES or normalized.startswith(_BLOCKED_COMMAND_ENV_PREFIXES):
+ if value == "":
+ # The adapter emits empty process-control values to clear inherited loader configuration.
+ continue
raise ValueError(
f"environment variable {key} can execute or alter code before confinement and is not allowed"
)
diff --git a/tests/test_harbor_local_mode.py b/tests/test_harbor_local_mode.py
index cdda2329..d679ed38 100644
--- a/tests/test_harbor_local_mode.py
+++ b/tests/test_harbor_local_mode.py
@@ -1974,6 +1974,15 @@ def test_runtime_injection_env_is_blocked_before_launcher(name: str, tmp_path: P
assert name in (result.stderr or "")
[email protected]("name", ["BASH_ENV", "ENV", "LD_PRELOAD", "PYTHONPATH"])
+def test_empty_runtime_injection_env_is_dropped(name: str) -> None:
+ assert SkillEvaluatorLocalEnvironment._filter_command_env({name: ""}, protected=set()) == {}
+
+
+def test_unblocked_empty_runtime_env_is_preserved() -> None:
+ assert SkillEvaluatorLocalEnvironment._filter_command_env({"EXAMPLE": ""}, protected=set()) == {"EXAMPLE": ""}
+
+
@pytest.mark.skipif(os.name == "nt", reason=_NATIVE_WINDOWS_LOCAL_REASON)
def test_persistent_runtime_injection_env_is_blocked_before_launcher(tmp_path: Path) -> None:
environment = _local_environment(tmp_path, persistent_env={"NODE_OPTIONS": "--require=/tmp/attack.js"})