Treat shell checks without expectations as harness errors

[luohui1]

Summary

• detect shell checks missing all expect_* keys before evaluating command output
• return error: true with a misconfiguration hint instead of an ordinary (no expectation) FAIL
• add a regression suite and document it in the verified test list

Closes #8.

Tests

• PATH=/c/tmp/bin:$PATH bash scripts/eval/tests/shell_no_expectation.sh
• PATH=/c/tmp/bin:$PATH bash scripts/eval/tests/shell_safety.sh
• PATH=/c/tmp/bin:$PATH bash -n scripts/eval/lib/score.sh
• PATH=/c/tmp/bin:$PATH bash -n scripts/eval/tests/shell_no_expectation.sh
• git diff --cached --check -- README.md scripts/eval/lib/score.sh scripts/eval/tests/shell_no_expectation.sh

[gemini-code-assist]

Code Review

This pull request introduces a validation check in the shell scoring logic to treat missing expectation fields as harness errors, accompanied by a new test suite and documentation updates. The review feedback recommends replacing a fragile grep-based YAML check with a robust check using already parsed variables in Bash, which avoids false positives and extra process spawning.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

[gemini-code-assist]

Using grep to parse YAML files is fragile and prone to false positives/negatives. For example, if the cmd field itself contains a multi-line string with the text expect_regex:, grep will incorrectly match it. Since you have already parsed and populated expect_regex, expect_min, and expect_exact using yq on lines 66-68, you can simply check if all of these variables are empty using Bash's built-in string checks. This is more robust, avoids spawning an extra grep process, and correctly handles YAML formatting variations.

Suggested change

	if ! grep -qE '^[[:space:]]*expect_(regex|min|exact)[[:space:]]*:' "$check_file"; then
	if [[ -z "$expect_regex" && -z "$expect_min" && -z "$expect_exact" ]]; then