hc
ced50ea2b0
Root cause: test-runner was giving overly optimistic results due to: 1. Context bias - knew the implementation, tended to defend it 2. No actual visual comparison - just wrote 'ACCEPTABLE' without looking 3. No structural validation - accepted 35x scale differences as 'acceptable' Solution: - New result-verifier agent that performs blind visual comparison - Strict pass/fail criteria for structural validation - Updated test-runner to use result-verifier for each figure - Clear guidelines: structural mismatches = FAIL, not ACCEPTABLE Test result: verifier correctly identified Fig3 as FAIL with 7 specific issues: - Wrong X-axis variable (channels vs power) - Wrong Y-axis scale (5x difference) - Wrong curve count (5 vs 4) - etc.
25 lines
442 B
JSON
25 lines
442 B
JSON
{
|
|
"$schema": "https://opencode.ai/config.json",
|
|
"default_agent": "paper-director",
|
|
"agent": {
|
|
"paper-director": {
|
|
"mode": "primary"
|
|
},
|
|
"paper-analyzer": {
|
|
"mode": "subagent"
|
|
},
|
|
"paper-image-extractor": {
|
|
"mode": "subagent"
|
|
},
|
|
"code-writer": {
|
|
"mode": "subagent"
|
|
},
|
|
"test-runner": {
|
|
"mode": "subagent"
|
|
},
|
|
"result-verifier": {
|
|
"mode": "subagent"
|
|
}
|
|
}
|
|
}
|