refactor: improve verification workflow with visual comparison

Major changes:
- paper-image-extractor: Generate reference_plots.py for visual verification
- paper-director: Add image understanding checkpoint with side-by-side comparison
- paper-analyzer: Add data source labeling with reliability levels
- code-writer: Change from TDD to VDD (Verification-Driven Development)
- test-runner: Generate comparison reports with images and explanations
- verification skill: Add difference classification system
- code-generation skill: Emphasize result independence

Key principles:
- Code results are authoritative, paper values are references
- Differences are expected and documented, not bugs to fix
- Visual comparison prioritized over exact numerical match
- Tests verify sanity (shape, gradient, range), not exact values
This commit is contained in:
hc
2026-03-31 19:55:36 +08:00
parent db731f6745
commit 5d5aee1f83
7 changed files with 683 additions and 270 deletions
+32
View File
@@ -17,6 +17,36 @@ Guidelines for translating paper descriptions into working PyTorch code.
2. **Testability**: Write code that can be unit tested
3. **Readability**: Prefer clarity over cleverness
4. **Modularity**: One component per file
5. **Independence**: Code logic based on paper methodology, NOT reverse-engineered from expected outputs
## Critical: Result Independence
The code must implement the **paper's described method**, not be reverse-engineered to match reference values.
### DO NOT:
```python
# WRONG: Using values from reference_plots.py as targets
expected_accuracy = 0.952 # Copied from paper figure
assert abs(accuracy - expected_accuracy) < 0.01 # This defeats the purpose
```
### DO:
```python
# CORRECT: Implement the method, let results be what they are
# Paper Section 4.1: "We use Adam with lr=1e-4"
optimizer = torch.optim.Adam(model.parameters(), lr=1e-4)
# Run training, record actual results
accuracy = evaluate(model, test_loader)
# This accuracy is authoritative - compare with paper in report
```
### Reference Values Are For Comparison Only
Values from `image_understanding.md` and `reference_plots.py` should:
- Be used in the **final report** for comparison
- **NOT** be used as assertion targets in tests
- **NOT** influence implementation decisions
## Paper-to-Code Mapping
@@ -199,3 +229,5 @@ Before completing a module:
- [ ] Example in docstring works
- [ ] No hardcoded dimensions (use params)
- [ ] Gradient flow verified (no in-place ops breaking autograd)
- [ ] **No reference values hardcoded as expected outputs**
- [ ] **Implementation based on paper method, not reverse-engineered from results**