141 lines
3.8 KiB
Markdown
141 lines
3.8 KiB
Markdown
---
|
|
name: paper-parsing
|
|
description: Use when analyzing ML/DL papers to ensure comprehensive extraction of all relevant information
|
|
---
|
|
|
|
# Paper Parsing Methodology
|
|
|
|
## Overview
|
|
|
|
Systematic approach to parsing ML/DL papers for replication. Emphasizes **completeness** and **openness** to avoid missing critical details.
|
|
|
|
**Announce at start:** "I'm using the paper-parsing skill to ensure comprehensive paper analysis."
|
|
|
|
## Core Philosophy
|
|
|
|
1. **Completeness over speed**: Better to extract too much than miss something
|
|
2. **Open-ended discovery**: Papers contain unique insights; don't force into templates
|
|
3. **Cross-reference**: Information appears in multiple places; cross-check
|
|
4. **Explicit uncertainty**: Mark unclear items rather than guessing
|
|
|
|
## Paper Sections Checklist
|
|
|
|
### Abstract
|
|
- [ ] Core contribution identified
|
|
- [ ] Key results/numbers extracted
|
|
- [ ] Problem domain understood
|
|
|
|
### Introduction
|
|
- [ ] Problem motivation clear
|
|
- [ ] Gap in existing work identified
|
|
- [ ] Proposed solution summarized
|
|
- [ ] Claimed contributions listed
|
|
|
|
### Related Work
|
|
- [ ] Key prior methods identified
|
|
- [ ] Differences from this work noted
|
|
- [ ] Potential baselines for comparison
|
|
|
|
### Method / Approach
|
|
- [ ] Architecture fully described
|
|
- [ ] All components identified
|
|
- [ ] Mathematical formulation complete
|
|
- [ ] Training procedure detailed
|
|
- [ ] Loss functions specified
|
|
- [ ] Hyperparameters listed
|
|
|
|
### Experiments
|
|
- [ ] Datasets listed with sizes
|
|
- [ ] Evaluation metrics defined
|
|
- [ ] Baseline comparisons noted
|
|
- [ ] Ablation studies cataloged
|
|
- [ ] Key numerical results extracted
|
|
|
|
### Appendix / Supplementary
|
|
- [ ] Additional implementation details
|
|
- [ ] Extended results
|
|
- [ ] Proofs or derivations
|
|
- [ ] Code references
|
|
|
|
## Information Extraction Patterns
|
|
|
|
### Architecture Details
|
|
|
|
Look for:
|
|
- Layer types and configurations
|
|
- Activation functions
|
|
- Normalization methods
|
|
- Attention mechanisms
|
|
- Skip connections
|
|
- Input/output dimensions
|
|
|
|
Common locations:
|
|
- Method section figures
|
|
- Architecture diagrams
|
|
- Table of hyperparameters
|
|
- Appendix implementation details
|
|
|
|
### Training Configuration
|
|
|
|
| Parameter | Typical Locations |
|
|
|-----------|-------------------|
|
|
| Learning rate | Experiments, Appendix |
|
|
| Batch size | Experiments, Appendix |
|
|
| Optimizer | Method, Appendix |
|
|
| Epochs | Experiments |
|
|
| Hardware | Experiments, Appendix |
|
|
| Training time | Experiments |
|
|
|
|
### Numerical Results
|
|
|
|
Extract from:
|
|
- Main results tables
|
|
- Comparison figures
|
|
- Ablation tables
|
|
- Training curves (approximate values)
|
|
|
|
Format as:
|
|
| Metric | Dataset | Value | Conditions |
|
|
|--------|---------|-------|------------|
|
|
| Accuracy | CIFAR-10 | 95.2% | ResNet-50 backbone |
|
|
|
|
## Common Omissions to Watch For
|
|
|
|
1. **Initialization**: Often in appendix or not mentioned
|
|
2. **Data augmentation**: May be standard but unspecified
|
|
3. **Early stopping criteria**: Often implied
|
|
4. **Evaluation protocol**: Train/val/test split details
|
|
5. **Random seeds**: Reproducibility details
|
|
6. **Software versions**: PyTorch, CUDA versions
|
|
|
|
## Quality Verification
|
|
|
|
Before completing analysis:
|
|
|
|
1. **Coverage check**: Every section reviewed?
|
|
2. **Consistency check**: Numbers match across sections?
|
|
3. **Completeness check**: Could someone implement from this?
|
|
4. **Ambiguity check**: Unclear items marked?
|
|
|
|
## Output Quality Markers
|
|
|
|
Good analysis:
|
|
- Specific numbers, not "good performance"
|
|
- Exact layer configs, not "standard ResNet"
|
|
- Explicit uncertainty markers
|
|
- Cross-references between sections
|
|
|
|
Poor analysis:
|
|
- Vague descriptions
|
|
- Missing hyperparameters
|
|
- No numerical targets
|
|
- Assumptions without noting them
|
|
|
|
## Red Flags
|
|
|
|
If you notice:
|
|
- "Implementation details in code" → Check GitHub link
|
|
- "Standard settings" → Look up the standard
|
|
- "Following [citation]" → May need to read that paper
|
|
- Inconsistent numbers → Note the discrepancy
|