[agent-performance] Documentation workflows: 0% PR merge rate (56 PRs closed without merge in 7 days) #16166
Description
Problem
Documentation-focused agent workflows created 56 PRs in the past 7 days with a 0% merge rate (all closed without merging). This represents significant wasted effort and indicates either quality issues, timing problems, or coordination failures.
Affected Workflows:
- Daily Doc Updater
- Instructions Janitor
- Documentation Unbloat
- Workflow Normalizer
- Glossary updates (weekly full scan)
- Spec updates (layout specification)
- Actions version updates
- Other documentation automation
Previous Period: 47 PRs with 0% merge rate (continuing pattern)
Data Sources:
- GitHub Search API (PRs created by github-actions, Feb 9-16)
- Agent Performance Report (week of Feb 9-16, 2026)
Impact
Severity: High
Scope: 8+ documentation workflows, 100+ PRs in 2 weeks
Effects:
- Wasted agent resources: Significant compute/API quota used with no value
- Reviewer fatigue: High PR volume overwhelming maintainers
- Missed improvements: Valid documentation changes not being applied
- Ecosystem inefficiency: Low ROI on documentation automation
- Agent effectiveness: 0% merge rate = 0% effectiveness
Cost estimate:
- 56 PRs × ~5 min/PR = ~4.5 hours agent time wasted in 7 days
- ~600+ hours annually if pattern continues
Root Cause Analysis
Hypothesis 1: Timing Issues (Most Likely)
- Many PRs created very recently (within 24-48 hours of this analysis)
- Not enough time for maintainer review
- May be closed before review due to superseding changes
Hypothesis 2: Manual Updates Supersede Agent PRs
- Maintainers apply fixes manually before reviewing agent PRs
- Agent PR becomes obsolete and gets closed
- Lack of coordination between manual and automated updates
Hypothesis 3: Volume Overwhelm
- 8+ overlapping documentation workflows
- ~8 PRs/day from documentation agents
- Too many small PRs instead of batched changes
- Reviewer bandwidth insufficient for volume
Hypothesis 4: Quality/Relevance Issues
- Agent changes may not align with maintainer priorities
- Changes may be too minor or cosmetic
- Updates may conflict with ongoing work
- PR descriptions may lack sufficient context
Hypothesis 5: Duplication
- Multiple workflows editing same files
- Conflicting changes between agents
- PRs superseded by other agent PRs
Investigation needed to determine which hypothesis explains the pattern.
Proposed Investigation Plan
Phase 1: PR Closure Analysis (2-3 hours)
-
Sample 20 closed PRs from the past 2 weeks:
- Categorize closure reasons (from PR comments/timeline)
- Identify common patterns
- Check for manual superseding changes
- Review PR quality and descriptions
-
Analyze PR content:
- What files are being changed most frequently?
- Are changes substantive or cosmetic?
- Do changes align with project priorities?
- Are multiple agents editing same files?
-
**Review timing (redacted)
- Time from PR creation to closure
- Time from PR creation to first review (if any)
- Compare with typical PR review times
- Check if PRs are closed in batches
Phase 2: Workflow Overlap Analysis (1-2 hours)
-
Map documentation workflows:
- Daily Doc Updater: Updates what?
- Instructions Janitor: Updates what?
- Documentation Unbloat: Updates what?
- Workflow Normalizer: Updates what?
- Glossary/Spec/Actions updates: Overlap?
-
Identify file-level conflicts:
- Which files are modified by multiple workflows?
- Do workflows coordinate or conflict?
- Are changes complementary or duplicate?
-
Analyze execution patterns:
- Daily vs. weekly schedules
- Triggers and conditions
- Coordination mechanisms (if any)
Phase 3: Maintainer Feedback (1 hour)
- Review recent PR comments for maintainer feedback
- Check for closure patterns (manual close, bot close, auto-close)
- Identify stated reasons for not merging
- Look for feature requests or improvement suggestions
Proposed Solutions
Solution 1: Consolidate Documentation Workflows (Recommended)
Problem addressed: Volume overwhelm, duplication, coordination
Approach:
- Merge 8+ documentation workflows into 2:
- Weekly Documentation Scan: Comprehensive updates once per week
- Daily Critical Updates: Only urgent documentation fixes
Benefits:
- Reduce PR volume by ~5-7x
- Eliminate duplication
- Batch related changes together
- Easier for maintainers to review
Estimated effort: 6-8 hours implementation
Solution 2: Improve Coordination with Manual Updates
Problem addressed: Manual superseding
Approach:
- Add checks for recent manual commits before creating PR
- Scan for open PRs touching same files
- Add "superseded by" detection
- Auto-close own PR if manual fix applied
Benefits:
- Reduce obsolete PRs
- Better coordination
- Less reviewer overhead
Estimated effort: 4-6 hours implementation
Solution 3: Batch Changes in Comprehensive PRs
Problem addressed: Volume overwhelm, fragmentation
Approach:
- Instead of many small PRs, create one comprehensive PR
- Group related changes (e.g., all glossary updates in one PR)
- Weekly batch runs instead of daily
- More context in PR descriptions
Benefits:
- Fewer PRs to review (8/day → 1-2/week)
- Better context for reviewers
- More likely to get attention
Estimated effort: 3-4 hours per workflow
Solution 4: Add Quality Gates and Prioritization
Problem addressed: Quality/relevance issues
Approach:
- Only create PR if changes meet threshold (e.g., >5 meaningful changes)
- Prioritize high-impact changes
- Add validation/testing before PR creation
- Improve PR descriptions with impact analysis
Benefits:
- Higher quality PRs
- Better aligned with priorities
- More likely to merge
Estimated effort: 4-5 hours implementation
Solution 5: Implement Deduplication Checks
Problem addressed: Duplication
Approach:
- Check for existing open PRs before creating new one
- Detect similar changes
- Consolidate into existing PR instead of creating new
- Add workflow coordination metadata
Benefits:
- Eliminate duplicate PRs
- Cleaner PR list
- Better tracking
Estimated effort: 2-3 hours implementation
Recommended Approach
Primary recommendation: Combine Solutions 1, 2, and 3
-
Consolidate workflows (Solution 1)
- Merge to 2 workflows (weekly + daily critical)
- Eliminate 6+ redundant workflows
- Timeline: Week 1-2
-
Batch changes (Solution 3)
- Weekly comprehensive PRs instead of daily small PRs
- Group related changes together
- Timeline: Week 2
-
Add coordination checks (Solution 2)
- Check for manual updates before creating PR
- Scan for conflicting open PRs
- Timeline: Week 3
-
Monitor and validate (4 weeks)
- Track merge rate improvement
- Measure reviewer satisfaction
- Adjust based on feedback
- Timeline: Weeks 4-7
Expected outcome:
- Merge rate: 0% → 50-70% (10-20x improvement)
- PR volume: ~8/day → 1-2/week (5-7x reduction)
- Reviewer overhead: ~30 min/day → ~20 min/week (10x reduction)
- Agent effectiveness: Significant improvement
Acceptance Criteria
- Investigation Phase 1-3 completed with findings documented
- Root causes identified and validated
- Solution approach selected and approved
- Implementation completed
- 4-week validation period completed
- Merge rate improved to > 50% (from 0%)
- PR volume reduced by > 50% (from ~8/day)
- No regression in documentation quality
- Maintainer feedback collected and addressed
Estimated Effort
Investigation: 4-6 hours
Implementation: 12-16 hours (depending on solution complexity)
Validation: 4 weeks monitoring
Total: 16-22 hours + validation period
Metrics to Track
Before (Baseline):
- Merge rate: 0%
- PRs created/week: 56
- PRs merged/week: 0
- Agent effectiveness: 0/100
After (Target):
- Merge rate: > 50%
- PRs created/week: < 15
- PRs merged/week: > 7
- Agent effectiveness: > 70/100
Additional Context
From Agent Performance Report:
Documentation PR Merge Rate - Critical Finding: 56 PRs created by agent workflows in past 7 days with 0% merge rate. Previous period had 47 PRs with 0% merge rate. Continuing pattern of low PR acceptance.
Comparison with other workflows:
- Code review PRs: Unknown merge rate (needs analysis)
- Refactoring PRs: Small sample but higher engagement
- Documentation PRs: 0% merge rate (critical issue)
References
- Agent Performance Report: Discussion (to be linked)
- Sample PRs: [docs] Update glossary - weekly full scan #16083 (glossary), [actions] Update GitHub Actions versions - February 16, 2026 #16067 (actions), [specs] Update layout specification - 2026-02-16 #16056 (specs), [docs] Update documentation for logs command and skip-bots/skip-roles features #16014 (docs), [cloclo] Fix locked error check: require 403 status AND locked message #15980 (cloclo)
- Analysis Run: §22072249270
Generated by Agent Performance Analyzer - Meta-Orchestrator
- expires on Feb 18, 2026, 5:40 PM UTC