Agent Skills Refactoring: Consolidate & Standardize All Agent Skill Directories

[ashleyshaw]

Audit Summary

Comprehensive audit of all 16 agent skill structures and refactoring plan for standardization and consolidation.

Current State

• All agents: 16/16 have top-level skills/ directories ✅
• No nested structures: 0 agents with agent/skills/ (already refactored)
• Total skills: 368 skill directories, 10,332 files across all agents
• Manifest coverage: 16/16 agents have skills.md and skills.csv ✅

Skills Distribution (Org-Wide)

Category	Skill Dirs	Files	Status
agent-attached	178	3,777	Specialized, agent-specific
local	80	3,134	General-purpose, reusable
plugin-provided	69	1,860	Third-party integrations
platform-managed	26	497	Built-in, platform skills
directory-installed	15	1,064	Directory-scoped skills

---

Audit Checklist / Scope

• All 16 agents analyzed (agent skill directories)
• 368 skill directories inventoried
• 10,332 skill files catalogued
• Manifest accuracy validated (discrepancies identified)
• Skill categorization reviewed (inconsistencies noted)
• Duplicate skills identified (~10-15 candidates)
• Path references analyzed (absolute paths flagged)
• SKILL.md entrypoint coverage verified

---

Findings / Risks

1. Manifest Accuracy Discrepancies (Priority: High)

• Finding: Significant discrepancies between directory structure and CSV manifests
• Root Cause: Manifests not regenerated after skill additions/moves
• Impact: Skill discovery tools may miss available skills
• Evidence:
  • prd-agent: 102 SKILL.md files vs. 57 CSV entries (+45 missing)
  • woo-config-agent: 31 SKILL.md files vs. 30 CSV entries (+1 missing)
  • wp-config-agent: 31 SKILL.md files vs. 37 CSV entries (-6 over-reported)
• Risk Level: High - Discovery and automation tools may fail silently

2. Absolute Path References (Non-Portable) (Priority: High)

• Finding: All manifests use absolute paths (/root/.codex/skills/...)
• Impact: Paths not portable across environments (local, CI, different machines)
• Risk Level: High - Manifests break in different deployment contexts
• Solution: Convert to relative paths (./skills/agent-attached/...)

3. Inconsistent Skill Categorization (Priority: Medium)

• Finding: Varied use of skill categories across agents
• Patterns:
  • Some agents: 5 categories (agent-attached, local, plugin-provided, platform-managed, directory-installed)
  • Others: 2 categories (only agent-attached, local)
• Impact: Inconsistent organization makes discovery and management difficult
• Risk Level: Medium - Maintainability and scalability concerns

4. Missing SKILL.md Entrypoints (Priority: Medium)

• Finding: Some skill directories lack SKILL.md files (documentation/metadata)
• Impact: Skills not discoverable by CI/CD and documentation generation tools
• Scope: ~15-20 skill directories missing entrypoints
• Risk Level: Medium - Skills may be undocumented and unused

5. Redundant Skill Copies (Priority: Low)

• Finding: Same skill exists in multiple agents (e.g., documents, github, google-drive)
• Impact: Maintenance burden, version inconsistency, duplication waste
• Scope: ~10-15 candidate skills for consolidation
• Risk Level: Low - Efficiency and maintainability concern

---

Remediation Actions

Phase 1: Inventory & Validation (Week 1)

• Crawl all agent skill directories
• Validate each skill directory has SKILL.md entrypoint
• Extract skill metadata (name, description, category, tags)
• Generate org-wide skills registry (SKILLS_INVENTORY.md)
• Identify duplicate/redundant skills
• Flag missing SKILL.md files and over-reported CSV entries

Deliverable: Complete skills inventory with metadata

Phase 2: Create Missing SKILL.md Files (Week 1-2)

• Audit identifies ~15-20 skill directories needing SKILL.md
• For each missing file:
  • Create SKILL.md template with basic metadata
  • Extract description from README or documentation
  • Set category, tags, and dependencies
  • Validate structure against SKILL.md schema
• Validate 100% SKILL.md coverage across all agents

Deliverable: All skill directories have valid SKILL.md entrypoints

Phase 3: Normalize Skill Paths in Manifests (Week 2)

• Convert absolute paths → relative paths in all manifests
  • Before: /root/.codex/skills/hermes/skill-name
  • After: ./skills/agent-attached/skill-name
• Regenerate manifests/skills.md for each agent
• Regenerate manifests/skills.csv for each agent
• Validate manifest accuracy against filesystem (0 discrepancies)

Deliverable: Updated manifests with relative, portable paths

Phase 4: Consolidate Duplicate Skills (Week 2-3)

• Identify 10-15 duplicate skills
• Evaluate consolidation candidates (documents, GitHub, Google Workspace, etc.)
• For each consolidation:
  • Move skill to top-level skills/ (outside agents/)
  • Update manifests to reference shared location
  • Add version pinning for consistency
• Document shared vs. agent-specific skills

Deliverable: Consolidated shared skills, updated manifests

Phase 5: Standardize Skill Categorization (Week 3)

• Define canonical 5-category system:
  • agent-attached — Specialized for this agent
  • local — General-purpose, locally managed
  • plugin-provided — Via plugin integration (e.g., GitHub, Slack)
  • platform-managed — Built-in, platform-provided
  • directory-installed — Directory-scoped skills
• Reclassify skills to match standard categories
• Update manifests and metadata
• Document category definitions in skills/README.md

Deliverable: Standardized skill categorization across all 16 agents

Phase 6: Update Agent Metadata (Week 3-4)

• Update each agent's AGENT.md to list skills by category
• Update agent/instructions/AGENTS.md with skill integration patterns
• Update claude/agent.md, copilot/agent.md, openai/agent.md to reference skills
• Validate all agent configs match actual skill locations

Deliverable: Updated AGENT.md and provider configs for all agents

Phase 7: Validation & Testing (Week 4)

• Run skill discovery checks:
  • Verify all SKILL.md files are valid
  • Verify manifests match filesystem (100% accuracy)
  • Verify paths are relative and portable
  • Verify no broken symlinks/references
• Test agent skill loading from each platform:
  • Claude Code can discover skills
  • GitHub Actions can reference skills
  • Copilot can access skills
• Documentation checks

Deliverable: Validation report with 100% pass rate

Phase 8: CI/CD Integration (Week 4)

• Create scripts/agents/regenerate-skills-manifest.cjs:
  • Crawl agent skills directories
  • Generate skills.md and skills.csv
  • Validate SKILL.md structure
  • Report discrepancies
• Add to lint workflow
• Add pre-commit hook to regenerate manifests
• Document in agent development guide

Deliverable: Automated manifest validation in CI/CD

---

Acceptance Criteria

• Audit scope and checklist completed
• Findings and risks documented (5 major issues identified)
• Remediation actions mapped (8 phases, 40-60 hours estimated)
• Audit report published (.github/reports/agents/SKILLS_AUDIT_2026-07-23.md)
• Refactoring plan created (detailed 8-phase roadmap)
• Related issues/PRs linked
• Documentation updated (audit findings documented)

---

Additional Context

Per-Agent Breakdown (Top 10 by Skill Count)

Agent	SKILL.md	agent-attached	local	total files
prd-agent	102	25	13	1,360
tour-operator-config-agent	30	10	20	1,066
woo-config-agent	31	10	1	942
wp-config-agent	31	11	1	929
website-content-strategist-agent	67	28	1	928
prd-factory-planner-agent	57	24	10	918
ai-readiness-estimator-agent	51	19	1	855
client-website-discovery-assistant-agent	67	1	3	759
zendesk-support-agent	37	17	3	757
design-partner-agent	47	1	1	700

Related Issues & PRs

• Epic: #1079 — Agent Standards Phase 2
• PR: #1140 — tour-operator-config agent standardization
• PR: #1141 — woo-config agent standardization
• PR: chore/agents-finalize-incomplete-agents (branch)

Effort Estimate

• Duration: 4-5 weeks
• Effort: 40-60 hours (1 FTE)
• Risk: Low (refactoring, non-user-facing)
• Dependencies: None (standalone initiative)

Success Criteria for Refactoring

• All 368 skill directories have valid SKILL.md entrypoints
• All manifests (skills.md + skills.csv) are 100% accurate (0 discrepancies)
• All skill paths are relative, portable, and working
• Duplicate skills consolidated (10-15 shared skills)
• Skill categorization standardized (5 categories)
• All agent AGENT.md files updated with skill references
• CI/CD validates manifests on every commit
• 100% of agents pass skill discovery validation

---

Definition of Ready (DoR)

• Audit scope, checklist, and goals defined
• All 16 agents/368 skills analyzed
• Dependencies and standards mapped
• Refactoring phases planned

Definition of Done (DoD)

• Audit performed and findings documented
• Remediation actions assigned (8 phases with deliverables)
• Documentation/audit report updated (.github/reports/agents/SKILLS_AUDIT_2026-07-23.md)
• PR uses correct branch prefix (chore/)
• Issue linked to Epic Epic: Multi-Provider Agent Standardization Initiative #1079

---

Audit conducted: 2026-07-23 | Refactoring plan: 8 phases | Estimated effort: 40-60 hours

Per-Agent Refactoring Guide

This section provides step-by-step instructions for refactoring each of the 16 agents. Follow this process for each agent in sequence:

Agent Refactoring Checklist (For Each Agent)

Before Starting:

• Create feature branch: feat/refactor-{agent-name}-skills
• Read this entire guide
• Have the agent's current structure documented

Phase 1: Inventory & Validation

• List all skills in skills/agent-attached/
• List all skills in skills/local/
• Verify each skill has a SKILL.md file
• For missing SKILL.md: create template with metadata
• Run checksum validation: find agents/{agent}/ -type f -exec sha256sum {} \;

Phase 2: Manifest Accuracy

• Review current manifests/skills.md
• Review current manifests/skills.csv
• Count SKILL.md files vs CSV entries
• Flag any discrepancies (overage/shortage)
• For each discrepancy: add/remove from CSV

Phase 3: Normalize Paths

• Open manifests/skills.md
• Find all paths with /root/.codex/skills/ (absolute)
• Replace with relative paths: ./skills/agent-attached/ or ./skills/local/
• Regenerate skills.csv with relative paths
• Validate all paths work with test -d checks

Phase 4: Categorize Skills

• For each skill directory, confirm category:
  • agent-attached/ — Specialized for this agent only
  • local/ — General-purpose, reusable
  • plugin-provided/ — Third-party integration (GitHub, Slack, etc)
  • platform-managed/ — Built-in, platform-provided
  • directory-installed/ — Directory-scoped skills
• Update manifests if categorization is wrong
• Document reasoning in manifests/

Phase 5: Validate SKILL.md Entrypoints

• For each skill directory without SKILL.md:
  • Create SKILL.md template with:

    ---
    name: {skill-name}
    title: {Descriptive Title}
    description: {One-paragraph description}
    category: {agent-attached|local|plugin-provided|platform-managed|directory-installed}
    tags: [{tag1}, {tag2}]
    ---

  • Extract description from README if available
  • Set appropriate category & tags
  • Validate structure against schema
• Verify 100% SKILL.md coverage for this agent

Phase 6: Update Agent Metadata

• Edit agents/{agent}/AGENT.md:
  • Add skills section listing all available skills by category
  • Example:

    ## Skills
    
    This agent accesses **X custom skills**:
    
    **Agent-Attached Skills (Y):**
    - skill-1
    - skill-2
    
    **Local Skills (Z):**
    - skill-a
    - skill-b
    
    See [manifests/skills.md](manifests/skills.md) for complete inventory.

Phase 7: Update Provider Instructions

• Edit agents/{agent}/claude/agent.md:
  • Add skills reference
  • Example: "Accesses Y agent-attached skills and Z local skills"
• Edit agents/{agent}/copilot/agent.md:
  • Document GitHub-specific skill integrations
• Edit agents/{agent}/openai/agent.md:
  • Document OpenAI function-calling skills

Phase 8: Update Core Instructions

• Edit agents/{agent}/agent/instructions/AGENTS.md:
  • Add "Skill Integration" section
  • Document how to invoke each skill category
  • Add examples of skill usage patterns
  • Example:

    ## Skill Integration
    
    ### Agent-Attached Skills
    These specialized skills are available for {agent}:
    - {skill-1}: {purpose}
    - {skill-2}: {purpose}
    
    Usage: Invoke by skill name in prompts
    
    ### Local Skills
    General-purpose skills available:
    - {skill-a}: {purpose}
    - {skill-b}: {purpose}
    
    ### Accessing Skills
    Skills are automatically available in the agent context.
    Reference them by: {{ skill-name }}

Phase 9: Regenerate Checksums

cd agents/{agent}
find . -type f ! -name ".DS_Store" ! -name "checksums.sha256" -exec sha256sum {} \; | sort > checksums.sha256

• Verify checksums generated

Phase 10: Validation Tests

• Skill discoverability: ls skills/*/SKILL.md | wc -l matches expected count
• Path validity: grep -r "/root/.codex" manifests/ returns nothing (no absolute paths)
• Manifest accuracy: CSV count = actual skill directories
• Metadata: AGENT.md lists all skills
• No broken symlinks: find skills -type l -! -exec test -e {} \;

Phase 11: Documentation Update

• Add entry to docs/AGENT_SKILL_REFACTORING.md:

  ### {Agent Name} (Status: ✅ Complete | ⏳ In Progress | ☐ Pending)
  - Skills analyzed: X total (Y agent-attached, Z local)
  - Issues resolved: [list]
  - PR: #{pr-number}
  - Completed: YYYY-MM-DD

Phase 12: Create & Test PR

• Commit all changes with message:

  refactor(agents): standardize {agent-name} skill manifests and integration
  
  - Validate all skill directories have SKILL.md entrypoints
  - Normalize paths: absolute → relative (./skills/...)
  - Regenerate manifests/skills.md and manifests/skills.csv
  - Update AGENT.md with skill integration guide
  - Update instructions with skill usage patterns
  - Regenerate checksums
  - Zero discrepancies between manifests and filesystem
  
  Related to: #1197 (Agent Skills Refactoring)
  

• Push branch and create PR
• Link to issue Agent Skills Refactoring: Consolidate & Standardize All Agent Skill Directories #1197
• Verify CI passes
• Request review
• Merge when approved

---

Agent Refactoring Order

Recommended sequence (by complexity/size):

1. ✅ prd-factory-planner-agent (57 SKILL.md files, 24 agent-attached + 10 local)
2. ⏳ tour-operator-config-agent (30 SKILL.md files, 10 agent-attached + 20 local)
3. ☐ woo-config-agent (31 SKILL.md files, 10 agent-attached + 1 local)
4. ☐ wp-config-agent (31 SKILL.md files, 11 agent-attached + 1 local)
5. ☐ website-content-strategist-agent (67 SKILL.md files, 28 agent-attached + 1 local)
6. ☐ prd-agent (102 SKILL.md files, 25 agent-attached + 13 local)
7. ☐ ai-readiness-estimator-agent (51 SKILL.md files, 19 agent-attached + 1 local)
8. ☐ client-website-discovery-assistant-agent (67 SKILL.md files, 1 agent-attached + 3 local)
9. ☐ zendesk-support-agent (37 SKILL.md files, 17 agent-attached + 3 local)
10. ☐ design-partner-agent (47 SKILL.md files, 1 agent-attached + 1 local)
11. ☐ linear-advisor-agent (42 SKILL.md files, 16 agent-attached + 10 local)
12. ☐ proposal-desk-agent (50 SKILL.md files, 6 agent-attached + 3 local)
13. ☐ website-scope-estimator-agent (49 SKILL.md files, 5 agent-attached + 10 local)
14. ☐ playwright-testing-agent (21 SKILL.md files, 2 agent-attached + 2 local)
15. ☐ harvest-analytical-agent (20 SKILL.md files, 0 agent-attached + 1 local)
16. ☐ pagespeed-agent (25 SKILL.md files, 3 agent-attached + 0 local)

---

Batch Refactoring Script

For Phase 1-2 batch work, use this script to validate all agents:

#!/bin/bash
# Validate all agents for manifest discrepancies

for agent_dir in agents/*/; do
  agent_name=$(basename "$agent_dir")
  
  # Count SKILL.md files
  skill_count=$(find "$agent_dir/skills" -name "SKILL.md" | wc -l)
  
  # Count CSV entries
  csv_count=$(tail -n +2 "$agent_dir/manifests/skills.csv" 2>/dev/null | wc -l)
  
  # Check for absolute paths
  absolute_paths=$(grep -r "/root/.codex" "$agent_dir/manifests/" 2>/dev/null | wc -l)
  
  # Show results
  status="✅"
  [ $skill_count -ne $csv_count ] && status="⚠️"
  [ $absolute_paths -gt 0 ] && status="❌"
  
  printf "%s %-45s SKILL.md: %3d | CSV: %3d | Abs paths: %d\n" \
    "$status" "$agent_name" "$skill_count" "$csv_count" "$absolute_paths"
done

---

Troubleshooting

Issue: SKILL.md not found for skill directory
→ Solution: Create minimal SKILL.md with frontmatter template

Issue: Manifest shows absolute paths
→ Solution: Use sed to replace /root/.codex/skills/ with ./skills/

Issue: CSV count doesn't match directory count
→ Solution: Regenerate CSV from filesystem using inventory script

Issue: Skills not discoverable in provider configs
→ Solution: Add to AGENT.md + update claude/copilot/openai/agent.md

---

Each agent refactoring takes 1-2 hours following this checklist. Total project: 16-32 hours.

[linear-code]

LS-1759

[github-actions]

✅ Template check passed after update. Thanks for fixing the issue body.

[github-actions]

⏱️ Aging and SLA annotation

• Age: 5 day(s)
• SLA state: Within SLA
• Thresholds: warn at 7 days, breach at 14 days
• Last updated: 2026-07-29T17:26:19.398Z

Maintained by project-meta-sync workflow.