Skip to main content
AI/MLjmagly

soul-validate

Validate a SOUL.md file against community best practices and quality criteria

Stars
141
Source
jmagly/aiwg
Updated
2026-05-31
Slug
jmagly--aiwg--soul-validate
View on GitHubRaw SKILL.md

// install — copy + paste into any project

mkdir -p .claude/skills && curl -fsSL https://raw.githubusercontent.com/jmagly/aiwg/HEAD/agentic/code/addons/aiwg-utils/skills/soul-validate/SKILL.md -o .claude/skills/soul-validate.md

Drops the SKILL.md into .claude/skills/soul-validate.md. Works with Claude Code, Cursor, and any agent that loads SKILL.md files from .claude/skills/.

soul-validate

Validate a SOUL.md file against community best practices and quality criteria.

Triggers

Alternate expressions and non-obvious activations (primary phrases are matched automatically from the skill description):

  • "soul score" → soul quality metrics
  • "check the soul" → soul validation

Behavior

When triggered, this skill performs a comprehensive quality assessment of a SOUL.md file, checking for completeness, specificity, and effectiveness.

Validation Process

  1. Locate SOUL.md — check ./SOUL.md, ./.aiwg/SOUL.md, or path from user
  2. Parse sections — identify which recommended sections are present
  3. Assess each section for quality criteria
  4. Report results with specific, actionable feedback

Section Checklist

Section Required Quality Criteria
Who I Am Yes Specific background, not generic. Mentions concrete experience.
Worldview Yes Beliefs specific enough to be falsifiable. Not platitudes.
Opinions Yes Organized by domain. Each opinion could be disagreed with.
Standards Recommended Concrete bar for completion. Names dangling threads this persona never leaves and how complete-vs-fast tension resolves. Not "I value quality."
Vocabulary Recommended Actual terms with personal definitions, not categories.
Boundaries Recommended Concrete refusals, not vague ethical statements.
Current Focus Optional Active projects, current thinking. Dated or contextual.
Influences Optional Specific people/works with what was taken from each.
Tensions Optional Genuine contradictions — shows self-awareness.
Pet Peeves Optional Specific triggers, not generic annoyances.
Interests Optional Deep interests that inform cross-domain thinking.

Backwards compatibility: Souls that predate the Standards section validate with a warning, not an error. Existing souls remain functional; /soul-enhance proposes Standards content. New souls created via /soul-create should populate the section.

Quality Tests

1. Prediction Test (Critical)

"Could someone reading this SOUL.md predict the agent's takes on new, unstated topics?"

  • Pass: Worldview + opinions are specific enough to interpolate
  • Fail: Too vague — anyone could have these opinions

2. Specificity Test

Check for vague language patterns that weaken the soul:

Vague (Fail) Specific (Pass)
"I have nuanced views on X" "I think X is overrated because Y"
"I value quality" "I'll delay a release to fix a flaky test"
"I aim for excellence" "Done = no dangling threads, real fix not workaround"
"I believe in completeness" "I never ship a fix without a regression test for it"
"I'm interested in technology" "I've spent 10 years on distributed systems"
"I believe in best practices" "Most 'best practices' are cargo cult"

Flag any sentence that could apply to anyone.

3. Context Budget Test

  • Under 5K tokens (~3,750 words): Pass
  • 5K-8K tokens: Warning — consider trimming
  • Over 8K tokens: Fail — too large for context budget

4. Anti-Pattern Detection

Flag common anti-patterns:

Anti-Pattern Example Fix
Generic positivity "I'm passionate about helping" Replace with specific beliefs
Exhaustive rules 50+ boundary statements Reduce to 5-10 core boundaries
No contradictions Suspiciously coherent persona Add 2-3 genuine tensions
Category-level vocabulary "technical terms" List actual terms with definitions
Missing opinions Worldview without takes Add domain-specific opinions

5. Companion File Check

Check for recommended companion files:

File Purpose Status
STYLE.md Writing patterns Check if exists
examples/good-outputs.md Calibration examples Check if exists
examples/bad-outputs.md Anti-pattern examples Check if exists

Output Format

Soul Validation Report
=======================

File: ./SOUL.md (~2,847 tokens)

Section Completeness
---------------------
  ✓ Who I Am          Present, specific
  ✓ Worldview         Present, 4 falsifiable beliefs
  ✓ Opinions          Present, organized by 3 domains
  ⚠ Standards         Missing — what's this persona's bar for "done"? (warning, not error)
  ✓ Vocabulary        Present, 12 terms defined
  ✗ Boundaries        Missing — what will the agent refuse?
  ✓ Current Focus     Present
  ✓ Influences        Present, 5 named with takeaways
  ✗ Tensions          Missing — what contradictions exist?
  ✓ Pet Peeves        Present, 4 specific triggers

Quality Tests
--------------
  ✓ Prediction test   Opinions are interpolatable
  ⚠ Specificity test  2 vague statements found:
    Line 14: "I value clean code" → too generic, what specifically?
    Line 31: "I have experience with many frameworks" → which ones?
  ✓ Context budget    ~2,847 tokens (under 5K limit)
  ✓ Anti-patterns     None detected

Companion Files
----------------
  ✗ STYLE.md               Not found
  ✗ examples/good-outputs.md   Not found
  ✗ examples/bad-outputs.md    Not found

Score: 7/10

Recommendations
----------------
1. Add a Boundaries section — define 5-10 concrete refusals
2. Add a Tensions section — list 2-3 genuine contradictions
3. Fix vague statements on lines 14 and 31
4. Consider creating calibration examples (good-outputs.md)
5. Consider creating a STYLE.md companion for writing patterns

Run /soul-enhance to auto-improve these issues.

Scoring

Score Meaning
9-10 Production-ready. Distinctive, interpolatable, well-bounded.
7-8 Good foundation. Missing sections or minor vagueness.
5-6 Needs work. Multiple missing sections or pervasive vagueness.
3-4 Weak. Mostly generic. Fails prediction test.
1-2 Placeholder. Not functional as a soul file.

Integration

With soul-create

After /soul-create, automatically offer validation:

SOUL.md created. Run /soul-validate to check quality.

With soul-enable

Before enabling, recommend validation if score < 7:

Warning: SOUL.md scores 5/10 on quality check.
Consider running /soul-validate and /soul-enhance before enabling.

Examples

# Validate project soul
/soul-validate

# Validate specific file
/soul-validate .aiwg/SOUL.md

# Validate agent soul
/soul-validate .claude/agents/test-engineer.soul.md

References

  • @$AIWG_ROOT/agentic/code/addons/aiwg-utils/skills/soul-create/SKILL.md — Soul creation
  • @$AIWG_ROOT/agentic/code/addons/aiwg-utils/commands/soul-enable.md — Soul enforcement
  • @$AIWG_ROOT/docs/soul-md-guide.md — SOUL.md integration guide
  • #437 — SOUL.md compatibility issue