Skip to main content
Generalopenai

metric-pack-designer

Design custom metric packs for plugin-eval so teams can add local evaluation rubrics that emit schema-compatible checks and metrics. Use when the user wants their own evaluation criteria or visualizations.

Stars
1,305
Source
openai/plugins
Updated
2026-05-30
Slug
openai--plugins--metric-pack-designer
View on GitHubRaw SKILL.md

// install — copy + paste into any project

mkdir -p .claude/skills && curl -fsSL https://raw.githubusercontent.com/openai/plugins/HEAD/plugins/plugin-eval/skills/metric-pack-designer/SKILL.md -o .claude/skills/metric-pack-designer.md

Drops the SKILL.md into .claude/skills/metric-pack-designer.md. Works with Claude Code, Cursor, and any agent that loads SKILL.md files from .claude/skills/.

Metric Pack Designer

Use this skill when the user wants to extend plugin-eval with a local rubric.

Workflow

  1. Clarify the custom rubric categories and target kinds.
  2. Define the smallest useful checks[] and metrics[] payload.
  3. Create a metric-pack manifest plus a script that prints JSON to stdout.
  4. Run the pack through plugin-eval analyze <path> --metric-pack <manifest.json>.

Design Rules

  • Keep IDs stable across runs so comparisons stay meaningful.
  • Emit only checks[], metrics[], and optional artifacts[].
  • Do not try to overwrite the core score or summary.
  • Prefer deterministic local signals over subjective text generation.

Reference

  • ../../references/metric-pack-manifest.md