← All skills

eval-llm-judge

85 / 100 · Low risk

Fine to try.

By Calibre-Labs/reforge-ai-evals (GitHub) · coding · 98 stars · scanned at commit 92afb10

Write LLM-as-judge evaluator prompts for AI products. Use this skill whenever you need to evaluate semantic properties of AI outputs that cannot be checked programmatically — output quality, tone, factual accuracy, edge case handling, reference alignment. Also use when auditing existing judge prompts for bias, vagueness, or compound criteria. Output is a judge prompt and a runner in the project's own scorer format (a Python runner when the project has none), saved where the project keeps its ev…

Works with any agent that supports SKILL.md skills.

What this skill can do

Install

Paste this into your terminal. It downloads exactly the version we scanned, not whatever is newest.

if [ -e ~/.agents/skills/'eval-llm-judge' ] || [ -L ~/.agents/skills/'eval-llm-judge' ]; then echo 'A skill folder named eval-llm-judge already exists — not replacing it' >&2; false; else d=$(mktemp -d) && git clone --quiet --filter=blob:none --no-checkout 'https://github.com/Calibre-Labs/reforge-ai-evals' "$d" && git -C "$d" checkout --quiet '92afb10f9a692da088134b6d27841f92318aa211' -- 'skills/eval-llm-judge' && mkdir -p ~/.agents/skills && cp -R "$d"/'skills/eval-llm-judge' ~/.agents/skills/'eval-llm-judge' && rm -rf "$d"; fi

Codex reads ~/.agents/skills directly. For Claude Code, Cursor, Copilot, Gemini CLI, Antigravity or OpenCode, also link it. For Claude Code: ln -s ~/.agents/skills/'eval-llm-judge' ~/.claude/skills/'eval-llm-judge' (other agents: ~/.cursor/skills, ~/.copilot/skills, ~/.gemini/skills, ~/.gemini/config/skills, ~/.config/opencode/skills). Or install through the finder skill, which does both.

Badge for your README

The badge for Calibre-Labs/reforge-ai-evals shows the score of its lowest-scoring skill.

[![SkillTrust](https://zalavijaysinh.github.io/skilltrust/badges/Calibre-Labs/reforge-ai-evals.svg)](https://zalavijaysinh.github.io/skilltrust/repos/Calibre-Labs/reforge-ai-evals/)

Score history