← All skills

llm-align

100 / 100 · Low risk

Fine to try.

By Calibre-Labs/reforge-ai-evals (GitHub) · coding · 98 stars · scanned at commit 92afb10

Analyze alignment between LLM judge scores and human labels in an eval dataset. Use this skill whenever someone wants to evaluate how well an LLM judge agrees with human reviewers, calculate TPR/TNR, investigate disagreements, or improve a scorer prompt. Trigger on phrases like: "calculate TPR/TNR", "judge alignment", "where does the LLM disagree with humans", "improve my scorer", "why is the LLM getting these wrong", "compare human labels to LLM scores", "evaluate my eval", or any time someone…

Works with any agent that supports SKILL.md skills.

What this skill can do

We found nothing in the files we read that matches our rules. That does not prove the skill is safe: see the limits of static scanning.

Install

Paste this into your terminal. It downloads exactly the version we scanned, not whatever is newest.

if [ -e ~/.agents/skills/'llm-align' ] || [ -L ~/.agents/skills/'llm-align' ]; then echo 'A skill folder named llm-align already exists — not replacing it' >&2; false; else d=$(mktemp -d) && git clone --quiet --filter=blob:none --no-checkout 'https://github.com/Calibre-Labs/reforge-ai-evals' "$d" && git -C "$d" checkout --quiet '92afb10f9a692da088134b6d27841f92318aa211' -- 'skills/llm-align' && mkdir -p ~/.agents/skills && cp -R "$d"/'skills/llm-align' ~/.agents/skills/'llm-align' && rm -rf "$d"; fi

Codex reads ~/.agents/skills directly. For Claude Code, Cursor, Copilot, Gemini CLI, Antigravity or OpenCode, also link it. For Claude Code: ln -s ~/.agents/skills/'llm-align' ~/.claude/skills/'llm-align' (other agents: ~/.cursor/skills, ~/.copilot/skills, ~/.gemini/skills, ~/.gemini/config/skills, ~/.config/opencode/skills). Or install through the finder skill, which does both.

Badge for your README

The badge for Calibre-Labs/reforge-ai-evals shows the score of its lowest-scoring skill.

[![SkillTrust](https://zalavijaysinh.github.io/skilltrust/badges/Calibre-Labs/reforge-ai-evals.svg)](https://zalavijaysinh.github.io/skilltrust/repos/Calibre-Labs/reforge-ai-evals/)

Score history