← All skills

align-human

95 / 100 · Low risk

Fine to try.

By agentscope-ai/OpenJudge (GitHub) · data · 1,101 stars · scanned at commit 03c4445

Use when the user has a judge/grader and human-labeled data, and wants to measure how well the judge agrees with humans, detect systematic biases, determine whether automatic evaluation can replace human review, or build a human-reduction roadmap. Also use when the user mentions calibration, TPR/TNR, judge validation, inter-rater agreement, Cohen's kappa, bias detection, or "is my automatic evaluation trustworthy." Merges the calibrate and align functions into one skill.

Works with any agent that supports SKILL.md skills.

What this skill can do

Install

Paste this into your terminal. It downloads exactly the version we scanned, not whatever is newest.

if [ -e ~/.agents/skills/'align-human' ] || [ -L ~/.agents/skills/'align-human' ]; then echo 'A skill folder named align-human already exists — not replacing it' >&2; false; else d=$(mktemp -d) && git clone --quiet --filter=blob:none --no-checkout 'https://github.com/agentscope-ai/OpenJudge' "$d" && git -C "$d" checkout --quiet '03c444567d5bb10429b10f4d2ad9df493b0edcc9' -- 'skills/eval_pipeline/03-align-human' && mkdir -p ~/.agents/skills && cp -R "$d"/'skills/eval_pipeline/03-align-human' ~/.agents/skills/'align-human' && rm -rf "$d"; fi

Codex reads ~/.agents/skills directly. For Claude Code, Cursor, Copilot, Gemini CLI, Antigravity or OpenCode, also link it. For Claude Code: ln -s ~/.agents/skills/'align-human' ~/.claude/skills/'align-human' (other agents: ~/.cursor/skills, ~/.copilot/skills, ~/.gemini/skills, ~/.gemini/config/skills, ~/.config/opencode/skills). Or install through the finder skill, which does both.

Badge for your README

The badge for agentscope-ai/OpenJudge shows the score of its lowest-scoring skill.

[![SkillTrust](https://zalavijaysinh.github.io/skilltrust/badges/agentscope-ai/OpenJudge.svg)](https://zalavijaysinh.github.io/skilltrust/repos/agentscope-ai/OpenJudge/)

Score history