← All skills
lm-evaluation-harness
65 / 100 · Review before installing
Check before installing. Some of what it can do is expected: a research skill usually looks things up on the internet.
By graniet/kheish (GitHub) · research · 213 stars · scanned at commit df50433
Evaluates LLMs across 60+ academic benchmarks (MMLU, HumanEval, GSM8K,
Works with any agent that supports SKILL.md skills.
What this skill can do
Downloads and installs other software
The author does not mention this.
Found in 3 places.
Runs commands on your computer
The author does not mention this.
Found in 27 places.
Show 17 more
- skills/mlops/evaluation/lm-evaluation-harness/SKILL.md, line 97
--tasks mmlu,gsm8k,hellaswag,truthfulqa,arc_challenge - skills/mlops/evaluation/lm-evaluation-harness/SKILL.md, line 104
lm_eval --model hf \ - skills/mlops/evaluation/lm-evaluation-harness/SKILL.md, line 113
lm_eval --model hf \ - skills/mlops/evaluation/lm-evaluation-harness/SKILL.md, line 121
lm_eval --model hf \ - skills/mlops/evaluation/lm-evaluation-harness/SKILL.md, line 131
lm_eval --model hf \ - skills/mlops/evaluation/lm-evaluation-harness/SKILL.md, line 196
CHECKPOINT_DIR=$1 - skills/mlops/evaluation/lm-evaluation-harness/SKILL.md, line 228
os.system(f"./eval_checkpoint.sh checkpoints step-{step}") - skills/mlops/evaluation/lm-evaluation-harness/SKILL.md, line 245
os.system(f"lm_eval --model hf --model_args pretrained={checkpoint_path} ...") - skills/mlops/evaluation/lm-evaluation-harness/SKILL.md, line 288
meta-llama/Llama-2-7b-hf - skills/mlops/evaluation/lm-evaluation-harness/SKILL.md, line 300
TASKS="mmlu,gsm8k,hellaswag,truthfulqa" - skills/mlops/evaluation/lm-evaluation-harness/SKILL.md, line 375
pip install vllm - skills/mlops/evaluation/lm-evaluation-harness/SKILL.md, line 381
lm_eval --model vllm \ - skills/mlops/evaluation/lm-evaluation-harness/SKILL.md, line 393
lm_eval --model hf \ - skills/mlops/evaluation/lm-evaluation-harness/SKILL.md, line 426
lm_eval --model vllm \ - skills/mlops/evaluation/lm-evaluation-harness/SKILL.md, line 432
--num_fewshot 0 # Instead of 5 - skills/mlops/evaluation/lm-evaluation-harness/SKILL.md, line 437
--tasks mmlu_stem # Only STEM subjects - skills/mlops/evaluation/lm-evaluation-harness/SKILL.md, line 444
--batch_size 1 # Or --batch_size auto
and 7 more places
Can call paid AI services billed to you
The author does not mention this.
Found in 2 places.
Connects to the internet
The author does not mention this. Expected for a research skill.
Found in 1 place.
Safer alternatives
Low-risk skills in the same category (research) that look related. Check that one does what you need.
Install
Paste this into your terminal. It downloads exactly the version we scanned, not whatever is newest.
if [ -e ~/.agents/skills/'lm-evaluation-harness' ] || [ -L ~/.agents/skills/'lm-evaluation-harness' ]; then echo 'A skill folder named lm-evaluation-harness already exists — not replacing it' >&2; false; else d=$(mktemp -d) && git clone --quiet --filter=blob:none --no-checkout 'https://github.com/graniet/kheish' "$d" && git -C "$d" checkout --quiet 'df50433f8d577f832c6812b9e67be8ddb766059d' -- 'skills/mlops/evaluation/lm-evaluation-harness' && mkdir -p ~/.agents/skills && cp -R "$d"/'skills/mlops/evaluation/lm-evaluation-harness' ~/.agents/skills/'lm-evaluation-harness' && rm -rf "$d"; fi
Codex reads ~/.agents/skills directly. For Claude Code, Cursor, Copilot, Gemini CLI, Antigravity or OpenCode, also link it. For Claude Code: ln -s ~/.agents/skills/'lm-evaluation-harness' ~/.claude/skills/'lm-evaluation-harness' (other agents: ~/.cursor/skills, ~/.copilot/skills, ~/.gemini/skills, ~/.gemini/config/skills, ~/.config/opencode/skills). Or install through the finder skill, which does both.
Badge for your README
The badge for graniet/kheish shows the score of its lowest-scoring skill.

[](https://zalavijaysinh.github.io/skilltrust/repos/graniet/kheish/)
Score history
| Date | Version | Score |
|---|
| 2026-11-12 | df50433 | 65 |