← All skills
evaluating-llms-harness
65 / 100 · Review before installing
Check before installing. Some of what it can do is expected: a coding skill usually runs build and test commands.
By StarryCod/cogitum (GitHub) · coding · 12 stars · scanned at commit a99de21
lm-eval-harness: benchmark LLMs (MMLU, GSM8K, etc.).
Works with any agent that supports SKILL.md skills.
What this skill can do
Downloads and installs other software
The author does not mention this.
Found in 3 places.
Runs commands on your computer
The author does not mention this. Expected for a coding skill.
Found in 27 places.
Show 17 more
- cogitum/data/skills/mlops/evaluation/lm-evaluation-harness/SKILL.md, line 75
--tasks mmlu,gsm8k,hellaswag,truthfulqa,arc_challenge - cogitum/data/skills/mlops/evaluation/lm-evaluation-harness/SKILL.md, line 82
lm_eval --model hf \ - cogitum/data/skills/mlops/evaluation/lm-evaluation-harness/SKILL.md, line 91
lm_eval --model hf \ - cogitum/data/skills/mlops/evaluation/lm-evaluation-harness/SKILL.md, line 99
lm_eval --model hf \ - cogitum/data/skills/mlops/evaluation/lm-evaluation-harness/SKILL.md, line 109
lm_eval --model hf \ - cogitum/data/skills/mlops/evaluation/lm-evaluation-harness/SKILL.md, line 174
CHECKPOINT_DIR=$1 - cogitum/data/skills/mlops/evaluation/lm-evaluation-harness/SKILL.md, line 206
os.system(f"./eval_checkpoint.sh checkpoints step-{step}") - cogitum/data/skills/mlops/evaluation/lm-evaluation-harness/SKILL.md, line 223
os.system(f"lm_eval --model hf --model_args pretrained={checkpoint_path} ...") - cogitum/data/skills/mlops/evaluation/lm-evaluation-harness/SKILL.md, line 266
meta-llama/Llama-2-7b-hf - cogitum/data/skills/mlops/evaluation/lm-evaluation-harness/SKILL.md, line 278
TASKS="mmlu,gsm8k,hellaswag,truthfulqa" - cogitum/data/skills/mlops/evaluation/lm-evaluation-harness/SKILL.md, line 353
pip install vllm - cogitum/data/skills/mlops/evaluation/lm-evaluation-harness/SKILL.md, line 359
lm_eval --model vllm \ - cogitum/data/skills/mlops/evaluation/lm-evaluation-harness/SKILL.md, line 371
lm_eval --model hf \ - cogitum/data/skills/mlops/evaluation/lm-evaluation-harness/SKILL.md, line 404
lm_eval --model vllm \ - cogitum/data/skills/mlops/evaluation/lm-evaluation-harness/SKILL.md, line 410
--num_fewshot 0 # Instead of 5 - cogitum/data/skills/mlops/evaluation/lm-evaluation-harness/SKILL.md, line 415
--tasks mmlu_stem # Only STEM subjects - cogitum/data/skills/mlops/evaluation/lm-evaluation-harness/SKILL.md, line 422
--batch_size 1 # Or --batch_size auto
and 7 more places
Can call paid AI services billed to you
The author does not mention this.
Found in 2 places.
Connects to the internet
The author does not mention this.
Found in 1 place.
Safer alternatives
Low-risk skills in the same category (coding) that look related. Check that one does what you need.
Install
Paste this into your terminal. It downloads exactly the version we scanned, not whatever is newest.
if [ -e ~/.agents/skills/'evaluating-llms-harness' ] || [ -L ~/.agents/skills/'evaluating-llms-harness' ]; then echo 'A skill folder named evaluating-llms-harness already exists — not replacing it' >&2; false; else d=$(mktemp -d) && git clone --quiet --filter=blob:none --no-checkout 'https://github.com/StarryCod/cogitum' "$d" && git -C "$d" checkout --quiet 'a99de210a5d82a24b7aa5020afa1f904f595d14c' -- 'cogitum/data/skills/mlops/evaluation/lm-evaluation-harness' && mkdir -p ~/.agents/skills && cp -R "$d"/'cogitum/data/skills/mlops/evaluation/lm-evaluation-harness' ~/.agents/skills/'evaluating-llms-harness' && rm -rf "$d"; fi
Codex reads ~/.agents/skills directly. For Claude Code, Cursor, Copilot, Gemini CLI, Antigravity or OpenCode, also link it. For Claude Code: ln -s ~/.agents/skills/'evaluating-llms-harness' ~/.claude/skills/'evaluating-llms-harness' (other agents: ~/.cursor/skills, ~/.copilot/skills, ~/.gemini/skills, ~/.gemini/config/skills, ~/.config/opencode/skills). Or install through the finder skill, which does both.
Badge for your README
The badge for StarryCod/cogitum shows the score of its lowest-scoring skill.

[](https://zalavijaysinh.github.io/skilltrust/repos/StarryCod/cogitum/)
Score history
| Date | Version | Score |
|---|
| 2026-11-08 | a99de21 | 85 |