← All skills

sentencepiece

80 / 100 · Low risk

Fine to try.

By KalarisLabs/research-agent-skills (GitHub) · research · 126 stars · scanned at commit 802bbea

Language-independent tokenizer treating text as raw Unicode. Supports BPE and Unigram algorithms. Fast (50k sentences/sec), lightweight (6MB memory), deterministic vocabulary. Used by T5, ALBERT, XLNet, mBART. Train on raw text without pre-tokenization. Use when you need multilingual support, CJK languages, or reproducible tokenization.

Works with any agent that supports SKILL.md skills.

Partially scanned — some files couldn't be read; we'll retry on the next scan

What this skill can do

Install

Paste this into your terminal. It downloads exactly the version we scanned, not whatever is newest.

if [ -e ~/.agents/skills/'sentencepiece' ] || [ -L ~/.agents/skills/'sentencepiece' ]; then echo 'A skill folder named sentencepiece already exists — not replacing it' >&2; false; else d=$(mktemp -d) && git clone --quiet --filter=blob:none --no-checkout 'https://github.com/KalarisLabs/research-agent-skills' "$d" && git -C "$d" checkout --quiet '802bbea44e551abac24d6856f83fc55149bbdba3' -- 'skills/sentencepiece' && mkdir -p ~/.agents/skills && cp -R "$d"/'skills/sentencepiece' ~/.agents/skills/'sentencepiece' && rm -rf "$d"; fi

Codex reads ~/.agents/skills directly. For Claude Code, Cursor, Copilot, Gemini CLI, Antigravity or OpenCode, also link it. For Claude Code: ln -s ~/.agents/skills/'sentencepiece' ~/.claude/skills/'sentencepiece' (other agents: ~/.cursor/skills, ~/.copilot/skills, ~/.gemini/skills, ~/.gemini/config/skills, ~/.config/opencode/skills). Or install through the finder skill, which does both.

Badge for your README

The badge for KalarisLabs/research-agent-skills shows the score of its lowest-scoring skill.

[![SkillTrust](https://zalavijaysinh.github.io/skilltrust/badges/KalarisLabs/research-agent-skills.svg)](https://zalavijaysinh.github.io/skilltrust/repos/KalarisLabs/research-agent-skills/)

Score history