← All skills

sentencepiece

80 / 100 · Low risk

Fine to try.

By artubss/SKILLS-CLAUDE-CODE (GitHub) · other · 11 stars · scanned at commit ac67ed8

Tokenizador agnóstico de linguagem que trata texto como Unicode bruto. Suporta algoritmos BPE e Unigram. Rápido (50k sentenças/seg), leve (6MB de memória), vocabulário determinístico. Usado por T5, ALBERT, XLNet, mBART. Treina em texto bruto sem pré-tokenização. Use quando você precisar de suporte multilíngue, linguagens CJK ou tokenização reproduzível.

Works with any agent that supports SKILL.md skills.

What this skill can do

Install

Paste this into your terminal. It downloads exactly the version we scanned, not whatever is newest.

if [ -e ~/.agents/skills/'sentencepiece' ] || [ -L ~/.agents/skills/'sentencepiece' ]; then echo 'A skill folder named sentencepiece already exists — not replacing it' >&2; false; else d=$(mktemp -d) && git clone --quiet --filter=blob:none --no-checkout 'https://github.com/artubss/SKILLS-CLAUDE-CODE' "$d" && git -C "$d" checkout --quiet 'ac67ed85889b15fa9e784e2a93a83f8916870698' -- 'skills/skills/ai-research/tokenization-sentencepiece' && mkdir -p ~/.agents/skills && cp -R "$d"/'skills/skills/ai-research/tokenization-sentencepiece' ~/.agents/skills/'sentencepiece' && rm -rf "$d"; fi

Codex reads ~/.agents/skills directly. For Claude Code, Cursor, Copilot, Gemini CLI, Antigravity or OpenCode, also link it. For Claude Code: ln -s ~/.agents/skills/'sentencepiece' ~/.claude/skills/'sentencepiece' (other agents: ~/.cursor/skills, ~/.copilot/skills, ~/.gemini/skills, ~/.gemini/config/skills, ~/.config/opencode/skills). Or install through the finder skill, which does both.

Badge for your README

The badge for artubss/SKILLS-CLAUDE-CODE shows the score of its lowest-scoring skill.

[![SkillTrust](https://zalavijaysinh.github.io/skilltrust/badges/artubss/SKILLS-CLAUDE-CODE.svg)](https://zalavijaysinh.github.io/skilltrust/repos/artubss/SKILLS-CLAUDE-CODE/)

Score history