← All skills

crawlee

80 / 100 · Low risk

Fine to try. Some of what it can do is expected: a coding skill usually runs build and test commands.

By LeoYeAI/agent-master-skills (GitHub) · coding · 2,186 stars · scanned at commit 8f88f9a

Expert guide for building web scrapers and crawlers using Crawlee (JavaScript/TypeScript and Python). Use this skill whenever the user wants to: scrape a website, build a web crawler, extract data from web pages, automate browser navigation, handle anti-bot blocking, manage proxies or sessions for scraping, use Playwright/Puppeteer/Cheerio/BeautifulSoup for web data extraction, crawl sitemaps, download files from URLs, or deploy a scraper to the cloud. Trigger even for loosely related phrases l…

Works with any agent that supports SKILL.md skills.

What this skill can do

Install

Paste this into your terminal. It downloads exactly the version we scanned, not whatever is newest.

if [ -e ~/.agents/skills/'crawlee' ] || [ -L ~/.agents/skills/'crawlee' ]; then echo 'A skill folder named crawlee already exists — not replacing it' >&2; false; else d=$(mktemp -d) && git clone --quiet --filter=blob:none --no-checkout 'https://github.com/LeoYeAI/agent-master-skills' "$d" && git -C "$d" checkout --quiet '8f88f9a0dd5a1075d4f0c7626f9d960844d57575' -- 'skills/crawlee' && mkdir -p ~/.agents/skills && cp -R "$d"/'skills/crawlee' ~/.agents/skills/'crawlee' && rm -rf "$d"; fi

Codex reads ~/.agents/skills directly. For Claude Code, Cursor, Copilot, Gemini CLI, Antigravity or OpenCode, also link it. For Claude Code: ln -s ~/.agents/skills/'crawlee' ~/.claude/skills/'crawlee' (other agents: ~/.cursor/skills, ~/.copilot/skills, ~/.gemini/skills, ~/.gemini/config/skills, ~/.config/opencode/skills). Or install through the finder skill, which does both.

Badge for your README

The badge for LeoYeAI/agent-master-skills shows the score of its lowest-scoring skill.

[![SkillTrust](https://zalavijaysinh.github.io/skilltrust/badges/LeoYeAI/agent-master-skills.svg)](https://zalavijaysinh.github.io/skilltrust/repos/LeoYeAI/agent-master-skills/)

Score history