universal-scraping-architect
Build a scraper or extractor for sites, PDFs or paginated APIs and validate the fields (Firecrawl or local Python). Load whenever the user wants data pulled from pages, files or an API, or a scraper broke.
Target: Software development & testing. Automatically collected · unverified.
Excerpt from the author’s documentation
Extracted from a fixed version of SKILL.md in its original language. This is the author’s documentation; installation, compatibility and runtime results have not been verified by this site.
Read the original English text
Topics in the source
- Universal Scraping Architect
- Before Starting
- How This Skill Works
- Mode 1: API-Driven (Firecrawl)
- Mode 2: Local Python (Traditional)
- Mode 3: Hybrid Pipeline
- The Extraction Pipeline
- Proactive Triggers
Design complete, robust data-extraction pipelines with intelligent routing, validation, and token-budget tracking — not brittle one-off scripts. Dependency Notice: BYOK (Bring Your Own Key) pattern for Firecrawl; API keys must only be loaded via environment variables. Per-script dependencies: | Script | Dependencies | Exact CLI | |---|---|---| | scripts/validateextraction.py | stdlib only | python3 scripts/validateextraction.py output.json --json | | scripts/firecrawlexample.py | firecrawl, requests (template; --sample runs offline) | python3 scripts/firecrawlexample.py --sample | | scripts/localbs4example.py | beautifulsoup4, pandas (template; --sample runs offline) | python3 scripts/localbs4example.py --sample | Check for context first: If project-context.md exists, read it before asking questions. Determine the target data format, scale of extraction, and deployment environment before writing any code. This skill supports 3 extraction modes based on intelligent routing: Use when the source is a public URL, heavily dynamic (JS/SPA), requires search-first discovery, or involves bulk crawling across a domain. Use when extracting from local files (PDF, Excel, CSV), the data is priva
This is an excerpt. Refer to the complete source and its current requirements.
- Source repository
- alirezarezvani/claude-skills
- Source file
- engineering/universal-scraping-architect/skills/universal-scraping-architect/SKILL.md
- Fixed revision
- 8f082544116fc0d014188e521ca40304cdfc86b0
Sources and evidence
This record was collected automatically. Its documentation, suitability and runtime behavior have not been manually verified.
Author: alirezarezvani/claude-skills · original repository metadata. Documentation review: Not reviewed. Last source check: 2026-11-09T14:01:46.934Z.
Categories are inferred from names, descriptions and paths. They do not establish compatibility.
License and attribution
MIT. Third-party content and copyright remain with their authors. Refer to the original license and terms.
Original license ↗Synchronization, copying and user-reported success do not mark a resource as runtime-tested.
Limitations and notes
- Collecting metadata does not mean this site has installed, adapted, runtime-tested or security-reviewed the resource.