Scrapademic
A high-stealth academic research scraper and CLI engineered to extract and structure Google Scholar publications into SQL, Markdown, CSV, and JSON.
The challenge
Academic scrapers routinely fail because Google Scholar employs strict CAPTCHA triggers, dynamic client-side pagination, and rate-limiting. Researchers were forced to manually copy-paste hundreds of publications or rely on abandoned, broken python scripts.
The approach
We built Scrapademic with a stealth headless driver that mirrors human browsing cadences. It automatically paginates by clicking dynamic triggers, parses bibliographic attributes into clean schema, and supports simultaneous export to SQL INSERT statements, Markdown tables, CSV spreadsheets, and normalized JSON.
What we made
- Published npm Library & CLI Utility
- Stealth DOM Automation Core
- Multi-Format Exporter (SQL, MD, JSON, CSV)
- Dynamic Citation Parser & Normalizer
- Comprehensive Developer Documentation
Built with
Node.js / Puppeteer Stealth / JavaScript / ES Modules / CLI Architecture / npm Registry
Credits
Open-source developer tool authored and maintained by the Certified Cool team.