Karpathy Uses Claude to Curate HackerTube from Hacker News and Releases Dataset

Frustrated with default YouTube recommendation algorithms, OpenAI co-founder and former Tesla AI director Andrej Karpathy built a community-driven alternative. Using Claude Opus 5.5 alongside the Mac Claude Code desktop application, the workflow parsed roughly 43 million comments across nearly 20 years on Hacker News. A script first flagged over 100,000 YouTube video mentions, which were subsequently filtered down to around 7,000 using a threshold requiring at least three different commenters across at least three distinct months, ultimately yielding 6,454 indexed videos. Claude was instructed to interpret comment context—differentiating recommendations from passing citations or critiques—generate video descriptions, categorize taxonomy, and swap default thumbnails for mid-video frames. Karpathy stated his own manual actions were limited to acquiring a YouTube API key, leasing a cloud server, and purchasing a domain name, leaving site construction, deployment, and security hardening to Claude within a two-hour build window. Karpathy released a 17.8MB data archive containing DATA.md, a specification file designed for AI agents. Users can load this directory into the Claude Code desktop application and execute prompts to build a local curation site aligned with specific topics, avoiding external hosting or API keys. Reported constraints note that running equivalent multi-agent processes from scratch requires substantial API quota, public deployments demand ongoing maintenance and security patches, and the DATA.md documentation notes occasional classification or summary inaccuracies.

Key takeaways

• Karpathy utilized Claude to filter 6,454 YouTube videos from 43 million Hacker News comments using a multi-user, multi-month recommendation threshold.
• Claude parsed comment sentiment, generated summaries, and handled deployment and security tasks, while Karpathy handled host, domain, and API setups.
• A 17.8MB dataset with a DATA.md agent guide was published, allowing individuals to prompt Claude Code to build local video curation sites.
• Operational considerations include high compute usage for full pipeline runs, ongoing public maintenance responsibilities, and minor AI annotation errors.