Fundus: News Crawler for Text and Image Extraction from CC-News

Fundus is a news crawler and corpus construction tool developed by flairNLP, updated as of March 2026. It enables the extraction of text and images from news sources using RSS feeds, sitemaps, and CommonCrawl data.

Use Cases

• Training NLP models on extracted news text content
• Developing image classification systems using extracted news images
• Analyzing publication patterns using RSS-sourced metadata

Strengths

• Supports CC-News integration for historical news access
• Provides dual-modality extraction for text and images
• Licensed under MIT for open redistribution

Limitations

• Requires active execution of Python scripts to generate data
• Susceptible to extraction failures on non-standard news site layouts

Provenance

Source

flairNLP GitHub repository

Collection Method

scraped

Freshness

Updated March 2026.

Geography

global