Open Data Monitor: Tracking Government Dataset Additions and Deletions

Splitgraph developed this tracking feed in 2023 to monitor the lifecycle of open government datasets across various portals. It records specific events when datasets are added or deleted, providing a historical log of data availability. The system utilizes metadata from Socrata-based platforms and was deployed using Seafowl and Fly.io.

Use Cases

• Analyzing the churn rate of government datasets by monitoring addition and deletion events
• Auditing the persistence of specific public datasets over time
• Researching the lifecycle and 'link rot' of open data across different municipal organizations

Strengths

• Captures dataset deletions which are typically purged from official government catalogs
• Aggregates metadata from multiple disparate Socrata-based open data portals
• Provides a temporal record of data publishing patterns

Limitations

• Schema and specific column names are not publicly documented in the source
• Temporal staleness due to the last update occurring in early 2023
• Platform bias toward Socrata-hosted instances

Provenance

Source

Splitgraph and Socrata-based open data portals

Collection Method

Automated monitoring of open data portal metadata feeds

Time Range

Ending April 2023

Freshness

The dataset was last updated in April 2023 and may not reflect recent changes in government portals.

Geography

Primarily United States