WTNIC: Web-Text Profiles for 300,000 Companies from Internet Archive

Eric Heiden at the University of Southern California proposes a database of 300,000 companies built from the Internet Archive Wayback Machine. The WTNIC project aims to support analyses of market structure and competition by creating company profiles from archived web pages. Machine learning techniques are intended to define company similarity based on their product and service offerings.

Use Cases

• Analyzing product market competition based on web-text similarity between companies.
• Studying market structure for private firms based on profiles from web archives.
• Researching innovation patterns based on company service offerings described online.

Strengths

• Proposes a database of 300,000 companies, a scale not previously available for private firms.
• Sources data from the Internet Archive Wayback Machine, a digital archive of the World Wide Web.

Limitations

• Column-level documentation is absent; field semantics must be inferred after download.
• Row count for the final database is unverified; the 300,000 figure is a project goal.
• License is closed, which restricts redistribution and commercial use.

Provenance

Source

University of Southern California

Collection Method

Proposes mining the Internet Archive Wayback Machine to build company profiles.