On Coreference Extraction from Identric’s Documents

Business Understanding:

Identrics would like to extract knowledge from unstructured text. One use case for that is to automate the process of information extraction from news articles. Some of Indentrics clients are from the finance industry and have a need to understand the impact of news on the valuation of certain, at the stock index listed, companies. To help their clients’ analysts derive faster to insights, Indentrics’ NLP Team uses coreferencing to extract all the mentions of an entity (like a company or person) in a news article and presents these information back to the analysts.

Data Understanding:

In approaching this problem outlined above, we are presented with two files (i.e. documents.csv and entities.csv). Each line outlined in documents.csv represents one row of the data containing the unprocessed corpus and the corresponding document ID. In entities.csv, the file contains a list of mentions that are manually tagged to indicate entities present in the documents corresponding to the document ID. The contents of the entities.csv file include following columns:

entity_id – column containing generated integer identifiers for entities
doc_id – column containing generated doc_id that references to the same doc_id in table `documents`.
word – string representation of mentioned entity
type – mentioned entity type. In this dump all entities are of type “ORGANIZATION”
start_offset – the position number of first entity character in document text
end_offset – the position number of last entity character in document text
Column entity_id has primary key constraint for the table. Column doc_id hasn’t foreign key constraint to doc_id in `documents` table.

Identrics Final