Tag: NLP

Posted shaipPosted in NewsNews

The Next Evolution of NLP: Why High-Quality Training Data Matters More Than Ever

Natural Language Processing (NLP) is accelerating the evolution of intelligent systems, yet the performance of NLP models continues to depend heavily on the quality of their training data. As organizations adopt LLMs and conversational AI at scale, they face challenges in contextual understanding, domain specificity, bias mitigation, and safety. High-quality, diverse, and human-annotated datasets have emerged as the key differentiator for achieving accuracy, reliability, and real-world applicability.

Shaip enables enterprises to meet these challenges through end-to-end data solutions—ranging from multilingual text collection to expert annotation and rigorous quality assurance. By delivering clean, contextual, compliant, and domain-rich datasets, Shaip empowers teams to build NLP systems that are more intelligent, ethical, and aligned with real-world use cases.

Posted vpekar, wariodoor Mentors: preslav, vsenderov, laura, alberto, giovanni7 CommentsPosted in Datathons SolutionsDatathons Solutions

Datathon – HackNews – Solution – Leopards

This is a Leopards team’s submission for the Propaganda Detection datathon. Key findings: the best performing classifier is logistic regression, operating on Word2vec representations of the sentences plus several designed features like the proportion of sentiment-bearing words in the sentence.

Posted vickyLeave a commentPosted in Team solutionsTeam solutions

case_onto_text_team_a_vicky

Although the data given to us has several snippets corresponding to each parent-subsidy pair, only some of the snippets reveal actual parent-subsidiary relationship. Therefore we felt that concatenating the snippets corresponding to each pair  into one single article and then training can give the model more information about which text snippet actually reveals the parent-subsidy relationship. A Bidirectional GRU models each sentence into a sentence vector and then two attention networks try to figure out the important words in each sentence and important sentences in each document. In addition to returning the probability of company 2 being a subsidiary of company 1 the model as returns important sentences which triggered its prediction. For instance when it says Orcale Corp is the parent of Microsys it can also return that
Orcale Corp’s Microsys customer support portal was seen communicating with a server known to be used by the carbanak gang, is the sentence which triggered its prediction.

Posted minimalistic, stefanl19, clstl, andreytagarev7 CommentsPosted in Team solutionsTeam solutions

CASE Ontotext, Team CENTROIDA

This paper presents a DNN-based approach to learn entities relations from distant-labeled free text. The proposed approach presents task-specific data cleaning, which despite effective in removing textual noise is preserving semantics necessary for the training process. The cleaned-up dataset is then used to build a number of bLSTM attention-based DNN models, hyper-tuned using recall as an optimization objective. The resulting models are then joined into an ensemble that deliver our best result