Cardiff University at SemEval-2020 Task 6: fine-tuning BERT for domain-specific definition classification

Jeawak, Shelan, Espinosa-Anke, Luis and Schockaert, Steven 2020. Cardiff University at SemEval-2020 Task 6: fine-tuning BERT for domain-specific definition classification. Presented at: International Workshop on Semantic Evaluation (SemEval 2020), Barcelona, Spain, 12-13 December 2020.

Preview

PDF - Accepted Post-Print Version Download (121kB) | Preview

Abstract

We describe the system submitted to SemEval-2020 Task 6, Subtask 1. The aim of this subtask is to predict whether a given sentence contains a definition or not. Unsurprisingly, we found that strong results can be achieved by fine-tuning a pre-trained BERT language model. In this paper,we analyze the performance of this strategy. Among others, we show that results can be improved by using a two-step fine-tuning process, in which the BERT model is first fine-tuned on the full training set, and then further specialized towards a target domain.

Item Type:	Conference or Workshop Item (Paper)
Status:	In Press
Schools:	Computer Science & Informatics
Date of First Compliant Deposit:	17 August 2020
Date of Acceptance:	26 June 2020
Last Modified:	17 Aug 2020 15:45
URI:	http://orca-mwe.cf.ac.uk/id/eprint/134231

Actions (repository staff only)

Edit Item