The first method to improve the MD performance is fine-tuning the four datasets on the Transformer-based pre-trained LLM DistilBERT (Sanh et al. 2019), which is an efficient variant of BERT. This model is smaller and faster while maintaining 97% of BERTs performance. The training for this Transformers approach is threefold: The model-specific tokenizer maps the dataset’s text tokens to indexes, the transformer converts these indexes to contextual embeddings and the pre-trained head is fine-tuned on the MD task (Wolf et al. 2020). The fine-tuning procedure consisted of training with 70% of the data and evaluating with the remaining 30% for each dataset. For hyperparameters, the batch size of 32, the learning rate of 2e-5 and 5 training-epochs yielded best results.