LlamBERT: Large-scale low-cost data annotation in NLP

Bálint Csanády, Lajos Muzsai, Péter Vedres, Zoltán Nádasdy, András Lukács

2024-03-23Text Classification Sentiment Analysis

Abstract

Large Language Models (LLMs), such as GPT-4 and Llama 2, show remarkable proficiency in a wide range of natural language processing (NLP) tasks. Despite their effectiveness, the high costs associated with their use pose a challenge. We present LlamBERT, a hybrid approach that leverages LLMs to annotate a small subset of large, unlabeled databases and uses the results for fine-tuning transformer encoders like BERT and RoBERTa. This strategy is evaluated on two diverse datasets: the IMDb review dataset and the UMLS Meta-Thesaurus. Our results indicate that the LlamBERT approach slightly compromises on accuracy while offering much greater cost-effectiveness.

Results

Task	Dataset	Metric	Value	Model
Sentiment Analysis	IMDb	Accuracy	96.68	RoBERTa-large with LlamBERT
Sentiment Analysis	IMDb	Accuracy	96.54	RoBERTa-large
Sentiment Analysis	IMDb	Accuracy	95.39	Llama-2-70b-chat (0-shot)

Related Papers

Making Language Model a Hierarchical Classifier and Generator2025-07-17 AdaptiSent: Context-Aware Adaptive Attention for Multimodal Aspect-Based Sentiment Analysis2025-07-17 AI Wizards at CheckThat! 2025: Enhancing Transformer-Based Embeddings with Sentiment for Subjectivity Detection in News Articles2025-07-15 DCR: Quantifying Data Contamination in LLMs Evaluation2025-07-15 SentiDrop: A Multi Modal Machine Learning model for Predicting Dropout in Distance Learning2025-07-14 GNN-CNN: An Efficient Hybrid Model of Convolutional and Graph Neural Networks for Text Representation2025-07-10 The Trilemma of Truth in Large Language Models2025-06-30 Robustness of Misinformation Classification Systems to Adversarial Examples Through BeamAttack2025-06-30