TasksSotADatasetsPapersMethodsSubmitAbout
Papers With Code 2

A community resource for machine learning research: papers, code, benchmarks, and state-of-the-art results.

Explore

Notable BenchmarksAll SotADatasetsPapersMethods

Community

Submit ResultsAbout

Data sourced from the PWC Archive (CC-BY-SA 4.0). Built by the community, for the community.

Papers/How to Fine-Tune BERT for Text Classification?

How to Fine-Tune BERT for Text Classification?

Chi Sun, Xipeng Qiu, Yige Xu, Xuanjing Huang

2019-05-14Text ClassificationSentiment AnalysisGeneral ClassificationLanguage Modelling
PaperPDFCodeCodeCode(official)CodeCodeCodeCodeCodeCodeCodeCodeCodeCodeCodeCode

Abstract

Language model pre-training has proven to be useful in learning universal language representations. As a state-of-the-art language model pre-training model, BERT (Bidirectional Encoder Representations from Transformers) has achieved amazing results in many language understanding tasks. In this paper, we conduct exhaustive experiments to investigate different fine-tuning methods of BERT on text classification task and provide a general solution for BERT fine-tuning. Finally, the proposed solution obtains new state-of-the-art results on eight widely-studied text classification datasets.

Results

TaskDatasetMetricValueModel
Sentiment AnalysisYelp Fine-grained classificationError28.62BERT_large+ITPT
Sentiment AnalysisYelp Fine-grained classificationError29.42BERT_base+ITPT
Sentiment AnalysisYelp Binary classificationError1.81BERT_large+ITPT
Sentiment AnalysisYelp Binary classificationError1.92BERT_base+ITPT
Sentiment AnalysisIMDbAccuracy95.79BERT_large+ITPT
Sentiment AnalysisIMDbAccuracy95.63BERT_base+ITPT
Text ClassificationSogou NewsAccuracy98.07BERT-ITPT-FiT
Text ClassificationTREC-6Error3.2BERT-ITPT-FiT
Text ClassificationDBpediaError0.68BERT-ITPT-FiT
Text ClassificationAG NewsError4.8BERT-ITPT-FiT
Text ClassificationYahoo! AnswersAccuracy77.62BERT-ITPT-FiT
ClassificationSogou NewsAccuracy98.07BERT-ITPT-FiT
ClassificationTREC-6Error3.2BERT-ITPT-FiT
ClassificationDBpediaError0.68BERT-ITPT-FiT
ClassificationAG NewsError4.8BERT-ITPT-FiT
ClassificationYahoo! AnswersAccuracy77.62BERT-ITPT-FiT

Related Papers

Visual-Language Model Knowledge Distillation Method for Image Quality Assessment2025-07-21Making Language Model a Hierarchical Classifier and Generator2025-07-17AdaptiSent: Context-Aware Adaptive Attention for Multimodal Aspect-Based Sentiment Analysis2025-07-17VisionThink: Smart and Efficient Vision Language Model via Reinforcement Learning2025-07-17The Generative Energy Arena (GEA): Incorporating Energy Awareness in Large Language Model (LLM) Human Evaluations2025-07-17Inverse Reinforcement Learning Meets Large Language Model Post-Training: Basics, Advances, and Opportunities2025-07-17Assay2Mol: large language model-based drug design using BioAssay context2025-07-16Describe Anything Model for Visual Question Answering on Text-rich Images2025-07-16