Instruction Tuning with Lexicons for Zero-Shot Style Classification

Abstract: Style is used to convey authors' intentions and attitudes. Despite the success of large pre-trained LLMs on style classification, prior work relies on fine-tuning with labeled examples. Prompting LLMs to classify style without fine-tuning is challenging because language styles can be difficult to define. In this study, we investigate the effectiveness of style lexicons as a means for instructing LLMs how to identify new styles that are unseen during training. Our experiments show that lexicon-based instructions improve transfer zero-shot performance significantly. We will release our code and data.