2000 character limit reached
SemEval-2024 Task 8: Weighted Layer Averaging RoBERTa for Black-Box Machine-Generated Text Detection (2402.15873v2)
Published 24 Feb 2024 in cs.CL
Abstract: This document contains the details of the authors' submission to the proceedings of SemEval 2024's Task 8: Multigenerator, Multidomain, and Multilingual Black-Box Machine-Generated Text Detection Subtask A (monolingual) and B. Detection of machine-generated text is becoming an increasingly important task, with the advent of LLMs. In this paper, we lay out how using weighted averages of RoBERTa layers lets us capture information about text that is relevant to machine-generated text detection.
- 2020. Intrinsic dimensionality explains the effectiveness of language model fine-tuning.
- 1997. Long short-term memory. Neural Computation, 9(8):1735–1780.
- 2021. Lora: Low-rank adaptation of large language models.
- 2019. Roberta: A robustly optimized bert pretraining approach.
- 2018. Deep contextualized word representations.
- 2022. Effect of scale on catastrophic forgetting in neural networks. In International Conference on Learning Representations.
- 2020. A primer in bertology: What we know about how bert works.
- 2023. M4: Multi-generator, multi-domain, and multi-lingual black-box machine-generated text detection.
- 2023. Adalora: Adaptive budget allocation for parameter-efficient fine-tuning.