Where It Really Matters: Few-Shot Environmental Conservation Media Monitoring for Low-Resource Languages (2402.11818v1)
Abstract: Environmental conservation organizations routinely monitor news content on conservation in protected areas to maintain situational awareness of developments that can have an environmental impact. Existing automated media monitoring systems require large amounts of data labeled by domain experts, which is only feasible at scale for high-resource languages like English. However, such tools are most needed in the global south where news of interest is mainly in local low-resource languages, and far fewer experts are available to annotate datasets sustainably. In this paper, we propose NewsSerow, a method to automatically recognize environmental conservation content in low-resource languages. NewsSerow is a pipeline of summarization, in-context few-shot classification, and self-reflection using LLMs. Using at most 10 demonstration example news articles in Nepali, NewsSerow significantly outperforms other few-shot methods and achieves comparable performance with models fully fine-tuned using thousands of examples. The World Wide Fund for Nature (WWF) has deployed NewsSerow for media monitoring in Nepal, significantly reducing their operational burden, and ensuring that AI tools for conservation actually reach the communities that need them the most. NewsSerow has also been deployed for countries with other languages like Colombia.
- Language Models are Few-Shot Learners. In Larochelle, H.; Ranzato, M.; Hadsell, R.; Balcan, M.; and Lin, H., eds., Advances in Neural Information Processing Systems, volume 33, 1877–1901. Curran Associates, Inc.
- Unsupervised Cross-lingual Representation Learning at Scale. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 8440–8451. Online: Association for Computational Linguistics.
- BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), 4171–4186. Minneapolis, Minnesota: Association for Computational Linguistics.
- ChatGPT outperforms crowd workers for text-annotation tasks. Proceedings of the National Academy of Sciences of the United States of America, 120.
- Data Study Group Final Report: WWF.
- Multi-Dimensional Evaluation of Text Summarization with In-Context Learning. In Findings of the Association for Computational Linguistics: ACL 2023. Toronto, Canada: Association for Computational Linguistics.
- Jang, E. 2023. Can LLMs Critique and Iterate on Their Own Outputs? evjang.com.
- Language Models (Mostly) Know What They Know. arXiv:2207.05221.
- NewsPanda: Media Monitoring for Timely Conservation Action. Proceedings of the AAAI Conference on Artificial Intelligence, 37(13): 15528–15536.
- Large Language Models are Zero-Shot Reasoners. ArXiv, abs/2205.11916.
- AlpacaEval: An Automatic Evaluator of Instruction-following Models. https://github.com/tatsu-lab/alpaca˙eval.
- Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing. CoRR, abs/2107.13586.
- RoBERTa: A Robustly Optimized BERT Pretraining Approach. arXiv:1907.11692.
- Decoupled Weight Decay Regularization. In International Conference on Learning Representations.
- Don’t Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarization. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, 1797–1807. Brussels, Belgium: Association for Computational Linguistics.
- Nenkova, A. 2005. Automatic Text Summarization of Newswire: Lessons Learned from the Document Understanding Conference. In AAAI Conference on Artificial Intelligence.
- Show Your Work: Scratchpads for Intermediate Computation with Language Models. CoRR, abs/2112.00114.
- OpenAI. 2023. GPT-4 Technical Report. ArXiv, abs/2303.08774.
- Reflexion: Language Agents with Verbal Reinforcement Learning. arXiv:2303.11366.
- Llama 2: Open Foundation and Fine-Tuned Chat Models. ArXiv, abs/2307.09288.
- Chain of Thought Prompting Elicits Reasoning in Large Language Models. In Oh, A. H.; Agarwal, A.; Belgrave, D.; and Cho, K., eds., Advances in Neural Information Processing Systems.
- Generating Sequences by Learning to Self-Correct. ArXiv, abs/2211.00053.