DISC-LawLLM: Fine-tuning Large Language Models for Intelligent Legal Services (2309.11325v2)
Abstract: We propose DISC-LawLLM, an intelligent legal system utilizing LLMs to provide a wide range of legal services. We adopt legal syllogism prompting strategies to construct supervised fine-tuning datasets in the Chinese Judicial domain and fine-tune LLMs with legal reasoning capability. We augment LLMs with a retrieval module to enhance models' ability to access and utilize external legal knowledge. A comprehensive legal benchmark, DISC-Law-Eval, is presented to evaluate intelligent legal systems from both objective and subjective dimensions. Quantitative and qualitative results on DISC-Law-Eval demonstrate the effectiveness of our system in serving various users across diverse legal scenarios. The detailed resources are available at https://github.com/FudanDISC/DISC-LawLLM.
- Baichuan-inc. 2023. Baichuan-13b. https://github.com/baichuan-inc/Baichuan-13B.
- Lexnlp: Natural language processing and information extraction for legal and regulatory texts. Research Handbook on Big Data Law.
- CAIL. 2020. Cail2020. https://github.com/china-ai-law-challenge/CAIL2020.
- CAIL. 2022. Cail2022. https://github.com/china-ai-law-challenge/CAIL2022.
- Joint entity and relation extraction for legal documents with legal feature enhancement. In Proceedings of the 28th International Conference on Computational Linguistics, pages 1561–1571, Barcelona, Spain (Online). International Committee on Computational Linguistics.
- Chatlaw: Open-source legal large language model with integrated external knowledge bases.
- Efficient and effective text encoding for chinese llama and alpaca. arXiv preprint arXiv:2304.08177.
- Glm: General language model pretraining with autoregressive blank infilling. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 320–335.
- Cjrc: A reliable human-annotated benchmark dataset for chinese judicial reading comprehension. In Chinese Computational Linguistics, pages 439–451, Cham. Springer International Publishing.
- Anne von der Lieth Gardner. 1987. An artificial intelligence approach to legal reasoning. MIT press.
- Lawyer llama technical report. ArXiv, abs/2305.15062.
- IDEA-CCNL. 2021. Fengshenbang-lm. https://github.com/IDEA-CCNL/Fengshenbang-LM.
- Incorporating argument-level interactions for persuasion comments evaluation using co-attention model. In Proceedings of the 27th International Conference on Computational Linguistics, pages 3703–3714.
- Discrete argument representation learning for interactive argument pair identification. arXiv preprint arXiv:1911.01621.
- Cong Jiang and Xiaolei Yang. 2023. Legal syllogism prompting: Teaching large language models for legal judgment prediction. arXiv preprint arXiv:2307.08321.
- Answering legal questions by learning neural attentive text representation. In Proceedings of the 28th International Conference on Computational Linguistics, pages 988–998.
- Haitao Li. 2023. Lexilaw. https://github.com/CSHaitao/LexiLaw.
- Lawgpt. https://github.com/LiuHC0428/LAW_GPT.
- Lecard: a legal case retrieval dataset for chinese law system. In Proceedings of the 44th international ACM SIGIR conference on research and development in information retrieval, pages 2342–2348.
- Meta. 2023. Llama. https://github.com/facebookresearch/llama.
- Crosslingual generalization through multitask finetuning.
- OpenAI. 2022. Chatgpt: Optimizing language models for dialogue.
- OpenAI. 2023. Gpt-4 technical report.
- Instruction tuning with gpt-4. arXiv preprint arXiv:2304.03277.
- Richard A Posner. 1990. The problems of jurisprudence. Harvard University Press.
- Deepspeed: System optimizations enable training deep learning models with over 100 billion parameters. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 3505–3506.
- Pengxiao Song. 2023. Lawgpt. https://github.com/pengxiao-song/LaWGPT.
- Yun Song and Zhongyu Wei. 2021. Inferring association between alcohol addiction and defendant’s emotion based on sound at court. Frontiers in Psychology, 12:669780.
- Self-instruct: Aligning language model with self generated instructions. arXiv preprint arXiv:2212.10560.
- CAIL2018: A large-scale legal dataset for judgment prediction. CoRR, abs/1807.02478.
- Jianxin Yang. 2023. Firefly. https://github.com/yangjianxin1/Firefly.
- Legal judgment prediction via multi-perspective bi-feedback network. arXiv preprint arXiv:1905.03969.
- LEVEN: A large-scale Chinese legal event detection dataset. In Findings of the Association for Computational Linguistics: ACL 2022, pages 183–201, Dublin, Ireland. Association for Computational Linguistics.
- Interpretable charge predictions for criminal cases: Learning to generate court views from fact descriptions. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 1854–1864, New Orleans, Louisiana. Association for Computational Linguistics.
- ymcui. 2023. Chinese-llama-alpaca-2. https://github.com/ymcui/Chinese-LLaMA-Alpaca-2.
- Overview of smp-cail2020-argmine: The interactive argument-pair extraction in judgement document challenge. Data Intelligence, 3(2):287–307.
- Chinese open instruction generalist: A preliminary release.
- Judging llm-as-a-judge with mt-bench and chatbot arena.
- How does nlp benefit legal system: A summary of legal artificial intelligence. arXiv preprint arXiv:2004.12158.
- Jec-qa: A legal-domain question answering dataset. In Proceedings of AAAI.