Decomposition for Enhancing Attention: Improving LLM-based Text-to-SQL through Workflow Paradigm (2402.10671v3)
Abstract: In-context learning of large-LLMs has achieved remarkable success in the field of natural language processing, while extensive case studies reveal that the single-step chain-of-thought prompting approach faces challenges such as attention diffusion and inadequate performance in complex tasks like text-to-SQL. To improve the contextual learning capabilities of LLMs in text-to-SQL, a workflow paradigm method is proposed, aiming to enhance the attention and problem-solving scope of LLMs through decomposition. Specifically, the information determination module for eliminating redundant information and the brand-new prompt structure based on problem classification greatly enhance the model's attention. Additionally, the inclusion of self-correction and active learning modules greatly expands the problem-solving scope of LLMs, hence improving the upper limit of LLM-based approaches. Extensive experiments conducted on three datasets demonstrate that our approach outperforms other methods by a significant margin. About 2-3 percentage point improvements compared to the existing baseline on the Spider Dev, Spider-Realistic, and Bird Dev datasets and new SOTA results on the Spider Test dataset are achieved. Our code is available on GitHub: \url{https://github.com/FlyingFeather/DEA-SQL}.
- LGESQL: Line graph enhanced text-to-SQL model with mixed local and non-local relations. In ACL.
- Shuaichen Chang and Eric Fosler-Lussier. 2023a. How to prompt llms for text-to-sql: A study in zero-shot, single-domain, and cross-domain settings. arXiv preprint arXiv:2305.11853.
- Shuaichen Chang and Eric Fosler-Lussier. 2023b. Selective demonstrations for cross-domain text-to-sql. In Findings of EMNLP.
- Structure-grounded pretraining for text-to-sql. In NAACL.
- A survey for in-context learning. arXiv preprint arXiv:2301.00234.
- C3: Zero-shot text-to-sql with chatgpt. arXiv preprint arXiv:2307.07306.
- Improving text-to-sql evaluation methodology. In ACL.
- Text-to-sql empowered by large language models: A benchmark evaluation. arXiv preprint arXiv:2308.15363.
- Towards complex text-to-sql in cross-domain database with intermediate representation. In ACL.
- S2sql: Injecting syntax to question-schema interaction graph encoder for text-to-sql parsers. In ACL.
- Re-examining the role of schema linking in text-to-sql. In EMNLP.
- Resdsql: Decoupling schema linking and skeleton parsing for text-to-sql. In AAAI.
- Graphix-t5: Mixing pre-trained transformers with graph-aware layers for text-to-sql parsing. In ACL.
- Few-shot aspect category sentiment analysis via meta-learning. ACM Transactions on Information Systems.
- A comprehensive evaluation of chatgpt’s zero-shot text-to-sql capability. arXiv preprint arXiv:2303.13547.
- OpenAI. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774.
- Training language models to follow instructions with human feedback. In NeurIPS.
- Mohammadreza Pourreza and Davood Rafiei. 2023. DIN-SQL: Decomposed in-context learning of text-to-SQL with self-correction. In NeurIPS.
- A survey on text-to-sql parsing: Concepts, methods, and future directions. arXiv preprint arXiv:2208.13629.
- Evaluating the text-to-sql capabilities of large language models. arXiv preprint arXiv:2204.00498.
- Picard: Parsing incrementally for constrained auto-regressive decoding from language models. In EMNLP.
- Exploring chain-of-thought style prompting for text-to-sql. In EMNLP.
- Rat-sql: Relation-aware schema encoding and linking for text-to-sql parsers. In ACL.
- Mac-sql: Multi-agent collaboration for text-to-sql. arXiv preprint arXiv:2312.11242.
- Chain-of-thought prompting elicits reasoning in large language models. In NeurIPS.
- A paradigm shift in machine translation: Boosting translation performance of large language models. arXiv preprint arXiv:2309.11674.
- Spider: A large-scale human-labeled dataset for complex and cross-domain semantic parsing and text-to-sql task. In EMNLP.
- Act-sql: In-context learning for text-to-sql with automatically-generated chain-of-thought. In Findings of EMNLP.
- Semantic evaluation for text-to-sql with distilled test suite. In EMNLP.