2000 character limit reached
A Differentiable Integer Linear Programming Solver for Explanation-Based Natural Language Inference (2404.02625v1)
Published 3 Apr 2024 in cs.CL, cs.AI, and cs.LG
Abstract: Integer Linear Programming (ILP) has been proposed as a formalism for encoding precise structural and semantic constraints for Natural Language Inference (NLI). However, traditional ILP frameworks are non-differentiable, posing critical challenges for the integration of continuous language representations based on deep learning. In this paper, we introduce a novel approach, named Diff-Comb Explainer, a neuro-symbolic architecture for explanation-based NLI based on Differentiable BlackBox Combinatorial Solvers (DBCS). Differently from existing neuro-symbolic solvers, Diff-Comb Explainer does not necessitate a continuous relaxation of the semantic constraints, enabling a direct, more precise, and efficient incorporation of neural representations into the ILP formulation. Our experiments demonstrate that Diff-Comb Explainer achieves superior performance when compared to conventional ILP solvers, neuro-symbolic black-box solvers, and Transformer-based encoders. Moreover, a deeper analysis reveals that Diff-Comb Explainer can significantly improve the precision, consistency, and faithfulness of the constructed explanations, opening new opportunities for research on neuro-symbolic architectures for explainable and transparent NLI in complex domains.
- Differentiable convex optimization layers. In Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc.
- Think you have solved question answering? try arc, the ai2 reasoning challenge. arXiv preprint arXiv:1803.05457.
- Combining retrieval, statistics, and inference to answer elementary science questions. In AAAI, pages 2580–2586. Citeseer.
- Transformers as soft reasoners over language. In Proceedings of the Twenty-Ninth International Conference on International Joint Conferences on Artificial Intelligence, pages 3882–3890.
- Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186.
- Worldtree: A corpus of explanation graphs for elementary science questions supporting multi-hop inference. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018).
- Billion-scale similarity search with gpus. CoRR, abs/1702.08734.
- Question answering via integer programming over semi-structured knowledge. In Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence, pages 1145–1152.
- Question answering as global reasoning over semantic abstractions. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 32.
- Unifiedqa: Crossing format boundaries with a single qa system. arXiv preprint arXiv:2005.00700.
- Answering complex questions using open information extraction. arXiv preprint arXiv:1704.05572.
- Exploiting explicit paths for multi-hop reading comprehension. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 2737–2747, Florence, Italy. Association for Computational Linguistics.
- L. Lovász and A. Schrijver. 1991. Cones of matrices and set-functions and 0-1 optimization. SIAM JOURNAL ON OPTIMIZATION, 1:166–190.
- Learning to attend on essential terms: An enhanced retriever-reader model for open-domain question answering. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 335–344.
- Comboptnet: Fit the right np-hard problem by learning integer programming constraints. In International Conference on Machine Learning, pages 8443–8453. PMLR.
- Differentiation of blackbox combinatorial solvers. In International Conference on Learning Representations.
- Sentence-bert: Sentence embeddings using siamese bert-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics.
- Prover: Proof generation for interpretable reasoning over rules. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 122–136.
- multiprover: Generating multiple proofs for improved interpretability in rule reasoning. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 3662–3677.
- Conceptnet 5.5: An open multilingual graph of general knowledge. In Thirty-first AAAI conference on artificial intelligence.
- Diff-Explainer: Differentiable Convex Optimization for Explainable Multi-hop Inference. Transactions of the Association for Computational Linguistics, 10:1103–1a119.
- A survey on explainability in machine reading comprehension. arXiv preprint arXiv:2010.00389.
- Explainable inference over grounding-abstract chains for science questions. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 1–12.
- Hybrid autoregressive inference for scalable multi-hop explanation regeneration. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 11403–11411.
- Unification-based reconstruction of multi-hop explanations for science questions. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 200–211.
- Huggingface’s transformers: State-of-the-art natural language processing. CoRR, abs/1910.03771.
- Worldtree v2: A corpus of science-domain structured explanations and inference patterns supporting multi-hop inference. In Proceedings of The 12th Language Resources and Evaluation Conference, pages 5456–5473.
- Alignment over heterogeneous embeddings for question answering. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 2681–2691.
- Quick and (not so) dirty: Unsupervised selection of justification sentences for multi-hop question answering. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 2578–2589, Hong Kong, China. Association for Computational Linguistics.
- Kg^ 2: Learning to reason science exam questions with contextual knowledge graph embeddings. arXiv preprint arXiv:1805.12393.