Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
110 tokens/sec
GPT-4o
56 tokens/sec
Gemini 2.5 Pro Pro
44 tokens/sec
o3 Pro
6 tokens/sec
GPT-4.1 Pro
47 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Adapting Knowledge for Few-shot Table-to-Text Generation (2302.12468v3)

Published 24 Feb 2023 in cs.CL

Abstract: Pretrained LLMs (PLMs) have made remarkable progress in table-to-text generation tasks. However, the lack of domain-specific knowledge makes it challenging to bridge the topological gap between tabular data and text, especially in real-world applications with limited resources. To mitigate the limitation of insufficient labeled data, we propose a novel framework: Adapt-Knowledge-to-Generate (AKG). The core insight of AKG is to adapt unlabeled domain-specific knowledge into the model, which brings at least three benefits: (1) it injects representation of normal table-related descriptions to bridge the topological gap between tabular data and texts; (2) it enables us to use large amounts of unlabeled domain-specific knowledge fully, which can alleviate the PLMs' inherent shortcomings of lacking domain knowledge; (3) it allows us to design various tasks to employ the domain-specific knowledge. Extensive experiments and analyses are conducted on three open-domain, few-shot natural language generation (NLG) data sets: Humans, Songs, and Books. Compared to previous state-of-the-art approaches, our model achieves superior performance in terms of both fluency and accuracy as judged by human and automatic evaluations.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (54)
  1. A. Gatt and E. Krahmer, “Survey of the state of the art in natural language generation: Core tasks, applications and evaluation,” Journal of Artificial Intelligence Research, vol. 61, pp. 65–170, 2018.
  2. Z. Chen, W. Chen, C. Smiley, S. Shah, I. Borova, D. Langdon, R. Moussa, M. Beane, T.-H. Huang, B. R. Routledge et al., “Finqa: A dataset of numerical reasoning over financial data,” in Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, 2021, pp. 3697–3711.
  3. M. Ghazvininejad, C. Brockett, M.-W. Chang, B. Dolan, J. Gao, W.-t. Yih, and M. Galley, “A knowledge-grounded neural conversation model,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 32, no. 1, 2018.
  4. W. Chen, M.-W. Chang, E. Schlinger, W. Y. Wang, and W. W. Cohen, “Open question answering over tables and text,” in International Conference on Learning Representations, 2020.
  5. H. He, A. Balakrishnan, M. Eric, and P. Liang, “Learning symmetric collaborative dialogue agents with dynamic knowledge graph embeddings,” in Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2017, pp. 1766–1776.
  6. S. Wiseman, S. M. Shieber, and A. M. Rush, “Challenges in data-to-document generation,” in Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, 2017, pp. 2253–2263.
  7. S. Murakami, A. Watanabe, A. Miyazawa, K. Goshima, T. Yanase, H. Takamura, and Y. Miyao, “Learning to generate market comments from stock prices,” in Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2017, pp. 1374–1384.
  8. S. A. Hasan and O. Farri, “Clinical natural language processing with deep learning,” in Data science for healthcare.   Springer, 2019, pp. 147–171.
  9. Z. Guo, J. Zhou, J. Qi, M. Yan, Z. He, G. Zheng, Z. Lin, X. Wang, and C. Zhou, “Towards controlled table-to-text generation with scientific reasoning,” arXiv preprint arXiv:2312.05402, 2023.
  10. R. Lebret, D. Grangier, and M. Auli, “Neural text generation from structured data with application to the biography domain,” in Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, 2016, pp. 1203–1213.
  11. H. Iso, Y. Uehara, T. Ishigaki, H. Noji, E. Aramaki, I. Kobayashi, Y. Miyao, N. Okazaki, and H. Takamura, “Learning to select, track, and generate for data-to-text,” in Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019, pp. 2102–2113.
  12. A. Parikh, X. Wang, S. Gehrmann, M. Faruqui, B. Dhingra, D. Yang, and D. Das, “Totto: A controlled table-to-text generation dataset,” in Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2020, pp. 1173–1186.
  13. Z. Chen, H. Eavani, W. Chen, Y. Liu, and W. Y. Wang, “Few-shot nlg with pre-trained language model,” in Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 2020, pp. 183–190.
  14. H. Gong, Y. Sun, X. Feng, B. Qin, W. Bi, X. Liu, and T. Liu, “Tablegpt: Few-shot table-to-text generation with table structure reconstruction and content matching,” in Proceedings of the 28th International Conference on Computational Linguistics, 2020, pp. 1978–1988.
  15. X. L. Li and P. Liang, “Prefix-tuning: Optimizing continuous prompts for generation,” in Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), 2021, pp. 4582–4597.
  16. Y. Luo, M. Lu, G. Liu, and S. Wang, “Few-shot table-to-text generation with prefix-controlled generator,” in Proceedings of the 29th International Conference on Computational Linguistics, 2022, pp. 6493–6504.
  17. Y. Su, Z. Meng, S. Baker, and N. Collier, “Few-shot table-to-text generation with prototype memory,” in Findings of the Association for Computational Linguistics: EMNLP 2021, 2021, pp. 910–917.
  18. E. Reiter and R. Dale, “Building applied natural language generation systems,” Natural Language Engineering, vol. 3, no. 1, pp. 57–87, 1997.
  19. P. Liang, M. I. Jordan, and D. Klein, “Learning semantic correspondences with less supervision,” in Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP, 2009, pp. 91–99.
  20. M. Walker, O. Rambow, and M. Rogati, “Spot: A trainable sentence planner,” in Second Meeting of the North American Chapter of the Association for Computational Linguistics, 2001.
  21. W. Lu, H. T. Ng, and W. S. Lee, “Natural language generation with tree conditional random fields,” in Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing.   Singapore: Association for Computational Linguistics, Aug. 2009, pp. 400–409.
  22. T. Liu, K. Wang, L. Sha, B. Chang, and Z. Sui, “Table-to-text generation by structure-aware seq2seq learning,” in Thirty-Second AAAI Conference on Artificial Intelligence, 2018.
  23. C. Gardent, A. Shimorina, S. Narayan, and L. Perez-Beltrachini, “The webnlg challenge: Generating text from rdf data,” in Proceedings of the 10th International Conference on Natural Language Generation, 2017, pp. 124–133.
  24. J. Novikova, O. Dusek, and V. Rieser, “The e2e dataset: New challenges for end-to-end generation,” in 18th Annual Meeting of the Special Interest Group on Discourse and Dialogue.   Association for Computational Linguistics, 2017, pp. 201–206.
  25. A. See, P. J. Liu, and C. D. Manning, “Get to the point: Summarization with pointer-generator networks,” in Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2017, pp. 1073–1083.
  26. H. Elsahar, C. Gravier, and F. Laforest, “Zero-shot question generation from knowledge graphs for unseen predicates and entity types,” in Proceedings of NAACL-HLT, 2018, pp. 218–228.
  27. B.-H. Tseng, F. Kreyssig, P. Budzianowski, I. Casanueva, Y.-C. Wu, S. Ultes, and M. Gasic, “Variational cross-domain natural language generation for spoken dialogue systems,” in Proceedings of the 19th Annual SIGdial Meeting on Discourse and Dialogue, 2018, pp. 338–343.
  28. V.-K. Tran and M. Le Nguyen, “Dual latent variable model for low-resource natural language generation in dialogue systems,” in Proceedings of the 22nd Conference on Computational Natural Language Learning, 2018, pp. 21–30.
  29. C. Finn, P. Abbeel, and S. Levine, “Model-agnostic meta-learning for fast adaptation of deep networks,” in International conference on machine learning.   PMLR, 2017, pp. 1126–1135.
  30. F. Mi, M. Huang, J. Zhang, and B. Faltings, “Meta-learning for low-resource natural language generation in task-oriented dialogue systems,” in Proceedings of the 28th International Joint Conference on Artificial Intelligence, 2019, pp. 3151–3157.
  31. H. Yao, Y.-x. Wu, M. Al-Shedivat, and E. Xing, “Knowledge-aware meta-learning for low-resource text classification,” in Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, 2021, pp. 1814–1821.
  32. Y. Su, D. Vandyke, S. Wang, Y. Fang, and N. Collier, “Plan-then-generate: Controlled data-to-text generation via planning,” in Findings of the Association for Computational Linguistics: EMNLP 2021, 2021, pp. 895–909.
  33. N. Houlsby, A. Giurgiu, S. Jastrzebski, B. Morrone, Q. De Laroussilhe, A. Gesmundo, M. Attariyan, and S. Gelly, “Parameter-efficient transfer learning for nlp,” in International Conference on Machine Learning.   PMLR, 2019, pp. 2790–2799.
  34. E. J. Hu, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, W. Chen et al., “Lora: Low-rank adaptation of large language models,” in International Conference on Learning Representations.
  35. H. Liu, D. Tam, M. Mohammed, J. Mohta, T. Huang, M. Bansal, and C. Raffel, “Few-shot parameter-efficient fine-tuning is better and cheaper than in-context learning,” in Advances in Neural Information Processing Systems.
  36. E. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, and W. Chen, “Lora: Low-rank adaptation of large language models,” 2021.
  37. L. Qin, X. Xu, L. Wang, Y. Zhang, and W. Che, “Modularized pre-training for end-to-end task-oriented dialogue,” IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2023.
  38. D. Emelin, D. Bonadiman, S. Alqahtani, Y. Zhang, and S. Mansour, “Injecting domain knowledge in language models for task-oriented dialogue systems,” arXiv preprint arXiv:2212.08120, 2022.
  39. J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” arXiv preprint arXiv:1810.04805, 2018.
  40. M. Lewis, Y. Liu, N. Goyal, M. Ghazvininejad, A. Mohamed, O. Levy, V. Stoyanov, and L. Zettlemoyer, “Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension,” in Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 2020, pp. 7871–7880.
  41. J. Ye, X. Chen, N. Xu, C. Zu, Z. Shao, S. Liu, Y. Cui, Z. Zhou, C. Gong, Y. Shen et al., “A comprehensive capability analysis of gpt-3 and gpt-3.5 series models,” arXiv preprint arXiv:2303.10420, 2023.
  42. P. Yang, H. Fang, and J. Lin, “Anserini: Enabling the use of lucene for information retrieval research,” in Proceedings of the 40th international ACM SIGIR conference on research and development in information retrieval, 2017, pp. 1253–1256.
  43. A. Białecki, R. Muir, G. Ingersoll, and L. Imagination, “Apache lucene 4,” in SIGIR 2012 workshop on open source information retrieval, 2012, p. 17.
  44. T. Wolf, L. Debut, V. Sanh, J. Chaumond, C. Delangue, A. Moi, P. Cistac, T. Rault, R. Louf, M. Funtowicz et al., “Transformers: State-of-the-art natural language processing,” in Proceedings of the 2020 conference on empirical methods in natural language processing: system demonstrations, 2020, pp. 38–45.
  45. D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” in 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, Y. Bengio and Y. LeCun, Eds., 2015. [Online]. Available: http://arxiv.org/abs/1412.6980
  46. S. Ma, P. Yang, T. Liu, P. Li, J. Zhou, and X. Sun, “Key fact as pivot: A two-stage model for low resource table-to-text generation,” in Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019, pp. 2047–2057.
  47. C. Raffel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y. Zhou, W. Li, P. J. Liu et al., “Exploring the limits of transfer learning with a unified text-to-text transformer.” J. Mach. Learn. Res., vol. 21, no. 140, pp. 1–67, 2020.
  48. W. Zhao, Y. Liu, Y. Wan, and P. Yu, “Attend, memorize and generate: Towards faithful table-to-text generation in few shots,” in Findings of the Association for Computational Linguistics: EMNLP 2021.   Punta Cana, Dominican Republic: Association for Computational Linguistics, Nov. 2021, pp. 4106–4117. [Online]. Available: https://aclanthology.org/2021.findings-emnlp.347
  49. Y. Wu, F. Wei, S. Huang, Y. Wang, Z. Li, and M. Zhou, “Response generation by context-aware prototype editing,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, no. 01, 2019, pp. 7281–7288.
  50. K. Papineni, S. Roukos, T. Ward, and W.-J. Zhu, “Bleu: a method for automatic evaluation of machine translation,” in Proceedings of the 40th annual meeting of the Association for Computational Linguistics, 2002, pp. 311–318.
  51. C.-Y. Lin, “Rouge: A package for automatic evaluation of summaries,” in Text summarization branches out, 2004, pp. 74–81.
  52. B. Dhingra, M. Faruqui, A. Parikh, M.-W. Chang, D. Das, and W. Cohen, “Handling divergent reference texts when evaluating table-to-text generation,” in Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019, pp. 4884–4895.
  53. A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, I. Sutskever et al., “Language models are unsupervised multitask learners.”
  54. J. Pfeiffer, A. Rücklé, C. Poth, A. Kamath, I. Vulić, S. Ruder, K. Cho, and I. Gurevych, “Adapterhub: A framework for adapting transformers,” in Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP 2020): Systems Demonstrations.   Online: Association for Computational Linguistics, 2020, pp. 46–54. [Online]. Available: https://www.aclweb.org/anthology/2020.emnlp-demos.7
User Edit Pencil Streamline Icon: https://streamlinehq.com
Authors (7)
  1. Zhixin Guo (8 papers)
  2. Minyxuan Yan (2 papers)
  3. Jiexing Qi (9 papers)
  4. Jianping Zhou (9 papers)
  5. Ziwei He (13 papers)
  6. Guanjie Zheng (37 papers)
  7. Xinbing Wang (98 papers)
Citations (2)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now