Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
149 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Tree-Averaging Algorithms for Ensemble-Based Unsupervised Discontinuous Constituency Parsing (2403.00143v2)

Published 29 Feb 2024 in cs.CL, cs.AI, and cs.LG

Abstract: We address unsupervised discontinuous constituency parsing, where we observe a high variance in the performance of the only previous model in the literature. We propose to build an ensemble of different runs of the existing discontinuous parser by averaging the predicted trees, to stabilize and boost performance. To begin with, we provide comprehensive computational complexity analysis (in terms of P and NP-complete) for tree averaging under different setups of binarity and continuity. We then develop an efficient exact algorithm to tackle the task, which runs in a reasonable time for all samples in our experiments. Results on three datasets show our method outperforms all baselines in all metrics; we also provide in-depth analyses of our approach.

Definition Search Book Streamline Icon: https://streamlinehq.com
References (49)
  1. Computational Complexity: A Modern Approach. Cambridge University Press, 2009. URL https://www.cambridge.org/us/catalogue/catalogue.asp?isbn=9780521424264.
  2. Mathematical Statistics: Basic Ideas and Selected Topics. CRC Press, 2015. URL https://bickel.stat.berkeley.edu/teaching.
  3. Rens Bod. From exemplar to grammar: A probabilistic analogy-based model of language learning. Cognitive Science, 33(5):752–793, 2009. URL https://onlinelibrary.wiley.com/doi/10.1111/j.1551-6709.2009.01031.x.
  4. The TIGER treebank. In Proceedings of the Workshop on Treebanks and Linguistic Theories, pages 24–41, 2002. URL https://api.semanticscholar.org/CorpusID:6209052.
  5. Leo Breiman. Bagging predictors. Machine Learning, 24:123–140, 1996a. URL https://link.springer.com/article/10.1007/BF00058655.
  6. Leo Breiman. Heuristics of instability and stabilization in model selection. The Annals of Statistics, 24(6):2350–2383, 1996b. URL https://projecteuclid.org/journals/annals-of-statistics/volume-24/issue-6/Heuristics-of-instability-and-stabilization-in-model-selection/10.1214/aos/1032181158.full.
  7. Unsupervised parsing via constituency tests. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, pages 4798–4808, 2020. URL https://aclanthology.org/2020.emnlp-main.389.
  8. Andrew Carnie. Syntax: A Generative Introduction. Wiley Blackwell, 2 edition, 2007. URL https://www.wiley.com/en-ca/Syntax:+A+Generative+Introduction,+4th+Edition-p-9781119569237.
  9. Eugene Charniak. A maximum-entropy-inspired parser. In the North American Chapter of the Annual Meeting of the Association for Computational Linguistics, pages 132–139, 2000. URL https://aclanthology.org/A00-2018.
  10. Alexander Clark. Unsupervised induction of stochastic context-free grammars using distributional clustering. In Proceedings of the Annual Meeting of the Association for Computational Linguistics Workshop on Computational Natural Language Learning, 2001. URL https://aclanthology.org/W01-0713.
  11. Data-oriented parsing with discontinuous constituents and function tags. Journal of Language Modelling, 4(1):57–111, 2016. URL https://jlm.ipipan.waw.pl/index.php/JLM/article/view/100.
  12. Discontinuous constituent parsing with pointer networks. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 7724–7731, 2020. URL https://ojs.aaai.org/index.php/AAAI/article/view/6275.
  13. Reducing discontinuous to continuous parsing with pointer network reordering. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, pages 10570–10578, 2021. URL https://aclanthology.org/2021.emnlp-main.825.
  14. Discontinuous grammar as a foreign language. Neurocomputing, 524:43–58, 2023. URL https://www.sciencedirect.com/science/article/pii/S092523122201551X?via%3Dihub.
  15. John Goldsmith. Unsupervised learning of the morphology of a natural language. Computational Linguistics, 27(2):153–198, 2001. URL https://direct.mit.edu/coli/article/27/2/153/1711/Unsupervised-Learning-of-the-Morphology-of-a.
  16. The Elements of Statistical Learning: Data Mining, Inference, and Prediction. Springer, 2009. URL https://link.springer.com/content/pdf/10.1007/978-0-387-84858-7_10.pdf.
  17. Computing partitions with applications to the knapsack problem. Journal of the Association for Computing Machinery, 21(2):277–292, 1974. URL https://dl.acm.org/doi/10.1145/321812.321823.
  18. Aravind K Joshi. Tree adjoining grammars: How much context-sensitivity is required to provide reasonable structural descriptions? In Natural Language Parsing, pages 206–250. 1985. URL https://www.cambridge.org/core/books/natural-language-parsing/tree-adjoining-grammars-how-much-contextsensitivity-is-required-to-provide-reasonable-structural-descriptions/81BFD6DAC6B0CB24A3042A06E964F2E1.
  19. Data-driven parsing with probabilistic linear context-free rewriting systems. In Proceedings of the International Conference on Computational Linguistics, pages 537–545, 2010. URL https://aclanthology.org/C10-1061.
  20. Neural unsupervised parsing beyond English. In Proceedings of the Workshop on Deep Learning Approaches for Low-Resource Natural Language Processing, pages 209–218, 2019. URL https://aclanthology.org/D19-6123.
  21. Compound probabilistic context-free grammars for grammar induction. In Proceedings of the Annual Meeting of the Association for Computational Linguistics, pages 2369–2385, 2019a. URL https://aclanthology.org/P19-1228.
  22. Unsupervised recurrent neural network grammars. In Proceedings of the North American Chapter of the Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pages 1105–1117, 2019b. URL https://aclanthology.org/N19-1114.
  23. Dan Klein. The Unsupervised Learning of Natural Language Structure. Stanford University, 2005. URL https://www.proquest.com/openview/4a4c32a46d9fab258d8114cede8311ed/1?pq-origsite=gscholar&cbl=18750&diss=y.
  24. A generative constituent-context model for improved grammar induction. In Proceedings of the Annual Meeting of the Association for Computational Linguistics, page 128–135, 2002. URL https://dl.acm.org/doi/10.3115/1073083.1073106.
  25. Minimum Bayes-risk decoding for statistical machine translation. In Proceedings of the Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics, pages 169–176, 2004. URL https://aclanthology.org/N04-1022.
  26. An imitation learning approach to unsupervised parsing. In Proceedings of the Annual Meeting of the Association for Computational Linguistics, pages 3485–3492, 2019. URL https://aclanthology.org/P19-1338.
  27. Contextual distortion reveals constituency: Masked language models are implicit parsers. In Proceedings of the Annual Meeting of the Association for Computational Linguistics, pages 5208–5222, 2023. URL https://aclanthology.org/2023.acl-long.285.
  28. Wolfgang Maier. Direct parsing of discontinuous constituents in German. In Proceedings of the NAACL HLT Workshop on Statistical Parsing of Morphologically-Rich Languages, pages 58–66, 2010. URL https://aclanthology.org/W10-1407.
  29. PLCFRS parsing revisited: Restricting the fan-out to two. In Proceedings of the International Workshop on Tree Adjoining Grammars and Related Formalisms, pages 126–134, 2012. URL https://aclanthology.org/W12-4615.
  30. James D. McCawley. Parentheticals and discontinuous constituent structure. Linguistic Inquiry, 13(1):91–106, 1982. URL http://www.jstor.org/stable/4178261.
  31. Helix: Unsupervised grammar induction for structured activity recognition. In Proceedings of the International Conference on Data Mining, pages 1194–1199, 2011. URL https://ieeexplore.ieee.org/abstract/document/6137337.
  32. Improved inference for unlexicalized parsing. In Proceedings of Human Language Technologies: The Conference of the North American Chapter of the Association for Computational Linguistics, pages 404–411, 2007. URL https://aclanthology.org/N07-1051.
  33. Ensemble distillation for unsupervised constituency parsing. arXiv preprint arXiv:2310.01717, 2023. URL https://arxiv.org/abs/2310.01717.
  34. Neural language modeling by jointly learning syntax and lexicon. In International Conference on Representation Learning, 2018a. URL https://openreview.net/forum?id=rkgOLb-0W.
  35. Straight to the tree: Constituency parsing with neural syntactic distance. In Proceedings of the Annual Meeting of the Association for Computational Linguistics, pages 1171–1180, 2018b. URL https://aclanthology.org/P18-1108.
  36. Ordered neurons: Integrating tree structures into recurrent neural networks. In International Conference on Representation Learning, 2019. URL https://openreview.net/forum?id=B1l6qiR5F7.
  37. Natural language to code translation with execution. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, pages 3533–3546, 2022. URL https://aclanthology.org/2022.emnlp-main.231.
  38. An annotation scheme for free word order languages. In Proceedings of the Applied Natural Language Processing, page 88–95, 1997. URL https://dl.acm.org/doi/10.3115/974557.974571.
  39. Unsupervised multilingual grammar induction. In Proceedings of the Joint Conference of the Annual Meeting of the Association for Computational Linguistics and the International Joint Conference on Natural Language Processing of the Asian Federation of Natural Language Processing, pages 73–81, 2009. URL https://aclanthology.org/P09-1009.
  40. Span-based LCFRS-2 parsing. In Proceedings of the International Conference on Parsing Technologies and the International Workshop on Parsing Technologies Shared Task on Parsing into Enhanced Universal Dependencies, pages 111–121, 2020. URL https://aclanthology.org/2020.iwpt-1.12.
  41. Thomas H Starks. An improved sign test for experiments in which neutral responses are possible. Technometrics, 21(4):525–530, 1979. URL http://www.jstor.org/stable/1268292.
  42. Loss minimization in parse reranking. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, pages 560–567, 2006. URL https://aclanthology.org/W06-1666.
  43. Masaru Tomita. Current Issues in Parsing Technology. Kluwer Academic Publishers, 1990. URL https://link.springer.com/book/10.1007/978-1-4615-3986-5.
  44. Large scale syntactic annotation of written dutch: Lassy. In Essential Speech and Language Technology for Dutch, pages 147–164. 2013. URL https://link.springer.com/chapter/10.1007/978-3-642-30910-6_9.
  45. Yannick Versley. Experiments with easy-first nonprojective constituent parsing. In Proceedings of the Joint Workshop on Statistical Parsing of Morphologically Rich Languages and Syntactic Analysis of Non-Canonical Languages, pages 39–53, 2014. URL https://aclanthology.org/W14-6104.
  46. Characterizing structural descriptions produced by various grammatical formalisms. In Proceedings of the Annual Meeting of the Association for Computational Linguistics, page 104–111, 1987. URL https://dl.acm.org/doi/10.3115/981175.981190.
  47. PCFGs can do better: Inducing probabilistic context-free grammars with many symbols. In Proceedings of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1487–1498, 2021. URL https://aclanthology.org/2021.naacl-main.117.
  48. Unsupervised discontinuous constituency parsing with mildly context-sensitive grammars. In Proceedings of the Annual Meeting of the Association for Computational Linguistics, pages 5747–5766, 2023. URL https://aclanthology.org/2023.acl-long.316.
  49. Daniel H. Younger. Recognition and parsing of context-free languages in time n3superscript𝑛3n^{3}italic_n start_POSTSUPERSCRIPT 3 end_POSTSUPERSCRIPT. Information and Control, 10(2):189–208, 1967. URL https://www.sciencedirect.com/science/article/pii/S001999586780007X?via%3Dihub.
Citations (1)
View on Semantic Scholar

Summary

We haven't generated a summary for this paper yet.

Ai Sparkles Streamline Icon: https://streamlinehq.com Summarize Now