2019 |
Zhang, Biao; Sennrich, Rico A Lightweight Recurrent Network for Sequence Modeling Inproceedings Korhonen, Anna; Traum, David; `a, {Lluís } M (Ed.): Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 1538–1548, Association for Computational Linguistics (ACL), 2019. @inproceedings{bfad036fcb75439ca27128cf9633c9ac, title = {A Lightweight Recurrent Network for Sequence Modeling}, author = {Biao Zhang and Rico Sennrich}, editor = {Anna Korhonen and David Traum and {Lluís } M{`a}rquez}, year = {2019}, date = {2019-01-01}, booktitle = {Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics}, pages = {1538–1548}, publisher = {Association for Computational Linguistics (ACL)}, abstract = {Recurrent networks have achieved great success on various sequential tasks with the assistance of complex recurrent units, but suffer from severe computational inefficiency due to weak parallelization. One direction to alleviate this issue is to shift heavy computations outside the recurrence. In this paper, we propose a lightweight recurrent network, or LRN.LRN uses input and forget gates to handle long-range dependencies as well as gradient vanishing and explosion, with all parameterrelated calculations factored outside the recurrence. The recurrence in LRN only manipulates the weight assigned to each token, tightly connecting LRN with self-attention networks.We apply LRN as a drop-in replacement of existing recurrent units in several neural sequential models. Extensive experiments on six NLP tasks show that LRN yields the best running efficiency with little or no loss in model performance. Source code is available at https://github.com/ bzhangGo/lrn. }, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } Recurrent networks have achieved great success on various sequential tasks with the assistance of complex recurrent units, but suffer from severe computational inefficiency due to weak parallelization. One direction to alleviate this issue is to shift heavy computations outside the recurrence. In this paper, we propose a lightweight recurrent network, or LRN.LRN uses input and forget gates to handle long-range dependencies as well as gradient vanishing and explosion, with all parameterrelated calculations factored outside the recurrence. The recurrence in LRN only manipulates the weight assigned to each token, tightly connecting LRN with self-attention networks.We apply LRN as a drop-in replacement of existing recurrent units in several neural sequential models. Extensive experiments on six NLP tasks show that LRN yields the best running efficiency with little or no loss in model performance. Source code is available at https://github.com/ bzhangGo/lrn. |
Grundkiewicz, Roman; Junczys-Dowmuntz, Marcin; Heafield, Kenneth Neural Grammatical Error Correction Systems with Unsupervised Pre-training on Synthetic Data Inproceedings Yannakoudakis, Helen; Kochmar, Ekaterina; Leacock, Claudia; Madnani, Nitin; á, Ildikó Pil; Zesch, Torsten (Ed.): Proceedings of the Fourteenth Workshop on Innovative Use of NLP for Building Educational Applications, pp. 252–263, Association for Computational Linguistics, 2019. @inproceedings{f4751308f4f8411692049c3aa4a37722, title = {Neural Grammatical Error Correction Systems with Unsupervised Pre-training on Synthetic Data}, author = {Roman Grundkiewicz and Marcin Junczys-Dowmuntz and Kenneth Heafield}, editor = {Helen Yannakoudakis and Ekaterina Kochmar and Claudia Leacock and Nitin Madnani and Ildikó Pil á and Torsten Zesch}, year = {2019}, date = {2019-01-01}, booktitle = {Proceedings of the Fourteenth Workshop on Innovative Use of NLP for Building Educational Applications}, pages = {252–263}, publisher = {Association for Computational Linguistics}, abstract = {Considerable effort has been made to address the data sparsity problem in neural grammatical error correction. In this work, we propose a simple and surprisingly effective unsupervised synthetic error generation method based onconfusion sets extracted from a spellchecker to increase the amount of training data. Synthetic data is used to pre-train a Transformer sequence-to-sequence model, which not only improves over a strong baseline trained on authenticerror-annotated data, but also enables the development of a practical GEC system in a scenario where little genuine error-annotated data is available. The developed systems placed first in the BEA19 shared task, achieving 69.47 and 64.24 F0:5 in the restricted and low-resource tracks respectively, both on the W&I+LOCNESS test set. On the popular CoNLL 2014 test set, we report state-of-theart results of 64.16 M2 for the submitted system, and 61.30 M2 for the constrained system trained on the NUCLE and Lang-8 data.}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } Considerable effort has been made to address the data sparsity problem in neural grammatical error correction. In this work, we propose a simple and surprisingly effective unsupervised synthetic error generation method based onconfusion sets extracted from a spellchecker to increase the amount of training data. Synthetic data is used to pre-train a Transformer sequence-to-sequence model, which not only improves over a strong baseline trained on authenticerror-annotated data, but also enables the development of a practical GEC system in a scenario where little genuine error-annotated data is available. The developed systems placed first in the BEA19 shared task, achieving 69.47 and 64.24 F0:5 in the restricted and low-resource tracks respectively, both on the W&I+LOCNESS test set. On the popular CoNLL 2014 test set, we report state-of-theart results of 64.16 M2 for the submitted system, and 61.30 M2 for the constrained system trained on the NUCLE and Lang-8 data. |
Silviu Vlad, ; Magdy, Walid Exploring Author Context for Detecting Intended vs Perceived Sarcasm Inproceedings Korhonen, Anna; Traum, David; `a, Lluís M (Ed.): Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 2854–2859, Association for Computational Linguistics (ACL), 2019. @inproceedings{559167e010eb42fd9768fd347a771048, title = {Exploring Author Context for Detecting Intended vs Perceived Sarcasm}, author = {{Silviu Vlad} Oprea and Walid Magdy}, editor = {Anna Korhonen and David Traum and Lluís M `a}, year = {2019}, date = {2019-01-01}, booktitle = {Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics}, pages = {2854–2859}, publisher = {Association for Computational Linguistics (ACL)}, abstract = {We investigate the impact of using author context on textual sarcasm detection. We define author context as the embedded representation of their historical posts on Twitter and suggest neural models that extract these representations. We experiment with two tweet datasets, one labelled manually for sarcasm, and the other via tag-based distant supervision. We achieve state-of-the-art performance on the second dataset, but not on the one labelled manually, indicating a difference between intended sarcasm, captured by distant supervision, and perceived sarcasm, captured by manual labelling.}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } We investigate the impact of using author context on textual sarcasm detection. We define author context as the embedded representation of their historical posts on Twitter and suggest neural models that extract these representations. We experiment with two tweet datasets, one labelled manually for sarcasm, and the other via tag-based distant supervision. We achieve state-of-the-art performance on the second dataset, but not on the one labelled manually, indicating a difference between intended sarcasm, captured by distant supervision, and perceived sarcasm, captured by manual labelling. |
Tang, Gongbo; Sennrich, Rico; Nivre, Joakim Understanding Neural Machine Translation by Simplification: The Case of Encoder-free Models Output type: Conference contribution Inproceedings Proceedings of the International Conference Recent Advances in Natural Language Processing, Association for Computational Linguistics (ACL), 2019. @inproceedings{b15de884a3dc4f59ac1ee6449fa714f5, title = {Understanding Neural Machine Translation by Simplification: The Case of Encoder-free Models Output type: Conference contribution}, author = {Gongbo Tang and Rico Sennrich and Joakim Nivre}, year = {2019}, date = {2019-01-01}, booktitle = {Proceedings of the International Conference Recent Advances in Natural Language Processing}, publisher = {Association for Computational Linguistics (ACL)}, abstract = {In this paper, we try to understand neural machine translation (NMT) via simplifying NMT architectures and training encoder-free NMT models. In an encoderfree model, the sums of word embeddings and positional embeddings represent the source. The decoder is a standard Transformer or recurrent neural network that directly attends to embeddings via attention mechanisms. Experimental results show (1) that the attention mechanism in encoder-free models acts as a strong feature extractor, (2) that the word embeddings in encoder-free models are competitive to those in conventional models, (3) that non-contextualized source representations lead to a big performance drop, and (4) that encoder-free models have different effects on alignment quality for German→English and Chinese→English.}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } In this paper, we try to understand neural machine translation (NMT) via simplifying NMT architectures and training encoder-free NMT models. In an encoderfree model, the sums of word embeddings and positional embeddings represent the source. The decoder is a standard Transformer or recurrent neural network that directly attends to embeddings via attention mechanisms. Experimental results show (1) that the attention mechanism in encoder-free models acts as a strong feature extractor, (2) that the word embeddings in encoder-free models are competitive to those in conventional models, (3) that non-contextualized source representations lead to a big performance drop, and (4) that encoder-free models have different effects on alignment quality for German→English and Chinese→English. |
Hosseini, Javad; Cohen, Shay; Johnson, Mark; Steedman, Mark Duality of Link Prediction and Entailment Graph Induction Inproceedings Korhonen, Anna; Traum, David; `a, Lluís M (Ed.): Proceedings of the 57th Annual Conference of the Association for Computational Linguistics (long papers), pp. 4736–4746, ACL Anthology, 2019, ISBN: 978-1-950737-48-2. @inproceedings{4e94495c5c35468386d4ed32c9e9b1f7, title = {Duality of Link Prediction and Entailment Graph Induction}, author = {Javad Hosseini and Shay Cohen and Mark Johnson and Mark Steedman}, editor = {Anna Korhonen and David Traum and Lluís M `a}, isbn = {978-1-950737-48-2}, year = {2019}, date = {2019-01-01}, booktitle = {Proceedings of the 57th Annual Conference of the Association for Computational Linguistics (long papers)}, volume = {1}, pages = {4736–4746}, publisher = {ACL Anthology}, abstract = {Link prediction and entailment graph induction are often treated as different problems. In this paper, we show that these two problems are actually complementary. We train a link prediction model on a knowledge graph of assertions extracted from raw text. We propose an entailment score that exploits the new facts discovered by the link prediction model, and then form entailment graphs between relations. We further use the learned entailments to predict improved link prediction scores. Our results show that the two tasks can benefit from each other. The new entailment score outperforms prior state-of-the-art results on a standard entialment dataset and the new link prediction scores show improvements over the raw link prediction scores.}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } Link prediction and entailment graph induction are often treated as different problems. In this paper, we show that these two problems are actually complementary. We train a link prediction model on a knowledge graph of assertions extracted from raw text. We propose an entailment score that exploits the new facts discovered by the link prediction model, and then form entailment graphs between relations. We further use the learned entailments to predict improved link prediction scores. Our results show that the two tasks can benefit from each other. The new entailment score outperforms prior state-of-the-art results on a standard entialment dataset and the new link prediction scores show improvements over the raw link prediction scores. |
Hahn, Michael; Keller, Frank; Bisk, Yonatan; Belinkov, Yonatan Character-based Surprisal as a Model of Reading Difficulty in the Presence of Errors Inproceedings Goel, Ashok; Seifert, Colleen; Freksa, Christian (Ed.): Proceedings of the 41st Annual Conference of the Cognitive Science Society, pp. 401–407, 2019, ISBN: 0-9911967-7-5. Abstract | BibTeX | Tags: computational modeling, errors, eye-tracking, himan reading, Neural Networks, surprisal @inproceedings{1a70a349190b4b13b6f8a68e36775721, title = {Character-based Surprisal as a Model of Reading Difficulty in the Presence of Errors}, author = {Michael Hahn and Frank Keller and Yonatan Bisk and Yonatan Belinkov}, editor = {Ashok Goel and Colleen Seifert and Christian Freksa}, isbn = {0-9911967-7-5}, year = {2019}, date = {2019-01-01}, booktitle = {Proceedings of the 41st Annual Conference of the Cognitive Science Society}, pages = {401--407}, abstract = {Intuitively, human readers cope easily with errors in text; typos, misspelling, word substitutions, etc. do not unduly disrupt natural reading. Previous work indicates that letter transpositions result in increased reading times, but it is unclear if this effect generalizes to more natural errors. In this paper, we report an eye-tracking study that compares two error types (letter transpositions and naturally occurring misspelling) and two error rates (10% or 50% of all words contain errors). We find that human readers show unimpaired comprehension in spite of these errors, but error words cause more reading difficulty than correct words. Also, transpositions are more difficult than misspellings, and a high error rate increases difficulty for all words, including correct ones. We then present a computational model that uses character-based (rather than traditional word-based) surprisal to account for these results. The model explains that transpositions are harder than misspellings because they contain unexpected letter combinations. It also explains the error rate effect: expectations about upcoming words are harder to compute when the context is degraded, leading to increased surprisal.}, keywords = {computational modeling, errors, eye-tracking, himan reading, Neural Networks, surprisal}, pubstate = {published}, tppubtype = {inproceedings} } Intuitively, human readers cope easily with errors in text; typos, misspelling, word substitutions, etc. do not unduly disrupt natural reading. Previous work indicates that letter transpositions result in increased reading times, but it is unclear if this effect generalizes to more natural errors. In this paper, we report an eye-tracking study that compares two error types (letter transpositions and naturally occurring misspelling) and two error rates (10% or 50% of all words contain errors). We find that human readers show unimpaired comprehension in spite of these errors, but error words cause more reading difficulty than correct words. Also, transpositions are more difficult than misspellings, and a high error rate increases difficulty for all words, including correct ones. We then present a computational model that uses character-based (rather than traditional word-based) surprisal to account for these results. The model explains that transpositions are harder than misspellings because they contain unexpected letter combinations. It also explains the error rate effect: expectations about upcoming words are harder to compute when the context is degraded, leading to increased surprisal. |
Mourad, Ahmed; Scholer, Falk; Magdy, Walid; Sanderson, Mark A Practical Guide for the Effective Evaluation of Twitter User Geolocation Journal Article ACM Transactions on Social Computing (TSC), 2019, ISSN: 2469-7818. @article{c09adc4e16e142348b590f77d7bef639, title = {A Practical Guide for the Effective Evaluation of Twitter User Geolocation}, author = {Ahmed Mourad and Falk Scholer and Walid Magdy and Mark Sanderson}, issn = {2469-7818}, year = {2019}, date = {2019-01-01}, journal = {ACM Transactions on Social Computing (TSC)}, publisher = {Association for Computing Machinery (ACM)}, abstract = {Geolocating Twitter users---the task of identifying their home locations---serves a wide range of community and business applications such as managing natural crises, journalism, and public health. Many approaches have been proposed for automatically geolocating users based on their tweets; at the same time, various evaluation metrics have been proposed to measure the effectiveness of these approaches, making it challenging to understand which of these metrics is the most suitable for this task. In this paper, we propose a guide for a standardized evaluation of Twitter user geolocation by analyzing fifteen models and two baselines in a controlled experimental setting. Models are evaluated using ten metrics over four geographic granularities. We use rank correlations to assess the effectiveness of these metrics. Our results demonstrate that the choice of effectiveness metric can have a substantial impact on the conclusions drawn from a geolocation system experiment, potentially leading experimenters to contradictory results about relative effectiveness. We show that for general evaluations, a range of performance metrics should be reported, to ensure that a complete picture of system effectiveness is conveyed. Given the global geographic coverage of this task, we specifically recommend evaluation at micro versus macro levels to measure the impact of the bias in distribution over locations. Although a lot of complex geolocation algorithms have been applied in recent years, a majority class baseline is still competitive at coarse geographic granularity. We propose a suite of statistical analysis tests, based on the employed metric, to ensure that the results are not coincidental.}, keywords = {}, pubstate = {published}, tppubtype = {article} } Geolocating Twitter users---the task of identifying their home locations---serves a wide range of community and business applications such as managing natural crises, journalism, and public health. Many approaches have been proposed for automatically geolocating users based on their tweets; at the same time, various evaluation metrics have been proposed to measure the effectiveness of these approaches, making it challenging to understand which of these metrics is the most suitable for this task. In this paper, we propose a guide for a standardized evaluation of Twitter user geolocation by analyzing fifteen models and two baselines in a controlled experimental setting. Models are evaluated using ten metrics over four geographic granularities. We use rank correlations to assess the effectiveness of these metrics. Our results demonstrate that the choice of effectiveness metric can have a substantial impact on the conclusions drawn from a geolocation system experiment, potentially leading experimenters to contradictory results about relative effectiveness. We show that for general evaluations, a range of performance metrics should be reported, to ensure that a complete picture of system effectiveness is conveyed. Given the global geographic coverage of this task, we specifically recommend evaluation at micro versus macro levels to measure the impact of the bias in distribution over locations. Although a lot of complex geolocation algorithms have been applied in recent years, a majority class baseline is still competitive at coarse geographic granularity. We propose a suite of statistical analysis tests, based on the employed metric, to ensure that the results are not coincidental. |
Alex, Beatrice; Grover, Claire; Tobin, Richard; Oberlander, Jon Geoparsing Historical and Contemporary Literary Text set in the City of Edinburgh Journal Article Language Resources and Evaluation, 2019, ISSN: 1574-020X, (Springer Compact Gold OA). Abstract | Links | BibTeX | Tags: @article{aa9912d3c46a48f4b08d1d4c494c97ec, title = {Geoparsing Historical and Contemporary Literary Text set in the City of Edinburgh}, author = {Beatrice Alex and Claire Grover and Richard Tobin and Jon Oberlander}, doi = {10.1007/s10579-019-09443-x}, issn = {1574-020X}, year = {2019}, date = {2019-01-01}, journal = {Language Resources and Evaluation}, publisher = {Springer Netherlands}, abstract = {While a reasonable amount of work has gone into automatic geoparsing text at the city or higher levels of granularity for different types of texts in different domains, there is relatively little research on geoparsing fine-grained locations such as building, green spaces and street names in text. This paper reports on how the Edinburgh Geoparser performs on this task for different types of literary text set in Edinburgh, the first UNESCO City of Literature. The non-copyrighted gold standard datasets created for this purpose are released along with this article.}, note = {Springer Compact Gold OA}, keywords = {}, pubstate = {published}, tppubtype = {article} } While a reasonable amount of work has gone into automatic geoparsing text at the city or higher levels of granularity for different types of texts in different domains, there is relatively little research on geoparsing fine-grained locations such as building, green spaces and street names in text. This paper reports on how the Edinburgh Geoparser performs on this task for different types of literary text set in Edinburgh, the first UNESCO City of Literature. The non-copyrighted gold standard datasets created for this purpose are released along with this article. |
Vasiljeva, Ieva; Gilroy, Sorcha; Lopez, Adam The problem with probabilistic DAG automata for semantic graphs Inproceedings Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics, pp. 902–911, Association for Computational Linguistics, 2019. @inproceedings{2ca4635e4fe942078ff9fe3640235e77, title = {The problem with probabilistic DAG automata for semantic graphs}, author = {Ieva Vasiljeva and Sorcha Gilroy and Adam Lopez}, year = {2019}, date = {2019-01-01}, booktitle = {Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics}, volume = {1}, pages = {902–911}, publisher = {Association for Computational Linguistics}, abstract = {Semantic representations in the form of directed acyclic graphs (DAGs) have been introduced in recent years, and to model them, we need probabilistic models of DAGs. One model that has attracted some attention is the DAG automaton, but it has not been studied as a probabilistic model. We show that some DAG automata cannot be made into useful probabilistic models by the nearly universal strategy of assigning weights to transitions. The problem affects single-rooted, multi-rooted, and unbounded-degree variants of DAG automata, and appears to be pervasive. It does not affect planar variants, but these are problematic for other reasons.}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } Semantic representations in the form of directed acyclic graphs (DAGs) have been introduced in recent years, and to model them, we need probabilistic models of DAGs. One model that has attracted some attention is the DAG automaton, but it has not been studied as a probabilistic model. We show that some DAG automata cannot be made into useful probabilistic models by the nearly universal strategy of assigning weights to transitions. The problem affects single-rooted, multi-rooted, and unbounded-degree variants of DAG automata, and appears to be pervasive. It does not affect planar variants, but these are problematic for other reasons. |
Aldayel, Abeer; Magdy, Walid Your Stance is Exposed! Analysing Possible Factors for Stance Detection on Social Media Inproceedings Proceedings for the 22nd ACM Conference on Computer-Supported Cooperative Work and Social Computing, 2019. Abstract | BibTeX | Tags: Opinion mining, Social Media, Stance detection @inproceedings{05ad933366944c3b81606a8be661ccff, title = {Your Stance is Exposed! Analysing Possible Factors for Stance Detection on Social Media}, author = {Abeer Aldayel and Walid Magdy}, year = {2019}, date = {2019-01-01}, booktitle = {Proceedings for the 22nd ACM Conference on Computer-Supported Cooperative Work and Social Computing}, volume = {1}, abstract = {To what extent user’s stance towards a given topic could be inferred? Most of the studies on stance detection have focused on analysing user’s posts on a given topic to predict the stance. However, the stance in social media can be inferred from a mixture of signals that might reflect user’s beliefs including posts and online interactions. This paper examines various online features of users to detect their stance towards different topics. We compare multiple set of features, including on-topic content, network interactions, user’s preferences, and online network connections. Our objective is to understand the online signals that can reveal the users’ stance.Experimentation is applied on tweets dataset from the SemEval stance detection task, which covers five topics. Results show that stance of a user can be detected with multiple signals of user’s online activity, including their posts on the topic, the network they interact with or follow, the websites they visit, and the content they like. The performance of the stance modelling using different network features are comparable with the state-of-the-art reported model that used textual content only. In addition, combining network and content features leads to the highest reported performance to date on the SemEval dataset with F-measure of 72.49%.We further present an extensive analysis to show how these different set of features can reveal stance. Our findings have distinct privacy implications, where they highlight that stance is strongly embedded in user’s online social network that, in principle, individuals can be profiled from their interactions and connections even when they do not post about the topic.}, keywords = {Opinion mining, Social Media, Stance detection}, pubstate = {published}, tppubtype = {inproceedings} } To what extent user’s stance towards a given topic could be inferred? Most of the studies on stance detection have focused on analysing user’s posts on a given topic to predict the stance. However, the stance in social media can be inferred from a mixture of signals that might reflect user’s beliefs including posts and online interactions. This paper examines various online features of users to detect their stance towards different topics. We compare multiple set of features, including on-topic content, network interactions, user’s preferences, and online network connections. Our objective is to understand the online signals that can reveal the users’ stance.Experimentation is applied on tweets dataset from the SemEval stance detection task, which covers five topics. Results show that stance of a user can be detected with multiple signals of user’s online activity, including their posts on the topic, the network they interact with or follow, the websites they visit, and the content they like. The performance of the stance modelling using different network features are comparable with the state-of-the-art reported model that used textual content only. In addition, combining network and content features leads to the highest reported performance to date on the SemEval dataset with F-measure of 72.49%.We further present an extensive analysis to show how these different set of features can reveal stance. Our findings have distinct privacy implications, where they highlight that stance is strongly embedded in user’s online social network that, in principle, individuals can be profiled from their interactions and connections even when they do not post about the topic. |
Stanojevic, Milos On the Computational Complexity of Head Movement and Affix Hopping Inproceedings Formal Grammar 2019, pp. 101–116, Springer-Verlag Berlin Heidelberg, 2019, ISBN: 978-3-662-59647-0. Abstract | Links | BibTeX | Tags: @inproceedings{8e97979ed10e4502a38d1ddbd63a2e66, title = {On the Computational Complexity of Head Movement and Affix Hopping}, author = {Milos Stanojevic}, doi = {10.1007/978-3-662-59648-7_7}, isbn = {978-3-662-59647-0}, year = {2019}, date = {2019-01-01}, booktitle = {Formal Grammar 2019}, pages = {101--116}, publisher = {Springer-Verlag Berlin Heidelberg}, series = {Lecture Notes in Computer Science (LNCS)}, abstract = {Head movement is a syntactic operation used in most generative syntactic analyses. However, its computational properties have not been extensively studied. Stabler (2001) formalises head movement in the framework of Minimalist Grammars by extending the item representation to allow for easy extraction of the head. This work shows that Stabler’s representation is in fact suboptimal because it causes higher polynomial parsing complexity. A new algorithm is derived for parsing head movement and affix hopping by changing the kinds of representations that the parser deals with. This algorithm has much better asymptotic worst-case runtime of O(n2k+5). This result makes parsing head movement and affix hopping computationally as efficient as parsing a single phrase movement.}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } Head movement is a syntactic operation used in most generative syntactic analyses. However, its computational properties have not been extensively studied. Stabler (2001) formalises head movement in the framework of Minimalist Grammars by extending the item representation to allow for easy extraction of the head. This work shows that Stabler’s representation is in fact suboptimal because it causes higher polynomial parsing complexity. A new algorithm is derived for parsing head movement and affix hopping by changing the kinds of representations that the parser deals with. This algorithm has much better asymptotic worst-case runtime of O(n2k+5). This result makes parsing head movement and affix hopping computationally as efficient as parsing a single phrase movement. |
Li, Bowen; Cheng, Jianpeng; Liu, Yang; Keller, Frank Dependency Grammar Induction with a Neural Variational Transition-based Parser Inproceedings Proceedings of the 33rd AAAI Conference on Artificial Intelligence, pp. 6658–6665, AAAI Press, 2019. Abstract | Links | BibTeX | Tags: @inproceedings{b49b520a2a4b43e88dc6d6ea62f23cb2, title = {Dependency Grammar Induction with a Neural Variational Transition-based Parser}, author = {Bowen Li and Jianpeng Cheng and Yang Liu and Frank Keller}, doi = {10.1609/aaai.v33i01.33016658}, year = {2019}, date = {2019-01-01}, booktitle = {Proceedings of the 33rd AAAI Conference on Artificial Intelligence}, volume = {33}, number = {1}, pages = {6658--6665}, publisher = {AAAI Press}, series = {Proceedings of the Thirty-Third AAAI Conference on Artificial Intelligence}, abstract = {Dependency grammar induction is the task of learning dependency syntax without annotated training data. Traditional graph-based models with global inference achieve state-ofthe-art results on this task but they require O(n3) run time. Transition-based models enable faster inference with O(n) time complexity, but their performance still lags behind. In this work, we propose a neural transition-based parser for dependency grammar induction, whose inference procedure utilizes rich neural features with O(n) time complexity. We train the parser with an integration of variational inference, posterior regularization and variance reduction techniques. The resulting framework outperforms previous unsupervised transition-based dependency parsers and achieves performance comparable to graph-based models, both on the English Penn Treebank and on the Universal Dependency Treebank. In an empirical comparison, we show that our approach substantially increases parsing speed over graph-based models.}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } Dependency grammar induction is the task of learning dependency syntax without annotated training data. Traditional graph-based models with global inference achieve state-ofthe-art results on this task but they require O(n3) run time. Transition-based models enable faster inference with O(n) time complexity, but their performance still lags behind. In this work, we propose a neural transition-based parser for dependency grammar induction, whose inference procedure utilizes rich neural features with O(n) time complexity. We train the parser with an integration of variational inference, posterior regularization and variance reduction techniques. The resulting framework outperforms previous unsupervised transition-based dependency parsers and achieves performance comparable to graph-based models, both on the English Penn Treebank and on the Universal Dependency Treebank. In an empirical comparison, we show that our approach substantially increases parsing speed over graph-based models. |
Corro, Caio; Titov, Ivan Differentiable Perturb-and-Parse: Semi-Supervised Parsing with a Structured Variational Autoencoder Inproceedings Seventh International Conference on Learning Representations (ICLR 2017), 2019. Abstract | BibTeX | Tags: dependency parsing, differentiable dynamic programming, semi-supervised learning, variational auto-encoder @inproceedings{240ff41b5e0541739cb30830cb50e12e, title = {Differentiable Perturb-and-Parse: Semi-Supervised Parsing with a Structured Variational Autoencoder}, author = {Caio Corro and Ivan Titov}, year = {2019}, date = {2019-01-01}, booktitle = {Seventh International Conference on Learning Representations (ICLR 2017)}, abstract = {Human annotation for syntactic parsing is expensive, and large resources are available only for a fraction of languages. A question we ask is whether one can leverage abundant unlabeled texts to improve syntactic parsers, beyond just using the texts to obtain more generalisable lexical features (i.e. beyond word embeddings). To this end, we propose a novel latent-variable generative model for semi-supervised syntactic dependency parsing. As exact inference is intractable, we introduce a differentiable relaxation to obtain approximate samples and compute gradients with respect to the parser parameters. Our method (Differentiable Perturb-and-Parse) relies on differentiable dynamic programming over stochastically perturbed edge scores. We demonstrate effectiveness of our approach with experiments on English, French and Swedish.}, keywords = {dependency parsing, differentiable dynamic programming, semi-supervised learning, variational auto-encoder}, pubstate = {published}, tppubtype = {inproceedings} } Human annotation for syntactic parsing is expensive, and large resources are available only for a fraction of languages. A question we ask is whether one can leverage abundant unlabeled texts to improve syntactic parsers, beyond just using the texts to obtain more generalisable lexical features (i.e. beyond word embeddings). To this end, we propose a novel latent-variable generative model for semi-supervised syntactic dependency parsing. As exact inference is intractable, we introduce a differentiable relaxation to obtain approximate samples and compute gradients with respect to the parser parameters. Our method (Differentiable Perturb-and-Parse) relies on differentiable dynamic programming over stochastically perturbed edge scores. We demonstrate effectiveness of our approach with experiments on English, French and Swedish. |
Constantin, Aurora; Lai, Catherine; Farrow, Elaine; Alex, Beatrice; Pel-Littel, Ruth; Henk Herman, ; Jeuring, Johan "Why is the Doctor a Man?" Reactions of Older Adults to a Virtual Training Doctor Inproceedings Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems, ACM, 2019, ISBN: 978-1-4503-5971-9. Abstract | Links | BibTeX | Tags: @inproceedings{1d7676350fe1412f8f639f515a0523e0, title = {"Why is the Doctor a Man?" Reactions of Older Adults to a Virtual Training Doctor}, author = {Aurora Constantin and Catherine Lai and Elaine Farrow and Beatrice Alex and Ruth Pel-Littel and {Henk Herman} Nap and Johan Jeuring}, doi = {10.1145/3290607.3312811}, isbn = {978-1-4503-5971-9}, year = {2019}, date = {2019-01-01}, booktitle = {Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems}, publisher = {ACM}, series = {CHI EA '19}, abstract = {Shared decision making (SDM) is increasingly considered as the best way to reach a treatment decision in a clinical environment. However, the use of SDM in practice can be obstructed by a number of factors, such as time constraints or lack of applicability due to patient characteristics. Our project, PrepDoc, explores how a Virtual Training Doctor (VTD) can help patients overcome some of these obstacles to experiencing effective SDM during doctor’s visits. In this paper, we report on user studies conducted with 19 participants in Scotland aged 65+. The goal of these studies was to identify the reactions of this audience to the PrepDoc system, evaluate its suitability within Scotland, and elicit suggestions to improve it. Our findings revealed that the idea of empowering people to participate in SDM using a virtual agent was positively received by all participants. However, the reactions to how this idea was implemented in the PrepDoc system varied greatly across participants. Based on this, our paper outlines recommendations for enhancing the user experience with VTDs, accommodating individual differences of older adults, and accounting for the national context.}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } Shared decision making (SDM) is increasingly considered as the best way to reach a treatment decision in a clinical environment. However, the use of SDM in practice can be obstructed by a number of factors, such as time constraints or lack of applicability due to patient characteristics. Our project, PrepDoc, explores how a Virtual Training Doctor (VTD) can help patients overcome some of these obstacles to experiencing effective SDM during doctor’s visits. In this paper, we report on user studies conducted with 19 participants in Scotland aged 65+. The goal of these studies was to identify the reactions of this audience to the PrepDoc system, evaluate its suitability within Scotland, and elicit suggestions to improve it. Our findings revealed that the idea of empowering people to participate in SDM using a virtual agent was positively received by all participants. However, the reactions to how this idea was implemented in the PrepDoc system varied greatly across participants. Based on this, our paper outlines recommendations for enhancing the user experience with VTDs, accommodating individual differences of older adults, and accounting for the national context. |
Saphra, Naomi; Lopez, Adam Understanding Learning Dynamics Of Language Models with SVCCA Inproceedings Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics, pp. 3257–3267, Association for Computational Linguistics, 2019. Abstract | Links | BibTeX | Tags: @inproceedings{14ac73fe4ca4417798da6ed4e55a96dd, title = {Understanding Learning Dynamics Of Language Models with SVCCA}, author = {Naomi Saphra and Adam Lopez}, doi = {10.18653/v1/n19-1329}, year = {2019}, date = {2019-01-01}, booktitle = {Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics}, volume = {1}, pages = {3257–3267}, publisher = {Association for Computational Linguistics}, abstract = {Research has shown that neural models implicitly encode linguistic features, but there has been no research showing how these encodings arise as the models are trained. We present the first study on the learning dynamics of neural language models, using a simple and flexible analysis method called Singular Vector Canonical Correlation Analysis (SVCCA), which enables us to compare learned representations across time and across models, without the need to evaluate directly on annotated data. We probe the evolution of syntactic, semantic, and topic representations and find that part-of-speech is learned earlier than topic; that recurrent layers become more similar to those of a tagger during training; and embedding layers less similar. Our results and methods could inform better learning algorithms for NLP models, possibly to incorporate linguistic information more effectively.}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } Research has shown that neural models implicitly encode linguistic features, but there has been no research showing how these encodings arise as the models are trained. We present the first study on the learning dynamics of neural language models, using a simple and flexible analysis method called Singular Vector Canonical Correlation Analysis (SVCCA), which enables us to compare learned representations across time and across models, without the need to evaluate directly on annotated data. We probe the evolution of syntactic, semantic, and topic representations and find that part-of-speech is learned earlier than topic; that recurrent layers become more similar to those of a tagger during training; and embedding layers less similar. Our results and methods could inform better learning algorithms for NLP models, possibly to incorporate linguistic information more effectively. |
Coavoux, Maximin; Cohen, Shay Discontinuous Constituency Parsing with a Stack-Free Transition System and a Dynamic Oracle Inproceedings Burstein, Jill; Doran, Christy; Solorio, Thamar (Ed.): Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics, pp. 204–217, Association for Computational Linguistics (ACL), 2019. Abstract | Links | BibTeX | Tags: @inproceedings{02c6a24c82a044e49c04d9ddd6d6190a, title = {Discontinuous Constituency Parsing with a Stack-Free Transition System and a Dynamic Oracle}, author = {Maximin Coavoux and Shay Cohen}, editor = {Jill Burstein and Christy Doran and Thamar Solorio}, doi = {10.18653/v1/n19-1018}, year = {2019}, date = {2019-01-01}, booktitle = {Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics}, volume = {1}, pages = {204–217}, publisher = {Association for Computational Linguistics (ACL)}, abstract = {We introduce a novel transition system for discontinuous constituency parsing. Instead of storing subtrees in a stack –i.e. a data structure with linear-time sequential access– the proposed system uses a set of parsing items, with constant-time random access. This change makes it possible to construct any discontinuous constituency tree in exactly 4n − 2 transitions for a sentence of length n, whereas existing systems need a quadratic number of transitions to derive some structures. At each parsing step, the parser considers every item in the set to be combined with a focus item and to construct a new constituent in a bottom-up fashion. The parsing strategy is based on the assumption that most syntactic structures can be parsed incrementally and that the set – the memory of the parser– remains reasonably small on average. Moreover, we introduce a dynamic oracle for the new transition system, and present the first experiments in discontinuous constituency parsing using a dynamic oracle. Our parser obtains state-of-the-art results on three English and German discontinuous treebanks.}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } We introduce a novel transition system for discontinuous constituency parsing. Instead of storing subtrees in a stack –i.e. a data structure with linear-time sequential access– the proposed system uses a set of parsing items, with constant-time random access. This change makes it possible to construct any discontinuous constituency tree in exactly 4n − 2 transitions for a sentence of length n, whereas existing systems need a quadratic number of transitions to derive some structures. At each parsing step, the parser considers every item in the set to be combined with a focus item and to construct a new constituent in a bottom-up fashion. The parsing strategy is based on the assumption that most syntactic structures can be parsed incrementally and that the set – the memory of the parser– remains reasonably small on average. Moreover, we introduce a dynamic oracle for the new transition system, and present the first experiments in discontinuous constituency parsing using a dynamic oracle. Our parser obtains state-of-the-art results on three English and German discontinuous treebanks. |
Douglas W, ; Carpuat, Marine; Galuscakova, Petra; Barrow, Joseph; Nair, Suraj; Niu, Xing; Shing, Han-Chin; Xu, Weijia; Zotkina, Elena; McKeown, Kathleen; Muresan, Smaranda; Efsun Selin, ; Eskander, Ramy; Kedzie, Chris; Virin, Yan; Dragomir R, ; Zhang, Rui; Mark J F, ; Ragni, Anton; Heafield, Kenneth Surprise Languages: Rapid-Response Cross-Language IR Inproceedings Nicola, ; Ishita, Emi; Kando, Noriko; Makoto P, ; Soboroff, Ian; Zhang, Min (Ed.): Proceedings of the Ninth International Workshop on Evaluating Information Access (EVIA 2019), pp. 23, ACM, 2019, ISBN: 978-4-86049-077-5. Abstract | BibTeX | Tags: Evaluation, Fast, Retrieval @inproceedings{8ff207da1f78474e9d3a7395e79bcf32, title = {Surprise Languages: Rapid-Response Cross-Language IR}, author = {{Douglas W } Oard and Marine Carpuat and Petra Galuscakova and Joseph Barrow and Suraj Nair and Xing Niu and Han-Chin Shing and Weijia Xu and Elena Zotkina and Kathleen McKeown and Smaranda Muresan and {Efsun Selin} Kayi and Ramy Eskander and Chris Kedzie and Yan Virin and {Dragomir R } Radev and Rui Zhang and {Mark J F} Gales and Anton Ragni and Kenneth Heafield}, editor = {{Nicola } Ferro and Emi Ishita and Noriko Kando and {Makoto P } Kato and Ian Soboroff and Min Zhang}, isbn = {978-4-86049-077-5}, year = {2019}, date = {2019-01-01}, booktitle = {Proceedings of the Ninth International Workshop on Evaluating Information Access (EVIA 2019)}, pages = {23}, publisher = {ACM}, abstract = {Sixteen years ago, the first "surprise language exercise" was conducted, in Cebuano. The evaluation goal of a surprise language exercise is to learn how well systems for a new language can be quickly built. This paper briefly reviews the history of surprise language exercises. Some details from the most recent surprise language exercise, in Lithuanian, are included to help to illustrate how the state of the art has advanced over this period.}, keywords = {Evaluation, Fast, Retrieval}, pubstate = {published}, tppubtype = {inproceedings} } Sixteen years ago, the first "surprise language exercise" was conducted, in Cebuano. The evaluation goal of a surprise language exercise is to learn how well systems for a new language can be quickly built. This paper briefly reviews the history of surprise language exercises. Some details from the most recent surprise language exercise, in Lithuanian, are included to help to illustrate how the state of the art has advanced over this period. |
Bansal, Sameer; Kamper, Herman; Livescu, Karen; Lopez, Adam; Goldwater, Sharon Pre-training on high-resource speech recognition improves low-resource speech-to-text translation Inproceedings Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics, pp. 58–68, Association for Computational Linguistics, 2019. Abstract | Links | BibTeX | Tags: @inproceedings{041a6277f28c475d976aca751268bf10, title = {Pre-training on high-resource speech recognition improves low-resource speech-to-text translation}, author = {Sameer Bansal and Herman Kamper and Karen Livescu and Adam Lopez and Sharon Goldwater}, doi = {10.18653/v1/n19-1006}, year = {2019}, date = {2019-01-01}, booktitle = {Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics}, volume = {1}, pages = {58–68}, publisher = {Association for Computational Linguistics}, abstract = {We present a simple approach to improve direct speech-to-text translation (ST) when the source language is low-resource: we pre-train the model on a high-resource automatic speech recognition (ASR) task, and then fine-tune its parameters for ST. We demonstrate that our approach is effective by pre-training on 300 hours of English ASR data to improve SpanishEnglish ST from 10.8 to 20.2 BLEU when only 20 hours of Spanish-English ST training data are available. Through an ablation study, we find that the pre-trained encoder (acoustic model) accounts for most of the improvement, despite the fact that the shared language in these tasks is the target language text, not the source language audio. Applying this insight, we show that pre-training on ASR helps ST even when the ASR language differs from both source and target ST languages: pre-training on French ASR also improves Spanish-English ST. Finally, we show that the approach improves performance on a true low-resource task: pre-training on a combination of English ASR and French ASR improves Mboshi-French ST, where only 4 hours of data are available, from 3.5 to 7.1 BLEU.}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } We present a simple approach to improve direct speech-to-text translation (ST) when the source language is low-resource: we pre-train the model on a high-resource automatic speech recognition (ASR) task, and then fine-tune its parameters for ST. We demonstrate that our approach is effective by pre-training on 300 hours of English ASR data to improve SpanishEnglish ST from 10.8 to 20.2 BLEU when only 20 hours of Spanish-English ST training data are available. Through an ablation study, we find that the pre-trained encoder (acoustic model) accounts for most of the improvement, despite the fact that the shared language in these tasks is the target language text, not the source language audio. Applying this insight, we show that pre-training on ASR helps ST even when the ASR language differs from both source and target ST languages: pre-training on French ASR also improves Spanish-English ST. Finally, we show that the approach improves performance on a true low-resource task: pre-training on a combination of English ASR and French ASR improves Mboshi-French ST, where only 4 hours of data are available, from 3.5 to 7.1 BLEU. |
Philip John, ; Wu, Honghan; Grover, Claire; Tobin, Richard; Talbot, Conn; Whalley, Heather; Whiteley, William; Alex, Beatrice 2019, (Second UK Healthcare Text Analytics Conference, HealTAC 2019 ; Conference date: 24-04-2019 Through 25-04-2019). Abstract | Links | BibTeX | Tags: @conference{0c8513cef26a4650a1e5f63f9fb5ff4a, title = {Named Entity Recognition for Electronic Health Records: A Comparison of Rule-based and Machine Learning Approaches}, author = {{Philip John} Gorinski and Honghan Wu and Claire Grover and Richard Tobin and Conn Talbot and Heather Whalley and William Whiteley and Beatrice Alex}, url = {http://healtex.org/healtac-2019/}, year = {2019}, date = {2019-01-01}, abstract = {This work investigates multiple approaches to Named Entity Recognition (NER) for text in Electronic Health Record (EHR) data. In particular, we look into the application of (i) rule-based, (ii) deep learning and (iii) transfer learning systems for the task of NER on brain imaging reports with a focus on records from patients with stroke. We explore the strengths and weaknesses of each approach, develop rules and train on a common dataset, and evaluate each system's performance on common test sets of Scottish radiology reports from two sources (brain imaging reports in ESS -- Edinburgh Stroke Study data collected by NHS Lothian as well as radiology reports created in NHS Tayside). Our comparison shows that a hand-crafted system is the most accurate way to automatically label EHR, but machine learning approaches can provide a feasible alternative where resources for a manual system are not readily available.}, note = {Second UK Healthcare Text Analytics Conference, HealTAC 2019 ; Conference date: 24-04-2019 Through 25-04-2019}, keywords = {}, pubstate = {published}, tppubtype = {conference} } This work investigates multiple approaches to Named Entity Recognition (NER) for text in Electronic Health Record (EHR) data. In particular, we look into the application of (i) rule-based, (ii) deep learning and (iii) transfer learning systems for the task of NER on brain imaging reports with a focus on records from patients with stroke. We explore the strengths and weaknesses of each approach, develop rules and train on a common dataset, and evaluate each system's performance on common test sets of Scottish radiology reports from two sources (brain imaging reports in ESS -- Edinburgh Stroke Study data collected by NHS Lothian as well as radiology reports created in NHS Tayside). Our comparison shows that a hand-crafted system is the most accurate way to automatically label EHR, but machine learning approaches can provide a feasible alternative where resources for a manual system are not readily available. |
Alex, Beatrice; Grover, Claire; Tobin, Richard; Sudlow, Catherine; Mair, Grant; Whiteley, William Text Mining Brain Imaging Reports Journal Article Journal of Biomedical Semantics, 2019, ISSN: 2041-1480. @article{74ee8f4642e94ae1b07aa99d5a0c1e70, title = {Text Mining Brain Imaging Reports}, author = {Beatrice Alex and Claire Grover and Richard Tobin and Catherine Sudlow and Grant Mair and William Whiteley}, issn = {2041-1480}, year = {2019}, date = {2019-01-01}, journal = {Journal of Biomedical Semantics}, publisher = {BioMed Central}, abstract = {Background: With the improvements to text mining technology and the availability of large unstructured Electronic Healthcare Records (EHR) datasets, it is now possible to extract structured information from raw text contained within EHR at reasonably high accuracy. We describe a text mining system for classifying radiologists’ reports of CT and MRI brain scans, assigning labels indicating occurrence and type of stroke, as well as other observations. Our system, the Edinburgh Information Extraction for Radiology reports (EdIE-R) system, which we describe here, was developed and tested on a collection of radiology reports. Methods: The work reported in this paper is based on 1,168 radiology reports from the Edinburgh Stroke Study (ESS) [1], a hospital-based register of stroke and transient ischaemic attack patients. We created manually annotations for this data in parallel with developing the rule-based EdIE-R system to identify phenotype information related to stroke in radiology reports. This process was iterative and domain expert feedback was considered at each iteration to adapt and tune the EdIE-R text mining system which identifies entities, negation and relations between entities in each report and determines report-level labels. Results:The inter-annotator agreement (IAA) for all types of annotations is high at 96.96 for entities, 96.46 for negation, 95.84 for relations and 94.02 for labels. The equivalent system scores on the blind test set are equally high at 95.49 for entities, 94.41 for negation, 98.27 for relations and 96.39 for labels for the first annotator and 96.86, 96.01, 96.53 and 92.61, respectively for the second annotator. Conclusion: Automated reading of such EHR data at such high levels of accuracies opens up avenues for population health monitoring and audit, and can provide a resource for epidemiological studies. We are in the process of validating EdIE-R in separate larger cohorts in NHS England and Scotland. The manually annotated ESS corpus will be available for research purposes on application.}, keywords = {}, pubstate = {published}, tppubtype = {article} } Background: With the improvements to text mining technology and the availability of large unstructured Electronic Healthcare Records (EHR) datasets, it is now possible to extract structured information from raw text contained within EHR at reasonably high accuracy. We describe a text mining system for classifying radiologists’ reports of CT and MRI brain scans, assigning labels indicating occurrence and type of stroke, as well as other observations. Our system, the Edinburgh Information Extraction for Radiology reports (EdIE-R) system, which we describe here, was developed and tested on a collection of radiology reports. Methods: The work reported in this paper is based on 1,168 radiology reports from the Edinburgh Stroke Study (ESS) [1], a hospital-based register of stroke and transient ischaemic attack patients. We created manually annotations for this data in parallel with developing the rule-based EdIE-R system to identify phenotype information related to stroke in radiology reports. This process was iterative and domain expert feedback was considered at each iteration to adapt and tune the EdIE-R text mining system which identifies entities, negation and relations between entities in each report and determines report-level labels. Results:The inter-annotator agreement (IAA) for all types of annotations is high at 96.96 for entities, 96.46 for negation, 95.84 for relations and 94.02 for labels. The equivalent system scores on the blind test set are equally high at 95.49 for entities, 94.41 for negation, 98.27 for relations and 96.39 for labels for the first annotator and 96.86, 96.01, 96.53 and 92.61, respectively for the second annotator. Conclusion: Automated reading of such EHR data at such high levels of accuracies opens up avenues for population health monitoring and audit, and can provide a resource for epidemiological studies. We are in the process of validating EdIE-R in separate larger cohorts in NHS England and Scotland. The manually annotated ESS corpus will be available for research purposes on application. |
Aldayel, Abeer; Magdy, Walid Assessing Sentiment of the Expressed Stance on Social Media Inproceedings Social Informatics, 2019. Abstract | BibTeX | Tags: Event analysis, Public opinion, Sentiment analysis, Social Media, Stance detection @inproceedings{50bb512e36e0414ebae4febc50a2c079, title = {Assessing Sentiment of the Expressed Stance on Social Media}, author = {Abeer Aldayel and Walid Magdy}, year = {2019}, date = {2019-01-01}, booktitle = {Social Informatics}, abstract = {Stance detection is the task of inferring viewpoint towards a given topic or entity either being supportive or opposing. One may express a viewpoint towards a topic by using positive or negative language. This paper examines how the stance is being expressed in social media according to the sentiment polarity. There has been a noticeable misconception of the similarity between the stance and sentiment when it comes to viewpoint discovery, where negative sentiment is assumed to mean against stance, and positive sentiment means in-favour stance. To analyze the relation between stance and sentiment, we construct a new dataset with four topics and examine how people express their viewpoint with regards these topics. We validate our results by carrying a further analysis of the popular stance benchmark SemEval stance dataset. Our analyses reveal that sentiment and stance are not highly aligned, and hence the simple sentiment polarity cannot be used solely to denote a stance toward a given topic.}, keywords = {Event analysis, Public opinion, Sentiment analysis, Social Media, Stance detection}, pubstate = {published}, tppubtype = {inproceedings} } Stance detection is the task of inferring viewpoint towards a given topic or entity either being supportive or opposing. One may express a viewpoint towards a topic by using positive or negative language. This paper examines how the stance is being expressed in social media according to the sentiment polarity. There has been a noticeable misconception of the similarity between the stance and sentiment when it comes to viewpoint discovery, where negative sentiment is assumed to mean against stance, and positive sentiment means in-favour stance. To analyze the relation between stance and sentiment, we construct a new dataset with four topics and examine how people express their viewpoint with regards these topics. We validate our results by carrying a further analysis of the popular stance benchmark SemEval stance dataset. Our analyses reveal that sentiment and stance are not highly aligned, and hence the simple sentiment polarity cannot be used solely to denote a stance toward a given topic. |
Emelin, Denis; Titov, Ivan; Sennrich, Rico Widening the Representation Bottleneck in Neural Machine Translation with Lexical Shortcuts Inproceedings Proceedings of the Fourth Conference on Machine Translation (WMT), Volume 1, pp. 102–115, Association for Computational Linguistics (ACL), 2019, ISBN: 978-1-950737-27-7. @inproceedings{40d7489e597a438d8ef8e079b07cab0a, title = {Widening the Representation Bottleneck in Neural Machine Translation with Lexical Shortcuts}, author = {Denis Emelin and Ivan Titov and Rico Sennrich}, isbn = {978-1-950737-27-7}, year = {2019}, date = {2019-01-01}, booktitle = {Proceedings of the Fourth Conference on Machine Translation (WMT), Volume 1}, volume = {1}, pages = {102–115}, publisher = {Association for Computational Linguistics (ACL)}, abstract = {The transformer is a state-of-the-art neural translation model that uses attention to iteratively refine lexical representations with information drawn from the surrounding context. Lexical features are fed into the first layer and propagated through a deep network of hidden layers. We argue that the need to represent and propagate lexical features in each layer limits the model’s capacity for learning and representing other information relevant to the task. To alleviate this bottleneck, we introduce gated shortcut connections between the embedding layer and each subsequent layer within the encoder and decoder. This enables the model to access relevant lexical content dynamically, without expending limited resources on storing it within intermediate states. We show that the proposed modification yields consistent improvements over a baseline transformer on standard WMT translation tasks in 5 translation directions (0.9 BLEU on average) and reduces the amount of lexical information passed along the hidden layers. We furthermore evaluate different ways to integrate lexical connections into the transformer architecture and present ablation experiments exploring the effect of proposed shortcuts on model behavior.1 1Our code is publicly available to aid the reproduction of the reported results: https://github.com/demelin/ transformer_lexical_shortcuts}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } The transformer is a state-of-the-art neural translation model that uses attention to iteratively refine lexical representations with information drawn from the surrounding context. Lexical features are fed into the first layer and propagated through a deep network of hidden layers. We argue that the need to represent and propagate lexical features in each layer limits the model’s capacity for learning and representing other information relevant to the task. To alleviate this bottleneck, we introduce gated shortcut connections between the embedding layer and each subsequent layer within the encoder and decoder. This enables the model to access relevant lexical content dynamically, without expending limited resources on storing it within intermediate states. We show that the proposed modification yields consistent improvements over a baseline transformer on standard WMT translation tasks in 5 translation directions (0.9 BLEU on average) and reduces the amount of lexical information passed along the hidden layers. We furthermore evaluate different ways to integrate lexical connections into the transformer architecture and present ablation experiments exploring the effect of proposed shortcuts on model behavior.1 1Our code is publicly available to aid the reproduction of the reported results: https://github.com/demelin/ transformer_lexical_shortcuts |
Casey, Arlene; Bennett, Mike; Tobin, Richard; Grover, Claire; Engelmann, Lukas; Alex, Beatrice Plague Dot Text: Text mining and annotation of outbreak reports of the Third Plague Pandemic (1894-1952) Inproceedings Proceedings of HistoInformatics 2019, CEUR Workshop Proceedings, 2019, (To be published in 12 Sep 2019). @inproceedings{4f0eab00d19c4fc4b46dec7d05ba872b, title = {Plague Dot Text: Text mining and annotation of outbreak reports of the Third Plague Pandemic (1894-1952)}, author = {Arlene Casey and Mike Bennett and Richard Tobin and Claire Grover and Lukas Engelmann and Beatrice Alex}, year = {2019}, date = {2019-01-01}, booktitle = {Proceedings of HistoInformatics 2019}, publisher = {CEUR Workshop Proceedings}, abstract = {The design of models that govern diseases in their relation to population is built on information and data gathered from past outbreaks. However, epidemic outbreaks are never captured in statistical data alone but are communicated by narratives, built on empirical observations. Outbreak reports discuss correlations between populations, locations and the disease to infer insights into causes, vectors and potential interventions. The problem with these narratives is usually the lack of consistent structure that allows for exploration of their collection as a whole. Our interdisciplinary research investigates more than 100 reports from the third plague pandemic (1894-1952) evaluating ways of building a corpus to extract and structure information through text mining and manual annotation. In this paper we discuss the progress of our exploratory project, how we enhance optical character recognition (OCR) methods to improve text capture, our approach to structure the narratives and identify relevant entities in the reports. The structured corpus is made available via Solr enabling search and analysis across the whole collection for future research dedicated e.g. to the identification of concepts. The corpus will enable researchers to analyse the reports collectively and allows for deep insights into the global epidemiological consideration of plague in the early 20th century.}, note = {To be published in 12 Sep 2019}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } The design of models that govern diseases in their relation to population is built on information and data gathered from past outbreaks. However, epidemic outbreaks are never captured in statistical data alone but are communicated by narratives, built on empirical observations. Outbreak reports discuss correlations between populations, locations and the disease to infer insights into causes, vectors and potential interventions. The problem with these narratives is usually the lack of consistent structure that allows for exploration of their collection as a whole. Our interdisciplinary research investigates more than 100 reports from the third plague pandemic (1894-1952) evaluating ways of building a corpus to extract and structure information through text mining and manual annotation. In this paper we discuss the progress of our exploratory project, how we enhance optical character recognition (OCR) methods to improve text capture, our approach to structure the narratives and identify relevant entities in the reports. The structured corpus is made available via Solr enabling search and analysis across the whole collection for future research dedicated e.g. to the identification of concepts. The corpus will enable researchers to analyse the reports collectively and allows for deep insights into the global epidemiological consideration of plague in the early 20th century. |
Matusevych, Yevgen; Stevenson, Suzanne Analyzing and modeling free word associations Inproceedings Kalish, Chuck; Rau, Marina; Rogers, Tim; Zhu, Jerry (Ed.): Proceedings of the 40th Annual Conference of the Cognitive Science Society, pp. 750–755, Cognitive Science Society, 2019, ISBN: 978-1-5108-7205-9. @inproceedings{67c371766e3d49f8bd3ec6ba240a4e8f, title = {Analyzing and modeling free word associations}, author = {Yevgen Matusevych and Suzanne Stevenson}, editor = {Chuck Kalish and Marina Rau and Tim Rogers and Jerry Zhu}, isbn = {978-1-5108-7205-9}, year = {2019}, date = {2019-01-01}, booktitle = {Proceedings of the 40th Annual Conference of the Cognitive Science Society}, volume = {1}, pages = {750–755}, publisher = {Cognitive Science Society}, abstract = {Human free association (FA) norms are believed to reflect the strength of links between words in the lexicon of an average speaker. Large-scale FA norms are commonly used as a data source both in psycholinguistics and in computational modeling. However, few studies aim to analyze FA norms themselves, and it is not known what are the most important factors that guide speakers’ lexical choices in the FA task. Here, we first provide a statistical analysis of a large-scale data set of English FA norms. Second, we argue that such analysis can inform existing computational models of semantic memory, and present a case study with the topic model to support this claim. Based on our analysis, we provide the topic model with dictionary-based knowledge about word synonymy/antonymy, and demonstrate that the resulting model predicts human FA responses better than the topic model without this information.}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } Human free association (FA) norms are believed to reflect the strength of links between words in the lexicon of an average speaker. Large-scale FA norms are commonly used as a data source both in psycholinguistics and in computational modeling. However, few studies aim to analyze FA norms themselves, and it is not known what are the most important factors that guide speakers’ lexical choices in the FA task. Here, we first provide a statistical analysis of a large-scale data set of English FA norms. Second, we argue that such analysis can inform existing computational models of semantic memory, and present a case study with the topic model to support this claim. Based on our analysis, we provide the topic model with dictionary-based knowledge about word synonymy/antonymy, and demonstrate that the resulting model predicts human FA responses better than the topic model without this information. |
Le, Phong; Titov, Ivan Boosting Entity Linking Performance by Leveraging Unlabeled Documents Inproceedings Korhonen, Anna; Traum, David; `a, Lluís M (Ed.): Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (long papers), pp. 1935–1945, ACL Anthology, 2019, ISBN: 978-1-950737-48-2. @inproceedings{36d264a136734efcbc8539cac762e713, title = {Boosting Entity Linking Performance by Leveraging Unlabeled Documents}, author = {Phong Le and Ivan Titov}, editor = {Anna Korhonen and David Traum and Lluís M `a}, isbn = {978-1-950737-48-2}, year = {2019}, date = {2019-01-01}, booktitle = {Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (long papers)}, volume = {1}, pages = {1935–1945}, publisher = {ACL Anthology}, abstract = {Modern entity linking systems rely on large collections of documents specifically annotated for the task (e.g., AIDA CoNLL). In contrast, we propose an approach which exploits only naturally occurring information: unlabeled documents and Wikipedia. Our approach consists of two stages. First, we construct a high recall list of candidate entities for each mention in an unlabeled document. Second, we use the candidate lists as weak supervision to constrain our document-level entity linking model. The model treats entities as latent variables and, when estimated on a collection of unlabelled texts, learns to choose entities relying both on local context of each mention and on coherence with other entities in the document. The resulting approach rivals fully-supervised state-of-the-art systems on standard test sets. It also approaches their performance in the very challenging setting: when tested on a test set sampled from the data used to estimate the supervised systems. By comparing to Wikipedia-only training of our model, we demonstrate that modeling unlabeled documents is beneficial.}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } Modern entity linking systems rely on large collections of documents specifically annotated for the task (e.g., AIDA CoNLL). In contrast, we propose an approach which exploits only naturally occurring information: unlabeled documents and Wikipedia. Our approach consists of two stages. First, we construct a high recall list of candidate entities for each mention in an unlabeled document. Second, we use the candidate lists as weak supervision to constrain our document-level entity linking model. The model treats entities as latent variables and, when estimated on a collection of unlabelled texts, learns to choose entities relying both on local context of each mention and on coherence with other entities in the document. The resulting approach rivals fully-supervised state-of-the-art systems on standard test sets. It also approaches their performance in the very challenging setting: when tested on a test set sampled from the data used to estimate the supervised systems. By comparing to Wikipedia-only training of our model, we demonstrate that modeling unlabeled documents is beneficial. |
Steedman, Mark Combinatory Categorial Grammar Book Chapter é, András Kert; Moravcsik, Edith; á, Csilla R (Ed.): Current Approaches to Syntax, pp. 389–420, De Gruyter Mouton, Germany, 2019. Abstract | Links | BibTeX | Tags: @inbook{18333df5e1ab44569ca7f310fddea7b2, title = {Combinatory Categorial Grammar}, author = {Mark Steedman}, editor = {András Kert é and Edith Moravcsik and Csilla R á}, doi = {10.1515/9783110540253-014}, year = {2019}, date = {2019-01-01}, booktitle = {Current Approaches to Syntax}, pages = {389–420}, publisher = {De Gruyter Mouton}, address = {Germany}, abstract = {Combinatory Categorial Grammar (CCG) is a radically lexicalized theory of grammar in which all language-specific information, including the linear order of heads, arguments, and adjuncts, is specified in the lexicon, from which it is projected onto sentences by language-independent universal type-dependent combinatory rules of low “slightly non-context-free” expressive power, applying to strictly adjacent phonologically-realised categories. Syntactic and phonological derivation are isomorphic, and are synchronously coupled with semantic composition in a purely type-dependent rule-to-rule relation.}, keywords = {}, pubstate = {published}, tppubtype = {inbook} } Combinatory Categorial Grammar (CCG) is a radically lexicalized theory of grammar in which all language-specific information, including the linear order of heads, arguments, and adjuncts, is specified in the lexicon, from which it is projected onto sentences by language-independent universal type-dependent combinatory rules of low “slightly non-context-free” expressive power, applying to strictly adjacent phonologically-realised categories. Syntactic and phonological derivation are isomorphic, and are synchronously coupled with semantic composition in a purely type-dependent rule-to-rule relation. |
Aldayel, Abeer; Magdy, Walid It is more than what you Say!: Leveraging User Online Activity for Improved Stance Detection Inproceedings 5th International Conference on Computational Social Science IC2S2, 2019. BibTeX | Tags: @inproceedings{954161ea616a4310953f6be5d0a67208, title = {It is more than what you Say!: Leveraging User Online Activity for Improved Stance Detection}, author = {Abeer Aldayel and Walid Magdy}, year = {2019}, date = {2019-01-01}, booktitle = {5th International Conference on Computational Social Science IC2S2}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } |
Le, Phong; Titov, Ivan Distant Learning for Entity Linking with Automatic Noise Detection Inproceedings Korhonen, Anna; Traum, David; `a, Lluís M (Ed.): Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (long papers), pp. 4081–4090, ACL Anthology, 2019, ISBN: 978-1-950737-48-2. @inproceedings{08ce16564fa746538ed8351d67444c05, title = {Distant Learning for Entity Linking with Automatic Noise Detection}, author = {Phong Le and Ivan Titov}, editor = {Anna Korhonen and David Traum and Lluís M `a}, isbn = {978-1-950737-48-2}, year = {2019}, date = {2019-01-01}, booktitle = {Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (long papers)}, volume = {1}, pages = {4081–4090}, publisher = {ACL Anthology}, abstract = {Accurate entity linkers have been produced for domains and languages where annotated data (i.e., texts linked to a knowledge base) is available. However, little progress has been made for the settings where no or very limited amounts of labeled data are present (e.g., legal or most scientific domains). In this work, we show how we can learn to link mentions without having any labeled examples, only a knowledge base and a collection of unannotated texts from the corresponding domain.In order to achieve this, we frame the task as a multi-instance learning problem and rely on surface matching to create initial noisy labels. As the learning signal is weak and our surrogate labels are noisy, we introduce a noise detection component in our model: it lets the model detect and disregard examples which are likely to be noisy. Our method, jointly learning to detect noise and link entities, greatly outperforms the surface matching baseline. For a subset of entity categories, it even approaches the performance of supervised learning.}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } Accurate entity linkers have been produced for domains and languages where annotated data (i.e., texts linked to a knowledge base) is available. However, little progress has been made for the settings where no or very limited amounts of labeled data are present (e.g., legal or most scientific domains). In this work, we show how we can learn to link mentions without having any labeled examples, only a knowledge base and a collection of unannotated texts from the corresponding domain.In order to achieve this, we frame the task as a multi-instance learning problem and rely on surface matching to create initial noisy labels. As the learning signal is weak and our surrogate labels are noisy, we introduce a noise detection component in our model: it lets the model detect and disregard examples which are likely to be noisy. Our method, jointly learning to detect noise and link entities, greatly outperforms the surface matching baseline. For a subset of entity categories, it even approaches the performance of supervised learning. |
Sennrich, Rico; Zhang, Biao Revisiting Low-Resource Neural Machine Translation: A Case Study Inproceedings Korhonen, Anna; Traum, David; `a, Lluís M (Ed.): Proceedings of the 57th Conference of the Association for Computational Linguistics, pp. 211–221, Association for Computational Linguistics (ACL), 2019. @inproceedings{ea3fe11ca37b482d9bcb432bb900fcb8, title = {Revisiting Low-Resource Neural Machine Translation: A Case Study}, author = {Rico Sennrich and Biao Zhang}, editor = {Anna Korhonen and David Traum and Lluís M `a}, year = {2019}, date = {2019-01-01}, booktitle = {Proceedings of the 57th Conference of the Association for Computational Linguistics}, pages = {211–221}, publisher = {Association for Computational Linguistics (ACL)}, abstract = {It has been shown that the performance of neural machine translation (NMT) drops starkly in low-resource conditions, underperforming phrase-based statistical machine translation (PBSMT) and requiring large amounts of auxiliary data to achieve competitive results. In this paper, we re-assess the validity of these results, arguing that they are the result of lack of system adaptation to low-resource settings. We discuss some pitfalls to be aware of when training low-resource NMT systems, and recent techniques that have shown to be especially helpful in low-resource settings, resulting in a set of best practices for low-resource NMT. In our experiments on German–English with different amounts of IWSLT14 training data, we show that, without the use of any auxiliary monolingual or multilingual data, an optimized NMT system can outperform PBSMT with far less data than previously claimed. We also apply these techniques to a low-resource Korean–English dataset, surpassing previously reported results by 4 BLEU.}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } It has been shown that the performance of neural machine translation (NMT) drops starkly in low-resource conditions, underperforming phrase-based statistical machine translation (PBSMT) and requiring large amounts of auxiliary data to achieve competitive results. In this paper, we re-assess the validity of these results, arguing that they are the result of lack of system adaptation to low-resource settings. We discuss some pitfalls to be aware of when training low-resource NMT systems, and recent techniques that have shown to be especially helpful in low-resource settings, resulting in a set of best practices for low-resource NMT. In our experiments on German–English with different amounts of IWSLT14 training data, we show that, without the use of any auxiliary monolingual or multilingual data, an optimized NMT system can outperform PBSMT with far less data than previously claimed. We also apply these techniques to a low-resource Korean–English dataset, surpassing previously reported results by 4 BLEU. |
Steedman, Mark Form-Independent Meaning Representation for Eventualities Book Chapter Truswell, Robert (Ed.): The Oxford Handbook of Event Structure, pp. 605–623, Oxford University Press, United Kingdom, 2019. @inbook{dd2c391d92d94a948e38d66fa4969cd0, title = {Form-Independent Meaning Representation for Eventualities}, author = {Mark Steedman}, editor = {Robert Truswell}, year = {2019}, date = {2019-01-01}, booktitle = {The Oxford Handbook of Event Structure}, pages = {605--623}, publisher = {Oxford University Press}, address = {United Kingdom}, abstract = {Natural language text such as that found on the web or in newspapers can nowadays be efficiently parsed, including building meaning representations or logical forms, with somewhat usable accuracy, at speeds of hundreds or even thousands of sentences a second. And Google is parsing everything we type at it. Nevertheless, we still don’t have real question-answering (QA), so that we can ask a question such as ‘Is the president in Washington today?’, have it mapped to an equivalent query, and get a precise answer. Such an answer could in principle be based on a semantic net or knowledge graph of eventualities, continually built and updated by semantic parsers reading the newspapers. Instead, we are still presented with a bunch of snippets from pages whose words and linkages may or may not answer our question when we ourselves do the reading. The central problem in using parsers to answer questions from unrestricted text like this is that the answer to our question is very likely to be there somewhere, but that it is almost certainly in a form which is not the same as that suggested by the form of our question. For example, the question ‘Is the president in Washington?’ is in fact answered by the statement in today’s paper that ‘The president has arrived at the White House’. However, understanding this requires inferences that ‘having arrived’ at a place at a time entails ‘being at’ that place at that time, that being at the White House entails being in Washington, and so on. We ourselves draw all of these inferences effortlessly when we read the latter sentence. However, the standard logical form for our question is something like present (in washington president), while that of the text is present (perfect (arrived whitehouse president)). Of course, the commonsense knowledge that links the two statements can be hand-engineered for specialized domains, in this case by the use of resources such as named-entity linkers, ontologies, and gazeteers, and inference rules linking arriving with being there. However, there is simply too much of it to hand-engineer the open domain. The chapter begins by briefly reviewing some early attempts to build such representations by hand. It then compares the two main alternative contemporary approaches to the discovery of hidden meaning-representations for relation-denoting content words. Section 21.3 then examines the extension of one of these approaches to the discovery of latent episodic relations such as temporal sequence and causality between such terms, and examines some extensions and limitations of the approach. A brief concluding section considers some broader implications for the theory of meaning, andits implications for practical tasks like question answering.}, keywords = {}, pubstate = {published}, tppubtype = {inbook} } Natural language text such as that found on the web or in newspapers can nowadays be efficiently parsed, including building meaning representations or logical forms, with somewhat usable accuracy, at speeds of hundreds or even thousands of sentences a second. And Google is parsing everything we type at it. Nevertheless, we still don’t have real question-answering (QA), so that we can ask a question such as ‘Is the president in Washington today?’, have it mapped to an equivalent query, and get a precise answer. Such an answer could in principle be based on a semantic net or knowledge graph of eventualities, continually built and updated by semantic parsers reading the newspapers. Instead, we are still presented with a bunch of snippets from pages whose words and linkages may or may not answer our question when we ourselves do the reading. The central problem in using parsers to answer questions from unrestricted text like this is that the answer to our question is very likely to be there somewhere, but that it is almost certainly in a form which is not the same as that suggested by the form of our question. For example, the question ‘Is the president in Washington?’ is in fact answered by the statement in today’s paper that ‘The president has arrived at the White House’. However, understanding this requires inferences that ‘having arrived’ at a place at a time entails ‘being at’ that place at that time, that being at the White House entails being in Washington, and so on. We ourselves draw all of these inferences effortlessly when we read the latter sentence. However, the standard logical form for our question is something like present (in washington president), while that of the text is present (perfect (arrived whitehouse president)). Of course, the commonsense knowledge that links the two statements can be hand-engineered for specialized domains, in this case by the use of resources such as named-entity linkers, ontologies, and gazeteers, and inference rules linking arriving with being there. However, there is simply too much of it to hand-engineer the open domain. The chapter begins by briefly reviewing some early attempts to build such representations by hand. It then compares the two main alternative contemporary approaches to the discovery of hidden meaning-representations for relation-denoting content words. Section 21.3 then examines the extension of one of these approaches to the discovery of latent episodic relations such as temporal sequence and causality between such terms, and examines some extensions and limitations of the approach. A brief concluding section considers some broader implications for the theory of meaning, andits implications for practical tasks like question answering. |
Li, Bowen; Mou, Lili; Keller, Frank An Imitation Learning Approach to Unsupervised Parsing Inproceedings Korhonen, Anna; Traum, David; `a, Lluís M (Ed.): Annual Meeting of the Association for Computational Linguistics, pp. 3485–3492, Association for Computational Linguistics, 2019, ISBN: 978-1-950737-48-2. @inproceedings{de274647730e4df0aba5f9cb86a34b4c, title = {An Imitation Learning Approach to Unsupervised Parsing}, author = {Bowen Li and Lili Mou and Frank Keller}, editor = {Anna Korhonen and David Traum and Lluís M `a}, isbn = {978-1-950737-48-2}, year = {2019}, date = {2019-01-01}, booktitle = {Annual Meeting of the Association for Computational Linguistics}, volume = {1}, pages = {3485–3492}, publisher = {Association for Computational Linguistics}, abstract = {Recently, there has been an increasing interest in unsupervised parsers that optimize semantically oriented objectives, typically using reinforcement learning. Unfortunately, the learned trees often do not match actual syntax trees well. Shen et al. (2018) propose a structured attention mechanism for language modeling (PRPN), which induces better syntactic structures but relies on ad hoc heuristics.Also, their model lacks interpretability as it is not grounded in parsing actions. In our work, we propose an imitation learning approach to unsupervised parsing, where we transfer the syntactic knowledge induced by the PRPN to a Tree-LSTM model with discrete parsing actions. Its policy is then refined by GumbelSoftmax training towards a semantically oriented objective. We evaluate our approach on the All Natural Language Inference dataset and show that it achieves a new state of the art in terms of parsing F-score, outperforming our base models, including the PRPN.}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } Recently, there has been an increasing interest in unsupervised parsers that optimize semantically oriented objectives, typically using reinforcement learning. Unfortunately, the learned trees often do not match actual syntax trees well. Shen et al. (2018) propose a structured attention mechanism for language modeling (PRPN), which induces better syntactic structures but relies on ad hoc heuristics.Also, their model lacks interpretability as it is not grounded in parsing actions. In our work, we propose an imitation learning approach to unsupervised parsing, where we transfer the syntactic knowledge induced by the PRPN to a Tree-LSTM model with discrete parsing actions. Its policy is then refined by GumbelSoftmax training towards a semantically oriented objective. We evaluate our approach on the All Natural Language Inference dataset and show that it achieves a new state of the art in terms of parsing F-score, outperforming our base models, including the PRPN. |
De Cao, Nicola ; Aziz, Wilker; Titov, Ivan Question Answering by Reasoning Across Documents with Graph Convolutional Networks Inproceedings Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics, pp. 2306–2317, Association for Computational Linguistics, 2019. Abstract | Links | BibTeX | Tags: @inproceedings{785dedf7c76345e3a8560ab18bf4bf82, title = {Question Answering by Reasoning Across Documents with Graph Convolutional Networks}, author = {Nicola {De Cao} and Wilker Aziz and Ivan Titov}, doi = {10.18653/v1/n19-1240}, year = {2019}, date = {2019-01-01}, booktitle = {Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics}, volume = {1}, pages = {2306--2317}, publisher = {Association for Computational Linguistics}, abstract = {Most research in reading comprehension has focused on answering questions based on individual documents or even single paragraphs. We introduce a neural model which integrates and reasons relying on information spread within documents and across multiple documents. We frame it as an inference problem on a graph. Mentions of entities are nodes of this graph while edges encode relations between different mentions (e.g., within- and crossdocument coreference). Graph convolutional networks (GCNs) are applied to these graphs and trained to perform multi-step reasoning. Our Entity-GCN method is scalable and compact, and it achieves state-of-the-art results on a multi-document question answering dataset, WIKIHOP (Welbl et al., 2018).}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } Most research in reading comprehension has focused on answering questions based on individual documents or even single paragraphs. We introduce a neural model which integrates and reasons relying on information spread within documents and across multiple documents. We frame it as an inference problem on a graph. Mentions of entities are nodes of this graph while edges encode relations between different mentions (e.g., within- and crossdocument coreference). Graph convolutional networks (GCNs) are applied to these graphs and trained to perform multi-step reasoning. Our Entity-GCN method is scalable and compact, and it achieves state-of-the-art results on a multi-document question answering dataset, WIKIHOP (Welbl et al., 2018). |
Stanojevic, Milos; Steedman, Mark CCG Parsing Algorithm with Incremental Tree Rotation Inproceedings Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pp. 228–239, Association for Computational Linguistics, 2019. @inproceedings{121cd30eb679406299a70ba7f2e367f2, title = {CCG Parsing Algorithm with Incremental Tree Rotation}, author = {Milos Stanojevic and Mark Steedman}, year = {2019}, date = {2019-01-01}, booktitle = {Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)}, volume = {1}, pages = {228–239}, publisher = {Association for Computational Linguistics}, abstract = {The main obstacle to incremental sentence processing arises from right-branching constituent structures, which are present in the majority of English sentences, as well as from optional constituents that adjoin on the right, such as right adjuncts and right conjuncts. In CCG, many right-branching derivations can be replaced by semantically equivalent left-branching incremental derivations. The problem of right-adjunction is more resistant to solution, and has been tackled in the past using revealing-based approaches that often rely either on the higher-order unification over lambda terms (Pareschi and Steedman, 1987) or heuristics over dependency representations that do not cover the whole CCGbank (Ambati et al., 2015).We propose a new incremental parsing algorithm for CCG following the same revealing tradition of work but having a purely syntactic approach that does not depend on access to a distinct level of semantic representation. This algorithm can cover the whole CCGbank, with greater incrementality and accuracy than previous proposals.}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } The main obstacle to incremental sentence processing arises from right-branching constituent structures, which are present in the majority of English sentences, as well as from optional constituents that adjoin on the right, such as right adjuncts and right conjuncts. In CCG, many right-branching derivations can be replaced by semantically equivalent left-branching incremental derivations. The problem of right-adjunction is more resistant to solution, and has been tackled in the past using revealing-based approaches that often rely either on the higher-order unification over lambda terms (Pareschi and Steedman, 1987) or heuristics over dependency representations that do not cover the whole CCGbank (Ambati et al., 2015).We propose a new incremental parsing algorithm for CCG following the same revealing tradition of work but having a purely syntactic approach that does not depend on access to a distinct level of semantic representation. This algorithm can cover the whole CCGbank, with greater incrementality and accuracy than previous proposals. |
Torr, John; Stanojevic, Milos; Steedman, Mark; Cohen, Shay Wide-Coverage Neural A* Parsing for Minimalist Grammars Inproceedings Korhonen, Anna; Traum, David; `a, Lluís M (Ed.): Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 2486–2505, ACL Anthology, 2019. @inproceedings{4cbfe4d11bda41eb84d5153483d0e5ac, title = {Wide-Coverage Neural A* Parsing for Minimalist Grammars}, author = {John Torr and Milos Stanojevic and Mark Steedman and Shay Cohen}, editor = {Anna Korhonen and David Traum and Lluís M `a}, year = {2019}, date = {2019-01-01}, booktitle = {Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)}, volume = {1}, pages = {2486–2505}, publisher = {ACL Anthology}, abstract = {Minimalist Grammars (Stabler, 1997) are a computationally oriented, and rigorous formalization of many aspects of Chomsky’s (1995) Minimalist Program. This paper presents the first ever application of this formalism to the task of realistic wide-coverage parsing. The parser uses a linguistically expressive yet highly constrained grammar together with an adaptation of the A* search algorithm currently used in CCG parsing (Lewis and Steedman, 2014; Lewis et al., 2016), with supertag probabilities provided by a bi-LSTM neural network supertagger trained on MGbank, a corpus of MG derivation trees. We report on some promising initial experimental results for overall dependency recovery as well as on the recovery of certain unbounded long distance dependencies. Finally, although like other MG parsers, ours has a high order polynomial worst case time complexity, we show that in practice its expected time complexity is O(n3). The parser is publicly available.11: https://github.com/mgparsing/astar_mg_parser}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } Minimalist Grammars (Stabler, 1997) are a computationally oriented, and rigorous formalization of many aspects of Chomsky’s (1995) Minimalist Program. This paper presents the first ever application of this formalism to the task of realistic wide-coverage parsing. The parser uses a linguistically expressive yet highly constrained grammar together with an adaptation of the A* search algorithm currently used in CCG parsing (Lewis and Steedman, 2014; Lewis et al., 2016), with supertag probabilities provided by a bi-LSTM neural network supertagger trained on MGbank, a corpus of MG derivation trees. We report on some promising initial experimental results for overall dependency recovery as well as on the recovery of certain unbounded long distance dependencies. Finally, although like other MG parsers, ours has a high order polynomial worst case time complexity, we show that in practice its expected time complexity is O(n3). The parser is publicly available.11: https://github.com/mgparsing/astar_mg_parser |
Liu, Yang; Titov, Ivan; Lapata, Maria Single Document Summarization as Tree Induction Inproceedings Burstein, Jill; Doran, Christy; Solorio, Thamar (Ed.): Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics, pp. 1745–1755, Association for Computational Linguistics (ACL), 2019. Abstract | Links | BibTeX | Tags: @inproceedings{3d1e85d70ae1481db516e9c219c43f01, title = {Single Document Summarization as Tree Induction}, author = {Yang Liu and Ivan Titov and Maria Lapata}, editor = {Jill Burstein and Christy Doran and Thamar Solorio}, doi = {10.18653/v1/n19-1173}, year = {2019}, date = {2019-01-01}, booktitle = {Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics}, volume = {1}, pages = {1745–1755}, publisher = {Association for Computational Linguistics (ACL)}, abstract = {In this paper we conceptualize single-document extractive summarization as a tree induction problem. In contrast to previous approaches (Marcu, 1999; Yoshida et al., 2014) which have relied on linguistically motivated document representations to generate summaries, our model induces a multi-root dependency tree while predicting the output summary. Each root node in the tree is a summary sentence, and the subtrees attached to it are sentences whose content relates to or explains the summary sentence. We design a new iterative refinement algorithm: it induces the trees through repeatedly refining the structures predicted by previous iterations. We demonstrate experimentally on two benchmark datasets that our summarizer1 performs competitively against state-of-the-art methods.1: Our code is publicly available at https://github.com/nlpyang/SUMO. }, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } In this paper we conceptualize single-document extractive summarization as a tree induction problem. In contrast to previous approaches (Marcu, 1999; Yoshida et al., 2014) which have relied on linguistically motivated document representations to generate summaries, our model induces a multi-root dependency tree while predicting the output summary. Each root node in the tree is a summary sentence, and the subtrees attached to it are sentences whose content relates to or explains the summary sentence. We design a new iterative refinement algorithm: it induces the trees through repeatedly refining the structures predicted by previous iterations. We demonstrate experimentally on two benchmark datasets that our summarizer1 performs competitively against state-of-the-art methods.1: Our code is publicly available at https://github.com/nlpyang/SUMO. |
Voita, Elena; Sennrich, Rico; Titov, Ivan When a Good Translation is Wrong in Context: Context-Aware Machine Translation Improves on Deixis, Ellipsis, and Lexical Cohesion Inproceedings Korhonen, Anna; Traum, David; `a, Lluís M (Ed.): Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (long papers), pp. 1198–1212, ACL Anthology, 2019, ISBN: 978-1-950737-48-2. @inproceedings{c0687c320c5546268300c44b834fb92d, title = {When a Good Translation is Wrong in Context: Context-Aware Machine Translation Improves on Deixis, Ellipsis, and Lexical Cohesion}, author = {Elena Voita and Rico Sennrich and Ivan Titov}, editor = {Anna Korhonen and David Traum and Lluís M `a}, isbn = {978-1-950737-48-2}, year = {2019}, date = {2019-01-01}, booktitle = {Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (long papers)}, pages = {1198–1212}, publisher = {ACL Anthology}, abstract = {Though machine translation errors caused by the lack of context beyond one sentence have long been acknowledged, the development of context-aware NMT systems is hampered by several problems. Firstly, standard metrics are not sensitive to improvements in consistency in document-level translations. Secondly, previous work on context-aware NMT assumed that the sentence-aligned parallel data consisted of complete documents while in most practical scenarios such document-level data constitutes only a fraction of the available parallel data. To address the first issue, we perform a human study on an English-Russian subtitles dataset and identify deixis, ellipsis and lexical cohesion as three main sources of inconsistency. We then create test sets targeting these phenomena. To address the second shortcoming, we consider a set-up in which a much larger amount of sentence-level data is available compared to that aligned at the document level. We introduce a model that is suitable for this scenario and demonstrate major gains over a context-agnostic baseline on our new benchmarks without sacrificing performance as measured with BLEU.1 1We release code and data sets at https://github.com/lena-voita/ good-translation-wrong-in-context.}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } Though machine translation errors caused by the lack of context beyond one sentence have long been acknowledged, the development of context-aware NMT systems is hampered by several problems. Firstly, standard metrics are not sensitive to improvements in consistency in document-level translations. Secondly, previous work on context-aware NMT assumed that the sentence-aligned parallel data consisted of complete documents while in most practical scenarios such document-level data constitutes only a fraction of the available parallel data. To address the first issue, we perform a human study on an English-Russian subtitles dataset and identify deixis, ellipsis and lexical cohesion as three main sources of inconsistency. We then create test sets targeting these phenomena. To address the second shortcoming, we consider a set-up in which a much larger amount of sentence-level data is available compared to that aligned at the document level. We introduce a model that is suitable for this scenario and demonstrate major gains over a context-agnostic baseline on our new benchmarks without sacrificing performance as measured with BLEU.1 1We release code and data sets at https://github.com/lena-voita/ good-translation-wrong-in-context. |
Hunter, Tim; Stanojevic, Milos; Stabler, Edward The Active-Filler Strategy in a Move-Eager Left-Corner Minimalist Grammar Parser Inproceedings Proceedings of the Workshop on Cognitive Modeling and Computational Linguistics, pp. 1–10, Association for Computational Linguistics, 2019. @inproceedings{2d98622198d1412e9d1c6e44ec341b40, title = {The Active-Filler Strategy in a Move-Eager Left-Corner Minimalist Grammar Parser}, author = {Tim Hunter and Milos Stanojevic and Edward Stabler}, year = {2019}, date = {2019-01-01}, booktitle = {Proceedings of the Workshop on Cognitive Modeling and Computational Linguistics}, pages = {1--10}, publisher = {Association for Computational Linguistics}, abstract = {Recent psycholinguistic evidence suggests that human parsing of moved elements is 'active', and perhaps even 'hyper-active': it seems that a leftward-moved object is related to a verbal position rapidly, perhaps even before the transitivity information associated with the verb is available to the listener. This paper presents a formal, sound and complete parser for Minimalist Grammars whose search space contains branching points that we can identify as the locus of the decision to perform this kind of active gap-finding. This brings formal models of parsing into closer contact with recent psycholinguistic theorizing than was previously possible.}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } Recent psycholinguistic evidence suggests that human parsing of moved elements is 'active', and perhaps even 'hyper-active': it seems that a leftward-moved object is related to a verbal position rapidly, perhaps even before the transitivity information associated with the verb is available to the listener. This paper presents a formal, sound and complete parser for Minimalist Grammars whose search space contains branching points that we can identify as the locus of the decision to perform this kind of active gap-finding. This brings formal models of parsing into closer contact with recent psycholinguistic theorizing than was previously possible. |
Koncel-Kedziorski, Rik; Bekal, Dhanush; Luan, Yi; Lapata, Maria; Hajishirzi, Hannaneh Text Generation from Knowledge Graphs with Graph Transformers Inproceedings Burstein, Jill; Doran, Christy; Solorio, Thamar (Ed.): Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics, pp. 2284–2293, Association for Computational Linguistics (ACL), 2019. @inproceedings{879a7e0a02ca4ea0b636f20466fef6ea, title = {Text Generation from Knowledge Graphs with Graph Transformers}, author = {Rik Koncel-Kedziorski and Dhanush Bekal and Yi Luan and Maria Lapata and Hannaneh Hajishirzi}, editor = {Jill Burstein and Christy Doran and Thamar Solorio}, year = {2019}, date = {2019-01-01}, booktitle = {Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics}, volume = {1}, pages = {2284–2293}, publisher = {Association for Computational Linguistics (ACL)}, abstract = {Generating texts which express complex ideas spanning multiple sentences requires a structured representation of their content (document plan), but these representations are prohibitively expensive to manually produce. In this work, we address the problem of generating coherent multi-sentence texts from the output of an information extraction system, and in particular a knowledge graph. Graphical knowledge representations are ubiquitous in computing, but pose a significant challenge for text generation techniques due to their non-hierarchical nature, collapsing of longdistance dependencies, and structural variety. We introduce a novel graph transforming encoder which can leverage the relational structure of such knowledge graphs without imposing linearization or hierarchical constraints. Incorporated into an encoder-decoder setup, we provide an end-to-end trainable system for graph-to-text generation that we apply to the domain of scientific text. Automatic and human evaluations show that our technique produces more informative texts which exhibit better document structure than competitive encoder-decoder methods.}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } Generating texts which express complex ideas spanning multiple sentences requires a structured representation of their content (document plan), but these representations are prohibitively expensive to manually produce. In this work, we address the problem of generating coherent multi-sentence texts from the output of an information extraction system, and in particular a knowledge graph. Graphical knowledge representations are ubiquitous in computing, but pose a significant challenge for text generation techniques due to their non-hierarchical nature, collapsing of longdistance dependencies, and structural variety. We introduce a novel graph transforming encoder which can leverage the relational structure of such knowledge graphs without imposing linearization or hierarchical constraints. Incorporated into an encoder-decoder setup, we provide an end-to-end trainable system for graph-to-text generation that we apply to the domain of scientific text. Automatic and human evaluations show that our technique produces more informative texts which exhibit better document structure than competitive encoder-decoder methods. |
Bergmanis, Toms; Goldwater, Sharon Data Augmentation for Context-Sensitive Neural Lemmatization Using Inflection Tables and Raw Text Inproceedings Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics, pp. 4119–4128, Association for Computational Linguistics, 2019. @inproceedings{f2faa085168d4d749a3bdb7febc208e1, title = {Data Augmentation for Context-Sensitive Neural Lemmatization Using Inflection Tables and Raw Text}, author = {Toms Bergmanis and Sharon Goldwater}, year = {2019}, date = {2019-01-01}, booktitle = {Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics}, pages = {4119–4128}, publisher = {Association for Computational Linguistics}, abstract = {Lemmatization aims to reduce the sparse data problem by relating the inflected forms of a word to its dictionary form. Using context can help, both for unseen and ambiguous words. Yet most context-sensitive approaches require full lemma-annotated sentences for training, which may be scarce or unavailable in low-resource languages. In addition (as shown here), in a low-resource setting, a lemmatizer can learn more from n labeled examples of distinct words (types) than from n (contiguous) labeled tokens, since the latter contain far fewer distinct types. To combine the efficiency of type-based learning with the benefits of context, we propose a way to train a context-sensitive lemmatizer with little or no labeled corpus data, using inflection tables from the UniMorph project and raw text examples from Wikipedia that provide sentence contexts for the unambiguous UniMorph examples. Despite these being unambiguous examples, the model successfully generalizes from them, leading to improved results (both overall, and especially on unseen words) in comparison to a baseline that does not use context.}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } Lemmatization aims to reduce the sparse data problem by relating the inflected forms of a word to its dictionary form. Using context can help, both for unseen and ambiguous words. Yet most context-sensitive approaches require full lemma-annotated sentences for training, which may be scarce or unavailable in low-resource languages. In addition (as shown here), in a low-resource setting, a lemmatizer can learn more from n labeled examples of distinct words (types) than from n (contiguous) labeled tokens, since the latter contain far fewer distinct types. To combine the efficiency of type-based learning with the benefits of context, we propose a way to train a context-sensitive lemmatizer with little or no labeled corpus data, using inflection tables from the UniMorph project and raw text examples from Wikipedia that provide sentence contexts for the unambiguous UniMorph examples. Despite these being unambiguous examples, the model successfully generalizes from them, leading to improved results (both overall, and especially on unseen words) in comparison to a baseline that does not use context. |
Corkery, Maria; Matusevych, Yevgen; Goldwater, Sharon Are we there yet? Encoder-decoder neural networks as cognitive models of English past tense inflection Inproceedings Korhonen, Anna; Traum, David; `a, Lluís M (Ed.): Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 3868–3877, Association for Computational Linguistics (ACL), 2019. @inproceedings{e71ebbd1fcd74d7ca3817742a204bc9f, title = {Are we there yet? Encoder-decoder neural networks as cognitive models of English past tense inflection}, author = {Maria Corkery and Yevgen Matusevych and Sharon Goldwater}, editor = {Anna Korhonen and David Traum and Lluís M `a}, year = {2019}, date = {2019-01-01}, booktitle = {Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics}, pages = {3868–3877}, publisher = {Association for Computational Linguistics (ACL)}, abstract = {The cognitive mechanisms needed to account for the English past tense have long been a subject of debate in linguistics and cognitive science. Neural network models were proposed early on, but were shown to have clear flaws. Recently, however, Kirov and Cotterell (2018) showed that modern encoder-decoder (ED) models overcome many of these flaws. They also presented evidence that ED models demonstrate humanlike performance in a nonce-word task. Here, we look more closely at the behaviour of their model in this task. We find that (1) the model exhibits instability across multiple simulations in terms of its correlation with human data, and (2) even when results are aggregated across simulations (treating each simulation as an individual human participant), the fit to the human data is not strong—worse than an older rule-based model. These findings hold up through several alternative training regimes and evaluation measures. Although other neural architectures might do better, we conclude that there is still insufficient evidence to claim that neural nets are a good cognitive model for this task.}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } The cognitive mechanisms needed to account for the English past tense have long been a subject of debate in linguistics and cognitive science. Neural network models were proposed early on, but were shown to have clear flaws. Recently, however, Kirov and Cotterell (2018) showed that modern encoder-decoder (ED) models overcome many of these flaws. They also presented evidence that ED models demonstrate humanlike performance in a nonce-word task. Here, we look more closely at the behaviour of their model in this task. We find that (1) the model exhibits instability across multiple simulations in terms of its correlation with human data, and (2) even when results are aggregated across simulations (treating each simulation as an individual human participant), the fit to the human data is not strong—worse than an older rule-based model. These findings hold up through several alternative training regimes and evaluation measures. Although other neural architectures might do better, we conclude that there is still insufficient evidence to claim that neural nets are a good cognitive model for this task. |
Corro, Caio; Titov, Ivan Learning Latent Trees with Stochastic Perturbations and Differentiable Dynamic Programming Inproceedings Korhonen, Anna; Traum, David; `a, Lluís M (Ed.): Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (long papers), pp. 5508–5521, ACL Anthology, 2019, ISBN: 978-1-950737-48-2. @inproceedings{486211b72d9e447697532a1cb40e3d85, title = {Learning Latent Trees with Stochastic Perturbations and Differentiable Dynamic Programming}, author = {Caio Corro and Ivan Titov}, editor = {Anna Korhonen and David Traum and Lluís M `a}, isbn = {978-1-950737-48-2}, year = {2019}, date = {2019-01-01}, booktitle = {Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (long papers)}, volume = {1}, pages = {5508–5521}, publisher = {ACL Anthology}, abstract = {We treat projective dependency trees as latent variables in our probabilistic model and induce them in such a way as to be beneficial for a downstream task, without relying on any direct tree supervision. Our approach relies on Gumbel perturbations and differentiable dynamic programming. Unlike previous approaches to latent tree learning, we stochastically sample global structures and our parser is fully differentiable.We illustrate its effectiveness on sentiment analysis and natural language inference tasks. We also study its properties on a synthetic structure induction task. Ablation studies emphasize the importance of both stochasticity and constraining latent structures to be projective trees.}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } We treat projective dependency trees as latent variables in our probabilistic model and induce them in such a way as to be beneficial for a downstream task, without relying on any direct tree supervision. Our approach relies on Gumbel perturbations and differentiable dynamic programming. Unlike previous approaches to latent tree learning, we stochastically sample global structures and our parser is fully differentiable.We illustrate its effectiveness on sentiment analysis and natural language inference tasks. We also study its properties on a synthetic structure induction task. Ablation studies emphasize the importance of both stochasticity and constraining latent structures to be projective trees. |
Chen, Lushi; Magdy, Walid; Wolters, Maria Online Community Engagement when Talking About Infidelity: The Case of Reddit Inproceedings 5th International Conference on Computational Social Science IC2S2, 2019. @inproceedings{2400a494fea34ec8b9f9b58317ce9259, title = {Online Community Engagement when Talking About Infidelity: The Case of Reddit}, author = {Lushi Chen and Walid Magdy and Maria Wolters}, year = {2019}, date = {2019-01-01}, booktitle = {5th International Conference on Computational Social Science IC2S2}, abstract = {Writing or talking about upsetting events on social media has many benefits for posters, es-pecially in forums such as Reddit that support pseudonymity. The act of writing in itself has therapeutic effects on mental health; posters can find others who share similar experiences, thus addressing feelings of loneliness and isolation; and posters can receive social support. Community engagement is a prerequisite for obtaining support on social media. In this study, we investigate community engagement for social media about a specific stressful life event, the experience of infidelity. What constitutes infidelity varies from person to person. Here, we focus on people reporting that a romantic partner has done something which the person perceives as an act of infidelity. We chose Reddit, as this platform encourages pseudony-mous discussion. Using unsupervised clustering, we create a typology of posts and commentsthat elicit different levels of community engagement and discuss implications for the management of online communities.}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } Writing or talking about upsetting events on social media has many benefits for posters, es-pecially in forums such as Reddit that support pseudonymity. The act of writing in itself has therapeutic effects on mental health; posters can find others who share similar experiences, thus addressing feelings of loneliness and isolation; and posters can receive social support. Community engagement is a prerequisite for obtaining support on social media. In this study, we investigate community engagement for social media about a specific stressful life event, the experience of infidelity. What constitutes infidelity varies from person to person. Here, we focus on people reporting that a romantic partner has done something which the person perceives as an act of infidelity. We chose Reddit, as this platform encourages pseudony-mous discussion. Using unsupervised clustering, we create a typology of posts and commentsthat elicit different levels of community engagement and discuss implications for the management of online communities. |
De Cao, Nicola ; Aziz, Wilker; Titov, Ivan Block Neural Autoregressive Flow Inproceedings Proceedings of the Thirty-Fifth Conference on Uncertainty in Artificial Intelligence, UAI 2019, 2019. @inproceedings{8dbe5dc639af43d89b3657931f4048ac, title = {Block Neural Autoregressive Flow}, author = {Nicola {De Cao} and Wilker Aziz and Ivan Titov}, year = {2019}, date = {2019-01-01}, booktitle = {Proceedings of the Thirty-Fifth Conference on Uncertainty in Artificial Intelligence, UAI 2019}, abstract = {Normalising flows (NFs) map two density functions via a differentiable bijection whose Jacobian determinant can be computed efficiently. Recently, as an alternative to handcrafted bijections, Huang et al. (2018) proposed neural autoregressive flow (NAF) which is a universal approximator for density functions. Their flow is a neural network (NN) whose parameters are predicted by another NN. The latter grows quadratically with the size of the former and thus an efficient technique for parametrization is needed. We propose block neural autoregressive flow (BNAF), a much more compact universal approximator of density functions, where we model a bijection directly using a single feedforward network. Invertibility is ensured by carefully designing each affine transformation with block matrices that make the flow autoregressive and (strictly) monotone. We compare B-NAF to NAF and other established flows on density estimation and approximate inference for latent variable models. Our proposed flow is competitive across datasets while using orders of magnitude fewer parameters.}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } Normalising flows (NFs) map two density functions via a differentiable bijection whose Jacobian determinant can be computed efficiently. Recently, as an alternative to handcrafted bijections, Huang et al. (2018) proposed neural autoregressive flow (NAF) which is a universal approximator for density functions. Their flow is a neural network (NN) whose parameters are predicted by another NN. The latter grows quadratically with the size of the former and thus an efficient technique for parametrization is needed. We propose block neural autoregressive flow (BNAF), a much more compact universal approximator of density functions, where we model a bijection directly using a single feedforward network. Invertibility is ensured by carefully designing each affine transformation with block matrices that make the flow autoregressive and (strictly) monotone. We compare B-NAF to NAF and other established flows on density estimation and approximate inference for latent variable models. Our proposed flow is competitive across datasets while using orders of magnitude fewer parameters. |
Voita, Elena; Talbot, David; Moiseev, Fedor; Sennrich, Rico; Titov, Ivan Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Pruned Inproceedings Korhonen, Anna; Traum, David; `a}rquez, Lluís { M (Ed.): Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (long papers), pp. 5797–5808, ACL Anthology, 2019, ISBN: 978-1-950737-48-2. @inproceedings{f18d8d1df51c4e70b00346093dae0b51, title = {Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Pruned}, author = {Elena Voita and David Talbot and Fedor Moiseev and Rico Sennrich and Ivan Titov}, editor = {Anna Korhonen and David Traum and Lluís { M{`a}rquez}}, isbn = {978-1-950737-48-2}, year = {2019}, date = {2019-01-01}, booktitle = {Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (long papers)}, volume = {1}, pages = {5797–5808}, publisher = {ACL Anthology}, abstract = {Multi-head self-attention is a key component of the Transformer, a state-of-the-art architecture for neural machine translation. In this work we evaluate the contribution made by individual attention heads in the encoder to the overall performance of the model and analyze the roles played by them. We find that the most important and confident heads play consistent and often linguistically-interpretable roles. When pruning heads using a method based on stochastic gates and a differentiable relaxation of the L0 penalty, we observe that specialized heads are last to be pruned. Our novel pruning method removes the vast majority of heads without seriously affecting performance.For example, on the English-Russian WMT dataset, pruning 38 out of 48 encoder heads results in a drop of only 0.15 BLEU.1 1 We release code athttps://github.com/lena-voita/the-story-of-heads.}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } Multi-head self-attention is a key component of the Transformer, a state-of-the-art architecture for neural machine translation. In this work we evaluate the contribution made by individual attention heads in the encoder to the overall performance of the model and analyze the roles played by them. We find that the most important and confident heads play consistent and often linguistically-interpretable roles. When pruning heads using a method based on stochastic gates and a differentiable relaxation of the L0 penalty, we observe that specialized heads are last to be pruned. Our novel pruning method removes the vast majority of heads without seriously affecting performance.For example, on the English-Russian WMT dataset, pruning 38 out of 48 encoder heads results in a drop of only 0.15 BLEU.1 1 We release code athttps://github.com/lena-voita/the-story-of-heads. |
Chapman-Rounds, Matt; Lucas, Christopher; Keller, Frank Inattentional Blindness in Visual Search Inproceedings Goel, Ashok; Colleen, ; Freksa, Christian (Ed.): Proceedings of the 41st Annual Conference of the Cognitive Science Society, pp. 2688–2694, 2019, ISBN: 0-9911967-7-5. Abstract | BibTeX | Tags: Bayesian Modelling, Conjunction Search, Inattentional Blindness, Predictive Processing, Visual Attention @inproceedings{fa469d40a26c4120b62f53472e6f6cd4, title = {Inattentional Blindness in Visual Search}, author = {Matt Chapman-Rounds and Christopher Lucas and Frank Keller}, editor = {Ashok Goel and {Colleen } Seifert and Christian Freksa}, isbn = {0-9911967-7-5}, year = {2019}, date = {2019-01-01}, booktitle = {Proceedings of the 41st Annual Conference of the Cognitive Science Society}, pages = {2688--2694}, abstract = {Models of visual saliency normally belong to one of two camps: models such as Experience Guided Search (E-GS), which emphasize top-down guidance based on task features, and models such as Attention as Information Maximisation (AIM), which emphasize the role of bottom-up saliency. In this paper, we show that E-GS and AIM are structurally similar and can be unified to create a general model of visual search which includes a generic prior over potential non-task related objects. We demonstrate that this model displays inattentional blindness, and that blindness can be modulated by adjusting the relative precisions of several terms within the model. At the same time, our model correctly accounts for a series of classical visual search results.}, keywords = {Bayesian Modelling, Conjunction Search, Inattentional Blindness, Predictive Processing, Visual Attention}, pubstate = {published}, tppubtype = {inproceedings} } Models of visual saliency normally belong to one of two camps: models such as Experience Guided Search (E-GS), which emphasize top-down guidance based on task features, and models such as Attention as Information Maximisation (AIM), which emphasize the role of bottom-up saliency. In this paper, we show that E-GS and AIM are structurally similar and can be unified to create a general model of visual search which includes a generic prior over potential non-task related objects. We demonstrate that this model displays inattentional blindness, and that blindness can be modulated by adjusting the relative precisions of several terms within the model. At the same time, our model correctly accounts for a series of classical visual search results. |
Liu, Jiangming; Shay B, ; Lapata, Mirella Discourse Representation Structure Parsing with Recurrent Neural Networks and the Transformer Model Inproceedings Proceedings of the IWCS Shared Task on Semantic Parsing, Association for Computational Linguistics, 2019. @inproceedings{82f53598d581429b8cbbd4ace88706e7, title = {Discourse Representation Structure Parsing with Recurrent Neural Networks and the Transformer Model}, author = {Jiangming Liu and {Shay B } Cohen and Mirella Lapata}, year = {2019}, date = {2019-01-01}, booktitle = {Proceedings of the IWCS Shared Task on Semantic Parsing}, publisher = {Association for Computational Linguistics}, abstract = {We describe the systems we developed for Discourse Representation Structure (DRS) parsing as part of the IWCS-2019 Shared Task of DRS Parsing.1 Our systems are based on sequence-to- sequence modeling. To implement our model, we use the open-source neural machine translation system implemented in PyTorch, OpenNMT-py. We experimented with a variety of encoder-decoder models based on recurrent neural networks and the Transformer model. We conduct experiments on the standard benchmark of the Parallel Meaning Bank (PMB 2.2). Our best system achieves a score of 84.8% F1 in the DRS parsing shared task.}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } We describe the systems we developed for Discourse Representation Structure (DRS) parsing as part of the IWCS-2019 Shared Task of DRS Parsing.1 Our systems are based on sequence-to- sequence modeling. To implement our model, we use the open-source neural machine translation system implemented in PyTorch, OpenNMT-py. We experimented with a variety of encoder-decoder models based on recurrent neural networks and the Transformer model. We conduct experiments on the standard benchmark of the Parallel Meaning Bank (PMB 2.2). Our best system achieves a score of 84.8% F1 in the DRS parsing shared task. |
Algotiml, Bushra; AbdelRahim A, ; Magdy, Walid Arabic Tweet-Act: Speech Act Recognition for Arabic Asynchronous Conversations Inproceedings Proceedings of the Fourth Arabic Natural Language Processing Workshop, 2019. @inproceedings{43b2bffe97354cc8824f2e9d3dce6945, title = {Arabic Tweet-Act: Speech Act Recognition for Arabic Asynchronous Conversations}, author = {Bushra Algotiml and {AbdelRahim A } Elmadany and Walid Magdy}, year = {2019}, date = {2019-01-01}, booktitle = {Proceedings of the Fourth Arabic Natural Language Processing Workshop}, abstract = {Speech acts are the actions that a speaker intends when performing an utterance within conversations. In this paper, we proposed speech act classification for asynchronous conversations on Twitter using multiple machine learning methods including SVM and deep neural networks. We applied the proposed methods on the ArSAS tweets dataset. The obtained results show that superiority of deep learning methods compared to SVMs, where Bi-LSTM managed to achieve an accuracy of 87.5% and a macro-averaged F1 score 61.5%. We believe that our results are the first to be reported on the task of speech-act recognition for asynchronous conversations on Arabic Twitter.}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } Speech acts are the actions that a speaker intends when performing an utterance within conversations. In this paper, we proposed speech act classification for asynchronous conversations on Twitter using multiple machine learning methods including SVM and deep neural networks. We applied the proposed methods on the ArSAS tweets dataset. The obtained results show that superiority of deep learning methods compared to SVMs, where Bi-LSTM managed to achieve an accuracy of 87.5% and a macro-averaged F1 score 61.5%. We believe that our results are the first to be reported on the task of speech-act recognition for asynchronous conversations on Arabic Twitter. |
Damonte, Marco; Cohen, Shay Structural Neural Encoders for AMR-to-text Generation Inproceedings Burstein, Jill; Doran, Christy; Solorio, Thamar (Ed.): Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics, pp. 3649–3658, Association for Computational Linguistics (ACL), 2019. @inproceedings{71dd94b10de2471482339d60ab151264, title = {Structural Neural Encoders for AMR-to-text Generation}, author = {Marco Damonte and Shay Cohen}, editor = {Jill Burstein and Christy Doran and Thamar Solorio}, year = {2019}, date = {2019-01-01}, booktitle = {Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics}, volume = {1}, pages = {3649–3658}, publisher = {Association for Computational Linguistics (ACL)}, abstract = {AMR-to-text generation is a problem recently introduced to the NLP community, in which the goal is to generate sentences from Abstract Meaning Representation (AMR) graphs. Sequence-to-sequence models can be used to this end by converting the AMR graphs to strings. Approaching the problem while working directly with graphs requires the use of graph-to-sequence models that encode the AMR graph into a vector representation. Such encoding has been shown to be beneficial in the past, and unlike sequential encoding, it allows us to explicitly capture reentrant structures in the AMR graphs. We investigate the extent to which reentrancies (nodes with multiple parents) have an impact on AMR-to-text generation by comparing graph encoders to tree encoders, where reentrancies are not preserved. We show that improvements in the treatment of reentrancies and long-range dependencies contribute to higher overall scores for graph encoders. Our best model achieves 24.40 BLEU on LDC2015E86, outperforming the state of the art by 1.1 points and 24.54 BLEU on LDC2017T10, outperforming the state of the art by 1.24 points.}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } AMR-to-text generation is a problem recently introduced to the NLP community, in which the goal is to generate sentences from Abstract Meaning Representation (AMR) graphs. Sequence-to-sequence models can be used to this end by converting the AMR graphs to strings. Approaching the problem while working directly with graphs requires the use of graph-to-sequence models that encode the AMR graph into a vector representation. Such encoding has been shown to be beneficial in the past, and unlike sequential encoding, it allows us to explicitly capture reentrant structures in the AMR graphs. We investigate the extent to which reentrancies (nodes with multiple parents) have an impact on AMR-to-text generation by comparing graph encoders to tree encoders, where reentrancies are not preserved. We show that improvements in the treatment of reentrancies and long-range dependencies contribute to higher overall scores for graph encoders. Our best model achieves 24.40 BLEU on LDC2015E86, outperforming the state of the art by 1.1 points and 24.54 BLEU on LDC2017T10, outperforming the state of the art by 1.24 points. |
Bastings, Joost; Aziz, Wilker; Titov, Ivan Interpretable Neural Predictions with Differentiable Binary Variables Inproceedings Korhonen, Anna; Traum, David; `a, Lluís M (Ed.): Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (long papers), pp. 2963–2977, ACL Anthology, 2019, ISBN: 978-1-950737-48-2. @inproceedings{05264a06f72a400b9d6a720218e10b96, title = {Interpretable Neural Predictions with Differentiable Binary Variables}, author = {Joost Bastings and Wilker Aziz and Ivan Titov}, editor = {Anna Korhonen and David Traum and Lluís M `a}, isbn = {978-1-950737-48-2}, year = {2019}, date = {2019-01-01}, booktitle = {Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (long papers)}, volume = {1}, pages = {2963–2977}, publisher = {ACL Anthology}, abstract = {The success of neural networks comes hand in hand with a desire for more interpretability. We focus on text classifiers and make them more interpretable by having them provide a justification—a rationale—for their predictions.We approach this problem by jointly training two neural network models: a latent model that selects a rationale (i.e. a short and informative part of the input text), and a classifier that learns from the words in the rationale alone. Previous work proposed to assign binary latent masks to input positions and to promote short selections via sparsityinducing penalties such as L0 regularisation.We propose a latent model that mixes discrete and continuous behaviour allowing at the same time for binary selections and gradient-based training without REINFORCE. In our formulation, we can tractably compute the expected value of penalties such as L0, which allows us to directly optimise the model towards a prespecified text selection rate. We show that our approach is competitive with previous work on rationale extraction, and explore further uses in attention mechanisms.}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } The success of neural networks comes hand in hand with a desire for more interpretability. We focus on text classifiers and make them more interpretable by having them provide a justification—a rationale—for their predictions.We approach this problem by jointly training two neural network models: a latent model that selects a rationale (i.e. a short and informative part of the input text), and a classifier that learns from the words in the rationale alone. Previous work proposed to assign binary latent masks to input positions and to promote short selections via sparsityinducing penalties such as L0 regularisation.We propose a latent model that mixes discrete and continuous behaviour allowing at the same time for binary selections and gradient-based training without REINFORCE. In our formulation, we can tractably compute the expected value of penalties such as L0, which allows us to directly optimise the model towards a prespecified text selection rate. We show that our approach is competitive with previous work on rationale extraction, and explore further uses in attention mechanisms. |
Abu Farha, Ibrahim ; Magdy, Walid Mazajak: An Online Arabic Sentiment Analyser Inproceedings Proceedings of the Fourth Arabic Natural Language Processing Workshop, 2019. @inproceedings{bc32529dad9f4d189b8939f68760e02d, title = {Mazajak: An Online Arabic Sentiment Analyser}, author = {Ibrahim {Abu Farha} and Walid Magdy}, year = {2019}, date = {2019-01-01}, booktitle = {Proceedings of the Fourth Arabic Natural Language Processing Workshop}, abstract = {Sentiment analysis (SA) is one of the most useful natural language processing applications. Literature is flooding with many papers and systems addressing this task, but most of the work is focused on English. In this paper, we present “Mazajak”, an online system for Arabic SA. The system is based on a deep learning model, which achieves state-of-theart results on many Arabic dialect datasets including SemEval 2017 and ASTD. The availability of such system should assist various applications and research that rely on sentiment analysis as a tool.}, keywords = {}, pubstate = {published}, tppubtype = {inproceedings} } Sentiment analysis (SA) is one of the most useful natural language processing applications. Literature is flooding with many papers and systems addressing this task, but most of the work is focused on English. In this paper, we present “Mazajak”, an online system for Arabic SA. The system is based on a deep learning model, which achieves state-of-theart results on many Arabic dialect datasets including SemEval 2017 and ASTD. The availability of such system should assist various applications and research that rely on sentiment analysis as a tool. |
2019 |
A Lightweight Recurrent Network for Sequence Modeling Inproceedings Korhonen, Anna; Traum, David; `a, {Lluís } M (Ed.): Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 1538–1548, Association for Computational Linguistics (ACL), 2019. |
Neural Grammatical Error Correction Systems with Unsupervised Pre-training on Synthetic Data Inproceedings Yannakoudakis, Helen; Kochmar, Ekaterina; Leacock, Claudia; Madnani, Nitin; á, Ildikó Pil; Zesch, Torsten (Ed.): Proceedings of the Fourteenth Workshop on Innovative Use of NLP for Building Educational Applications, pp. 252–263, Association for Computational Linguistics, 2019. |
Exploring Author Context for Detecting Intended vs Perceived Sarcasm Inproceedings Korhonen, Anna; Traum, David; `a, Lluís M (Ed.): Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 2854–2859, Association for Computational Linguistics (ACL), 2019. |
Understanding Neural Machine Translation by Simplification: The Case of Encoder-free Models Output type: Conference contribution Inproceedings Proceedings of the International Conference Recent Advances in Natural Language Processing, Association for Computational Linguistics (ACL), 2019. |
Duality of Link Prediction and Entailment Graph Induction Inproceedings Korhonen, Anna; Traum, David; `a, Lluís M (Ed.): Proceedings of the 57th Annual Conference of the Association for Computational Linguistics (long papers), pp. 4736–4746, ACL Anthology, 2019, ISBN: 978-1-950737-48-2. |
Character-based Surprisal as a Model of Reading Difficulty in the Presence of Errors Inproceedings Goel, Ashok; Seifert, Colleen; Freksa, Christian (Ed.): Proceedings of the 41st Annual Conference of the Cognitive Science Society, pp. 401–407, 2019, ISBN: 0-9911967-7-5. |
A Practical Guide for the Effective Evaluation of Twitter User Geolocation Journal Article ACM Transactions on Social Computing (TSC), 2019, ISSN: 2469-7818. |
Geoparsing Historical and Contemporary Literary Text set in the City of Edinburgh Journal Article Language Resources and Evaluation, 2019, ISSN: 1574-020X, (Springer Compact Gold OA). |
The problem with probabilistic DAG automata for semantic graphs Inproceedings Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics, pp. 902–911, Association for Computational Linguistics, 2019. |
Your Stance is Exposed! Analysing Possible Factors for Stance Detection on Social Media Inproceedings Proceedings for the 22nd ACM Conference on Computer-Supported Cooperative Work and Social Computing, 2019. |
On the Computational Complexity of Head Movement and Affix Hopping Inproceedings Formal Grammar 2019, pp. 101–116, Springer-Verlag Berlin Heidelberg, 2019, ISBN: 978-3-662-59647-0. |
Dependency Grammar Induction with a Neural Variational Transition-based Parser Inproceedings Proceedings of the 33rd AAAI Conference on Artificial Intelligence, pp. 6658–6665, AAAI Press, 2019. |
Differentiable Perturb-and-Parse: Semi-Supervised Parsing with a Structured Variational Autoencoder Inproceedings Seventh International Conference on Learning Representations (ICLR 2017), 2019. |
"Why is the Doctor a Man?" Reactions of Older Adults to a Virtual Training Doctor Inproceedings Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems, ACM, 2019, ISBN: 978-1-4503-5971-9. |
Understanding Learning Dynamics Of Language Models with SVCCA Inproceedings Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics, pp. 3257–3267, Association for Computational Linguistics, 2019. |
Discontinuous Constituency Parsing with a Stack-Free Transition System and a Dynamic Oracle Inproceedings Burstein, Jill; Doran, Christy; Solorio, Thamar (Ed.): Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics, pp. 204–217, Association for Computational Linguistics (ACL), 2019. |
Surprise Languages: Rapid-Response Cross-Language IR Inproceedings Nicola, ; Ishita, Emi; Kando, Noriko; Makoto P, ; Soboroff, Ian; Zhang, Min (Ed.): Proceedings of the Ninth International Workshop on Evaluating Information Access (EVIA 2019), pp. 23, ACM, 2019, ISBN: 978-4-86049-077-5. |
Pre-training on high-resource speech recognition improves low-resource speech-to-text translation Inproceedings Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics, pp. 58–68, Association for Computational Linguistics, 2019. |
2019, (Second UK Healthcare Text Analytics Conference, HealTAC 2019 ; Conference date: 24-04-2019 Through 25-04-2019). |
Text Mining Brain Imaging Reports Journal Article Journal of Biomedical Semantics, 2019, ISSN: 2041-1480. |
Assessing Sentiment of the Expressed Stance on Social Media Inproceedings Social Informatics, 2019. |
Widening the Representation Bottleneck in Neural Machine Translation with Lexical Shortcuts Inproceedings Proceedings of the Fourth Conference on Machine Translation (WMT), Volume 1, pp. 102–115, Association for Computational Linguistics (ACL), 2019, ISBN: 978-1-950737-27-7. |
Plague Dot Text: Text mining and annotation of outbreak reports of the Third Plague Pandemic (1894-1952) Inproceedings Proceedings of HistoInformatics 2019, CEUR Workshop Proceedings, 2019, (To be published in 12 Sep 2019). |
Analyzing and modeling free word associations Inproceedings Kalish, Chuck; Rau, Marina; Rogers, Tim; Zhu, Jerry (Ed.): Proceedings of the 40th Annual Conference of the Cognitive Science Society, pp. 750–755, Cognitive Science Society, 2019, ISBN: 978-1-5108-7205-9. |
Boosting Entity Linking Performance by Leveraging Unlabeled Documents Inproceedings Korhonen, Anna; Traum, David; `a, Lluís M (Ed.): Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (long papers), pp. 1935–1945, ACL Anthology, 2019, ISBN: 978-1-950737-48-2. |
Combinatory Categorial Grammar Book Chapter é, András Kert; Moravcsik, Edith; á, Csilla R (Ed.): Current Approaches to Syntax, pp. 389–420, De Gruyter Mouton, Germany, 2019. |
It is more than what you Say!: Leveraging User Online Activity for Improved Stance Detection Inproceedings 5th International Conference on Computational Social Science IC2S2, 2019. |
Distant Learning for Entity Linking with Automatic Noise Detection Inproceedings Korhonen, Anna; Traum, David; `a, Lluís M (Ed.): Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (long papers), pp. 4081–4090, ACL Anthology, 2019, ISBN: 978-1-950737-48-2. |
Revisiting Low-Resource Neural Machine Translation: A Case Study Inproceedings Korhonen, Anna; Traum, David; `a, Lluís M (Ed.): Proceedings of the 57th Conference of the Association for Computational Linguistics, pp. 211–221, Association for Computational Linguistics (ACL), 2019. |
Form-Independent Meaning Representation for Eventualities Book Chapter Truswell, Robert (Ed.): The Oxford Handbook of Event Structure, pp. 605–623, Oxford University Press, United Kingdom, 2019. |
An Imitation Learning Approach to Unsupervised Parsing Inproceedings Korhonen, Anna; Traum, David; `a, Lluís M (Ed.): Annual Meeting of the Association for Computational Linguistics, pp. 3485–3492, Association for Computational Linguistics, 2019, ISBN: 978-1-950737-48-2. |
Question Answering by Reasoning Across Documents with Graph Convolutional Networks Inproceedings Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics, pp. 2306–2317, Association for Computational Linguistics, 2019. |
CCG Parsing Algorithm with Incremental Tree Rotation Inproceedings Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pp. 228–239, Association for Computational Linguistics, 2019. |
Wide-Coverage Neural A* Parsing for Minimalist Grammars Inproceedings Korhonen, Anna; Traum, David; `a, Lluís M (Ed.): Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 2486–2505, ACL Anthology, 2019. |
Single Document Summarization as Tree Induction Inproceedings Burstein, Jill; Doran, Christy; Solorio, Thamar (Ed.): Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics, pp. 1745–1755, Association for Computational Linguistics (ACL), 2019. |
When a Good Translation is Wrong in Context: Context-Aware Machine Translation Improves on Deixis, Ellipsis, and Lexical Cohesion Inproceedings Korhonen, Anna; Traum, David; `a, Lluís M (Ed.): Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (long papers), pp. 1198–1212, ACL Anthology, 2019, ISBN: 978-1-950737-48-2. |
The Active-Filler Strategy in a Move-Eager Left-Corner Minimalist Grammar Parser Inproceedings Proceedings of the Workshop on Cognitive Modeling and Computational Linguistics, pp. 1–10, Association for Computational Linguistics, 2019. |
Text Generation from Knowledge Graphs with Graph Transformers Inproceedings Burstein, Jill; Doran, Christy; Solorio, Thamar (Ed.): Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics, pp. 2284–2293, Association for Computational Linguistics (ACL), 2019. |
Data Augmentation for Context-Sensitive Neural Lemmatization Using Inflection Tables and Raw Text Inproceedings Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics, pp. 4119–4128, Association for Computational Linguistics, 2019. |
Are we there yet? Encoder-decoder neural networks as cognitive models of English past tense inflection Inproceedings Korhonen, Anna; Traum, David; `a, Lluís M (Ed.): Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 3868–3877, Association for Computational Linguistics (ACL), 2019. |
Learning Latent Trees with Stochastic Perturbations and Differentiable Dynamic Programming Inproceedings Korhonen, Anna; Traum, David; `a, Lluís M (Ed.): Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (long papers), pp. 5508–5521, ACL Anthology, 2019, ISBN: 978-1-950737-48-2. |
Online Community Engagement when Talking About Infidelity: The Case of Reddit Inproceedings 5th International Conference on Computational Social Science IC2S2, 2019. |
Block Neural Autoregressive Flow Inproceedings Proceedings of the Thirty-Fifth Conference on Uncertainty in Artificial Intelligence, UAI 2019, 2019. |
Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Pruned Inproceedings Korhonen, Anna; Traum, David; `a}rquez, Lluís { M (Ed.): Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (long papers), pp. 5797–5808, ACL Anthology, 2019, ISBN: 978-1-950737-48-2. |
Inattentional Blindness in Visual Search Inproceedings Goel, Ashok; Colleen, ; Freksa, Christian (Ed.): Proceedings of the 41st Annual Conference of the Cognitive Science Society, pp. 2688–2694, 2019, ISBN: 0-9911967-7-5. |
Discourse Representation Structure Parsing with Recurrent Neural Networks and the Transformer Model Inproceedings Proceedings of the IWCS Shared Task on Semantic Parsing, Association for Computational Linguistics, 2019. |
Arabic Tweet-Act: Speech Act Recognition for Arabic Asynchronous Conversations Inproceedings Proceedings of the Fourth Arabic Natural Language Processing Workshop, 2019. |
Structural Neural Encoders for AMR-to-text Generation Inproceedings Burstein, Jill; Doran, Christy; Solorio, Thamar (Ed.): Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics, pp. 3649–3658, Association for Computational Linguistics (ACL), 2019. |
Interpretable Neural Predictions with Differentiable Binary Variables Inproceedings Korhonen, Anna; Traum, David; `a, Lluís M (Ed.): Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (long papers), pp. 2963–2977, ACL Anthology, 2019, ISBN: 978-1-950737-48-2. |
Mazajak: An Online Arabic Sentiment Analyser Inproceedings Proceedings of the Fourth Arabic Natural Language Processing Workshop, 2019. |