Vector Space Semantic Parsing: A Framework for Compositional ...

More documents

Recommendations

Info

Production and Manufacturing by Omnipress, Inc. 2600 Anderson Street Madison, WI 53704 USA c○2013 The Association for Computational Linguistics Order copies of this and other ACL proceedings from: Association for Computational Linguistics (ACL) 209 N. Eighth Street Stroudsburg, PA 18360 USA Tel: +1-570-476-8006 Fax: +1-570-476-0860 acl@aclweb.org ISBN: 978-1-937284-67-1 ii
Introduction In recent years, there has been a growing interest in algorithms that learn a continuous representation for words, phrases, or documents. For instance, one can see latent semantic analysis (Landauer and Dumais, 1997) and latent Dirichlet allocation (Blei et al. 2003) as a mapping of documents or words into a continuous lower dimensional topic-space. Another example, continuous word vector-space models (Sahlgren 2006, Reisinger 2012, Turian et al., 2010, Huang et al., 2012) represent word meanings with vectors that capture semantic and syntactic information. These representations can be used to induce similarity measures by computing distances between the vectors, leading to many useful applications, such as information retrieval (Schuetze 1992, Manning et al., 2008), search query expansions (Jones et al., 2006), document classification (Sebastiani, 2002) and question answering (Tellex et al., 2003). On the fundamental task of language modeling, many hard clustering approaches have been proposed such as Brown clustering (Brown et al.,1992) or exchange clustering (Martin et al.,1998). These algorithms can provide desparsification and can be seen as examples of unsupervised pre-training. However, they have not been shown to consistently outperform models based on Kneser-Ney smoothed language models which have at their core discrete n-gram representations. On the contrary, one influential proposal that uses the idea of continuous vector spaces for language modeling is that of neural language models (Bengio et al., 2003, Mikolov 2012). In these approaches, n-gram probabilities are estimated using a continuous representation of words in lieu of standard discrete representations, using a neural network that performs both the projection and the probability estimate. They report state of the art performance on several well studied language modeling datasets. Other neural network based models that use continuous vector representations achieve state of the art performance in speech recognition applications (Schwenk, 2007, Dahl et al. 2011), multitask learning, NER and POS tagging (Collobert et al., 2011) or sentiment analysis (Socher et al. 2011). Moreover, in (Le et al., 2012), a continuous space translation model was introduced and its use in a large scale machine translation system yielded promising results in the last WMT evaluation. Despite the success of single word vector space models, they are severely limited since they do not capture compositionality, the important quality of natural language that allows speakers to determine the meaning of a longer expression based on the meanings of its words and the rules used to combine them (Frege, 1892). This prevents them from gaining a deeper understanding of the semantics of longer phrases or sentences. Recently, there has been much progress in capturing compositionality in vector spaces, e.g., (Pado and Lapata 2007; Erk and Pado 2008; Mitchell and Lapata, 2010; Baroni and Zamparelli, 2010; Zanzotto et al., 2010; Yessenalina and Cardie, 2011; Grefenstette and Sadrzadeh 2011). The work of Socher et al. 2012 compares several of these approaches on supervised tasks and for phrases of arbitrary type and length. Another different trend of research belongs to the family of spectral methods. The motivation in that context is that working in a continuous space allows for the design of algorithms that are not plagued with the local minima issues that discrete latent space models (e.g. HMM trained with EM) tend to suffer from (Hsu et al. 2008). In fact, this motivation strikes with the conventional justification behind vector space models from the neural network literature, which are usually motivated as a way of tackling data sparsity issues. This apparent dichotomy is interesting and has not been investigated yet. Finally, spectral methods have recently been developed for word representation learning (Dhillon et al. 2011), dependency parsing (Dhillon et al. 2012) and probabilistic context-free grammars (Cohen et al. 2012). In this workshop, we bring together researchers who are interested in how to learn continuous vector space models, their compositionality and how to use this new kind of representation in NLP applications. The goal is to review the recent progress and propositions, to discuss the challenges, to identify promising future research directions and the next challenges for the NLP community. iii
Page 1: ACL 2013 51st Annual Meeting of the
Page 7: Table of Contents Vector Space Sema
Page 10 and 11: Poster session (continued) 12:30 Lu
Page 12 and 13: Input: Log. Form: “red ball”
Page 14 and 15: the NP/N λx.x red N/N λx.A red x
Page 16 and 17: dicted and target distributions, an
Page 18 and 19: typed objects lie in the same vecto
Page 20 and 21: Peter D. Turney. 2006. Similarity o
Page 22 and 23: (1999), who suggested that the real
Page 24 and 25: mation. In addition, using a new se
Page 26 and 27: Figure 6: Positive rate, negative r
Page 28 and 29: References Rens Bod, Remko Scha, an
Page 30 and 31: A Structured Distributional Semanti
Page 32 and 33: Figure 1: Sample sentences & triple
Page 34 and 35: el as its three axes. We explore th
Page 36 and 37: as well as our framing of word comp
Page 38 and 39: References Marco Baroni and Roberto
Page 40 and 41: Letter N-Gram-based Input Encoding
Page 42 and 43: Hidden my house my house Visibl
Page 44 and 45: … m my y h se us Hidden Visible m
Page 46 and 47: line of the systems presented in Ta
Page 48 and 49: References Yoshua Bengio, Réjean D
Page 50 and 51: Transducing Sentences to Syntactic
Page 52 and 53:
t s “We booked the flight” →
Page 54 and 55:
D 3(s) = ❀ PRP + ❀ V + DT ❀ +
Page 56 and 57:
Output Model λ = 0 λ = 0.2 λ = 0
Page 58 and 59:
References James A. Anderson. 1973.
Page 60 and 61:
General estimation and evaluation o
Page 62 and 63:
Guevara (2010) and Zanzotto et al.
Page 64 and 65:
⃗p = A u ⃗v where A u is a matr
Page 66 and 67:
stituents. For this reason, we must
Page 68 and 69:
References Hervé Abdi and Lynne Wi
Page 70 and 71:
GOOD VERTICAL safe out raise level
Page 72 and 73:
HLBL HLBL SENNA 4lang original scal
Page 74 and 75:
Determining Compositionality of Wor
Page 76 and 77:
ability of a word in the investigat
Page 78 and 79:
terminer) wheel” and “reinvent
Page 80 and 81:
ing LSA are depicted. Discussion As
Page 82 and 83:
WSM Measure wAvg(of ρ) ρAN-VO-SV
Page 84 and 85:
“Not not bad” is not “bad”:
Page 86 and 87:
activation functions in recursive n
Page 88 and 89:
logic experiment in Socher et al. (
Page 90 and 91:
tinction without the need to move t
Page 92 and 93:
T. K. Landauer and S. T. Dumais. 19
Page 94 and 95:
This differs from the classical Ran
Page 96 and 97:
Hungarian Algorithm First cosine si
Page 98 and 99:
Features: MSRpar MSRvid SMTeuroparl
Page 100 and 101:
Joseph Reisinger and Raymond J. Moo
Page 102 and 103:
phrase meaning in vector space. •
Page 104 and 105:
By Bayes’ rule, p(x|a, n; Θ) ∝
Page 106 and 107:
4.2 Learning from distributional re
Page 108 and 109:
W = (w 1 , w 2 , · · · , w n ) a
Page 110 and 111:
Aggregating Continuous Word Embeddi
Page 112 and 113:
trieval, Sivic and Zisserman propos
Page 114 and 115:
Another advantage is that the propo
Page 116 and 117:
e 50 100 200 300 400 500 CLEF 4.0 6
Page 118 and 119:
Y. Bengio, J. Louradour, R. Collobe
Page 120 and 121:
Answer Extraction by Recursive Pars
Page 122 and 123:
Algorithm 1: Auto-encoders co-train
Page 124 and 125:
NP NP ) , ADJP , NP ( NP - NP Engli
Page 126 and 127:
System Short Answer Short Answer Sh
Page 128 and 129:
References Y. Bengio and R. Ducharm
Page 130 and 131:
problem of paragraphs, of tence, mo
Page 132 and 133:
m1 k 4 only on the length of s and
Page 134 and 135:
Dialogue Act Label Example Train (%
Page 136 and 137:
[Calhoun et al.2010] Sasha Calhoun,
show all

Vector Space Semantic Parsing: A Framework for Compositional ...

Create successful ePaper yourself

Delete template?

Save as template?