Vector Space Semantic Parsing: A Framework for Compositional ...

More documents

Recommendations

Info

Poster session (continued) 12:30 Lunch Break Towards Dynamic Word Sense Discrimination with Random Indexing Hans Moen, Erwin Marsi and Björn Gambäck (14:00) Oral session 2 14:00 INVITED TALK: Learning to Ground Meaning in the Visual World by Mirella Lapata 15:00 A Generative Model of Vector Space Semantics Jacob Andreas and Zoubin Ghahramani 15:20 Aggregating Continuous Word Embeddings for Information Retrieval Stephane Clinchant and Florent Perronnin 15:40 Coffee Break 16:00 Answer Extraction by Recursive Parse Tree Descent Christopher Malon and Bing Bai 16:20 Recurrent Convolutional Neural Networks for Discourse Compositionality Nal Kalchbrenner and Phil Blunsom (16:40) Panel Discussion x
Vector Space Semantic Parsing: A Framework for Compositional Vector Space Models Jayant Krishnamurthy Carnegie Mellon University 5000 Forbes Avenue Pittsburgh, PA 15213 jayantk@cs.cmu.edu Tom M. Mitchell Carnegie Mellon University 5000 Forbes Avenue Pittsburgh, PA 15213 tom.mitchell@cmu.edu Abstract We present vector space semantic parsing (VSSP), a framework for learning compositional models of vector space semantics. Our framework uses Combinatory Categorial Grammar (CCG) to define a correspondence between syntactic categories and semantic representations, which are vectors and functions on vectors. The complete correspondence is a direct consequence of minimal assumptions about the semantic representations of basic syntactic categories (e.g., nouns are vectors), and CCG’s tight coupling of syntax and semantics. Furthermore, this correspondence permits nonuniform semantic representations and more expressive composition operations than previous work. VSSP builds a CCG semantic parser respecting this correspondence; this semantic parser parses text into lambda calculus formulas that evaluate to vector space representations. In these formulas, the meanings of words are represented by parameters that can be trained in a task-specific fashion. We present experiments using noun-verbnoun and adverb-adjective-noun phrases which demonstrate that VSSP can learn composition operations that RNN (Socher et al., 2011) and MV-RNN (Socher et al., 2012) cannot. 1 Introduction Vector space models represent the semantics of natural language using vectors and operations on vectors (Turney and Pantel, 2010). These models are most commonly used for individual words and short phrases, where vectors are created using distributional information from a corpus. Such models achieve impressive performance on standardized tests (Turney, 2006; Rapp, 2003), correlate well with human similarity judgments (Griffiths et al., 2007), and have been successfully applied to a number of natural language tasks (Collobert et al., 2011). While vector space representations for individual words are well-understood, there remains much uncertainty about how to compose vector space representations for phrases out of their component words. Recent work in this area raises many important theoretical questions. For example, should all syntactic categories of words be represented as vectors, or are some categories, such as adjectives, different? Using distinct semantic representations for distinct syntactic categories has the advantage of representing the operational nature of modifier words, but the disadvantage of more complex parameter estimation (Baroni and Zamparelli, 2010). Also, does semantic composition factorize according to a constituency parse tree (Socher et al., 2011; Socher et al., 2012)? A binarized constituency parse cannot directly represent many intuitive intra-sentence dependencies, such as the dependence between a verb’s subject and its object. What is needed to resolve these questions is a comprehensive theoretical framework for compositional vector space models. In this paper, we observe that we already have such a framework: Combinatory Categorial Grammar (CCG) (Steedman, 1996). CCG provides a tight mapping between syntactic categories and semantic types. If we assume that nouns, sentences, and other basic syntactic categories are represented by vectors, this mapping prescribes semantic types for all other syntactic categories. 1 For example, we get that adjectives are functions from noun vectors to noun vectors, and that prepo- 1 It is not necessary to assume that sentences are vectors. However, this assumption simplifies presentation and seems like a reasonable first step. CCG can be used similarly to explore alternative representations. 1 Proceedings of the Workshop on Continuous Vector Space Models and their Compositionality, pages 1–10, Sofia, Bulgaria, August 9 2013. c○2013 Association for Computational Linguistics
Page 1 and 2: ACL 2013 51st Annual Meeting of the
Page 3: Introduction In recent years, there
Page 7: Table of Contents Vector Space Sema
Page 12 and 13: Input: Log. Form: “red ball”
Page 14 and 15: the NP/N λx.x red N/N λx.A red x
Page 16 and 17: dicted and target distributions, an
Page 18 and 19: typed objects lie in the same vecto
Page 20 and 21: Peter D. Turney. 2006. Similarity o
Page 22 and 23: (1999), who suggested that the real
Page 24 and 25: mation. In addition, using a new se
Page 26 and 27: Figure 6: Positive rate, negative r
Page 28 and 29: References Rens Bod, Remko Scha, an
Page 30 and 31: A Structured Distributional Semanti
Page 32 and 33: Figure 1: Sample sentences & triple
Page 34 and 35: el as its three axes. We explore th
Page 36 and 37: as well as our framing of word comp
Page 38 and 39: References Marco Baroni and Roberto
Page 40 and 41: Letter N-Gram-based Input Encoding
Page 42 and 43: Hidden my house my house Visibl
Page 44 and 45: … m my y h se us Hidden Visible m
Page 46 and 47: line of the systems presented in Ta
Page 48 and 49: References Yoshua Bengio, Réjean D
Page 50 and 51: Transducing Sentences to Syntactic
Page 52 and 53: t s “We booked the flight” →
Page 54 and 55: D 3(s) = ❀ PRP + ❀ V + DT ❀ +
Page 56 and 57: Output Model λ = 0 λ = 0.2 λ = 0
Page 58 and 59: References James A. Anderson. 1973.
Page 60 and 61:
General estimation and evaluation o
Page 62 and 63:
Guevara (2010) and Zanzotto et al.
Page 64 and 65:
⃗p = A u ⃗v where A u is a matr
Page 66 and 67:
stituents. For this reason, we must
Page 68 and 69:
References Hervé Abdi and Lynne Wi
Page 70 and 71:
GOOD VERTICAL safe out raise level
Page 72 and 73:
HLBL HLBL SENNA 4lang original scal
Page 74 and 75:
Determining Compositionality of Wor
Page 76 and 77:
ability of a word in the investigat
Page 78 and 79:
terminer) wheel” and “reinvent
Page 80 and 81:
ing LSA are depicted. Discussion As
Page 82 and 83:
WSM Measure wAvg(of ρ) ρAN-VO-SV
Page 84 and 85:
“Not not bad” is not “bad”:
Page 86 and 87:
activation functions in recursive n
Page 88 and 89:
logic experiment in Socher et al. (
Page 90 and 91:
tinction without the need to move t
Page 92 and 93:
T. K. Landauer and S. T. Dumais. 19
Page 94 and 95:
This differs from the classical Ran
Page 96 and 97:
Hungarian Algorithm First cosine si
Page 98 and 99:
Features: MSRpar MSRvid SMTeuroparl
Page 100 and 101:
Joseph Reisinger and Raymond J. Moo
Page 102 and 103:
phrase meaning in vector space. •
Page 104 and 105:
By Bayes’ rule, p(x|a, n; Θ) ∝
Page 106 and 107:
4.2 Learning from distributional re
Page 108 and 109:
W = (w 1 , w 2 , · · · , w n ) a
Page 110 and 111:
Aggregating Continuous Word Embeddi
Page 112 and 113:
trieval, Sivic and Zisserman propos
Page 114 and 115:
Another advantage is that the propo
Page 116 and 117:
e 50 100 200 300 400 500 CLEF 4.0 6
Page 118 and 119:
Y. Bengio, J. Louradour, R. Collobe
Page 120 and 121:
Answer Extraction by Recursive Pars
Page 122 and 123:
Algorithm 1: Auto-encoders co-train
Page 124 and 125:
NP NP ) , ADJP , NP ( NP - NP Engli
Page 126 and 127:
System Short Answer Short Answer Sh
Page 128 and 129:
References Y. Bengio and R. Ducharm
Page 130 and 131:
problem of paragraphs, of tence, mo
Page 132 and 133:
m1 k 4 only on the length of s and
Page 134 and 135:
Dialogue Act Label Example Train (%
Page 136 and 137:
[Calhoun et al.2010] Sasha Calhoun,
show all

Vector Space Semantic Parsing: A Framework for Compositional ...

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?