Notes on computational linguistics.pdf - UCLA Department of ...

More documents

Recommendations

Info

Stabler - Lx 185/209 2003 With this code we need only about 2.58 bits per symbol (126) Consider 12123333123333123312 Here we have P(1) = P(2) = 1 1 4 and P(3) = 2 ,sotheentropyis1.5/bitspersymbol. The sequence has length 20, so we should be able to encode it with 30 bits. However, consider blocks of 2. P(12) = 1 1 , P(33) = , and the entropy is 1 bit/symbol. 2 2 For the sequence of 10 blocks of 2, we need only 10 bits. So it is often worth looking for structure in larger and larger blocks. 8.1.17 Kraft’s inequality and Shannon’s theorem (127) MacMillan: (128) Kraft (1949): If uniquely decodable code C has K codewords of lengths l1,...,lK then K 2 −li ≤ 1. i=1 If a sequence l1,...,lK satisfies the previous inequality, then there is a uniquely decodable code C that has K codewords of lengths l1,...,lK (129) Shannon’s theorem. Using the definition of Hr in (116), Shannon (1948) proves the following famous theorem which specifies the information-theoretic limits of data compression: Suppose that X is a first order source with outcomes (or outputs) ΩX. Encoding the characters of ΩX in a code with characters Γ where |Γ |=r>1 requires an average of Hr (X) characters of Γ per character of ΩX. Furthermore, for any real number ɛ>0, there is a code that uses an average of Hr (X) + ɛ characters of Γ per character of ΩX. 8.1.18 String edits and other varieties of sequence comparison Overviews of string edit distance methods are provided in Hall and Dowling (1980) and Kukich (1992). Masek and Paterson (1980) present a fast algorithm for computing string edit distances. Ristad (1997) and Ristad and Yianilos (1996) consider the problem of learning string edit distances. 8.2 Probabilisitic context free grammars and parsing 8.2.1 PCFGs (130) A probabilistic context free grammar (PCFG) where G =〈Σ,N,(→), S,P〉, 1. Σ,N are finite, nonempty sets, 2. S is some symbol in N, 3. the binary relation (→) ⊆ N × (Σ ∪ N) ∗ is also finite (i.e. it has finitely many pairs), 159
Stabler - Lx 185/209 2003 4. the function P : (→) → [0, 1] maps productions to real numbers in the closed interval between 0 and 1 in such a way that P(c → β) = 1 〈c,β〉∈(→) We will often write the probabilities assigned to each production on the arrow: c p → β (131) The probability of a parse is the product of the probabilities of the productions in the parse (132) The probability of a string is the sum of the probabilities of its parses 160
Page 1 and 2:
Notes on computati
Page 3 and 4:
Stabler - Lx 185/209 2003 Linguisti
Page 5 and 6:
Stabler - Lx 185/209 2003 1 Setting
Page 7 and 8:
Stabler - Lx 185/209 2003 1.2 Propo
Page 9 and 10:
Stabler - Lx 185/209 2003 (8) Pitfa
Page 11 and 12:
Stabler - Lx 185/209 2003 In fact,
Page 13 and 14:
Stabler - Lx 185/209 2003 10. Call
Page 15 and 16:
Stabler - Lx 185/209 2003 compute s
Page 17 and 18:
Stabler - Lx 185/209 2003 (11) Pred
Page 19 and 20:
Stabler - Lx 185/209 2003 1.6 The l
Page 21 and 22:
Stabler - Lx 185/209 2003 L0 and L1
Page 23 and 24:
Stabler - Lx 185/209 2003 Exercises
Page 25 and 26:
Stabler - Lx 185/209 2003 3 more ex
Page 27 and 28:
Stabler - Lx 185/209 2003 2 Recogni
Page 29 and 30:
Stabler - Lx 185/209 2003 (8) Then
Page 31 and 32:
Stabler - Lx 185/209 2003 (13) The
Page 33 and 34:
Page 35 and 36:
Stabler - Lx 185/209 2003 Problem (
Page 37 and 38:
Stabler - Lx 185/209 2003 1 ?- [col
Page 39 and 40:
Stabler - Lx 185/209 2003 lex(’th
Page 41 and 42:
Stabler - Lx 185/209 2003 2 ?- ([
Page 43 and 44:
Stabler - Lx 185/209 2003 (3) Dalry
Page 45 and 46:
Stabler - Lx 185/209 2003 3.3 Recog
Page 47 and 48:
Stabler - Lx 185/209 2003 (22) Supp
Page 49 and 50:
Stabler - Lx 185/209 2003 (27) With
Page 51 and 52:
Stabler - Lx 185/209 2003 (31) tcl/
Page 53 and 54:
Stabler - Lx 185/209 2003 A c-comma
Page 55 and 56:
Stabler - Lx 185/209 2003 (46) Does
Page 57 and 58:
Stabler - Lx 185/209 2003 m. The id
Page 59 and 60:
Stabler - Lx 185/209 2003 4 Brief d
Page 61 and 62:
Stabler - Lx 185/209 2003 (50) Nem
Page 63 and 64:
Stabler - Lx 185/209 2003 5 Trees,
Page 65 and 66:
Stabler - Lx 185/209 2003 children(
Page 67 and 68:
Stabler - Lx 185/209 2003 5.3 Movem
Page 69 and 70:
Stabler - Lx 185/209 2003 a. d e c
Page 71 and 72:
Stabler - Lx 185/209 2003 children(
Page 73 and 74:
Stabler - Lx 185/209 2003 adjoin_no
Page 75 and 76:
Stabler - Lx 185/209 2003 (37) With
Page 77 and 78:
Stabler - Lx 185/209 2003 Mates’
Page 79 and 80:
Stabler - Lx 185/209 2003 81. ((A
Page 81 and 82:
Stabler - Lx 185/209 2003 6.2 LR pa
Page 83 and 84:
Stabler - Lx 185/209 2003 6.3 LC pa
Page 85 and 86:
Stabler - Lx 185/209 2003 (20) Like
Page 87 and 88:
Stabler - Lx 185/209 2003 6.4 All t
Page 89 and 90:
Stabler - Lx 185/209 2003 (26) GLC
Page 91 and 92:
Stabler - Lx 185/209 2003 (33) GLC
Page 93 and 94:
Stabler - Lx 185/209 2003 The secon
Page 95 and 96:
Stabler - Lx 185/209 2003 6.5.2 Bot
Page 97 and 98:
Stabler - Lx 185/209 2003 6.6 Asses
Page 99 and 100:
Stabler - Lx 185/209 2003 (56) Stru
Page 101 and 102:
Stabler - Lx 185/209 2003 6.6.4 A d
Page 103 and 104:
Stabler - Lx 185/209 2003 1. Downlo
Page 105 and 106:
Stabler - Lx 185/209 2003 It is ess
Page 107 and 108:
Stabler - Lx 185/209 2003 inference
Page 109 and 110: Stabler - Lx 185/209 2003 1 (’SBA
Page 111 and 112: Stabler - Lx 185/209 2003 7.2 Tree
Page 113 and 114: Stabler - Lx 185/209 2003 (14) With
Page 115 and 116: Stabler - Lx 185/209 2003 /* earley
Page 117 and 118: Stabler - Lx 185/209 2003 8 Stochas
Page 119 and 120: Stabler - Lx 185/209 2003 8.1.1 Cor
Page 121 and 122: Stabler - Lx 185/209 2003 to punctu
Page 123 and 124: Stabler - Lx 185/209 2003 jane aust
Page 125 and 126: Stabler - Lx 185/209 2003 where usu
Page 127 and 128: Stabler - Lx 185/209 2003 0.1 0.09
Page 129 and 130: Stabler - Lx 185/209 2003 We get al
Page 131 and 132: Stabler - Lx 185/209 2003 Word leng
Page 133 and 134: Stabler - Lx 185/209 2003 8.1.4 Pro
Page 135 and 136: Stabler - Lx 185/209 2003 8.1.5 Ran
Page 137 and 138: Stabler - Lx 185/209 2003 Matrix ar
Page 139 and 140: Stabler - Lx 185/209 2003 (65) To a
Page 141 and 142: Stabler - Lx 185/209 2003 octave:18
Page 143 and 144: Stabler - Lx 185/209 2003 added pro
Page 145 and 146: Stabler - Lx 185/209 2003 P(q1 ...q
Page 147 and 148: Stabler - Lx 185/209 2003 c. Finall
Page 149 and 150: Stabler - Lx 185/209 2003 This is p
Page 151 and 152: Stabler - Lx 185/209 2003 (100) Abn
Page 153 and 154: Stabler - Lx 185/209 2003 3. If des
Page 155 and 156: Stabler - Lx 185/209 2003 It will b
Page 157 and 158: Stabler - Lx 185/209 2003 Entropy (
Page 159: Stabler - Lx 185/209 2003 One indir
Page 163 and 164: Stabler - Lx 185/209 2003 8.2.3 Ass
Page 165 and 166: Stabler - Lx 185/209 2003 (142) We
Page 167 and 168: Stabler - Lx 185/209 2003 8.4 Next
Page 169 and 170: Stabler - Lx 185/209 2003 9.1 “Mi
Page 171 and 172: Stabler - Lx 185/209 2003 (4) More
Page 173 and 174: Stabler - Lx 185/209 2003 This is a
Page 175 and 176: Stabler - Lx 185/209 2003 (7) Let
Page 177 and 178: Stabler - Lx 185/209 2003 (9) Let
Page 179 and 180: Stabler - Lx 185/209 2003 dP3 maria
Page 181 and 182: Stabler - Lx 185/209 2003 The 4 Eng
Page 183 and 184: Stabler - Lx 185/209 2003 9.1.4 Fou
Page 185 and 186: Stabler - Lx 185/209 2003 (21) The
Page 187 and 188: Stabler - Lx 185/209 2003 :- [’pp
Page 189 and 190: Stabler - Lx 185/209 2003 9.2.2 Som
Page 191 and 192: Stabler - Lx 185/209 2003 who laugh
Page 193 and 194: Stabler - Lx 185/209 2003 Kayne ass
Page 195 and 196: Stabler - Lx 185/209 2003 (32) Vari
Page 197 and 198: Stabler - Lx 185/209 2003 dP t 3 v
Page 199 and 200: Stabler - Lx 185/209 2003 10 Toward
Page 201 and 202: Stabler - Lx 185/209 2003 We can en
Page 203 and 204: Stabler - Lx 185/209 2003 Head move
Page 205 and 206: Stabler - Lx 185/209 2003 D which w
Page 207 and 208: Stabler - Lx 185/209 2003 CP C’ C
Page 209 and 210: Stabler - Lx 185/209 2003 CP C’ C
Page 211 and 212:
Stabler - Lx 185/209 2003 that::=T
Page 213 and 214:
Stabler - Lx 185/209 2003 T t C C S
Page 215 and 216:
Stabler - Lx 185/209 2003 10.3.4 AP
Page 217 and 218:
Stabler - Lx 185/209 2003 T t C C C
Page 219 and 220:
Stabler - Lx 185/209 2003 10.3.6 Co
Page 221 and 222:
Stabler - Lx 185/209 2003 10.4 Modi
Page 223 and 224:
Stabler - Lx 185/209 2003 10.5 Summ
Page 225 and 226:
Stabler - Lx 185/209 2003 10.5.1 Re
Page 227 and 228:
Page 229 and 230:
Stabler - Lx 185/209 2003 10.6.3 Mu
Page 231 and 232:
Stabler - Lx 185/209 2003 10.6.5 Pi
Page 233 and 234:
Stabler - Lx 185/209 2003 CP C’ C
Page 235 and 236:
Stabler - Lx 185/209 2003 but also
Page 237 and 238:
Stabler - Lx 185/209 2003 And if yo
Page 239 and 240:
Stabler - Lx 185/209 2003 sentence:
Page 241 and 242:
Stabler - Lx 185/209 2003 Example:
Page 243 and 244:
Stabler - Lx 185/209 2003 15.1 Mono
Page 245 and 246:
Stabler - Lx 185/209 2003 Example:
Page 247 and 248:
Stabler - Lx 185/209 2003 16 Harder
Page 249 and 250:
Stabler - Lx 185/209 2003 A first,
Page 251 and 252:
Stabler - Lx 185/209 2003 non-demon
Page 253 and 254:
Stabler - Lx 185/209 2003 16.4 Scop
Page 255 and 256:
Stabler - Lx 185/209 2003 16.5 Infe
Page 257 and 258:
Stabler - Lx 185/209 2003 Extra cre
Page 259 and 260:
Stabler - Lx 185/209 2003 CP C’ C
Page 261 and 262:
Stabler - Lx 185/209 2003 (5) In su
Page 263 and 264:
Stabler - Lx 185/209 2003 (7) Anoth
Page 265 and 266:
Stabler - Lx 185/209 2003 17.2.1 A
Page 267 and 268:
Page 269 and 270:
Stabler - Lx 185/209 2003 Reference
Page 271 and 272:
Stabler - Lx 185/209 2003 Cornell,
Page 273 and 274:
Stabler - Lx 185/209 2003 Hale, Joh
Page 275 and 276:
Stabler - Lx 185/209 2003 Kraft, L.
Page 277 and 278:
Stabler - Lx 185/209 2003 Pollock,
Page 279 and 280:
Stabler - Lx 185/209 2003 Stabler,
Page 281 and 282:
Index (x, y), openintervalfromx to
Page 283 and 284:
Stabler - Lx 185/209 2003 Herbrand,
Page 285:
Stabler - Lx 185/209 2003 Seki, Hir
show all

Notes on computational linguistics.pdf - UCLA Department of ...

Create successful ePaper yourself

Delete template?

Save as template?