Domain Testing: Divide and Conquer - Testing Education

More documents

Recommendations

Info

Report on experiment on teaching specific testing techniques. Pat McGee 2 January 2004 Subjects There were 18 subjects in the first group. To date, I have only received files for 17 of them. The files for the 18 th subject seem to have gone missing. There were five subjects in the second group. Identification of variables Of those 17 subjects in the first group, 12 attempted to answer the dimension question, 12 attempted to answer the type question, and 9 attempted to answer the direction question. Of the second group, all five attempted to answer the all three of those questions. Three of them answered the dimensions question in part b, but not in part a. On the dimension and type question, it looked to me like the subjects exhibited poor test-taking behavior. The subjects that answered these questions mostly answered them correctly. I don't see any reason to believe that these subjects were taught poorly. On the direction question, just over half the subjects in the first group attempted to answer it at all. Of those, all but two subjects identified all the fields as inputs. Two of the subjects identified some but not all of the input/output fields as outputs. Of those two, one was in each of the first two sessions. None identified any fields as input/output. In the second group, all five identified all the variables as input only. From this, I conclude that this was not taught well. It seems to have been taught somewhat better in the first two sessions than the three after that. Equivalence class and boundary value analysis I believe that the problem presented was somewhat different from the simple cases presented in the training. The problem presented had four interrelated controls which basically represented a 2-D space with special points. McGeeReport v1.doc 1 03 January 2004 09:01
The material presented in training was mostly about 1-D spaces. So, in many respects, this problem was a good test in that it required performance at level 6 of Bloom's taxonomy. Unfortunately, none of the subjects made this conceptual leap. They tried to treat the problem as if it were two independent 1-D spaces. This led them to propose tests that I thought were not very powerful. They did well on the outside boundaries of each dimension, since these could be tested independently. They did well on the enumerated variables, correctly identifying each value as something they needed to test. Testing the numeric inputs showed some conceptual problems. Almost all of the students tested the inputs for field lengths of 5 and 32. Nowhere in the documentation that I saw claimed either one as a limit. The program seems to enforce 5 as a limit, but whether this is according to any specification or not, I can't tell. I think this was an invented limit because they couldn't deal with the concept of an unknowable limit. When dealing with experienced testers, I think it is a good thing to write test cases as specifications of constraints that the test cases must fit. I expect experienced testers to be able to generate good test cases from specifications. However, when dealing with inexperienced testers, I don't have that same expectation. Until I see some test cases that they generate from a specification, I don't know whether a specific person understands enough about what makes a test case good to generate good ones. Many of the subjects wrote more generic specifications for test cases instead of generating specific values. So, without more data, I can't tell whether they did this knowing how to translate that into good test cases, or whether they did it because they didn't know how to write good test cases and could figure out some specifications. Only one (1) subject wrote test cases with almost all test cases having specific values. Six (6) wrote them with most values specific, while fifteen (15) wrote them with few values specific. For the seven who wrote at least mostly test cases with specific values instead of specifications, the test cases were good. For the fifteen who didn't, I have no way of knowing whether they could actually generate good test cases for those specifications. McGeeReport v1.doc 2 03 January 2004 09:01
Page 1 and 2:
Domain Testing: Divide and Conquer
Page 3 and 4:
We the undersigned committee hereby
Page 5 and 6:
Table of Contents Chapter 1: Introd
Page 7 and 8:
Chapter 3: Instructional Design and
Page 9 and 10:
5.04 Facilities Used for the Experi
Page 11 and 12:
List of Tables Table 6.01 Question
Page 13 and 14:
Acknowledgements • Dr. Cem Kaner,
Page 15 and 16:
1.01 Problem Description Chapter 1:
Page 17 and 18:
There are two general approaches to
Page 19 and 20:
effective testing with reduced effo
Page 21 and 22:
Other researchers have used the fea
Page 23 and 24:
July 22, 2003; Frankl, Hamlet, Litt
Page 25 and 26:
Chapter 2: Domain Testing: A Litera
Page 27 and 28:
Black-box testers may or may not re
Page 29 and 30:
For instance, in the above example,
Page 31 and 32:
Myers (1979) suggested associating
Page 33 and 34:
In the functional approach to doing
Page 35 and 36:
in the correctness of the program.
Page 37 and 38:
Whittaker and Jorgensen (2002) desc
Page 39 and 40:
listed above. o Worst Case Testing
Page 41 and 42:
He referred to the number of test c
Page 43 and 44:
Hutcheson (2003) described boundary
Page 45 and 46:
2.02.03.02 Special Value Testing Jo
Page 47 and 48:
change has caused something that us
Page 49 and 50:
Hence, a row in an orthogonal array
Page 51 and 52:
combinations of valid equivalence c
Page 53 and 54:
Chapter 3: Instructional Design and
Page 55 and 56:
Carey, 1985; Dick et al., 2001; Dri
Page 57 and 58:
and how they apply the knowledge. T
Page 59 and 60:
Keeping the questionnaires anonymou
Page 61 and 62:
objectives of the instruction. Bein
Page 63 and 64:
software testing courses before, si
Page 65 and 66:
accomplished learning. This also he
Page 67 and 68:
available and integrate them with t
Page 69 and 70:
substance of the materials to make
Page 71 and 72:
4. Expertise-oriented approaches, w
Page 73 and 74:
specifically said, “A pretest is
Page 75 and 76:
instructional design itself. Accord
Page 77 and 78:
• Appendix G: Day 2 Lecture • A
Page 79 and 80:
• Reference materials • Linking
Page 81 and 82:
Every exercise question and every q
Page 83 and 84:
Chapter 5: Experiment Design 5.01 O
Page 85 and 86:
This first version of the advertise
Page 87 and 88:
o Numeric variables � Integer �
Page 89 and 90:
learners should be sufficient to te
Page 91 and 92:
Standard Deviation: “A measure de
Page 93 and 94:
Question 2 (10 points): • Test A:
Page 95 and 96:
Page 97 and 98:
Page 99 and 100:
Page 101 and 102:
Page 103 and 104:
Page 105 and 106:
Page 107 and 108:
Page 109 and 110:
Table 6.10: Final Scores - Pretest
Page 111 and 112:
Chart 6.01: Paper-Based Pretest and
Page 113 and 114:
6.02 Performance Test Results Appen
Page 115 and 116:
11 12 0 0 9. How would you rate you
Page 117 and 118:
commented that they liked the fact
Page 119 and 120:
11. What was the least useful part
Page 121 and 122:
appreciated the fact that a lot of
Page 123 and 124:
Most of them commented that they be
Page 125 and 126:
48% Q4: How would you rate your ove
Page 127 and 128:
Chapter 7: Conclusion 7.01 Domain T
Page 129 and 130:
7.01.02 Minuses Kaner (2003) also c
Page 131 and 132:
work is strong evidence that the st
Page 133 and 134:
Boland, P. J., Singh, H., & Cukic,
Page 135 and 136:
Clarke, L. A., Hassell, J., & Richa
Page 137 and 138:
Hajnal, À., & Forgács, I. (1998).
Page 139 and 140:
Jeng, B., & Weyuker, E. J. (1994, J
Page 141 and 142:
Mayrhauser, A. V., Walls, J., & Mra
Page 143 and 144:
Schroeder, P. J., & Korel, B. (2000
Page 145 and 146:
Zhu, H., Hall, P. A. V., & May, J.
Page 147 and 148:
Appendix A: Reference Material# 1 f
Page 149 and 150:
What is an input variable? An input
Page 151 and 152:
Step 6: Determine whether the varia
Page 153 and 154:
Domain Testing Equivalence Class an
Page 155 and 156:
Partition the input domain into dif
Page 157 and 158:
domain specific special condition b
Page 159 and 160:
Appendix C: Reference Material# 3 f
Page 161 and 162:
What is All-Pairs Combination Testi
Page 163 and 164:
If ‘n’ is the total number of v
Page 165 and 166:
Step 3: Step 4: Step 5: Test case#
Page 167 and 168:
Test case# a(3) b(2) c(2) d(2) e(2)
Page 169 and 170:
Step 9: • The only two pairs miss
Page 171 and 172:
Appendix D: Equivalence Class Analy
Page 173 and 174:
Appendix A: Equivalence Class Analy
Page 175 and 176:
Page 177 and 178:
Page 179 and 180:
Page 181 and 182:
Page 183 and 184:
Appendix E: Guidelines for All-Pair
Page 185 and 186:
Appendix B: Guidelines for filling
Page 187 and 188:
Appendix F: Day 1 Lecture (Introduc
Page 189 and 190:
Why do we need to test software? To
Page 191 and 192:
Analysis To assert that the output
Page 193 and 194:
Analysis The total possible combina
Page 195 and 196:
What is Domain Testing? Domain test
Page 197 and 198:
Domain testing-Terminology Boundary
Page 199 and 200:
Summary Our goal is to not only do
Page 201 and 202:
Session 2: Domain Testing Sowmya Pa
Page 203 and 204:
Testing Integer fields � Example
Page 205 and 206:
Testing Integer fields Example 1
Page 207 and 208:
Page 209 and 210:
Page 211 and 212:
Variable(s) Equivalence Class(es)
Page 213 and 214:
Page 215 and 216:
Page 217 and 218:
Page 219 and 220:
Testing Integer fields Example 2 St
Page 221 and 222:
© Sowmya Padmanabhan, 2003 19
Page 223 and 224:
Word Problems � Example 3: SunTru
Page 225 and 226:
Word Problems Example 3 � Credit
Page 227 and 228:
Word Problems Example 3 � ‘cred
Page 229 and 230:
Word Problems Example 3 � $400
Page 231 and 232:
Word Problems Example 3 � The inp
Page 233 and 234:
Word Problems Example 3 � Failure
Page 235 and 236:
Variable(s) Equivalence Class(es)
Page 237 and 238:
Word Problems Example 4 � Step 1:
Page 239 and 240:
Page 241 and 242:
Page 243 and 244:
Page 245 and 246:
Page 247 and 248:
Word Problems Example 4 Step 6: Par
Page 249 and 250:
Word Problems � Example 5: The pa
Page 251 and 252:
Word Problems Example 5 � User, P
Page 253 and 254:
Word Problems Example 5 � ‘widt
Page 255 and 256:
Word Problems Example 5 � 1
Page 257 and 258:
Word Problems Example 5 � The inp
Page 259 and 260:
Word Problems Example 5 � Failure
Page 261 and 262:
Variable (s) ‘width’ Equivalenc
Page 263 and 264:
Testing multiple ranges � The pro
Page 265 and 266:
Testing multiple ranges Example 6
Page 267 and 268:
Page 269 and 270:
Page 271 and 272:
Page 273 and 274:
Page 275 and 276:
Page 277 and 278:
Word Problems Example 6 Step 6: Par
Page 279 and 280:
Analyzing non-numbers � Let us fi
Page 281 and 282:
Standard ASCII The first 32 charact
Page 283 and 284:
52 4 53 5 54 6 55 7 56 8 57 9 58 :
Page 285 and 286:
108 l 109 m 110 n 111 o 112 p 113 q
Page 287 and 288:
ASCII characters � ASCII files:
Page 289 and 290:
Why bother about ASCII characters i
Page 291 and 292:
Analyzing non-numbers Example 6 �
Page 293 and 294:
Analyzing non-numbers Example 6 Par
Page 295 and 296:
Variable Equivalence classes tax-in
Page 297 and 298:
Testing String fields � We encoun
Page 299 and 300:
Testing String fields Example 7 �
Page 301 and 302:
Page 303 and 304:
Page 305 and 306:
Testing String fields Example 7 Ste
Page 307 and 308:
Page 309 and 310:
25 EM End of medium 26 SUB Substitu
Page 311 and 312:
80 P 81 Q 82 R 83 S 84 T 85 U 86 V
Page 313 and 314:
Extended ASCII Characters Reference
Page 315 and 316:
Session 3: Domain Testing Sowmya Pa
Page 317 and 318:
Testing multidimensional variables
Page 319 and 320:
Testing multidimensional variables
Page 321 and 322:
General analysis of multidimensiona
Page 323 and 324:
Multidimensional numeric fields Exa
Page 325 and 326:
Page 327 and 328:
Page 329 and 330:
Page 331 and 332:
Page 333 and 334:
Page 335 and 336:
Multidimensional string fields Exam
Page 337 and 338:
Page 339 and 340:
Page 341 and 342:
Page 343 and 344:
Page 345 and 346:
Multidimensional string fields �
Page 347 and 348:
Page 349 and 350:
Page 351 and 352:
Page 353 and 354:
Page 355 and 356:
Page 357 and 358:
Page 359 and 360:
Page 361 and 362:
Testing Enumerated fields � Range
Page 363 and 364:
Testing Enumerated fields � Examp
Page 365 and 366:
Testing Enumerated fields Example 4
Page 367 and 368:
Page 369 and 370:
Page 371 and 372:
Page 373 and 374:
Identifying variables of a given pr
Page 375 and 376:
Page 377 and 378:
Page 379 and 380:
Page 381 and 382:
Page 383 and 384:
Page 385 and 386:
Page 387 and 388:
Developing test cases for given pro
Page 389 and 390:
Page 391 and 392:
Page 393 and 394:
Page 395 and 396:
Page 397 and 398:
Page 399 and 400:
Page 401 and 402:
Page 403 and 404:
Page 405 and 406:
Page 407 and 408:
Page 409 and 410:
Page 411 and 412:
Page 413 and 414:
Page 415 and 416:
Page 417 and 418:
Page 419 and 420:
Page 421 and 422:
Page 423 and 424:
Page 425 and 426:
Page 427 and 428:
Page 429 and 430:
Page 431 and 432:
Page 433 and 434:
Variable Dimension Equivalence clas
Page 435 and 436:
Variable Dimension Equivalence clas
Page 437 and 438:
Look In Variable Equivalence classe
Page 439 and 440:
For ASCII sub-range of allowed grou
Page 441 and 442:
Files of Type Variable Equivalence
Page 443 and 444:
Appendix I: Day 4 Lecture (Day 4 Le
Page 445 and 446:
Testing in combination-Why? � All
Page 447 and 448:
All possible combinations testing
Page 449 and 450:
All-Pairs Combination Testing � M
Page 451 and 452:
All-Pairs Combination Testing � I
Page 453 and 454:
All-Pairs Combination Testing � 7
Page 455 and 456:
All-Pairs Combination Testing Examp
Page 457 and 458:
Page 459 and 460:
Page 461 and 462:
Page 463 and 464:
Page 465 and 466:
Page 467 and 468:
Page 469 and 470:
Page 471 and 472:
Page 473 and 474:
Page 475 and 476:
Application of all-pairs combinatio
Page 477 and 478:
Page 479 and 480:
Page 481 and 482:
Page 483 and 484:
Page 485 and 486:
Page 487 and 488:
Page 489 and 490:
Page 491 and 492:
Page 493 and 494:
Page 495 and 496:
Page 497 and 498:
Iteration 3: Test case# Y (4) X (3)
Page 499 and 500:
Iteration 3: Test case# a (4) b (3)
Page 501 and 502:
Iteration 6: Test case# a (4) b (3)
Page 503 and 504:
Look In Variable Equivalence classe
Page 505 and 506:
For ASCII sub-range of allowed grou
Page 507 and 508:
Files of Type Variable Equivalence
Page 509 and 510:
Iteration 2 Test case# LI (6) E(4)
Page 511 and 512:
Appendix J: Day 2 Exercises (Day 2
Page 513 and 514:
d. Partition the input domain into
Page 515 and 516:
Page 517:
f. Partition the input domain into
Page 520:
Page 523:
Page 526:
Page 529 and 530:
d. Next, identify the risks associa
Page 532:
Non-Numbers Identify the risks asso
Page 535:
Page 538 and 539:
Day 3 Exercises Date: ____________
Page 540 and 541:
g. Partition the input domain into
Page 542 and 543:
e. What is the input domain? f. Nex
Page 544 and 545:
Enumerated fields 3. In the online
Page 546 and 547:
. In Internet Explorer, in the File
Page 548 and 549:
Alignment
Page 550 and 551:
Border
Page 552 and 553:
Protection
Page 555 and 556:
Analyzing given programs/functions
Page 557 and 558:
3. Is the variable multidimensional
Page 559 and 560:
iv. Next, identify the risks associ
Page 561 and 562:
6. For the ‘Insert Table’ funct
Page 563 and 564:
3. Is the variable multidimensional
Page 565 and 566:
iv. Next, identify the risks associ
Page 568 and 569:
Appendix L: Day 4 Exercises (Day 4
Page 570 and 571:
iv. Determine the number of pairs i
Page 572 and 573:
v. Build the all-pairs combination
Page 574 and 575:
4. Develop a series of combination
Page 576 and 577:
Domain Testing-Test A 1 (100 points
Page 578 and 579:
3. A string field/variable ‘s’
Page 580 and 581:
5. The screenshot of the ‘Login
Page 582 and 583:
7. An I-20 VISA 2 program for inter
Page 584 and 585:
9. ZLTech has a web-based student r
Page 586 and 587:
2. A floating-point field/variable
Page 588 and 589:
4. DVD Collections, Inc. has a shop
Page 590 and 591:
6. There are four variables ‘a’
Page 592 and 593:
8. XYZ 3 credit cards offer a ‘ca
Page 594 and 595:
Appendix N: Performance Test (Perfo
Page 596 and 597:
Performance Test (Revised) (100 poi
Page 598 and 599:
Date: _________________ Evaluation
Page 600 and 601:
Page 602 and 603:
Page 604 and 605:
Page 606 and 607:
Page 608 and 609: Appendix P: Instructional Objective
Page 610 and 611: Appendix Q: Bloom’s Taxonomy of L
Page 612 and 613: 4. Analysis • seeing patterns •
Page 614 and 615: Day 2 Examples Day 3 Examples Trace
Page 616 and 617: Day 4 Exercises Example Level HOO I
Page 618 and 619: Training Schedule Day Instructional
Page 620 and 621: STUDENT APPLICATION RESEARCH INVOLV
Page 622 and 623: 8. Explain how your proposed study
Page 624 and 625: CONSENT FORM: DOMAIN TESTING EXPERI
Page 626 and 627: Appendix V: Call for Participation:
Page 628 and 629: Appendix W: Call for Participation:
Page 630 and 631: Appendix X: Performance Tests’ Ev
Page 632 and 633: • If one student had provided a g
Page 634 and 635: o You could type more or less quick
Page 636 and 637: There's no rationale for these test
Page 638 and 639: page size, perhaps via print or pri
Page 640 and 641: • Using the variables that this s
Page 642 and 643: • In the all-pairs analysis, 7 va
Page 644 and 645: Analysis of Student Performance in
Page 646 and 647: tester with a couple years experien
Page 648 and 649: APPENDIX A In this analysis, I used
Page 650 and 651: esolution: incrementer buttons: .1
Page 652 and 653: A different orientation may be used
Page 654 and 655: Slides orientation vs. notes orient
Page 656 and 657: localized versions of Windows alter
Page 660 and 661: All-pairs analysis The results of t
show all

Domain Testing: Divide and Conquer - Testing Education

Create successful ePaper yourself

Delete template?

Save as template?